Text intention reconstruction method and device, equipment, storage medium and computer product

Abstract

The invention discloses a text intention reconstruction method and device, equipment, a storage medium and a computer product, and relates to the technical field of natural language processing, and the method comprises the following steps: obtaining an intention reconstruction sample set; the intention reconstruction sample set comprises input sample data and output sample data; wherein the output sample data carries a semantic flag bit; performing instruction supervision fine tuning on the pre-training model based on the intention reconstruction sample set and the training target loss function to obtain a text intention reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution condition of the semantic flag bits; and inputting the to-be-reconstructed text into the text intention reconstruction model to obtain a text intention reconstruction result. According to the method, the strategy is updated based on the semantic flag bit loss function, the quality of text intention reconstruction is improved, and efficient training, robust generalization and remarkable improvement of response quality of the text intention reconstruction model are achieved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and particularly to a method, device, equipment, storage medium and computer product for text intention reconstruction. Background Art

[0002] The multi-turn dialogue system is a key component of intelligent question answering and human-computer interaction. Its core goal is to accurately understand the user's true intention and generate high-quality responses. In recent years, the Retrieval-Augmented Generation (RAG) model has significantly improved the accuracy and interpretability of responses by introducing external knowledge and context information. However, user queries often pose challenges to intention understanding due to issues such as incomplete information, vague expression, or unclear reference. Traditional retrieval methods are difficult to effectively model the context and are prone to retrieval errors or response biases.

[0003] To alleviate the impact of context loss on intention recognition, some methods attempt to introduce the full history of conversations into the current query for intention rewriting. Although the rewriting effect is improved to a certain extent, it results in high memory overhead and information redundancy, interfering with the model's judgment. At the same time, the sliding window strategy has limited ability in dealing with cross-turn references and is still prone to understanding biases and information omissions, affecting the stability and usability of the system. In addition, the current model highly depends on high-quality labeled data, and existing datasets generally lack structured and semantically clear multi-turn intention rewriting resources, restricting the generalization ability and rewriting effect of the model.

[0004] Therefore, there is an urgent need for a new method that can accurately model semantic consistency and context association, efficiently perform intention rewriting while ensuring the generation quality, reduce system resource consumption, and improve the understanding and response capabilities of the dialogue system. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device, equipment, storage medium and computer product for text intention reconstruction, aiming to solve the technical problems of low generation quality and high resource consumption in text intention reconstruction in related technologies.

[0006] To achieve the above purpose, this application proposes a method for text intention reconstruction. The method includes: Obtain an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; among them, the output sample data carries a semantic flag bit; Based on the intention reconstruction sample set and the training objective loss function, perform instruction supervision fine-tuning on the pre-trained model to obtain a text intention reconstruction model; where the loss weight of the training objective loss function is determined based on the distribution of the semantic flag bits; Input the text to be reconstructed into the text intention reconstruction model to obtain the text intention reconstruction result.

[0007] In one embodiment, the input sample data includes query question information, historical conversation information, and prompt words, and the output sample data includes question rewriting information, question intention information, and semantic similarity flag bits; the semantic similarity flag bits are determined based on the relevance between the historical conversation information and the query question information, and the question rewriting information carries an intention reconstruction flag bit.

[0008] In one embodiment, before the step of obtaining the intention reconstruction sample set, the following steps are further included: Obtain the initial Q&A text data, and divide the initial Q&A text data into a dialogue data set and a historical conversation data set; Based on the dialogue data set and the prompt words, generate multiple rounds of user Q&A pairs and corresponding question intention information through a large language model to construct a user Q&A data set; the user Q&A data set also includes question rewriting information and semantic similarity flag bits; Based on the historical conversation data set and the user Q&A data set, construct an intention reconstruction sample set.

[0009] In one embodiment, the steps of obtaining the text intention reconstruction model by performing instruction supervision fine-tuning on the pre-trained model based on the intention reconstruction sample set and the training target loss function include: Perform word segmentation on the intention reconstruction samples to obtain sample word sequences; Call the pre-trained model to predict the sample word sequences to obtain word sequence prediction results; Based on the training target loss function, determine the difference between the intention reconstruction samples and the word sequence prediction results to obtain the model loss; the loss weight of the training target loss function is updated based on the distribution of the semantic similarity flag bits and the intention reconstruction flag bits; Perform instruction supervision fine-tuning on the pre-trained model based on the model loss to obtain the text intention reconstruction model.

[0010] In one embodiment, the steps of constructing an intention reconstruction sample set based on the historical conversation data set and the user Q&A data set include: Randomly extract multiple rounds of historical conversation data from the historical conversation data set; Extract the query question information in one round of user Q&A pairs in sequence from the user Q&A data set; Based on the historical conversation data, the query question information, and the prompt words corresponding to the user Q&A pairs, form a set of input sample data; Based on the question intention information, question rewriting information, and semantic similarity flag bits corresponding to the query question information, form a set of output sample data; Construct an intention reconstruction sample set based on multiple groups of input sample data and output sample data.

[0011] In one embodiment, after the step of constructing the intention reconstruction sample set, the method further includes the steps of: Perform outlier detection on the text lengths of the intention reconstruction samples; Remove the text data that exceeds the preset outlier range from the intention reconstruction samples; Obtain an intention reconstruction sample set containing valid data.

[0012] In a second aspect, to achieve the above object, the present application further provides a text intention reconstruction device, which includes: An acquisition module, configured to acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein, the output sample data carries a semantic flag bit; A model fine-tuning module, configured to perform instruction supervision fine-tuning on a pre-trained model based on the intention reconstruction sample set and a training objective loss function to obtain a text intention reconstruction model; wherein the loss weight of the training objective loss function is determined based on the distribution of the semantic flag bits; An output module, configured to input the text to be reconstructed into the text intention reconstruction model to obtain a text intention reconstruction result.

[0013] In a third aspect, to achieve the above object, the present application further provides a text intention reconstruction device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the above text intention reconstruction method.

[0014] In a fourth aspect, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the above text intention reconstruction method are implemented.

[0015] In a fifth aspect, to achieve the above object, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the above text intention reconstruction method are implemented.

[0016] One or more technical solutions proposed by the present application have at least the following technical effects: This application provides a method for reconstructing text intent. First, high-quality training data including input sample data and output sample data is constructed. For the output sample data set, semantic flag bits are introduced, and combined with the instruction supervision fine-tuning framework, to guide the pre-trained model to focus on semantic key tokenization tokens. During the training process, a loss function update strategy based on semantic flag bits is incorporated to update the training objective loss function, so as to achieve weighted amplification or suppression of key tokenization tokens. Compared with the traditional full-volume historical splicing or sliding window strategy, this application significantly reduces the memory and computational overhead, weakens redundant interference, strengthens semantic consistency modeling and key content attention. In addition, by using the loss function update strategy, the attention to key tokenization is increased, the quality of text intent reconstruction is improved, and efficient training, robust generalization and significant improvement in response quality of the text intent reconstruction model are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a flowchart of the method for reconstructing text intent in an embodiment of this application.

[0020] Figure 2 It is a flowchart of the construction method of the intent reconstruction sample set in a specific embodiment of this application.

[0021] Figure 3 It is a flowchart of the training method of the text intent reconstruction model in a specific embodiment of this application.

[0022] Figure 4 It is an implementation diagram of the loss weight update in a specific embodiment of this application.

[0023] Figure 5 It is a structural diagram of the text intent reconstruction device of this application.

[0024] Figure 6 It is a structural diagram of the text intent reconstruction equipment of this application.

[0025] The realization of the purpose, functional characteristics and advantages of this application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not used to limit this application.

[0027] To better understand the technical solutions of this application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0028] Based on the problems mentioned in the background art, the embodiments of this application provide a text intention reconstruction method. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the text intention reconstruction method of this application.

[0029] In the embodiments of this application, the design idea of the text intention reconstruction method is as follows: First, construct an input sample including inquiry question information, historical dialogue information, and prompt words, and an output sample including question rewriting information, question intention information, and semantic similarity flag bits. Subsequently, use the training objective loss function determined based on the distribution of the semantic similarity flag bits and the intention reconstruction flag bits to perform instruction supervision fine-tuning on the pre-trained model to obtain a text intention reconstruction model, and input the text to be reconstructed into this model to generate a high-quality intention rewriting result.

[0030] Specifically, in this embodiment, the text intention reconstruction method includes steps S10 to S30: Step S10, obtain an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein, the output sample data carries a semantic flag bit.

[0031] The input sample data includes inquiry question information, historical dialogue information, and prompt words, and the output sample data includes question rewriting information, question intention information, and semantic similarity flag bits; the semantic similarity flag bit is determined based on the relevance between the historical dialogue information and the inquiry question information, and the question rewriting information carries an intention reconstruction flag bit.

[0032] It should be noted that the intention reconstruction sample set includes input sample data and output sample data, and the semantic flag bits include the intention reconstruction flag bit and the semantic similarity flag bit. Specifically, the intention reconstruction sample set is a training sample set for training a text intention reconstruction model, which includes input sample data and output sample data. The input sample data is composed of query question information, historical dialogue information, and prompt words. The query question information (query) represents the original question text currently proposed by the user in the dialogue. The historical dialogue information (history) represents several historical interaction records extracted from the previous round or multiple rounds of dialogue, which is used to simulate the context interference or support in the real scenario. The prompt word (Prompt) is used to help the model clarify the output form. The output sample data is composed of question rewriting information, question intention information, and semantic similarity flag bits. Among them, the question rewriting information is a normalized sentence that removes ambiguity and complements the context after the "query question information" is reconstructed by intention, and at the same time carries the intention reconstruction flag bit (such as the "query" flag bit) at the key word segmentation position, which is used as the basis during training. The question intention information is the query question intention implied by the rewritten question. The semantic similarity flag bit is represented as a boolean value or a graded label (true / false), which is used to indicate whether there is a semantic association between the current "query question" and the attached "historical dialogue", and is used as the basis during training.

[0033] In a feasible implementation manner, before step S10, steps A10 to A30 are further included: Step A10, obtaining initial Q&A text data, and dividing the initial Q&A text data into a dialogue data set and a historical dialogue data set; Step A20, based on the dialogue data set and the prompt words, generating multiple rounds of user Q&A pairs and corresponding question intention information through a large language model, and constructing a user Q&A data set; the user Q&A data set also includes question rewriting information and semantic similarity flag bits; Step A30, constructing an intention reconstruction sample set based on the historical dialogue data set and the user Q&A data set.

[0034] Among them, step A30 includes steps A31 to A35: Step A31, randomly extracting multiple rounds of historical dialogue data from the historical dialogue data set.

[0035] Step A32, sequentially extracting the query question information in one round of user Q&A pairs from the user Q&A data set; Step A33, constructing a set of input sample data based on the historical dialogue data, the query question information, and the prompt words corresponding to the user Q&A pair; Step A34, constructing a set of output sample data based on the question intention information, the question rewriting information, and the semantic similarity flag bit corresponding to the query question information; Step A35: Based on multiple groups of input sample data and output sample data, construct an intention reconstruction sample set.

[0036] It should be noted that in this embodiment, the initial Q&A text data is unprocessed dialogue or Q&A corpus, such as the set of user-system interaction records collected from search logs, FAQs, and encyclopedia dialogues. The dialogue data set is extracted from the initial data and used for subsequent processing to generate user Q&A pairs. The historical dialogue data set is extracted from the same initial data and specifically used as the old-round dialogue of "noise" or context interference, independent of the data of the new Q&A pairs being generated. Prompt: A short text that provides format and task guidance for fine-tuning or inference of large language models, such as "Please generate the next round of user questions and their intentions based on the following dialogue". Large language models, such as Qwen, OpenAI GPT, ChatGPT, etc., are used to generate new Q&A pairs and intention labels based on prompts and existing corpora. Multi-round user Q&A pairs are generated by large language models and are more than one round of Q&A dialogue, used to simulate continuous dialogue in real business scenarios.

[0037] Specifically, in this embodiment, the original Q&A corpus is first split into "dialogue data" and "historical dialogue data", and multi-round Q&A pairs and corresponding intentions, rewritten and semantic association labels are generated on the dialogue data with the help of large language models and prompts; then randomly extract historical noise fragments and a new round of user Q&A pairs generated, construct the input with "historical dialogue + original question + prompt", and construct the output with "rewritten text + intention label + relevance flag", synthesize multiple groups of samples, and finally summarize to obtain a high-quality intention reconstruction sample set. This process realizes the integrated construction from the original corpus to the structured and labeled rewritten samples, providing a data basis for subsequent weighted fine-tuning and efficient intention rewriting models.

[0038] In a specific example, refer to Figure 2 , Figure 2 is a schematic flow diagram of the construction method of the intention reconstruction sample set.

[0039] For ease of understanding, the method for constructing the intention reconstruction sample set can be: collect dialogue-encyclopedia data and randomly divide it into dialogue data and historical dialogue data. For the dialogue data, by using the APIs provided by Qwen, OpenAI, etc., and using prompts to generate user Q&A pairs and the intentions of the current questions. Preferably, in this example, Qwen-plus is adopted, and a total of three rounds of Q&A pairs and the intentions of the current questions are generated. The data examples are as follows: Among them, and respectively represent the questions and answers in the dialogue data; based on , and the prompt are generated using Qwen-plus , and ; represents a new question generated based on the previous round of question and answer, represents the answer to the current new question, represents the intention of the current question.

[0040] For historical conversation data, randomly extract historical conversation information in a simulated real conversation scenario from the constructed historical conversation database. In the intention reconstruction sample set constructed in this example, at most five rounds of user conversation information are saved for historical conversation data.

[0041] To improve the supervision ability of training data for intention information, this example designs and introduces two flag bits, the intention reconstruction flag bit "query" and the semantic similarity flag bit "Similarity", in the training samples. Among them, "query" represents the normalized expression of the user's current question after intention reconstruction; "Similarity" is used to indicate whether the current question has semantic association with the historical conversation. These two flag bits are used as the basis for semantic judgment in the training stage.

[0042] The intention reconstruction sample set is finally divided into input sample data representing user input and output sample data representing model output. The input sample data consists of the query question, historical conversation information (history), and the prompt. The construction steps are to randomly extract two to five rounds of data from the historical noise database and combine the question in a round of question and answer pair taken in sequence from the conversation database. The output sample data consists of the intention of the current question and semantic relevance. The construction steps are to take the question intention corresponding to the question in the question and answer pair generated by the model, the rewritten information of the question after intention reconstruction in the normalized expression, mark the intention rewritten part with the intention reconstruction flag bit "query", and add the semantic similarity flag bit "Similarity". The representation form of the semantic similarity flag bit is: when all the historical conversation information corresponding to the query question information is noise data, the "Similarity" flag bit of the current sample is semantically irrelevant and the value is false; when the query question information is the current conversation data appended in the historical conversation, the "Similarity" flag bit is semantically relevant and the value is true.

[0043] Furthermore, after step A30, steps A40 to A60 are also included: Step A40, perform outlier detection on the text length of the intention reconstruction samples.

[0044] Step A50, removing text data that exceeds a preset outlier range from the intended reconstruction sample.

[0045] Step A60, obtaining an intention reconstruction sample set containing valid data.

[0046] Specifically, this implementation adopts a text cleaning and outlier removal strategy. In a specific example, IQR (Interquartile Range) is used to detect outliers on the length of the text. The 25% quantile (Q1) of the data is calculated, that is, the value at the 25% position in the sorted data, and the 75% quantile (Q3) is calculated, that is, the value at the 75% position in the sorted data. For a set of sorted data sets D, which has N elements, the calculation formula for the quantile is: in, Represents the fractional digit of the calculated data, That is, the first elements, k is the percentile (Q1 is 25, Q3 is 75), N is the total number of elements in the dataset.

[0047] Calculate the interquartile range (IQR), which is the difference between Q3 and Q1. The calculation formula is: IQR=Q3-Q1 According to IQR, the upper and lower boundaries of the data are calculated using the following formula: Here, IQR is the interquartile range.

[0048] The lower boundary value is obtained by calculation and upper boundary value . Values ​​outside these two boundaries are considered outliers.

[0049] Eliminate outliers that exceed this range and filter out data that exceeds the upper and lower boundaries. The filtering condition formula is: in, is the valid data after filtering. is the lower boundary value, is the upper boundary value.

[0050] After filtering the data, the remaining valid data Used for subsequent intention reconstruction tasks to ensure the validity and stability of the data.

[0051] Step S20: Perform instruction supervision fine-tuning on the pre-trained model based on the intention reconstruction sample set and the training objective loss function to obtain a text intention reconstruction model. The loss weight of the training objective loss function is determined based on the distribution of the semantic similarity flag bit and the intention reconstruction flag bit.

[0052] Step S30: Input the text to be reconstructed into the text intention reconstruction model to obtain the text intention reconstruction result.

[0053] It should be noted that the training objective loss function is a function used to measure the error between the model output and the target output. The pre-trained model is a basic model that has undergone self-supervised learning on a large-scale general text (such as Qwen2.5-14B-Instruct), which has rich language understanding and generation capabilities, but has not been specifically optimized for the intention rewriting task. Instruction supervision fine-tuning is to further train the model with downstream task data in the form of "example input + expected output" on the basis of the pre-trained model, so that it can learn to respond to specific formats and task instructions. The text intention reconstruction model is the model obtained after the above instruction fine-tuning, which can receive an input containing the original question, historical context, and prompt words, and generate a rewritten result with a standardized expression and intention label. Further, in this embodiment, when calculating the overall loss through the loss weight, different amplification / suppression coefficients are assigned to different tokenizations of tokens to emphasize the model's learning on key semantic positions and samples highly relevant to history. The loss weight is determined based on the distribution of the semantic similarity flag bit and the intention reconstruction flag bit. In one example, the distribution can refer to the distribution of tokens containing the intention reconstruction flag bit in the entire training set, and the loss weights corresponding to different types of tokens can be adjusted according to this distribution.

[0054] In a feasible embodiment, step S20 includes steps B10 to B40: Step B10: Perform tokenization on the intention reconstruction samples to obtain a sample word sequence.

[0055] Step B20: Invoke the pre-trained model to predict the sample word sequence to obtain a word sequence prediction result.

[0056] Step B30: Based on the training objective loss function, determine the difference between the intention reconstruction samples and the word sequence prediction result to obtain the model loss. Among them, the loss weight of the training objective loss function is updated based on the distribution of the semantic similarity flag bit and the intention reconstruction flag bit.

[0057] Step B40: Perform instruction supervision fine-tuning on the pre-trained model based on the model loss to obtain a text intention reconstruction model.

[0058] Among them, step B30 also includes steps B31 to B34: Step B31: Obtain the preset loss weight configuration information.

[0059] Step B32: Based on the preset loss weight configuration information, determine the loss weights of the sample word sequences corresponding to the intent reconstruction flag bit and the semantic similarity flag bit.

[0060] Step B33: Update the training objective loss function based on the loss weights of the sample word sequences corresponding to the intent reconstruction flag bit and the semantic similarity flag bit.

[0061] Step B34: Based on the training objective loss function, determine the difference between the intent reconstruction sample and the sample prediction result to obtain the model loss.

[0062] It should be noted that the preset loss weight configuration information can be stored in a configuration file, and the data format of this configuration file can be in json format. Specifically, the preset loss weight configuration information can include the following content: the scaling weight of the intent reconstruction flag bit: "query" and the semantic similarity flag bit: "Similarity"; the scaling weight of the token after the intent reconstruction flag bit and the semantic similarity flag bit.

[0063] In traditional text generation tasks, the weights of each output token in the loss function are equal, and tokens that play a key role in semantic expression cannot be effectively distinguished. In this embodiment, by reading the preset loss weight configuration information from the configuration file, the training objective loss function is determined, and then after the loss function calculation and update, the context and the loss function value are returned, realizing the dynamic update of the loss function during the training process, so as to be more focused on the tokenized tokens that play a key role in semantic expression during the training process, thereby improving the quality of text intent reconstruction.

[0064] This embodiment is the specific training step of the text intent reconstruction model. For the convenience of understanding, the following is illustrated with a specific example: Refer to Figure 3 and Figure 4 , Figure 3 which is the overall logic schematic diagram of this example, Figure 4 and

[0065] is the implementation schematic diagram of loss weight update.

[0066] Specifically, for an intent reconstruction sample, first, it is tokenized by the tokenizer Tokenizer(text) to obtain the following tokenized token sequence x，where t is the sequence length.

[0067] For a given token sequence of word segmentation , through the Embedding layer, each token is mapped to a vector for training the pre-trained model.

[0068] The mapped vector is input into the Transformer block Layer for token prediction Subsequently, based on the training objective loss function, the difference between the intent reconstruction sample and the word sequence prediction result is determined to obtain the model loss.

[0069] Specifically, the basic calculation formula of the training objective loss function is as follows: where is the true token at the t-th position; is the prediction probability of the model for the true token at the t-th position based on the previous t - 1 positions; T is the sequence length (or the number of valid tokens, excluding padding positions).

[0070] Finally, based on the model loss, the pre-trained model is fine-tuned with instruction supervision to obtain the text intent reconstruction model.

[0071] During this process, additional low-rank matrices A and B are introduced through the LoRA algorithm, so that the updated weight matrix reduces the video memory usage and speeds up the training. Specifically as follows: where is the updated weight matrix, is the original weight matrix in the pre-trained model, is the introduced incremental weight; is the rank, is the scaling factor, A and B are low-rank matrices, usually the dimensions of these matrices are much smaller than the dimension of W, and r is relatively small; in this example, r is set to 8 and α is 16.

[0072] The training parameters for model training can include the maximum character length max_seq_length, the training batch size, the learning rate learning_rate, and the number of training epochs num_train_epochs. For example, set max_seq_length = 4096, train_batch_size = 16, learning_rate = 1e-5, num_train_epoch = 5. In this way, after training according to these training parameters and using the first training sample, a text intent reconstruction model can be obtained.

[0073] Furthermore, when performing token prediction for word segmentation, the loss function weights are also adjusted to improve the rewriting effect of the intent reconstruction model.

[0074] Specifically, when constructing training samples, two key flag bits "query" and "Similarity" are introduced. Among them, "query" represents the expression after the intent reconstruction of the current user's question, and "Similarity" indicates the semantic relevance between the current question and the historical conversation, with a value of true or false. These two flag bits provide semantic basis for the weighted mechanism of the loss weights.

[0075] It can be understood that in traditional text generation tasks, the weights of each output token in the loss function are equal, and tokens that are crucial for semantic expression cannot be effectively distinguished. In this example, for the generated intent reconstruction samples, a loss weight adjustment strategy based on semantic functions is adopted to scale the training weights of the token loss function after the flag bits "query" and "Similarity".

[0076] The specific calculation formula is as follows, where represents the loss weight corresponding to each token.

[0077] where is the t-th target token, is the prediction probability of the model for this token, is the loss weight coefficient corresponding to this token; T is the total number of valid tokens in the output sequence.

[0078] Specifically, on the basis of the basic calculation method of the training target loss function, the model will use the previously marked token flag bits to assign corresponding weights to each token. Tokens with higher weights will have a greater influence in the loss function, thus prompting the model to focus more on the learning of these tokens.

[0079] Such as Figure 4As shown, the preset loss weight configuration information is loaded through a configuration file to obtain the scaling weight. An example setting of the preset loss weight configuration information can be: the flag token scaling weight of the intent reconstruction flag bit "query" and the semantic similarity flag bit "Similarity" is 1.2, and the token scaling weight after the intent reconstruction flag bit and the semantic similarity flag bit is 2. After loss calculation, the updated context and loss function value are finally returned.

[0080] Taking the query "What are the application scenarios of Taylor's formula?" as an example, first, it is processed through modules such as the large model Tokenizer and embedding vector representation, and Attention fusion is generated through the transformer module. Finally, the output data format is as follows: Output = {"query": text information, "Similarity": "true / false"} After obtaining the output text, through the loss scale configuration information, the information of the scaled token and the scaling weight are obtained, and finally the updated loss function and context are returned for subsequent model training.

[0081] It can be understood that by providing a text intent reconstruction method, this application first constructs high-quality training data containing input sample data and output sample data by introducing historical dialogue fragments and data cleaning steps. An intent reconstruction flag bit is provided for the reconstructed query question information, and combined with the instruction supervised fine-tuning framework, it guides the pre-trained model to focus on the semantic key segmentation tokens; during the training process, a loss weight update strategy based on the flag bit is incorporated to amplify or suppress the key segmentation tokens. Compared with the traditional full-scale historical splicing or sliding window strategy, this method significantly reduces the memory and computational overhead, weakens the redundant interference, strengthens the semantic consistency modeling and key content attention, and uses the loss weight update strategy to improve the quality of text intent reconstruction, realizing the efficient training, robust generalization of the text intent reconstruction model and a significant improvement in the response quality.

[0082] It should be noted that the above examples are only used to understand this application and do not constitute a limitation on the text intent reconstruction method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0083] This application also provides a text intent reconstruction device. Please refer to Figure 5 , the text intent reconstruction device includes: An acquisition module 10, configured to acquire an intent reconstruction sample set; the intent reconstruction sample set includes input sample data and output sample data; wherein, the output sample data carries a semantic flag bit.

[0084] The model fine-tuning module 20 is used to perform instruction supervision fine-tuning on the pre-trained model based on the intention reconstruction sample set and the training objective loss function to obtain a text intention reconstruction model; wherein the loss weight of the training objective loss function is determined based on the distribution of semantic flag bits.

[0085] The output module 30 is used to input the text to be reconstructed into the text intention reconstruction model to obtain a text intention reconstruction result.

[0086] The text intention reconstruction device provided by this application adopts the text intention reconstruction method in the above-mentioned embodiment, and can solve the technical problems of low generation quality and high resource consumption in the related technology in text intention reconstruction. Compared with the related technology, the beneficial effects of the text intention reconstruction device provided by this application are the same as those of the text intention reconstruction method provided by the above-mentioned embodiment, and other technical features in the text intention reconstruction device are the same as those disclosed in the method of the above-mentioned embodiment, and will not be elaborated here.

[0087] This application provides a text intention reconstruction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the text intention reconstruction method in the above-mentioned embodiment.

[0088] Next, refer to Figure 6 , which shows a schematic structural diagram of a text intention reconstruction device suitable for implementing the embodiments of this application. The text intention reconstruction device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The text intention reconstruction device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0089] As Figure 6As shown, the text intention reconstruction device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the text intention reconstruction device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the text intention reconstruction device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a text intention reconstruction device having various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0090] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0091] The text intention reconstruction device provided by the present application adopts the text intention reconstruction method in the above embodiments, and can solve the technical problems of low generation quality and large resource consumption in the related art in text intention reconstruction. Compared with the related art, the beneficial effects of the text intention reconstruction device provided by the present application are the same as those of the text intention reconstruction method provided by the above embodiments, and other technical features in the text intention reconstruction device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0092] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0093] The above are only the specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

[0094] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the text intention reconstruction method in the above embodiments.

[0095] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0096] The above computer-readable storage medium can be included in the text intention reconstruction device; it can also exist separately without being assembled into the text intention reconstruction device.

[0097] The above computer-readable storage medium carries one or more programs, which, when executed by the text intention reconstruction device, cause the text intention reconstruction device to: obtain the password requirement information of the target user. Select a standard text intention reconstruction scheme consistent with the password requirement information from a preset cryptographic library. The standard text intention reconstruction scheme includes a standard cryptographic algorithm, a standard cryptographic protocol, and a standard cryptographic scheme. Output the standard text intention reconstruction scheme.

[0098] Computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0100] The modules described in the embodiments of the present application may be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0101] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned text intention reconstruction method, which can solve the technical problems of low generation quality and high resource consumption in text intention reconstruction in the related art. Compared with the related art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the text intention reconstruction method provided by the above embodiments, and will not be elaborated here.

[0102] This application also provides a computer program product, including a computer program, which when executed by a processor, implements the steps of the text intention reconstruction method as described above.

[0103] The computer program product provided by this application can solve the technical problems of low generation quality and high resource consumption in text intention reconstruction in the related art. Compared with the related art, the beneficial effects of the computer program product provided by this application are the same as those of the text intention reconstruction method provided by the above embodiments, and will not be elaborated here.

[0104] The above are only partial embodiments of this application, and do not limit the patent scope of this application. Any equivalent structural transformation made by using the content of the specification and drawings of this application under the technical concept of this application, or direct / indirect application in other related technical fields, is included in the patent protection scope of this application.

Claims

1. A method for text intention reconstruction, characterized in that, The method described above includes: Obtaining an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein, the output sample data carries a semantic flag bit; Performing instruction supervision fine-tuning on a pre-trained model based on the intention reconstruction sample set and a training objective loss function to obtain a text intention reconstruction model; wherein the loss weight of the training objective loss function is determined based on the distribution of the semantic flag bits; Inputting the text to be reconstructed into the text intention reconstruction model to obtain a text intention reconstruction result.

2. The method according to claim 1, characterized in that, The input sample data includes query question information, historical dialogue information, and prompt words, and the output sample data includes question rewriting information, question intention information, and a semantic similarity flag bit; the semantic similarity flag bit is determined based on the relevance between the historical dialogue information and the query question information, and the question rewriting information carries an intention reconstruction flag bit.

3. The method according to claim 2, wherein The step of performing instruction supervision fine-tuning on the pre-trained model based on the intention reconstruction sample set and the training objective loss function to obtain the text intention reconstruction model includes: Performing word segmentation on the intention reconstruction samples to obtain sample word sequences; Invoking the pre-trained model to predict the sample word sequences to obtain word sequence prediction results; Based on the training objective loss function, determining the difference between the intention reconstruction samples and the word sequence prediction results to obtain a model loss; the loss weight of the training objective loss function is updated based on the distribution of the semantic similarity flag bits and the intention reconstruction flag bits; Performing instruction supervision fine-tuning on the pre-trained model based on the model loss to obtain the text intention reconstruction model.

4. The method according to claim 1, wherein Before the step of obtaining the intention reconstruction sample set, there is also a step: Obtaining initial Q&A text data, and dividing the initial Q&A text data into a dialogue data set and a historical dialogue data set; Based on the dialogue data set and prompt words, generating multiple rounds of user Q&A pairs and corresponding question intention information through a large language model to construct a user Q&A data set; the user Q&A data set also includes question rewriting information and a semantic similarity flag bit; Based on the historical dialogue data set and the user Q&A data set, constructing the intention reconstruction sample set.

5. The method according to claim 4, characterized in that, The step of constructing the intention reconstruction sample set based on the historical dialogue data set and the user Q&A data set includes: Randomly extracting multiple rounds of historical dialogue data from the historical dialogue data set; Sequentially extracting the query question information in one round of user Q&A pairs from the user Q&A data set; Based on the historical dialogue data, the query question information, and the prompt words corresponding to the user Q&A pairs, forming a set of input sample data; Based on the question intention information, question rewriting information, and semantic similarity flag bit corresponding to the query question information, forming a set of output sample data; Based on multiple sets of the input sample data and the output sample data, constructing the intention reconstruction sample set.

6. The method according to claim 4, wherein After the step of constructing the intention reconstruction sample set, there is also a step: Performing outlier detection on the text lengths of the intention reconstruction samples; Eliminate text data that exceeds a preset outlier range from the intention reconstruction sample; Obtain a set of intent reconstruction samples containing valid data.

7. A text intention reconstruction device, characterized in that The device comprises: An acquisition module, used to acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag; A model fine-tuning module, used to perform instruction-supervised fine-tuning on the pre-trained model based on the intent reconstruction sample set and the training target loss function to obtain a text intent reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of the semantic flag bits; The output module is used to input the text to be reconstructed into the text intention reconstruction model to obtain the text intention reconstruction result.

8. A text intention reconstruction device, characterized in that, The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the text intention reconstruction method according to any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the text intent reconstruction method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that, The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the text intention reconstruction method according to any one of claims 1 to 6 are implemented.

Images