Text intent reconstruction method, device, equipment, storage medium and computer product

Abstract

The present application discloses a method, apparatus, device, storage medium, and computer product for text intent reconstruction, which relates to the field of natural language processing technology. The method comprises: obtaining an intent reconstruction sample set; the intent reconstruction sample set comprises input sample data and output sample data; wherein the output sample data carries a semantic flag; performing instruction-supervised fine-tuning on a pre-trained model based on the intent reconstruction sample set and a training target loss function to obtain a text intent reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of the semantic flag; inputting the text to be reconstructed into the text intent reconstruction model to obtain a text intent reconstruction result. The present application utilizes an update strategy based on a semantic flag loss function to improve the quality of text intent reconstruction, thereby achieving efficient training, robust generalization, and significant improvement in response quality of the text intent reconstruction model.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a method, apparatus, device, storage medium, and computer product for reconstructing text intent. Background Art

[0002] Multi-turn dialogue systems are a key component of intelligent question-answering and human-computer interaction. Their core goal is to accurately understand the user's true intent and generate high-quality responses. In recent years, retrieval-augmented generation (RAG) models have significantly improved the accuracy and interpretability of responses by incorporating external knowledge and contextual information. However, user queries often pose challenges to understanding intent due to incomplete information, ambiguous expressions, or unclear references. Traditional retrieval methods struggle to effectively model context, leading to retrieval errors and response bias.

[0003] To mitigate the impact of missing context on intent recognition, some methods attempt to incorporate the entire historical conversation into the current query for intent rewriting. While this improves rewriting performance to some extent, it also results in high memory usage and information redundancy, interfering with model judgment. Furthermore, sliding window strategies are limited in their ability to handle cross-turn references, and are still prone to misunderstandings and information omissions, impacting system stability and usability. Furthermore, current models rely heavily on high-quality annotated data, while existing datasets generally lack structured, semantically clear multi-turn intent rewriting resources, limiting the model's generalization and rewriting effectiveness.

[0004] Therefore, there is an urgent need for a new method that can accurately model semantic consistency and contextual associations, efficiently perform intent rewriting while ensuring generation quality, reduce system resource consumption, and improve the understanding and responsiveness of the dialogue system. Summary of the Invention

[0005] The main purpose of this application is to provide a text intent reconstruction method, device, equipment, storage medium and computer product, aiming to solve the technical problems of low generation quality and high resource consumption in related technologies in text intent reconstruction.

[0006] To achieve the above objectives, this application proposes a text intent reconstruction method, which includes:

[0007] Obtaining an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag bit;

[0008] Based on the intent reconstruction sample set and the training target loss function, the pre-trained model is fine-tuned through instruction supervision to obtain a text intent reconstruction model. The loss weight of the training target loss function is determined based on the distribution of semantic landmarks.

[0009] Input the text to be reconstructed into the text intent reconstruction model to obtain the text intent reconstruction result.

[0010] In one embodiment, the input sample data includes inquiry question information, historical conversation information and prompt words, and the output sample data includes question rewriting information, question intent information and a semantic similarity flag; the semantic similarity flag is determined based on the degree of relevance between the historical conversation information and the inquiry question information, and the question rewriting information carries an intent reconstruction flag.

[0011] In one embodiment, before the step of obtaining the intended reconstruction sample set, the method further includes the following steps:

[0012] Obtain initial question-and-answer text data, and divide the initial question-and-answer text data into a conversation dataset and a historical conversation dataset;

[0013] Based on the conversation dataset and prompt words, a large language model is used to generate multiple rounds of user question-answer pairs and corresponding question intent information to construct a user question-answer dataset. The user question-answer dataset also includes question rephrasing information and semantic similarity flags.

[0014] Based on the historical conversation dataset and user question and answer dataset, an intent reconstruction sample set is constructed.

[0015] In one embodiment, the steps of performing instruction-supervised fine-tuning on the pre-trained model based on the intent reconstruction sample set and the training target loss function to obtain the text intent reconstruction model include:

[0016] Perform word segmentation on the intent reconstruction sample to obtain a sample word sequence;

[0017] Call the pre-trained model to predict the sample word sequence and obtain the word sequence prediction result;

[0018] Based on the training target loss function, determining the difference between the intention reconstruction sample and the word sequence prediction result to obtain the model loss; the loss weight of the training target loss function is updated based on the distribution of the semantic similarity flag and the intention reconstruction flag;

[0019] The pre-trained model is fine-tuned through instruction supervision based on the model loss to obtain a text intent reconstruction model.

[0020] In one embodiment, the step of constructing an intent reconstruction sample set based on a historical conversation dataset and a user question and answer dataset includes:

[0021] Randomly extract multiple rounds of historical dialogue data from the historical dialogue dataset;

[0022] Sequentially extract the question information in a round of user question-answer pairs from the user question-answer dataset;

[0023] A set of input sample data is formed based on historical conversation data, question information, and prompt words corresponding to user question and answer pairs;

[0024] Based on the question intention information, question rewriting information and semantic similarity flag corresponding to the query information, a set of output sample data is formed;

[0025] Based on multiple sets of input sample data and output sample data, an intention reconstruction sample set is constructed.

[0026] In one embodiment, after the step of constructing the intended reconstruction sample set, the method further includes the following steps:

[0027] Perform outlier detection on the text length of the intended reconstruction sample;

[0028] Eliminate text data that exceeds the preset outlier range from the intent reconstruction sample;

[0029] Obtain a set of intent reconstruction samples containing valid data.

[0030] In a second aspect, to achieve the above-mentioned objectives, the present application further provides a text intent reconstruction device, comprising:

[0031] An acquisition module is used to acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag bit;

[0032] The model fine-tuning module is used to perform instruction-supervised fine-tuning on the pre-trained model based on the intent reconstruction sample set and the training target loss function to obtain a text intent reconstruction model; the loss weight of the training target loss function is determined based on the distribution of semantic flags;

[0033] The output module is used to input the text to be reconstructed into the text intent reconstruction model to obtain the text intent reconstruction result.

[0034] On the third aspect, in order to achieve the above-mentioned purpose, the present application continues to provide a text intent reconstruction device, which includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor, and the computer program is configured to implement the steps of the above-mentioned text intent reconstruction method.

[0035] Fourthly, in order to achieve the above-mentioned purpose, the present application continues to provide a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the above-mentioned text intent reconstruction method are implemented.

[0036] In the fifth aspect, in order to achieve the above-mentioned purpose, the present application continues to provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the above-mentioned text intent reconstruction method.

[0037] One or more technical solutions proposed in this application have at least the following technical effects:

[0038] This application provides a text intent reconstruction method. First, high-quality training data including input sample data and output sample data is constructed. For the output sample data set, semantic flags are introduced, and the instruction supervision fine-tuning framework is combined to guide the pre-training model to focus on semantic key word segmentation tokens. During the training process, a loss function update strategy based on semantic flags is incorporated to update the training target loss function to achieve weighted amplification or suppression of key word segmentation tokens. Compared with the traditional full-history splicing or sliding window strategy, this application significantly reduces memory and computing overhead, weakens redundant interference, and strengthens semantic consistency modeling and key content attention. In addition, by using the loss function update strategy, the focus on key word segmentations is increased, the quality of text intent reconstruction is improved, and efficient training, robust generalization, and significant improvement in response quality of the text intent reconstruction model are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 Schematic diagram of the flow of the text intention reconstruction method in one embodiment of the present application.

[0042] Figure 2 This is a flowchart of a method for constructing a sample set intended to be reconstructed in a specific implementation example of this application.

[0043] Figure 3 This is a flowchart of a text intent reconstruction model training method in a specific implementation example of this application.

[0044] Figure 4 This is a schematic diagram of the implementation of loss weight update in a specific implementation example of this application.

[0045] Figure 5 This is a schematic diagram of the structure of the reconstruction device intended for this application.

[0046] Figure 6 This is a schematic diagram of the structure of the equipment intended for reconstruction in this application.

[0047] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] Based on the problems mentioned in the background technology, the embodiment of the present application provides a text intention reconstruction method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the text intention reconstruction method of this application.

[0051] In an embodiment of the present application, the design idea of the text intent reconstruction method is: first, construct an input sample containing question information, historical dialogue information and prompt words, and an output sample containing question rewriting information, question intent information and semantic similarity flags, and then perform instruction supervision fine-tuning on the pre-trained model based on the training target loss function determined based on the distribution of semantic similarity flags and intent reconstruction flags to obtain a text intent reconstruction model, and input the text to be reconstructed into the model to generate high-quality intent rewriting results.

[0052] Specifically, in this embodiment, the text intention reconstruction method includes steps S10 to S30:

[0053] Step S10: Acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag.

[0054] The input sample data includes question information, historical dialogue information and prompt words, and the output sample data includes question rewriting information, question intent information and semantic similarity flag; the semantic similarity flag is determined based on the degree of relevance between historical dialogue information and question information, and the question rewriting information carries the intent reconstruction flag.

[0055] It should be noted that the intent reconstruction sample set includes input sample data and output sample data, and the semantic flags include intent reconstruction flags and semantic similarity flags. Specifically, the intent reconstruction sample set is a training sample set used to train the text intent reconstruction model. It includes input sample data and output sample data. The input sample data consists of query information, historical conversation information, and prompts. The query information (query) represents the original question text posed by the user in the current conversation. Historical conversation information (history) represents several historical interaction records extracted from the previous round or rounds of conversation, which is used to simulate contextual interference or support in real-world scenarios. Prompts (prompts) help the model clarify the output format. The output sample data consists of question rewriting information, question intent information, and semantic similarity flags. The question rewriting information is a standardized sentence that reconstructs the query information based on intent, removes ambiguity, and completes the context. It also carries intent reconstruction flags (such as the "query" flag) at key word positions as a basis for training. The question intent information is the query intent implied by the rewritten question. The semantic similarity flag is represented as a Boolean value or a hierarchical label (true / false), which is used to indicate whether the current "asked question" has a semantic association with the attached "historical conversation" as a basis for training.

[0056] In a feasible implementation manner, before step S10, steps A10 to A30 are further included:

[0057] Step A10: obtaining initial question-and-answer text data, and dividing the initial question-and-answer text data into a conversation data set and a historical conversation data set;

[0058] Step A20: Based on the conversation dataset and prompt words, a large language model is used to generate multiple rounds of user question-answer pairs and corresponding question intent information to construct a user question-answer dataset; the user question-answer dataset also includes question rephrasing information and semantic similarity flags;

[0059] Step A30: construct an intent reconstruction sample set based on the historical conversation dataset and the user question and answer dataset.

[0060] Wherein, step A30 includes steps A31 to A35:

[0061] Step A31: randomly extract multiple rounds of historical conversation data from the historical conversation dataset.

[0062] Step A32, sequentially extracting the question information in a round of user question and answer pairs from the user question and answer dataset;

[0063] Step A33: construct a set of input sample data based on the historical conversation data, the question information, and the prompt words corresponding to the user question and answer pairs;

[0064] Step A34: constructing a set of output sample data based on the question intention information, question rewriting information, and semantic similarity flag corresponding to the query information;

[0065] Step A35: construct an intention reconstruction sample set based on multiple groups of input sample data and output sample data.

[0066] It should be noted that in this embodiment, the initial question-and-answer text data refers to unprocessed conversations or question-and-answer corpora, such as user-system interaction records collected from search logs, FAQs, and encyclopedia conversations. The conversation dataset is data extracted from the initial data and used for subsequent processing to generate user question-and-answer pairs. The historical conversation dataset is data from older conversation rounds extracted from the same initial data, specifically used as "noise" or contextual interference, independent of the new question-and-answer pairs being generated. Prompts: Short text that provides formatting and task guidance for fine-tuning or inference of a large language model, such as "Please generate the next round of user questions and their intents based on the following conversation." Large language models, such as Qwen, OpenAI GPT, and ChatGPT, are used to generate new question-and-answer pairs and intent labels based on prompts and existing corpora. Multi-round user question-and-answer pairs are question-and-answer conversations exceeding one round generated by the large language model, used to simulate continuous conversations in real-world business scenarios.

[0067] Specifically, this implementation first splits the original question-and-answer corpus into "conversation data" and "historical conversation data." Leveraging a large language model and prompts, it generates multiple rounds of question-and-answer pairs, along with corresponding intent, rewritten, and semantic relevance labels, from the conversation data. It then randomly extracts historical noise fragments and a newly generated round of user question-and-answer pairs, constructing the input from "historical conversation + original question + prompt" and the output from "rewritten text + intent label + relevance marker." This synthesis of multiple sets of samples ultimately yields a high-quality set of intent reconstruction samples. This process achieves an integrated construction from the original corpus to structured, tokenized rewritten samples, providing the data foundation for subsequent weighted fine-tuning and efficient intent rewriting models.

[0068] In a specific example, referring to Figure 2 , Figure 2 Flowchart of how to construct a sample set for intent reconstruction.

[0069] For ease of understanding, constructing an intent reconstruction sample set can be done by collecting conversation-encyclopedia data and randomly dividing it into conversation data and historical conversation data. For conversation data, using APIs provided by Qwen, OpenAI, and other platforms, prompt words are used to generate user question-answer pairs and the intent of the current question. This example preferably uses Qwen-plus, generating three rounds of question-answer pairs and the intent of the current question. The data example is as follows:

[0070]

[0071] in, and Represent the questions and answers in the dialogue data respectively; based on 、 And the prompt word prompt is generated using Qwen-plus 、 and ; Represents a new question generated based on the previous round of questions and answers, Represents the answer to the current new question, Represents the intent of the current question.

[0072] For historical conversation data, we randomly extract historical conversation information from the constructed historical conversation database, simulating real-world conversation scenarios. In this example, the intent reconstruction sample set contains at most five rounds of user conversation information.

[0073] To improve the training data's ability to monitor intent information, this example introduces two flags into the training samples: the intent reconstruction flag "query" and the semantic similarity flag "Similarity." "Query" represents the normalized representation of the user's current question after reconstructing the intent; "Similarity" indicates whether the current question has a semantic connection with previous conversations. These two flags serve as the basis for semantic judgment during the training phase.

[0074] The intention reconstruction sample set is ultimately divided into input sample data representing user input and output sample data representing model output. It consists of a query, historical conversation information (history), and a prompt. The construction steps are to randomly extract two to five rounds of data from the historical noise database, and sequentially take the questions in a round of question-answer pairs from the conversation database to combine them. Output sample data By the intention of the current question The construction steps are to take the question intent corresponding to the question in the question-answer pair generated by the model, rewrite the question information in a standardized expression after intent reconstruction, mark the rewritten part with the intent reconstruction flag "query", and add the semantic similarity flag "Similarity". The semantic similarity flag is represented as follows: when the historical conversation information corresponding to the query information is all noise data, the "Similarity" flag of the current sample is semantically irrelevant and has a value of false. When the query information is the current conversation data appended to the historical conversation, the "Similarity" flag is semantically relevant and has a value of true.

[0075] Furthermore, after step A30, steps A40 to A60 are also included:

[0076] Step A40: performing outlier detection on the text length of the intended reconstruction sample.

[0077] Step A50: remove text data that exceeds a preset outlier range from the intent reconstruction sample.

[0078] Step A60: Obtain an intention reconstruction sample set containing valid data.

[0079] Specifically, this implementation uses a text cleanup and outlier removal strategy. In one specific example, the IQR (Interquartile Range) is used to detect outliers on the length of the text. The 25th percentile (Q1), which is the value at the 25th percentile in the sorted data, and the 75th percentile (Q3), which is the value at the 75th percentile in the sorted data, are calculated. For a sorted data set D with N elements, the quantile calculation formula is:

[0080]

[0081] in, Represents the fractional digit of the calculated data, That is, the first elements, k is the percentile (Q1 is 25, Q3 is 75), N is the total number of elements in the dataset.

[0082] Calculate the interquartile range (IQR), which is the difference between Q3 and Q1. The calculation formula is:

[0083] IQR=Q3-Q1

[0084] The upper and lower boundaries of the data are calculated based on IQR. The calculation formula is:

[0085]

[0086] Here, IQR is the interquartile range.

[0087] The lower boundary value is obtained by calculation and upper boundary value . Values outside these two boundaries are considered outliers.

[0088] Eliminate outliers outside the range and filter out data that exceeds the upper and lower boundaries. The filtering condition formula is:

[0089]

[0090] in, is the valid data after filtering, is the lower boundary value, is the upper boundary value.

[0091] After filtering the data, the remaining valid data Used for subsequent intent reconstruction tasks to ensure the validity and stability of the data.

[0092] Step S20, based on the intent reconstruction sample set and the training target loss function, the pre-trained model is fine-tuned with instruction supervision to obtain a text intent reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of the semantic similarity flag and the intent reconstruction flag.

[0093] Step S30: input the text to be reconstructed into the text intention reconstruction model to obtain the text intention reconstruction result.

[0094] It should be noted that the training target loss function measures the error between the model output and the target output. The pre-trained model is a basic model (such as Qwen2.5-14B-Instruct) that has undergone self-supervised learning on large-scale general text. It possesses rich language understanding and generation capabilities, but has not yet been specifically optimized for the intent rewriting task. Instruction-supervised fine-tuning involves further training the pre-trained model using data from downstream tasks in the form of "example input + expected output" to learn to respond to specific formats and task instructions. The text intent reconstruction model is the model obtained after the aforementioned instruction fine-tuning. It accepts an input consisting of the original question, historical context, and prompt words, and generates a rewritten result with standardized expression and intent labeling. Furthermore, this embodiment uses loss weights to assign different amplification / suppression coefficients to different segmentation tokens when calculating the overall loss, emphasizing model learning on key semantic locations and samples with high historical relevance. The loss weights are determined based on the distribution of semantic similarity flags and intent reconstruction flags. In one example, the distribution may refer to the distribution of tokens containing the intention reconstruction flag in the entire training set, and the loss weights corresponding to different types of tokens may be adjusted based on the distribution.

[0095] In a feasible implementation, step S20 includes steps B10 to B40:

[0096] Step B10: perform word segmentation processing on the intention reconstruction sample to obtain a sample word sequence.

[0097] Step B20: Call the pre-trained model to predict the sample word sequence to obtain the word sequence prediction result.

[0098] Step B30, based on the training target loss function, determines the difference between the intent reconstruction sample and the word sequence prediction result to obtain the model loss, wherein the loss weight of the training target loss function is updated based on the distribution of the semantic similarity flag and the intent reconstruction flag.

[0099] Step B40: Perform instruction-supervised fine-tuning on the pre-trained model based on the model loss to obtain a text intent reconstruction model.

[0100] Wherein, step B30 also includes steps B31 to B34:

[0101] Step B31, obtain preset loss weight configuration information.

[0102] Step B32: Based on the preset loss weight configuration information, determine the loss weight of the sample word sequence corresponding to the intention reconstruction flag and the semantic similarity flag.

[0103] Step B33: Update the training target loss function based on the loss weights of the sample word sequences corresponding to the intention reconstruction flag and the semantic similarity flag.

[0104] Step B34: Based on the training target loss function, determine the difference between the intended reconstruction sample and the sample prediction result to obtain the model loss.

[0105] It should be noted that the preset loss weight configuration information can be stored in a configuration file, which can be in JSON format. Specifically, the preset loss weight configuration information can include the following: the intent reconstruction flag: "query", the semantic similarity flag: "Similarity" scaling weight; and the scaling weight of the word segmentation token after the intent reconstruction flag and the semantic similarity flag are set.

[0106] In traditional text generation tasks, each output token has equal weight in the loss function, which fails to effectively distinguish tokens that play a key role in semantic expression. This implementation reads the preset loss weight configuration information from the configuration file, determines the training target loss function, and then returns the context and loss function value after the loss function calculation and update. This realizes the dynamic update of the loss function during the training process, so that the training process can focus more on the word segmentation tokens that play a key role in semantic expression, thereby improving the quality of text intent reconstruction.

[0107] This implementation is a specific training step for the text intent reconstruction model. For ease of understanding, the following is an example:

[0108] Reference Figure 3 and Figure 4 , Figure 3 is the overall logic diagram of this example. Figure 4 Schematic diagram of the implementation of loss weight update.

[0109] In this example, the pre-trained model is a Transformer-based Decoder Only architecture. Preferably, this example performs intent reconstruction training based on Qwen2.5-14-Instruct.

[0110] Specifically, for an intent reconstruction sample, it is first segmented by the Tokenizer (text) to obtain the following segmentation token sequence x， Where t is the sequence length.

[0111]

[0112] For a given token sequence , after the word embedding layer (Embedding), each token Mapped to a vector , to train the pre-trained model.

[0113]

[0114] Input the mapping vector into the Transformer block layer for token prediction

[0115] Subsequently, based on the training objective loss function, the difference between the intent reconstruction sample and the word sequence prediction result is determined to obtain the model loss.

[0116] Specifically, the basic calculation formula of the training target loss function is as follows:

[0117]

[0118] in, is the real token at position t; is the model's predicted probability of the true token at the t-th position based on the previous t-1 positions; T is the sequence length (or the number of valid tokens, excluding padding positions).

[0119] Finally, the pre-trained model is fine-tuned with instruction supervision based on the model loss to obtain the text intent reconstruction model.

[0120] During this process, additional low-rank matrices A and B are introduced through the LoRA algorithm, so that the updated weight matrix reduces video memory usage and speeds up training. The details are as follows:

[0121]

[0122] in, is the updated weight matrix, is the original weight matrix in the pre-trained model, is the incremental weight introduced; For order, is the scaling factor, A and B are low-rank matrices. Usually, the dimensions of these matrices are much smaller than the dimension of W, and r is relatively small; in this example, r is set to 8 and α is 16.

[0123] The training parameters for model training can include the maximum character length (max_seq_length), the training batch size (training_batch_size), the learning rate (learning_rate), and the number of training epochs (num_train_epochs). For example, setting max_seq_length = 4096, train_batch_size = 16, learning_rate = 1e-5, and num_train_epoch = 5 will generate a text intent reconstruction model after training with the first training sample according to these training parameters.

[0124] Furthermore, when predicting word segmentation tokens, the loss function weight is adjusted to improve the rewriting effect of the intent reconstruction model.

[0125] Specifically, when constructing training samples, two key flags, "query" and "Similarity," are introduced. "Query" represents the current user's question after reconstructing the intent, while "Similarity" indicates the semantic relevance between the current question and historical conversations, taking values of true or false. These two flags provide semantic basis for the loss weighting mechanism.

[0126] It is understandable that in traditional text generation tasks, each output token has equal weight in the loss function, which fails to effectively distinguish tokens that play a key role in semantic expression. This example uses a semantic function-based loss weight adjustment strategy for the generated intent reconstruction samples, and scales the training weights of the token loss function after the flags "query" and "Similarity".

[0127] The specific calculation formula is as follows, where: Indicates the loss weight corresponding to each token.

[0128]

[0129] in is the t-th target token, is the model’s predicted probability for the token, is the loss weight coefficient corresponding to the token; T is the total number of valid tokens in the output sequence.

[0130] Specifically, based on the basic calculation method of the training target loss function, the model will use the previously marked token flags to assign corresponding weights to each token. Tokens with higher weights have a greater influence on the loss function, which drives the model to focus more on learning these tokens.

[0131] like Figure 4 As shown, the preset loss weight configuration information is loaded through the configuration file to obtain the scaling weight. The example setting of the preset loss weight configuration information can be: the scaling weight of the flag bit token of the intent reconstruction flag "query" and the semantic similarity flag "Similarity" is 1.2, and the scaling weight of the token after the intent reconstruction flag and the semantic similarity flag is 2. After the loss calculation, the updated context and loss function value are finally returned.

[0132] Take the query "What are the application scenarios of Taylor's formula?" as an example. First, it is processed by the large model's Tokenizer and Embedding vector representation modules. Then, it is processed by the Transformer module to generate Attention fusion. The final output data format is as follows:

[0133] Output = {“query”: text information, “Similarity”: “true / false”}

[0134] After obtaining the output text, the loss scale configuration information is used to obtain the scaling token information and scaling weights, and finally the updated loss function and context are returned for subsequent model training.

[0135] It can be understood that the present application provides a method for text intent reconstruction. First, by introducing historical conversation fragments and data cleaning steps, high-quality training data containing input sample data and output sample data is constructed. The intent reconstruction flag is provided for the reconstructed question information, and the instruction supervision fine-tuning framework is combined to guide the pre-training model to focus on semantic key word segmentation tokens. The loss weight update strategy based on the flag is incorporated into the training process to weightedly amplify or suppress the key word segmentation tokens. Compared with the traditional full-history splicing or sliding window strategy, this method greatly reduces memory and computing overhead, weakens redundant interference, strengthens semantic consistency modeling and key content attention, and uses the loss weight update strategy to improve the quality of text intent reconstruction, thereby achieving efficient training, robust generalization, and significant improvement in response quality of the text intent reconstruction model.

[0136] It should be noted that the above examples are only used to understand this application and do not constitute a limitation on the method of reconstructing the text of this application. More simple transformations based on this technical concept are all within the scope of protection of this application.

[0137] This application also provides a text intention reconstruction device, please refer to Figure 5 , the text intention reconstruction device includes:

[0138] The acquisition module 10 is used to acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag.

[0139] The model fine-tuning module 20 is used to perform instruction-supervised fine-tuning on the pre-trained model based on the intent reconstruction sample set and the training target loss function to obtain a text intent reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of semantic flags.

[0140] The output module 30 is used to input the text to be reconstructed into the text intention reconstruction model to obtain the text intention reconstruction result.

[0141] The text intent reconstruction device provided in this application, which utilizes the text intent reconstruction method described in the aforementioned embodiment, can address the technical issues of low generation quality and high resource consumption in related technologies regarding text intent reconstruction. Compared to related technologies, the text intent reconstruction device provided in this application offers the same beneficial effects as the text intent reconstruction method described in the aforementioned embodiment. Other technical features of the text intent reconstruction device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0142] The present application provides a text intent reconstruction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the text intent reconstruction method in the above embodiment.

[0143] Reference below Figure 6 , which shows a schematic structural diagram of a text intent reconstruction device suitable for implementing the embodiments of the present application. The text intent reconstruction device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The text intention reconstruction device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0144] like Figure 6As shown, the text intent reconstruction device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the text intent reconstruction device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the text intent reconstruction device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a text intent reconstruction device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or have alternatively.

[0145] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0146] The text intent reconstruction device provided by this application, which employs the text intent reconstruction method described in the aforementioned embodiment, can address the technical issues of low generation quality and high resource consumption in related technologies for text intent reconstruction. Compared with related technologies, the beneficial effects of the text intent reconstruction device provided by this application are the same as those of the text intent reconstruction method described in the aforementioned embodiment. Other technical features of this text intent reconstruction device are the same as those disclosed in the method described in the preceding embodiment and are not further elaborated here.

[0147] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0148] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0149] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the text intent reconstruction method in the above-mentioned embodiment.

[0150] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0151] The computer-readable storage medium may be included in the text intent reconstruction device; or it may exist independently without being assembled into the text intent reconstruction device.

[0152] The computer-readable storage medium carries one or more programs. When executed by a text intent reconstruction device, the one or more programs cause the text intent reconstruction device to: obtain password requirement information of a target user; select a standard text intent reconstruction solution consistent with the password requirement information from a preset cryptographic library. The standard text intent reconstruction solution includes a standard cryptographic algorithm, a standard cryptographic protocol, and a standard cryptographic scheme; and output the standard text intent reconstruction solution.

[0153] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0156] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned text intent reconstruction method. This computer-readable storage medium can address the technical issues of low generation quality and high resource consumption in related art text intent reconstruction. Compared to related art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the text intent reconstruction method provided in the aforementioned embodiments, and are not further elaborated here.

[0157] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned text intent reconstruction method when executed by a processor.

[0158] The computer program product provided in this application can address the technical issues of low generation quality and high resource consumption in related technologies for text intent reconstruction. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the text intent reconstruction method provided in the above-mentioned embodiments, and are not further elaborated here.

[0159] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A text intention reconstruction method, characterized in that: The method includes: Acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag bit; Based on the intention reconstruction sample set and the training target loss function, the pre-trained model is fine-tuned by instruction supervision to obtain a text intention reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of the semantic flag bits; Inputting the text to be reconstructed into the text intention reconstruction model to obtain a text intention reconstruction result; The input sample data includes inquiry question information, historical conversation information, and prompt words; the output sample data includes question rewriting information, question intent information, and a semantic similarity flag; the semantic similarity flag is determined based on the degree of relevance between the historical conversation information and the inquiry question information, and the question rewriting information carries an intent reconstruction flag; The step of fine-tuning the pre-trained model through instruction supervision based on the intent reconstruction sample set and the training target loss function to obtain the text intent reconstruction model includes: Perform word segmentation on the intent reconstruction sample to obtain a sample word sequence; Calling the pre-trained model to predict the sample word sequence to obtain a word sequence prediction result; Based on the training target loss function, determining the difference between the intention reconstruction sample and the word sequence prediction result to obtain the model loss; the loss weight of the training target loss function is updated based on the distribution of the semantic similarity flag and the intention reconstruction flag; Performing instruction-supervised fine-tuning on the pre-trained model based on the model loss to obtain the text intent reconstruction model; The step of determining the difference between the intention reconstruction sample and the word sequence prediction result based on the training target loss function to obtain the model loss includes: Get the preset loss weight configuration information; Based on the preset loss weight configuration information, determine the loss weight of the sample word sequence corresponding to the intention reconstruction flag and the semantic similarity flag; Update the training objective loss function based on the loss weights of the sample word sequences corresponding to the intention reconstruction flag and the semantic similarity flag; Based on the training target loss function, the difference between the intention reconstruction sample and the sample prediction result is determined to obtain the model loss; The specific calculation formula for the model loss is: in, Where is the tth target token, is the model’s predicted probability for the token, is the weight coefficient corresponding to the token; T is the total number of valid tokens in the output sequence.

2. The method according to claim 1, wherein Before the step of obtaining the intention reconstruction sample set, the following steps are also included: Obtaining initial question-and-answer text data, and dividing the initial question-and-answer text data into a conversation dataset and a historical conversation dataset; Based on the conversation dataset and prompt words, a large language model is used to generate multiple rounds of user question-answer pairs and corresponding question intent information to construct a user question-answer dataset; the user question-answer dataset also includes question rewriting information and semantic similarity flags; Based on the historical conversation dataset and the user question and answer dataset, the intention reconstruction sample set is constructed.

3. The method according to claim 2, wherein The step of constructing the intention reconstruction sample set based on the historical conversation dataset and the user question and answer dataset includes: Randomly extracting multiple rounds of historical conversation data from the historical conversation dataset; Sequentially extracting question information from a round of user question-answer pairs from the user question-answer dataset; Constructing a set of input sample data based on the historical conversation data, the inquiry question information, and the prompt words corresponding to the user question and answer pairs; Based on the question intention information, question rewriting information and semantic similarity flag corresponding to the query information, a set of output sample data is formed; The intention reconstruction sample set is constructed based on multiple groups of the input sample data and the output sample data.

4. The method according to claim 2, wherein After the step of constructing the intention reconstruction sample set, the method further includes the following steps: Perform outlier detection on the text length of the intended reconstruction sample; Eliminate text data that exceeds a preset outlier range from the intent reconstruction sample; Obtain a set of intent reconstruction samples containing valid data.

5. A text intention reconstruction device, characterized in that: The device comprises: An acquisition module is used to acquire an intention reconstruction sample set; the intention reconstruction sample set includes input sample data and output sample data; wherein the output sample data carries a semantic flag bit; A model fine-tuning module is used to perform instruction-supervised fine-tuning on the pre-trained model based on the intent reconstruction sample set and the training target loss function to obtain a text intent reconstruction model; wherein the loss weight of the training target loss function is determined based on the distribution of the semantic flag bits; An output module, configured to input the text to be reconstructed into the text intention reconstruction model to obtain a text intention reconstruction result; The input sample data includes inquiry question information, historical conversation information, and prompt words; the output sample data includes question rewriting information, question intent information, and a semantic similarity flag; the semantic similarity flag is determined based on the degree of relevance between the historical conversation information and the inquiry question information, and the question rewriting information carries an intent reconstruction flag; The model fine-tuning module is also used to perform word segmentation processing on the intent reconstruction sample to obtain a sample word sequence; Calling the pre-trained model to predict the sample word sequence to obtain a word sequence prediction result; Based on the training target loss function, determining the difference between the intention reconstruction sample and the word sequence prediction result to obtain the model loss; the loss weight of the training target loss function is updated based on the distribution of the semantic similarity flag and the intention reconstruction flag; Performing instruction-supervised fine-tuning on the pre-trained model based on the model loss to obtain the text intent reconstruction model; The step of determining the difference between the intention reconstruction sample and the word sequence prediction result based on the training target loss function to obtain the model loss includes: Get the preset loss weight configuration information; Based on the preset loss weight configuration information, determine the loss weight of the sample word sequence corresponding to the intention reconstruction flag and the semantic similarity flag; Update the training objective loss function based on the loss weights of the sample word sequences corresponding to the intention reconstruction flag and the semantic similarity flag; Based on the training target loss function, the difference between the intention reconstruction sample and the sample prediction result is determined to obtain the model loss; The specific calculation formula for the model loss is: in, Where is the tth target token, is the model’s predicted probability for the token, is the weight coefficient corresponding to the token; T is the total number of valid tokens in the output sequence.

6. A text intention reconstruction device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the text intention reconstruction method according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the text intent reconstruction method according to any one of claims 1 to 4 are implemented.

8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the text intention reconstruction method according to any one of claims 1 to 4 are implemented.

Images