User intention compression method and device and electronic equipment

Abstract

The invention discloses a user intention compression method and device and electronic equipment, and relates to the field of data processing. In the method, a training data set is obtained, and the training data set comprises rule synthesis data, dialogue annotation data and diversity synthesis data; training a preset language model based on the training data set, and performing parameter adjustment on the preset language model through a first preset matrix and a second preset matrix in the training process to obtain a compression model for compressing the user intention; acquiring a plurality of historical conversations of the user and current input of the user; splicing the plurality of historical conversations and the current input to obtain target content; and inputting the target content into the compression model to obtain a user intention compression result. By implementing the technical scheme provided by the invention, the accuracy of user intention recognition is improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and particularly to a method, apparatus, and electronic device for compressing user intents. Background Art

[0002] With the rapid development of information technology and the field of artificial intelligence, multi-turn dialogue systems, such as intelligent customer service and virtual assistants, have been widely used in daily life and business operations. These systems need to effectively process and understand the dialogue intents of users to provide accurate and personalized services. With the popularization of dialogue systems, users expect to obtain faster and more accurate responses, which poses higher requirements for the intelligent processing capabilities of the systems.

[0003] Currently, although existing multi-turn dialogue systems can handle basic queries and interactions, they often perform poorly in understanding complex user intents, especially in long-term and multi-topic conversations. Traditional dialogue systems usually rely on static scripts and limited preset responses, and it is difficult to adapt to the diversity and dynamic changes of user needs. In addition, when processing long conversations containing redundant and non-critical information, these systems often cannot accurately extract the current intent of the user, resulting in inaccurate recognition of user intents.

[0004] Therefore, there is an urgent need for a method, apparatus, and electronic device for compressing user intents. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for compressing user intents, which improves the accuracy of user intent recognition.

[0006] In a first aspect of this application, a method for compressing user intents is provided. The method includes: obtaining a training data set, where the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data; training a preset language model based on the training data set, and adjusting the parameters of the preset language model through a first preset matrix and a second preset matrix during the training process to obtain a compression model for compressing user intents; obtaining multiple historical conversations of a user and the current input of the user; concatenating the multiple historical conversations and the current input to obtain a target content; and inputting the target content into the compression model to obtain a user intent compression result.

[0007] By adopting the above technical solution, by obtaining a training data set including rule synthesis data, dialogue annotation data, and diversity synthesis data, training a preset language model, and introducing a first preset matrix and a second preset matrix to adjust the model parameters, an efficient and accurate intent compression model is obtained. This intent compression model can make full use of the historical dialogue and the context information of the current input, splice to obtain the target content, and input it into the compression model to generate a compact and highly concentrated user intent representation. This intent compression method can not only filter out redundant and irrelevant information, but also accurately capture the core needs of users, providing high-quality input for subsequent task processing. At the same time, the compressed intent representation greatly reduces the overhead of data transmission and storage, improving scalability and real-time performance. Through the user intent compression method of the present invention, efficient, accurate, and low-cost user intent understanding and task processing can be achieved.

[0008] Optionally, in the obtaining of the training data set, obtaining the dialogue annotation data specifically includes: obtaining multiple groups of historical dialogue data, each group of the historical dialogue data including multiple historical dialogues and a current input; using a preset summary model to perform semantic extraction on the historical dialogues and the current input in each group of the historical dialogue data to generate a preliminary annotation result; sending the preliminary annotation result to an artificial annotation platform for an artificial annotator to check and correct the preliminary annotation result to obtain an artificial annotation result; generating the dialogue annotation data based on the preliminary annotation result and the artificial annotation result.

[0009] By adopting the above technical solution, when obtaining the training data set, a semi-automatic method for generating dialogue annotation data is adopted, effectively improving the quality and efficiency of the annotation data. First, semantic extraction is performed on the historical dialogue data through a preset summary model to generate a preliminary annotation result, reducing the workload of manual annotation. Then, the preliminary annotation result is sent to the artificial annotation platform, and the artificial annotator checks and corrects it to ensure the accuracy and consistency of the annotation result. Finally, the final dialogue annotation data is generated based on the preliminary annotation result and the artificial annotation result. This semi-automatic annotation method makes full use of the semantic extraction ability of the machine learning model and the professional knowledge of manual annotation, ensuring the high quality of the dialogue annotation data while improving the annotation efficiency.

[0010] Optionally, in the obtaining of the training data set, obtaining the rule synthesis data specifically includes: constructing a dialogue template library based on the historical dialogue data, the dialogue template library including multiple dialogue templates, each of the dialogue templates consisting of multiple rounds of dialogues; for each of the dialogue templates, generating the rule synthesis data through a preset code script in a way of missing core content and rhetorical questions.

[0011] By adopting the above technical solutions, when obtaining the training dataset, a rule-based synthetic data generation method based on dialogue templates is introduced, effectively expanding the scale and diversity of the training data. First, a dialogue template library is constructed based on historical dialogue data. Each dialogue template consists of multiple rounds of dialogues, capturing the typical structure and process of the dialogue. Then, for each dialogue template, through a preset code script, rule-based synthetic data is generated in the way of missing core content and rhetorical questions. This data synthesis method can automatically generate a large number of dialogues with certain regularity according to the dialogue templates, covering a variety of possible user intents and dialogue scenarios. By adding rule-based synthetic data, the intent compression model can learn richer and more comprehensive dialogue patterns and intent expression methods, improving the generalization ability of the model.

[0012] Optionally, in the process of obtaining the training dataset, obtaining diverse synthetic data specifically includes: merging the rule-based synthetic data and the dialogue annotation data to obtain merged data; using a preset data augmentation algorithm to perform augmentation processing on the merged data to obtain augmented data, where the preset data augmentation algorithm includes generating synonymous sentences, generating rhetorical questions, and adding random noise; inputting the augmented data into a preset large model and generating the diverse synthetic data based on the preset large model.

[0013] By adopting the above technical solutions, when obtaining the training dataset, a diverse synthetic data generation method combining data augmentation and large model generation is used, further improving the richness and diversity of the training data. First, the rule-based synthetic data and the dialogue annotation data are merged to obtain a high-quality initial dataset. Then, data augmentation algorithms such as generating synonymous sentences, generating rhetorical questions, and adding random noise are used to perform augmentation processing on the merged data to generate augmented data with similar semantics but diverse expression forms. Finally, the augmented data is input into a preset large model to generate diverse synthetic data. By combining data augmentation and large model generation, a large amount of high-quality training data is obtained, covering various variants and possible forms of user intent expression. Furthermore, rich semantic information and expression patterns are provided for model training, enabling the model to better understand and adapt to the diversity of user intents, and improving the practical application ability of the model.

[0014] Optionally, the process of concatenating the multiple historical dialogues and the current input to obtain the target content specifically includes: performing chronological sorting on the multiple historical dialogues according to the timestamp information of the historical dialogues to obtain a historical dialogue sequence; traversing the historical dialogue sequence and identifying the historical dialogues other than the last round of historical dialogue in the historical dialogue sequence as target historical dialogues; replacing the assistant reply content in the target historical dialogues with a preset placeholder to obtain a target historical dialogue sequence; concatenating the current input to the end of the target historical dialogue sequence to obtain the target content.

[0015] By adopting the above technical solution, target content with concentrated information and complete meaning is generated, providing high-quality input for intent compression. First, chronological sorting is performed according to the timestamp information of the historical conversation to ensure the temporal continuity and causal relationship of the conversation. The assistant's reply content in the target historical conversation is replaced with a preset placeholder, highlighting the user's intent expression and simplifying the context information. Finally, the current input is concatenated to the end of the target historical conversation sequence to obtain the complete target content. Through the concatenation method of the present invention, the generated target content not only retains the key context information of the historical conversation but also highlights the user's intent of the current input, forming an intent representation with concentrated information and prominent focus. Such high-quality target content provides accurate and targeted input for subsequent intent compression, helping to improve the effect and efficiency of intent compression.

[0016] Optionally, based on the training data set, a preset language model is trained, and during the training process, the parameters of the preset language model are adjusted through a first preset matrix and a second preset matrix to obtain a compression model for compressing user intents. Specifically, it includes: obtaining the preset language model, which is a pre-trained transformer model, and the preset language model includes multiple self-attention layers, and each self-attention layer has an initial weight matrix; training the preset language model based on the training data set to obtain the number of training rounds; determining whether the number of training rounds is greater than or equal to a preset number of training rounds; if it is determined that the number of training rounds is greater than or equal to the preset number of training rounds, stop training to obtain the compression model.

[0017] By adopting the above technical solution, by introducing parameter matrices, on the basis of retaining the original structure of the preset language model, the expression ability and generalization performance of the model are improved. First, a pre-trained transformer model is obtained as the preset language model, and the rich language knowledge and context information learned by it on a large-scale corpus are utilized. Then, the preset language model is fine-tuned based on the training data set, and the model parameters are optimized through multiple rounds of iteration to make it adapt to the intent compression task. During the training process, the first preset matrix and the second preset matrix are introduced to adjust the parameters of the self-attention layer of the model. Through the method of low-rank matrix decomposition, without increasing the complexity of the model, the representation space of the model is expanded. By setting an appropriate number of training rounds, the training progress of the model is controlled to avoid overfitting. Finally, a superior-performing intent compression model is obtained, which can compress the user's intent while maximizing the retention of the semantic information and context features of the original intent.

[0018] Optionally, training the preset language model based on the training data set to obtain the number of training rounds specifically includes: inputting the training data set into the preset language model for forward propagation calculation; for each self-attention layer in the preset language model, calculating the product of the first preset matrix and the second preset matrix through matrix multiplication to obtain a low-rank update matrix; adding the low-rank update matrix to the initial weight matrix of each self-attention layer to obtain a target weight matrix; replacing the initial weight matrix with the target weight matrix and continuing the forward propagation calculation; counting the number of times of forward propagation calculation and using the number as the number of training rounds.

[0019] By adopting the above technical solution, by introducing the first preset matrix and the second preset matrix, the model parameters are dynamically adjusted during the training process of the intent compression model, improving the model's expression ability and generalization performance. In each training step, the training data is input into the preset language model, and the prediction result of the model is obtained through forward propagation calculation. For each self-attention layer in the model, the product of the first preset matrix and the second preset matrix is calculated through matrix multiplication to obtain a low-rank update matrix. Then, the low-rank update matrix is added to the initial weight matrix of the self-attention layer to obtain the updated target weight matrix, which is used to replace the original weight matrix. Through this method of low-rank matrix adjustment, without increasing the model complexity, the model's representation space is expanded, enabling the model to learn richer semantic features and intent representations. At the same time, by counting the number of times of forward propagation calculation, the current number of training rounds is obtained, providing a monitoring metric for model training.

[0020] In the second aspect of the present application, a user intent compression device is provided. The device includes: an acquisition module and a processing module, where: the acquisition module is used to acquire a training data set, and the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data; the processing module is used to train a preset language model based on the training data set and adjust the parameters of the preset language model through a first preset matrix and a second preset matrix during the training process to obtain a compression model for compressing user intents; the acquisition module is further used to acquire multiple historical conversations of the user and the current input of the user; the processing module is further used to splice the multiple historical conversations and the current input to obtain a target content; the processing module is further used to input the target content into the compression model to obtain a user intent compression result.

[0021] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the electronic device executes the method described in any one of the above.

[0022] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0023] In summary, one or more technical solutions provided by the present application have at least the following technical effects or advantages: 1. By obtaining a training data set containing rule synthesis data, dialogue annotation data, and diversity synthesis data, training a preset language model, and introducing a first preset matrix and a second preset matrix to adjust the model parameters, an efficient and accurate intention compression model is obtained. This intention compression model can make full use of the context information of historical conversations and current inputs, splice to obtain the target content, and input it into the compression model to generate a compact and highly concentrated user intention representation. This intention compression method can not only filter out redundant and irrelevant information, but also accurately capture the core needs of users, providing high-quality input for subsequent task processing. At the same time, the compressed intention representation greatly reduces the overhead of data transmission and storage, improving scalability and real-time performance. Through the user intention compression method of the present invention, efficient, accurate, and low-cost user intention understanding and task processing can be achieved. Description of the Drawings

[0024] Figure 1 is a flowchart of a user intention compression method disclosed in an embodiment of the present application; Figure 2 is a module diagram of a user intention compression device disclosed in an embodiment of the present application; Figure 3 is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0025] Description of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Embodiments

[0026] To enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments.

[0027] In the description of the embodiments of this application, words such as "for example" or "for illustration" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0028] In the description of the embodiments of this application, the meaning of the term "a plurality" refers to two or more. For example, a plurality of systems refers to two or more systems, and a plurality of screen terminals refers to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] This application provides a user intention compression method. Refer to Figure 1 , Figure 1 is a schematic flow chart of a user intention compression method provided by an embodiment of this application. This method is applied to a server. The server is a server that executes a user intention compression program. The server can be a single server, or a server cluster composed of multiple servers, or a cloud computing service center. This method includes steps S101 to S105. The above steps are as follows: Step S101: Obtain a training data set, where the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data.

[0030] In step S101, when obtaining the training data set and obtaining the dialogue annotation data, it specifically includes: obtaining multiple groups of historical dialogue data, where each group of historical dialogue data includes multiple historical dialogues and a current input; using a preset summary model to perform semantic extraction on the historical dialogues and the current input in each group of historical dialogue data to generate a preliminary annotation result; sending the preliminary annotation result to an artificial annotation platform so that artificial annotators can check and correct the preliminary annotation result to obtain an artificial annotation result; generating dialogue annotation data based on the preliminary annotation result and the artificial annotation result.

[0031] Specifically, the server retrieves multiple sets of historical conversation data from a database or other data sources. Each set of historical conversation data includes multiple rounds of conversation records between the user and a chatbot or customer service staff, as well as the user's current input. These historical conversation data can be from real user interaction logs or artificial conversation data collected specifically for training the model.

[0032] Next, the server processes each set of historical conversation data using a preset summarization model. The preset summarization model can be a sequence-to-sequence (seq2seq) model based on deep learning, such as LSTM, or a pre-trained language model based on the Transformer architecture, such as BERT. The server takes the historical conversations and the current input in each set of historical conversation data as inputs, performs semantic extraction and information condensation through the preset summarization model, and generates corresponding preliminary annotation results.

[0033] After generating the preliminary annotation results, the server sends them to the manual annotation platform. The manual annotation platform is a web-based annotation tool that allows multiple annotators to log in and collaborate simultaneously. The annotators view each set of historical conversation data and the corresponding preliminary annotation results through the platform interface, and check and correct the preliminary annotation results.

[0034] Finally, the server generates the final conversation annotation data based on the preliminary annotation results and the manual annotation results. The server compares the preliminary annotation results with the manual annotation results. For the parts that are exactly the same, they are directly incorporated into the conversation annotation data; for the parts with differences, the server adopts the manual annotation results. At the same time, the server also records and analyzes the differences between the preliminary annotation results and the manual annotation results for evaluating and improving the performance of the preset summarization model.

[0035] For example, assume that a set of historical conversation data is as follows: User: I want to book a flight from Beijing to Shanghai, departing tomorrow.

[0036] Customer Service: Okay, I can query the flight information from Beijing to Shanghai for you tomorrow. What time period of the flight do you need? User: A flight between 8:00 am and 10:00 am will be fine.

[0037] Customer Service: According to your requirements, I found the following two flights: Air China CA1893, departing from Beijing Capital Airport at 08:30 and arriving at Shanghai Hongqiao Airport at 10:20; China Eastern Airlines MU5183, departing from Beijing Daxing Airport at 09:15 and arriving at Shanghai Pudong Airport at 11:00. Which flight do you need to book? User: I choose the Air China flight. Thank you.

[0038] For the above dialogue data, the preliminary annotation results generated by the preset summary model may be: "The user needs to book a flight ticket from Beijing to Shanghai tomorrow morning." After manual annotation, the annotation results may be corrected to: "The user booked the Air China flight CA1893 departing from Beijing Capital Airport at 08:30 tomorrow and arriving at Shanghai Hongqiao Airport at 10:20." The server integrates the preliminary annotation results and the manual annotation results to obtain the final dialogue annotation data, which is used as the corpus for subsequent training of the compression model.

[0039] In a possible implementation, in obtaining the training data set, obtaining rule synthesis data specifically includes: constructing a dialogue template library based on historical dialogue data, where the dialogue template library includes multiple dialogue templates, and each dialogue template consists of multiple rounds of dialogue; for each dialogue template, through a preset code script, generating rule synthesis data in the way of missing core content and rhetorical questions.

[0040] Specifically, the server constructs a dialogue template library based on historical dialogue data. The server obtains multiple user dialogue records from a database or other data sources and analyzes these dialogue data. The server uses natural language processing technology to identify key information elements in the dialogue data, such as user intentions, entities, keywords, etc. Then, the server clusters the dialogues with similar structures and similar intentions, extracts the common patterns among them, and forms dialogue templates. Each dialogue template consists of multiple rounds of dialogue, reflecting the typical interaction process and information structure of a certain type of dialogue.

[0041] Next, for each dialogue template in the dialogue template library, the server generates rule synthesis data through a preset code script. The preset code script is a set of programs customized according to dialogue generation rules and strategies, and can instantiate the dialogue template in different ways. Among them, missing core content and rhetorical questions are two data synthesis strategies included in the preset code script.

[0042] For the missing core content strategy, the preset code script will identify some key information elements missing in the dialogue template and generate the complete dialogue context. Then, the preset code script will automatically construct a user question related to the missing information according to the internal logic and prior knowledge of the dialogue template, as the user input for the current dialogue turn.

[0043] For example, for the ticket booking dialogue template, the preset code script may generate the following incomplete dialogue: User: I want to book a flight ticket to Shanghai, departing tomorrow.

[0044] Customer service: Okay, I can query the flight information to Shanghai for you tomorrow. May I ask which city you are departing from? User: [Destination missing] Customer service: Excuse me, you haven't told me your departure city yet. To provide you with accurate flight information, may I ask where you are departing from to Shanghai? In this way of lacking core content, the server generates a large number of dialogue samples that need to supplement key information, which helps the compression model learn how to accurately grasp the user's intention and guide the user to provide necessary information when the information is incomplete.

[0045] For the rhetorical question strategy, the preset code script will insert some user questions that are irrelevant to the current topic or in conflict with the information on the basis of the dialogue template, simulating the situation where the user's intention deviates or transfers. Then, the preset code script will automatically generate a customer service reply that guides the user back to the original topic according to the dialogue template and domain common sense, so that the conversation gets back on track.

[0046] For example, for the ticket booking dialogue template, the preset code script may insert a question about the weather and generate the following dialogue: User: I want to book a flight from Beijing to Shanghai tomorrow.

[0047] Customer service: Okay, I can check the flight information from Beijing to Shanghai for you tomorrow. What time period of the flight do you need? User: What's the weather like in Shanghai tomorrow? [Irrelevant question] Customer service: The weather forecast for Shanghai tomorrow is cloudy turning to sunny, and the temperature is suitable. However, we are currently discussing your flight needs. To better serve you, what time of flight do you hope to take? By introducing these rhetorical questions with intention deviation, the server trains the compression model on how to accurately identify the core intention in the face of confusion or interference, and timely guide the conversation back to the theme to avoid being troubled by irrelevant information.

[0048] Finally, the server integrates a large number of dialogue samples automatically synthesized through the above steps to form a rule synthesis dataset. This dataset covers various dialogue scenarios and intention transfer situations that require special processing, and can effectively improve the compression model's ability to handle complex conversations.

[0049] In a possible implementation manner, in the acquisition of the training dataset, to obtain diverse synthetic data, it specifically includes: merging the rule synthesis data and the dialogue annotation data to obtain merged data; using a preset data augmentation algorithm to perform augmentation processing on the merged data to obtain augmented data, and the preset data augmentation algorithm includes generating synonymous sentences, generating rhetorical questions, and adding random noise; inputting the augmented data into a preset large model and performing text generation based on the preset large model to obtain diverse synthetic data.

[0050] Specifically, the server performs a merging process on the rule synthesis data and the dialogue annotation data. The server reads dialogue samples from the rule synthesis data and the dialogue annotation data respectively, and integrates them according to a unified format and standard. During the merging process, the server will perform necessary cleaning and filtering on the data, removing duplicate, invalid, or incorrectly formatted samples to ensure the quality of the merged data. At the same time, the server will also randomly shuffle the merged data to eliminate possible order biases and increase the diversity of the data. Finally, the server obtains a merged dataset containing rule-synthesized dialogues and manually annotated dialogues, which serves as the basis for subsequent data augmentation.

[0051] Next, the server uses a preset data augmentation algorithm to perform augmentation processing on the merged data. The preset data augmentation algorithm generates new training samples by transforming and perturbing the original data. In this embodiment, the server uses three data augmentation algorithms: synonym sentence generation, rhetorical question generation, and random noise addition.

[0052] For synonym sentence generation, the server uses natural language processing techniques, such as word list replacement and syntactic transformation, to automatically rewrite the dialogue text in the merged data into synonymous sentences with similar semantics but different expressions. For example, for the original dialogue "I want to book a flight ticket from Beijing to Shanghai tomorrow", the server may generate a synonymous sentence "I plan to book a flight ticket from Beijing to Shanghai the next day". By introducing expression diversity, the server helps the compression model learn multiple representations of semantics and improves the model's adaptability to vocabulary and sentence pattern changes.

[0053] For rhetorical question generation, based on the merged data, the server automatically constructs some rhetorical questions related to the original dialogue but with different questioning methods. These rhetorical questions may raise doubts about specific information points in the original dialogue or pursue the original intention from different angles. For example, for the original dialogue "I want to book a flight ticket from Beijing to Shanghai tomorrow", the server may generate rhetorical questions such as "May I ask if you need a one-way ticket or a round-trip ticket?" or "Do you have any specific requirements for the flight time?" By introducing rhetorical questions, the server trains the compression model on how to accurately grasp the user's core intention and give appropriate clarification or guidance when dealing with diverse inquiries.

[0054] For random noise addition, the server randomly inserts some noise information, such as typos, grammar errors, and irrelevant words, into the text of the merged data to simulate various interference factors in real conversations. For example, modify "I want to book a flight ticket from Beijing to Shanghai tomorrow" to "I want to book a flight ticket from Beijing to Shanghai tomorrow". By introducing this random noise, the server enhances the anti-interference ability of the compression model, enabling it to still maintain good intention understanding and response capabilities when faced with incorrect, ambiguous, or incomplete user inputs.

[0055] Finally, the server inputs the enhanced data after data enhancement processing into a preset large model, and utilizes the language understanding and generation capabilities of the preset large model to further expand the diversity of dialogue samples. The preset large model refers to a large-scale language model pre-trained on a vast amount of corpus, such as GPT, BERT, etc., which has text generation and completion capabilities. The server takes the enhanced data as input, performs text generation through the preset large model, and obtains a series of new dialogue samples that are related to the original dialogue intention but have a more diverse expression form.

[0056] For example, for the enhanced dialogue "I plan to book a flight ticket from Beijing to Shanghai the next day", the large model may generate the following new samples: "I need to purchase a one-way ticket from Beijing to Shanghai, and the departure date is tomorrow." "Please help me arrange a flight from Capital Airport to Hongqiao Airport tomorrow." "I plan to travel by flight directly from Beijing to Shanghai tomorrow. Can you check it for me?" Through the text generation of the preset large model, the server further enriches the expression methods and language styles of the dialogue samples, enabling the compression model to learn more natural and user-friendly intention expressions.

[0057] Step S102: Based on the training data set, train the preset language model, and during the training process, adjust the parameters of the preset language model through the first preset matrix and the second preset matrix to obtain a compression model for compressing user intentions.

[0058] In step S102, based on the training data set, train the preset language model, and during the training process, adjust the parameters of the preset language model through the first preset matrix and the second preset matrix to obtain a compression model for compressing user intentions, which specifically includes: obtaining the preset language model. The preset language model is a pre-trained transformer model, and the preset language model includes multiple self-attention layers, and each self-attention layer has an initial weight matrix; training the preset language model based on the training data set to obtain the number of training rounds; determining whether the number of training rounds is greater than or equal to the preset number of training rounds; if it is determined that the number of training rounds is greater than or equal to the preset number of training rounds, stop training to obtain the compression model.

[0059] Specifically, the server obtains the preset language model as the starting point of training. The preset language model is a pre-trained transformer model that has been pre-trained on a large-scale corpus and has good language understanding and representation capabilities. The preset language model contains multiple self-attention layers, and each self-attention layer has its initial weight matrix, and these initial weight matrices are learned during the pre-training stage.

[0060] Next, the server fine-tunes the pre-set language model using the constructed training dataset. The server divides the training dataset into a training set and a validation set according to a certain ratio, and converts it into an input format acceptable to the model, such as a token sequence. Then, the server performs multiple rounds of training on the pre-set language model using the training dataset, and each training round traverses the entire training set once. In each training step, the server randomly extracts a batch of data from the training set, inputs it into the pre-set language model, calculates the predicted output of the model through forward propagation, and uses a loss function (such as cross-entropy loss) to measure the difference between the predicted output and the true label. The server updates the weight parameters of the model using an optimization algorithm according to the gradient of the loss function, so that it gradually adapts to the task of compressing user intentions.

[0061] During the training process, in order to further improve the expression ability and generalization performance of the model, the server introduces a first pre-set matrix and a second pre-set matrix to adjust the model parameters. After each training round, the server uses the validation set data to evaluate the current model, and calculates the loss value and evaluation metrics (such as accuracy, F1 value, etc.) of the model on the validation set. The server records the evaluation results of each training round and dynamically adjusts hyperparameters such as the learning rate to optimize the training process of the model.

[0062] The server continuously performs the above training process until the pre-set number of training rounds is reached. The pre-set number of training rounds is a hyperparameter that can be set according to factors such as task difficulty, data scale, and model complexity to balance training time and model performance, and this application does not make any limitations in this regard. After each training round, the server determines whether the current number of training rounds is greater than or equal to the pre-set number of training rounds. If the pre-set number of training rounds reaches or exceeds the pre-set number of training rounds, the server stops the training process and saves the current model as the final compressed model.

[0063] In a possible implementation manner, training the pre-set language model based on the training dataset to obtain the number of training rounds specifically includes: inputting the training dataset into the pre-set language model for forward propagation calculation; for each self-attention layer in the pre-set language model, calculating the product of the first pre-set matrix and the second pre-set matrix through matrix multiplication to obtain a low-rank update matrix; adding the low-rank update matrix to the initial weight matrix of each self-attention layer to obtain a target weight matrix; replacing the initial weight matrix with the target weight matrix and continuing forward propagation calculation; counting the number of times of forward propagation calculation and using the number as the number of training rounds.

[0064] Specifically, the server inputs the training dataset into a pre-set language model and starts the forward propagation calculation. The server divides the training dataset into batches of a certain size and processes one batch of data at a time. For each batch, the server converts it into an input format acceptable to the model, such as a token sequence, an attention mask, etc., and passes it into the input layer of the pre-set language model.

[0065] Next, the server adjusts the parameters of each self-attention layer in the pre-set language model. The pre-set language model usually contains multiple self-attention layers, and each self-attention layer has its initial weight matrix for calculating the attention weights between queries, keys, and values. During the forward propagation process, the server uses the first pre-set matrix and the second pre-set matrix to perform a low-rank update on the weight matrix of each self-attention layer.

[0066] Specifically, for each self-attention layer, the server first calculates the product of the first pre-set matrix and the second pre-set matrix through matrix multiplication to obtain a low-rank update matrix. The shape of the first pre-set matrix is (d1, d2), and the shape of the second pre-set matrix is (d2, d1), where d1 is the hidden layer dimension of the pre-set language model and d2 is the compressed hidden layer dimension (usually less than d1). The server multiplies the first pre-set matrix by the second pre-set matrix to obtain a low-rank update matrix with a shape of (d1, d1) and a rank of d2.

[0067] Then, the server performs an element-wise addition operation on the low-rank update matrix and the initial weight matrix of this self-attention layer to obtain a target weight matrix. The target weight matrix has the same shape (d1, d1) as the initial weight matrix, but its values have been adjusted by the low-rank update matrix. In this way, without changing the structure of the self-attention layer, the server introduces additional learnable parameters, enabling the model to compress user intentions while retaining more context information and semantic features.

[0068] Next, the server replaces the initial weight matrix of this self-attention layer with the target weight matrix and continues the forward propagation calculation. The server uses the updated weight matrix to calculate the attention weights between queries, keys, and values, and applies the attention weights to the value vectors to obtain the output representation of this self-attention layer. The server passes the output representation to the subsequent feed-forward neural network and other self-attention layers until the entire forward propagation process is completed to obtain the predicted output of the model.

[0069] The server repeats the above process, processing each batch in the training dataset in turn until the entire training dataset is traversed. Each time the entire training dataset is traversed, it is considered to complete one training epoch. The server counts the number of forward propagation calculations and uses it as the current training epoch number.

[0070] For example, assume that the pre-trained language model is BERT-base, the dimension d1 of its hidden layer is 768, and the compressed dimension d2 of the hidden layer is 384. The server randomly initializes a first pre-set matrix with a shape of (768, 384) and a second pre-set matrix with a shape of (384, 768). During the forward propagation process, for each self-attention layer in BERT-base, the server multiplies the first pre-set matrix by the second pre-set matrix to obtain a low-rank update matrix with a shape of (768, 768). Then, the server adds the low-rank update matrix to the initial weight matrix of this self-attention layer to obtain a target weight matrix, and replaces the initial weight matrix with the target weight matrix. The server uses the updated weight matrix to calculate the attention weights and continues the forward propagation calculation. The server processes each batch in the training dataset in sequence and counts the number of processed batches. Assume that the training dataset contains 100,000 samples and the batch size is 32, then it takes about 100,000 / 32 = 3125 batches to complete one training epoch. The server divides the number of processed batches by 3125 to obtain the current training epoch.

[0071] Step S103: Obtain multiple historical conversations of the user and the user's current input.

[0072] In step S103, the server establishes a connection with the user and assigns a unique session identifier to the user. This session identifier is used to identify the user's identity and conversation context, ensuring that the server can correctly obtain and manage the user's historical conversation records. When the user first establishes a connection with the server, the server creates a new session and initializes the historical conversation of this session to be empty.

[0073] Next, the server continuously listens for the user's input and processes it as the current input. The user's input can be transmitted to the server through various channels, such as a text input box, speech recognition, image upload, etc. Then, the server associates the user's current input with the user's historical conversations to construct a complete conversation context. The server first obtains the user's historical conversation records in this dialog box according to the session identifier and arranges them in chronological order.

[0074] Step S104: Concatenate the multiple historical conversations and the current input to obtain the target content.

[0075] In step S104, multiple historical conversations and the current input are concatenated to obtain the target content, which specifically includes: sorting the multiple historical conversations in chronological order according to the timestamp information of the historical conversations to obtain a historical conversation sequence; traversing the historical conversation sequence and identifying the historical conversations in the historical conversation sequence except for the last round of historical conversation as the target historical conversations; replacing the assistant's response content in the target historical conversations with a preset placeholder to obtain a target historical conversation sequence; and concatenating the current input to the end of the target historical conversation sequence to obtain the target content.

[0076] Specifically, the server sorts the multiple historical conversations in chronological order according to the timestamp information of the historical conversations to obtain a historical conversation sequence. Each historical conversation contains timestamp information that records the specific time when the conversation occurred. The server extracts the timestamp of each historical conversation and sorts the historical conversations in chronological order to ensure that the conversations in the historical conversation sequence are arranged in the actual order of occurrence. This step ensures the temporal continuity and causal relationship of the historical conversations and provides the correct context order for subsequent intent compression.

[0077] Next, the server traverses the historical conversation sequence and identifies the historical conversations in the historical conversation sequence except for the last round of historical conversation as the target historical conversations. In the intent compression task, usually the user's recent conversation history is concerned, while the earlier conversation content is ignored. Then, the server replaces the assistant's response content in the target historical conversations with a preset placeholder to obtain a target historical conversation sequence. In the intent compression task, the main focus is on the user's intent and needs, and the assistant's response content is not necessary for compressing the intent. Therefore, the server replaces the assistant's response content in the target historical conversations with a preset placeholder (such as [Assistant]) to simplify the context information and highlight the user's intent. This step can not only reduce unnecessary information interference but also reduce the computational complexity of the subsequent intent compression model.

[0078] Finally, the server concatenates the current input to the end of the target historical conversation sequence to obtain the final target content. The current input represents the user's latest intent and needs and is the key object for intent compression. The server appends the current input to the end of the target historical conversation sequence to form a complete conversation context. This target content contains the user's previous conversation history and the current input, providing rich context information and sufficient input data for the subsequent intent compression model.

[0079] For example, assume the server obtains the following historical conversation sequence: [2023-05-10 10:00:00] User: I want to book a flight ticket.

[0080] [2023-05-10 10:00:05] Assistant: Hello, I'm glad to serve you. May I ask about your departure and destination for the reservation? [2023-05-10 10:00:10] User: I want to fly from Beijing to Shanghai.

[0081] [2023-05-10 10:00:15] Assistant: Okay, may I ask about your expected departure date? [2023-05-10 10:00:20] User: I plan to depart tomorrow.

[0082] [2023-05-10 10:00:25] Assistant: There are many flights from Beijing to Shanghai tomorrow. Which time slot of the flight do you prefer? The server selects two rounds of conversations except the last round as the target historical conversation, and replaces the assistant's reply content in the target historical conversation with the placeholder [Assistant], obtaining the target historical conversation sequence: [2023-05-10 10:00:00] User: I want to book a plane ticket.

[0083] [Assistant].

[0084] [2023-05-10 10:00:10] User: I want to fly from Beijing to Shanghai.

[0085] [Assistant].

[0086] [2023-05-10 10:00:20] User: I plan to depart tomorrow.

[0087] [2023-05-10 10:00:25] Assistant: There are many flights from Beijing to Shanghai tomorrow. Which time slot of the flight do you prefer? Finally, the server concatenates the current input "Preferably a flight between 10 am and 12 pm?" to the end of the target historical conversation sequence, obtaining the final target content.

[0088] Step S105: Input the target content into the compression model to obtain the compressed result of the user's intention.

[0089] In step S105, the server performs necessary preprocessing on the target content and converts it into an input format acceptable to the compression model. This step includes natural language processing tasks such as word segmentation, part-of-speech tagging, and named entity recognition to extract key information elements from the target content. The server can use a pre-trained word segmentation model to segment the target content and convert it into a sequence of word chunks. At the same time, the server can also perform part-of-speech tagging and named entity recognition on the target content, marking information such as keywords and entity mentions to provide richer feature information for subsequent intent compression.

[0090] Next, the server inputs the preprocessed target content into the compression model and starts intent compression. The compression model is the model trained in the previous step. The server inputs the sequence of word chunks of the target content into the input layer of the compression model and extracts the deep semantic representation of the target content through a multi-layer self-attention mechanism and a feed-forward neural network. In this process, the compression model will comprehensively consider features such as context information, keywords, and entities in the target content and generate a highly condensed semantic vector as the compressed representation of the user intent.

[0091] When generating the compressed representation, the server can control the degree of compression by setting the compression ratio. The compression ratio determines the dimension size of the compressed semantic vector, usually a percentage of the length of the input content. For example, if the target content contains 100 word chunks and the compression ratio is set to 20%, the dimension of the compressed semantic vector is 20. The server can adjust the compression ratio according to actual needs and model performance to balance the accuracy and efficiency of intent compression.

[0092] Finally, the server uses the semantic vector generated by the compression model as the user intent compression result for subsequent task processing. The user intent compression result is a real-valued vector with a fixed dimension, which compactly represents the core content of the user intent and filters out redundant and irrelevant information. The server can pass the intent compression result to downstream task models such as dialogue management, semantic matching, and knowledge base query to support more efficient and accurate task processing. At the same time, the server can also store the intent compression result in a database or cache for tracking and analyzing historical intents, providing data support for personalized services and user portraits.

[0093] For example, for the target content obtained in the previous step, the server first preprocesses the target content to obtain the following sequence of word chunks: ['I', 'intend', 'to', 'depart', 'tomorrow', '.', '[Assistant]', 'Preferably', 'a', 'flight', 'between', '10', 'am', 'and', '12', 'pm'] Then, the server inputs this sequence of word chunks into the compression model. Assuming a compression ratio of 30%, the dimension of the compressed semantic vector is 8. The compression model extracts the key information of the target content through the self-attention mechanism and the feed-forward neural network, and generates an 8-dimensional real-valued vector as the compression result of the user intention: [0.25, -0.12, 0.45, 0.78, -0.03, 0.61, 0.14, -0.39] The server passes the compression result of the user intention to downstream task models, such as flight query, cabin selection, and discount judgment, to support efficient and accurate task processing. At the same time, the server can also store this intention compression result in the user intention history database for subsequent personalized services and user behavior analysis.

[0094] Referring to Figure 2 , this application also provides a user intention compression device, which is a server. The server includes an acquisition module 201 and a processing module 202. The acquisition module 201 is used to acquire a training data set, and the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data. The processing module 202 is used to train a preset language model based on the training data set, and adjust the parameters of the preset language model through a first preset matrix and a second preset matrix during the training process to obtain a compression model for compressing the user intention. The acquisition module 201 is also used to acquire multiple historical conversations of the user and the current input of the user. The processing module 202 is also used to splice the multiple historical conversations and the current input to obtain the target content. The processing module 202 is also used to input the target content into the compression model to obtain the compression result of the user intention.

[0095] In a possible implementation manner, when the acquisition module 201 acquires the dialogue annotation data in the training data set, it specifically includes: The acquisition module 201 acquires multiple groups of historical conversation data, and each group of historical conversation data includes multiple historical conversations and the current input. The processing module 202 uses a preset summary model to perform semantic extraction on the historical conversations and the current input in each group of historical conversation data to generate a preliminary annotation result. The processing module 202 sends the preliminary annotation result to the manual annotation platform so that the manual annotator can check and correct the preliminary annotation result to obtain the manual annotation result. The processing module 202 generates the dialogue annotation data based on the preliminary annotation result and the manual annotation result.

[0096] In a possible implementation manner, the obtaining module 201 obtains rule synthesis data from the training data set, specifically including: the processing module 202 constructs a dialogue template library based on historical dialogue data, the dialogue template library includes a plurality of dialogue templates, and each dialogue template consists of multiple rounds of dialogues; for each dialogue template, the processing module 202 generates rule synthesis data in the way of missing core content and rhetorical questions through a preset code script.

[0097] In a possible implementation manner, the obtaining module 201 obtains diversity synthesis data from the training data set, specifically including: the processing module 202 merges the rule synthesis data and the dialogue annotation data to obtain merged data; the processing module 202 uses a preset data augmentation algorithm to perform augmentation processing on the merged data to obtain augmented data, and the preset data augmentation algorithm includes generating synonymous sentences, generating rhetorical questions, and adding random noise; the processing module 202 inputs the augmented data into a preset large model and performs text generation based on the preset large model to obtain diversity synthesis data.

[0098] In a possible implementation manner, the processing module 202 splices a plurality of historical dialogues and the current input to obtain target content, specifically including: the processing module 202 performs chronological sorting on the plurality of historical dialogues according to the timestamp information of the historical dialogues to obtain a historical dialogue sequence; the processing module 202 traverses the historical dialogue sequence and confirms the historical dialogues other than the last round of historical dialogue in the historical dialogue sequence as target historical dialogues; the processing module 202 replaces the assistant reply content in the target historical dialogues with a preset placeholder to obtain a target historical dialogue sequence; the processing module 202 splices the current input to the end of the target historical dialogue sequence to obtain target content.

[0099] In a possible implementation manner, the processing module 202 trains a preset language model based on the training data set and adjusts the parameters of the preset language model through a first preset matrix and a second preset matrix during the training process to obtain a compression model for compressing user intentions, specifically including: the obtaining module 201 obtains a preset language model, the preset language model is a pre-trained transformer model, the preset language model includes a plurality of self-attention layers, and each self-attention layer has an initial weight matrix; the processing module 202 trains the preset language model based on the training data set to obtain the number of training rounds; the processing module 202 determines whether the number of training rounds is greater than or equal to a preset number of training rounds; if the processing module 202 determines that the number of training rounds is greater than or equal to the preset number of training rounds, it stops training to obtain a compression model.

[0100] In a possible implementation, the processing module 202 trains a preset language model based on a training data set to obtain the number of training rounds, specifically including: the processing module 202 inputs the training data set into the preset language model for forward propagation calculation; for each self-attention layer in the preset language model, the processing module 202 calculates the product of a first preset matrix and a second preset matrix through matrix multiplication to obtain a low-rank update matrix; the processing module 202 adds the low-rank update matrix to the initial weight matrix of each self-attention layer to obtain a target weight matrix; the processing module 202 replaces the initial weight matrix with the target weight matrix and continues with the forward propagation calculation; the processing module 202 counts the number of times of forward propagation calculation and uses the number as the number of training rounds.

[0101] It should be noted that: when the device provided in the above embodiment realizes its functions, only the division of the above function modules is used for illustration. In actual applications, the above functions can be allocated to different function modules according to needs, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be elaborated here.

[0102] This application also provides an electronic device. Refer to Figure 3 , Figure 3 is a schematic structural diagram of an electronic device provided in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0103] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0104] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0105] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface).

[0106] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, it performs various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0107] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. Optionally, the memory 305 may further be at least one storage device located far from the aforementioned processor 301. Referring to Figure 3 , as a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface module, and an application program for a user intention compression method.

[0108] In Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for the user to input and obtain the data input by the user; while the processor 301 can be used to call the application program stored in the memory 305 for a user intention compression method. When executed by one or more processors 301, the electronic device 300 is caused to execute one or more of the methods as described in the foregoing embodiments. It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0109] The present application also provides a computer-readable storage medium storing instructions. When executed by one or more processors 301, the electronic device 300 is caused to execute one or more of the methods as described in the foregoing embodiments.

[0110] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] In several implementation manners provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some service interfaces. The indirect couplings or communication connections of the devices or units can be in electrical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] In addition, the functional units in the respective embodiments of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0114] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, mobile hard disks, magnetic disks, or optical discs.

[0115] The above are only exemplary embodiments of the present disclosure, and the scope of the present disclosure cannot be limited thereby. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. After considering the specification and the disclosure of the practical truth, those skilled in the art will easily think of other implementation schemes of the present disclosure.

[0116] This application aims to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A user intention compression method, characterized in that, The method includes: Obtaining a training data set, where the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data; Based on the training data set, training a preset language model, and during the training process, adjusting the parameters of the preset language model through a first preset matrix and a second preset matrix to obtain a compression model for compressing user intents; Obtaining multiple historical conversations of the user and the current input of the user; Concatenating the multiple historical conversations and the current input to obtain a target content; Inputting the target content into the compression model to obtain a compressed result of the user intent.

2. The method according to claim 1, characterized in that In the process of obtaining the training data set, obtaining the dialogue annotation data specifically includes: Obtaining multiple groups of historical conversation data, where each group of historical conversation data includes multiple historical conversations and a current input; Using a preset summary model to perform semantic extraction on the historical conversations and the current input in each group of historical conversation data to generate a preliminary annotation result; Sending the preliminary annotation result to an artificial annotation platform so that artificial annotators can check and correct the preliminary annotation result to obtain an artificial annotation result; Generating the dialogue annotation data based on the preliminary annotation result and the artificial annotation result.

3. The method according to claim 2, wherein In the process of obtaining the training data set, obtaining the rule synthesis data specifically includes: Constructing a dialogue template library based on the historical conversation data, where the dialogue template library includes multiple dialogue templates, and each dialogue template consists of multiple rounds of conversations; For each of the dialogue templates, generating the rule synthesis data through a preset code script in a way of missing core content and rhetorical questions.

4. The method according to claim 3, characterized in that In the process of obtaining the training data set, obtaining the diversity synthesis data specifically includes: Merging the rule synthesis data and the dialogue annotation data to obtain merged data; Using a preset data augmentation algorithm to perform augmentation processing on the merged data to obtain augmented data, where the preset data augmentation algorithm includes generating synonymous sentences, generating rhetorical questions, and adding random noise; Inputting the augmented data into a preset large model and performing text generation based on the preset large model to obtain the diversity synthesis data.

5. The method according to claim 1, wherein The process of concatenating the multiple historical conversations and the current input to obtain the target content specifically includes: According to the timestamp information of the historical conversations, performing chronological sorting on the multiple historical conversations to obtain a historical conversation sequence; Traversing the historical conversation sequence and identifying the historical conversations in the historical conversation sequence except for the last round of historical conversation as target historical conversations; Replacing the assistant response content in the target historical conversations with a preset placeholder to obtain a target historical conversation sequence; Concatenating the current input to the end of the target historical conversation sequence to obtain the target content.

6. The method according to claim 1, wherein The process of training a preset language model based on the training data set and adjusting the parameters of the preset language model through a first preset matrix and a second preset matrix during the training process to obtain a compression model for compressing user intents specifically includes: Obtain the preset language model, where the preset language model is a pre-trained Transformer model, and the preset language model includes multiple self-attention layers, and each self-attention layer has an initial weight matrix; Train the preset language model based on the training data set to obtain the number of training rounds; Determine whether the number of training rounds is greater than or equal to a preset number of training rounds; If it is determined that the number of training rounds is greater than or equal to the preset number of training rounds, stop training to obtain the compressed model.

7. The method according to claim 6, wherein The training the preset language model based on the training data set to obtain the number of training rounds specifically includes: Input the training data set into the preset language model for forward propagation calculation; For each self-attention layer in the preset language model, calculate the product of the first preset matrix and the second preset matrix through matrix multiplication to obtain a low-rank update matrix; Add the low-rank update matrix to the initial weight matrix of each self-attention layer to obtain a target weight matrix; Replace the initial weight matrix with the target weight matrix and continue with forward propagation calculation; Count the number of times of forward propagation calculation and use the number as the number of training rounds.

8. A user intention compression device, characterized in that, The device includes an acquisition module (201) and a processing module (202), where: The acquisition module (201) is used to acquire a training data set, and the training data set includes rule synthesis data, dialogue annotation data, and diversity synthesis data; The processing module (202) is used to train a preset language model based on the training data set, and adjust the parameters of the preset language model through a first preset matrix and a second preset matrix during training to obtain a compressed model for compressing user intentions; The acquisition module (201) is further used to acquire multiple historical conversations of the user and the current input of the user; The processing module (202) is further used to splice the multiple historical conversations and the current input to obtain a target content; The processing module (202) is further used to input the target content into the compressed model to obtain a compressed result of the user intention.

9. An electronic device, characterized in that, It includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions, the user interface (303) and the network interface (304) are used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1-7 is executed.

Images