A method and system for improving task inference speed

Abstract

This application discloses a method and system for improving task inference speed, relating to the field of model processing. The method includes: sorting the original intent list by semantic similarity, so that semantically similar intents are arranged adjacently in the list; generating a special string with high uniqueness to uniquely identify the sorted intent list; performing a first-stage memory fine-tuning and a second-stage business fine-tuning; wherein, the first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special string and the intent list, and fine-tuning the basic large language model to enable the model to remember the mapping relationship; the second-stage business fine-tuning includes: fine-tuning the model that has completed memory fine-tuning based on an intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios; during the model inference stage, embedding the special string into prompt words to replace the complete intent list.

Description

Technical Field

[0001] This application relates to the field of model processing technology, and in particular to a method and system for improving task inference speed. Background Technology

[0002] In scenarios such as intelligent customer service in the printing industry, intent recognition is a key step in achieving accurate responses. As the number of intent categories grows to nearly a thousand, existing solutions face significant bottlenecks: traditional fine-tuning solutions based on pre-trained models (such as BERT) experience a surge in parameters during large-scale classification, resulting in high training and inference costs and poor maintainability; solutions based on large language model prompt words suffer from a dramatic increase in the number of prompt word tokens due to the excessive length of the intent list, leading to high computational overhead and slow response speed; while solutions based on intent grouping can compress the number of tokens, they rely on manual classification standards, posing a risk of error accumulation and affecting the final classification accuracy.

[0003] Therefore, there is an urgent need for a technical solution that can achieve a balance between classification scale, inference speed and classification accuracy. Summary of the Invention

[0004] This application provides a method and system for improving task reasoning speed to solve the above-mentioned problems.

[0005] On the one hand, this application provides a method to improve the inference speed of a task, applied to the classification of customer consultation intents in printing. The method includes the following steps: Step S1: Sort the original intent list by semantic similarity, so that semantically similar intents are arranged adjacently in the list; Step S2: Generate a special string with high uniqueness to uniquely identify the intent list after sorting in Step S1; Step S3: Perform a first-stage memory fine-tuning and a second-stage business fine-tuning; wherein, the first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special string and the intent list, and fine-tuning the basic large language model to enable the model to remember the mapping relationship; the second-stage business fine-tuning includes: fine-tuning the model that has completed memory fine-tuning based on the intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios; Step S4: In the model inference stage, embed the special string into the prompt words to replace the complete intent list, reduce the number of input tokens, and improve the inference speed.

[0006] In one implementation of this application, step S1 specifically involves: representing each intent using a pre-trained vector encoder; calculating the similarity matrix between each intent vector; and sorting the intents using a clustering algorithm based on the similarity matrix, so that semantically similar intents are placed adjacent to each other in the list.

[0007] In one implementation of this application, step S2 specifically involves: concatenating a predefined fixed prefix string, a timestamp, and a custom encoding to form a source string; wherein the fixed prefix is ​​used to identify the string as an identifier of the intent list; performing hash encoding on the source string to generate a unique string of fixed length as the special string; wherein the timestamp is accurate to the second, and the custom encoding includes the project identifier and the intent list version number.

[0008] In one implementation of this application, the question-and-answer dataset constructed in the first stage of memory fine-tuning expands to include at least one of the following question-and-answer types: direct query type, reverse query type, fuzzy query type, and multi-round follow-up question type; wherein, the direct query type queries the content of the intent list corresponding to the special string, the reverse query type queries the special string corresponding to the list containing several intents, the fuzzy query type queries the user intent category represented by the special string, and the multi-round follow-up question type gradually reveals the mapping relationship between the special string and the intent list through multiple rounds of dialogue.

[0009] In one implementation of this application, the first stage of memory fine-tuning uses the low-rank adaptation method LoRA to fine-tune the parameters. During the fine-tuning process, the mapping accuracy is periodically evaluated on the validation set. When the mapping accuracy continuously reaches a preset threshold and the model's generality decreases by no more than a preset range, the memory fine-tuning is deemed complete.

[0010] In one implementation of this application, after the first stage of memory fine-tuning is completed, the LoRA weights are merged with the base model weights to form an intermediate model that can be directly inferred.

[0011] In one implementation of this application, the dataset used for the second stage of business fine-tuning comes from real business dialogue logs and manually constructed edge scenario samples; the annotation results of the dataset undergo consistency verification to ensure annotation quality; the business fine-tuning also adopts low-rank adaptation technology, with the optimization goal of improving the accuracy of intent classification in business scenarios.

[0012] In one implementation of this application, in step S4, the prompt word structure includes: a user message field; an instruction target field, which contains a reference to the list of intents identified by the special string; a prompt information field, used to guide the model to correctly understand the special string and perform the classification task; and an output format field, used to standardize the output format of the classification results.

[0013] In one implementation of this application, in step S1, a hierarchical clustering algorithm is used to group and sort the intent vectors, so that semantically similar intents are continuously distributed in the list, thereby improving the model's learning efficiency for dense semantic regions.

[0014] On the other hand, this application also provides a system for improving task inference speed. The system includes: an intent list processing module for sorting the original intent list by semantic similarity, so that semantically similar intents are arranged adjacently in the list; a special string generation module for generating a highly unique special string to uniquely identify the standardized intent list; a two-stage fine-tuning module for performing a first-stage memory fine-tuning and a second-stage business fine-tuning; wherein, the first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special string and the intent list, and fine-tuning the basic large language model to enable the model to memorize the mapping relationship; the second-stage business fine-tuning includes: fine-tuning the model that has completed memory fine-tuning based on an intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios; and an inference interface module for embedding the special string into prompt words during the model inference stage to replace the complete intent list, reduce the number of input tokens, and improve inference speed.

[0015] This application provides a method and system for improving task reasoning speed, which has the following beneficial effects: 1. Significantly improved inference efficiency: By using special strings to replace the complete intent list, the number of tokens in the prompt words is greatly reduced, thereby effectively reducing the computational load on the model and improving the inference speed by about 50%, meeting the real-time response requirements in high-concurrency scenarios.

[0016] 2. Excellent classification accuracy: Under the two-stage fine-tuning mechanism, the model further adapts to the real business distribution based on the memory mapping relationship, so that the large-scale classification accuracy is very close to that of the complete list solution, which is significantly better than the traditional grouping method and avoids the error propagation problem.

[0017] 3. Achieving the best balance between speed and accuracy: While almost without sacrificing classification performance, it simultaneously reduced the number of input tokens by 84% and the inference latency by 50%, overcoming the technical contradiction of previous solutions that had to make trade-offs between speed and accuracy.

[0018] 4. The model's general capabilities are maintained: By adopting efficient parameter fine-tuning technology, while focusing on improving task performance, the original general reasoning and language understanding capabilities of the large language model are largely preserved, ensuring that it can still flexibly adapt to a variety of downstream scenarios. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a method for improving task inference speed provided in this application embodiment; Figure 2 This is a system composition diagram for improving task inference speed, provided as an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a method and system for improving task inference speed. The technical solutions proposed in this application will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 A flowchart illustrating a method for improving task inference speed, provided as an embodiment of this application. Figure 1 As shown, the method mainly includes the following steps: Step S1: Sort the original intent list by semantic similarity so that semantically similar intents are arranged adjacently in the list.

[0023] To ensure that semantically similar intents are grouped adjacently in the list, cosine similarity is used for sorting. The specific steps are as follows: First, each intent is encoded using the Qwen2.5 vector encoder to obtain a vector for each intent. The similarity matrix between the vectors is calculated. Then, the intents represented by the vectors are sorted based on hierarchical clustering.

[0024] Step S2: Generate a special string with high uniqueness to uniquely identify the standardized intent list.

[0025] Define a special string to represent the entire list of standardized intents, and its generation rules are as follows, which aims to open up a completely new "memory space" for the model.

[0026] Encoding rules: Timestamp: YYYYMMDD_HHMMSS format (e.g., 20250717_022242), accurate to the second.

[0027] Custom encoding: consists of "project identifier (2-4 letters)" + "intent list version number (2 digits)" (e.g., SJKY_01).

[0028] Concatenation and MD5 encoding: Concatenate a fixed prefix, timestamp, and custom encoding into a source string (e.g., IntentList_20240717_022242_SJKY_01), and then use the MD5 encryption algorithm to generate a 32-bit lowercase string as the final special string.

[0029] The principle of memory space: MD5 strings generated in this way are highly unique, which greatly reduces the probability of collisions with words already present in the model's pre-training corpus. This makes the model tend to recognize it as a unique identifier without inherent semantics, making it easier to establish strong mapping weights between this identifier and the large list of intents in the model parameters during subsequent fine-tuning.

[0030] Step S3: Perform the first stage of memory fine-tuning and the second stage of business fine-tuning; wherein, the first stage of memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special strings and the intent list, and fine-tuning the basic large language model to enable the model to memorize the mapping relationship; the second stage of business fine-tuning includes: fine-tuning the model that has completed memory fine-tuning based on the intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios.

[0031] To enable the model to learn the correspondence between special strings and the intent list, a fine-tuning dataset containing 1,000 question-answer pairs was generated by calling the Qwen2.5-72B-Instruct model.

[0032] The question-and-answer dataset constructed in the first stage of memory fine-tuning includes at least one of the following question-and-answer types: direct query type, reverse query type, fuzzy query type, and multi-round follow-up question type; wherein, the direct query type queries the content of the intent list corresponding to the special string, the reverse query type queries the special string corresponding to the list containing several intents, the fuzzy query type queries the user intent category represented by the special string, and the multi-round follow-up question type gradually reveals the mapping relationship between the special string and the intent list through multiple rounds of dialogue.

[0033] Call parameters: Set temperature coefficient (temperature) = 0.0 during generation to ensure the determinism of generated content, maximum generation length (max_length) = 2048, top_p = 0.9.

[0034] Furthermore, during the first stage of memory fine-tuning, the question-answering dataset generated in the previous step is used to fine-tune the basic large model using LoRA. The core goal is to enable the model to "memorize" the correspondence between keywords and the "intent list". LoRA core parameters: r=16, alpha=32, learning rate=2e-4, batch size=2, cumulative gradient steps=1, epochs=5. Fine-tuning is performed using the MS-Swift framework developed based on Modelscope. Training success evaluation criteria: After every 100 steps, the mapping accuracy is evaluated on the validation set. When the accuracy is not lower than 95% for 5 consecutive evaluations, and the model's accuracy on general inference tasks (such as GSM8K math problems) decreases by no more than 5%, the training is considered successful, and the intermediate model ModelA is obtained.

[0035] After the first stage of memory fine-tuning is completed, the LoRA weights are merged with the base model weights to form an intermediate model that can be directly inferred. To obtain an independent and efficient inference model, the LoRA weights generated in the first stage of fine-tuning are merged with the original LLM weights. This generates an independent Model A that can run without an additional LoRA adapter.

[0036] Furthermore, a business scenario intent classification dataset is constructed. To adapt Model A to real-world business scenarios, a high-quality intent classification dataset is created. Dataset specifications: A total of 10,000 samples, of which 8,000 are derived from real dialogue logs of intelligent customer service from January to June 2024, and the other 2,000 are manually constructed edge scenario samples (e.g., the vague expression "Where is my order?" corresponding to "Query order logistics").

[0037] Labeling standards: Four senior AI customer service training and labeling engineers independently labeled the samples back-to-back. The consistency of the labeling was tested by the Kappa coefficient (K≥0.92). All discrepancies were discussed and labeled one by one.

[0038] The second phase involves fine-tuning the business dataset constructed in the previous step. Model A is then fine-tuned a second time using LoRA, with the goal of improving its intent classification performance in real-world scenarios.

[0039] Parameter differences: To fine-tune on business data and avoid overfitting, the learning rate was reduced to 1e-5 and the number of training rounds was set to 5.

[0040] Optimization goal: On a business scenario-specific intent classification test set (1000 samples), the final intent classification accuracy of Model B should be no less than 90%, and the improvement should be no less than 5% compared to Model A.

[0041] Step S4: During the model inference stage, the special string is embedded in the prompt words to replace the complete intent list, reducing the number of input tokens and improving inference speed.

[0042] The above is a method for improving task inference speed provided by embodiments of this application. Based on the same inventive concept, embodiments of this application also provide a system for improving task inference speed. Figure 2 A system composition diagram for improving task inference speed is provided in the embodiments of this application, such as... Figure 2 As shown, the system mainly includes: an intent list processing module 201, used to sort the original intent list by semantic similarity, so that semantically similar intents are arranged adjacently in the list; a special string generation module 202, used to generate a special string with high uniqueness, used to uniquely identify the standardized intent list; a two-stage fine-tuning module 203, used to perform a first-stage memory fine-tuning and a second-stage business fine-tuning; wherein, the first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special string and the intent list, and performing instruction fine-tuning on the basic large language model so that the model remembers the mapping relationship; the second-stage business fine-tuning includes: based on the intent classification dataset of real business scenarios, further fine-tuning the model that has completed memory fine-tuning to improve its classification performance in specific business scenarios; and an inference interface module 204, used to embed the special string into prompt words during the model inference stage, replacing the complete intent list, reducing the number of input tokens, and improving inference speed.

[0043] In this application, the experimental environment and comparison benchmarks are as follows: Test dataset: A business test set that has no overlap with the training set, containing 1000 different intent categories and a total of 5000 test samples.

[0044] Hardware environment: Single consumer-grade NVIDIA GeForce RTX 4090 (24G) graphics card.

[0045] Software environment: PyTorch 2.1.0, Transformers 4.35.2, PEFT 0.7.1.

[0046] Comparison with baseline models (both based on Qwen2.5-7B-Instruct): Benchmark 1: Native Model (Complete Intent List): Zero-shot inference is performed by providing the complete list of 1000 intents in the Prompt, representing the theoretical upper limit of accuracy.

[0047] Benchmark 2: Native Model (Two-Step Intent List): This reasoning approach, which involves first grouping intents and then performing a two-step classification, is a common compromise for speed optimization.

[0048] Comparison of quantification effects: The performance of each option on key evaluation indicators is shown in the table below:

[0049] Based on the above experimental data, the beneficial effects of the present invention are specifically reflected in: Inference efficiency is doubled, achieving optimal speed: Compared to benchmark 1, which provides a complete list of intents in the Prompt, the inference latency of the model in this invention is reduced from 780ms to 390ms, an efficiency improvement of 50%. At the same time, it also significantly outperforms benchmark 2 (550ms) which uses a two-step method, becoming the fastest response solution among the three schemes.

[0050] The model achieves an optimal balance in classification accuracy: It reaches 90.1% accuracy. Compared to the highest accuracy of benchmark 1 (91.5%), the difference is only 1.4 percentage points, successfully avoiding significant accuracy loss that could result from model fine-tuning. However, compared to benchmark 2 (82.0%), which suffered a severe drop in accuracy due to error propagation, the accuracy is significantly improved by 8.1 percentage points (a relative improvement of approximately 10%).

[0051] The optimal trade-off between speed and accuracy is achieved: Overall, the solution of this invention achieves a 50% increase in inference speed and an 84% saving in tokens at a very small cost to accuracy (a reduction of 1.4 percentage points), while effectively avoiding the serious accuracy decline problem caused by the traditional two-step method, thus achieving the best balance between speed and accuracy in large-scale classification tasks.

[0052] The general capabilities of the model were effectively maintained: After deep fine-tuning of the model, the performance score of the model on general inference capability test sets such as GSM8K only slightly decreased from 78.0% to 76.5%, and the loss of general capabilities was controlled within a very small range (reduced by 1.5 percentage points), which proves that the solution has successfully achieved a good balance between "task specialization" and "maintaining general capabilities".

[0053] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0054] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for improving task reasoning speed, applied to the task of classifying customer inquiry intentions in printing, characterized in that, The method includes the following steps: Step S1: Sort the original intent list by semantic similarity so that semantically similar intents are arranged adjacently in the list; Step S2: Generate a highly unique special string to uniquely identify the intent list after sorting in Step S1; specifically, concatenate a predefined fixed prefix string, a timestamp, and a custom code to form a source string; wherein, the fixed prefix is ​​used to identify the string as an identifier of the intent list; perform hash encoding on the source string to generate a fixed-length unique string as the special string; wherein, the timestamp is accurate to the second, and the custom code includes the project identifier and the intent list version number; Step S3: Perform the first-stage memory fine-tuning and the second-stage business fine-tuning; wherein, the first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special strings and the intent list, and fine-tuning the basic large language model to enable the model to memorize the mapping relationship; the second-stage business fine-tuning includes: fine-tuning the model that has completed memory fine-tuning based on the intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios; the question-answering dataset constructed in the first-stage memory fine-tuning includes at least one of the following question-answering types: direct query type, reverse query type, fuzzy query type, and multi-turn follow-up question type; the first-stage memory fine-tuning uses the low-rank adaptation method LoRA to fine-tune parameters, and during the fine-tuning process, the mapping accuracy is evaluated periodically on the validation set; when the mapping accuracy continuously reaches a preset threshold and the model's general capability decreases by no more than a preset range, the memory fine-tuning is deemed complete; Step S4: During the model inference stage, the special string is embedded in the prompt words to replace the complete intent list, reducing the number of input tokens and improving inference speed.

2. The method for improving task reasoning speed according to claim 1, characterized in that, Step S1 specifically includes: Each intent is represented by a vector using a pre-trained vector encoder; Calculate the similarity matrix between each intent vector; Based on the similarity matrix, a hierarchical clustering algorithm is used to group the intent vectors and then sort them, so that semantically similar intents are continuously distributed in the list, thereby improving the model's learning efficiency for dense semantic regions.

3. The method for improving task reasoning speed according to claim 1, characterized in that, The direct query type queries the intent list content corresponding to the special string; the reverse query type queries the special string corresponding to a list containing several intents; the fuzzy query type queries the user intent category represented by the special string; and the multi-round follow-up question type gradually reveals the mapping relationship between the special string and the intent list through multiple rounds of dialogue.

4. The method for improving task reasoning speed according to claim 1, characterized in that, After the first stage of memory fine-tuning is completed, the LoRA weights are merged with the base model weights to form an intermediate model that can be directly inferred.

5. The method for improving task reasoning speed according to claim 1, characterized in that, The dataset used in the second phase of business fine-tuning comes from real business dialogue logs and manually constructed edge scenario samples; the annotation results of the dataset have undergone consistency checks to ensure annotation quality. The business fine-tuning also employs low-rank adaptation technology, with the optimization goal of improving the accuracy of intent classification in business scenarios.

6. The method for improving task reasoning speed according to claim 1, characterized in that, In step S4, the prompt word structure includes: User message fields; The instruction target field contains a reference to a list of intents identified by the special string; The prompt information field is used to guide the model to correctly understand the special string and perform the classification task; The output format field is used to standardize the output format of the classification results.

7. A system for improving task reasoning speed, employing the method for improving task reasoning speed as described in claim 1, characterized in that, The system includes: The intent list processing module is used to sort the original intent list by semantic similarity, so that semantically similar intents are arranged adjacently in the list; The special string generation module is used to generate a highly unique special string to uniquely identify the standardized intent list. A two-stage fine-tuning module is used to perform a first-stage memory fine-tuning and a second-stage business fine-tuning. The first-stage memory fine-tuning includes constructing a question-answering dataset based on the mapping relationship between the generated special strings and the intent list, and fine-tuning the basic large language model to enable the model to memorize the mapping relationship. The second-stage business fine-tuning includes fine-tuning the model that has completed memory fine-tuning based on an intent classification dataset of real business scenarios to improve its classification performance in specific business scenarios. The inference interface module is used to embed the special string into the prompt words during the model inference stage, replacing the complete intent list, reducing the number of input tokens, and improving inference speed.

Images