A method for training an intention recognition model using automatically synthesized data

Abstract

This invention discloses an automatic data synthesis method for training an intent recognition model, belonging to the field of recognition technology. The method includes the following steps: Step 1: Extract all menu names from the business system, vectorize them using a text vector model, and store them in a vector database to obtain a basic vector library; Step 2: Create original training data using differentiated methods for multiple types of intents, and integrate them after manual correction and supplementation to obtain a standard initial dataset; Step 3: Build a dual-classifier model based on the standard initial dataset, sequentially extract text and keyword features, and call the basic vector library to generate menu features. Merge text, keyword, and menu features, and combine iterative training and data augmentation to complete the dual-classifier model training. This invention uses multi-path automatic generation of training samples, coupled with validation set-driven closed-loop data augmentation, significantly reducing the workload of manual annotation and enabling cold-start training of models in small-sample scenarios.

Description

Technical Field

[0001] This invention relates to the field of recognition technology, and more specifically to a method for training an intention recognition model using automatically synthesized data. Background Technology

[0002] With the increasing prevalence of AI applications, the importance of intent recognition models is also growing. The main function of an intent recognition model is to select the function that best suits the user's needs from among the system's functions based on the user's natural language input, routing the input to that function. If the intent recognition is incorrect, the user's input will be routed to the wrong system function, preventing the request from being fulfilled. Furthermore, intent recognition is only an entry point to the overall AI system functionality; if it consumes too much runtime, it will negatively impact the user experience.

[0003] Training intent recognition models faces several challenges: 1. Model developers lack sufficient training datasets. Such datasets often require extensive manual annotation and collection, consuming significant manpower and time. 2. Using a single text classification model limits its ability to identify user-similar intents based on the existing dataset's data distribution, lacking awareness of existing system functions. Adding a function not present in the training set may lead to intent recognition errors, necessitating a reconstructed dataset and model retraining. 3. If the system includes knowledge base question-and-answer functionality, and the knowledge base contains rich and diverse content, the intent "query knowledge base" will become a scattered, unpredictable OOD (out-of-distribution) label type, making it difficult for the model to learn. This invention proposes a small-sample-trainable, high-precision, low-inference-latency intent recognition model with native system function awareness capabilities, while effectively handling OOD data scenarios.

[0004] Based on this, the present invention designs an automatic data synthesis intention recognition model training method to solve the above problems. Summary of the Invention

[0005] To address the aforementioned shortcomings of existing technologies, this invention provides a method for training an intention recognition model using automatically synthesized data.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for training an intention recognition model using automatically synthesized data includes the following steps: Step 1: Extract all menu names from the business system, vectorize them using a text vector model, and store them in a vector database to obtain the basic vector library; Step 2: Create original training data using differentiated methods for different types of intents, and integrate them after manual correction and supplementation to obtain a standard initial dataset; Step 3: Build a dual classifier model based on the standard initial dataset, extract text and keyword features in sequence, and call the basic vector library to generate menu features. Combine text, keyword and menu features, and with iterative training and data augmentation, complete the training of the dual classifier model. Step 4: Train Rag Classifier using the standard initial dataset and the data augmented dataset, and simultaneously use low-rank adaptation technique to perform supervised fine-tuning on the large language model to obtain a dedicated intent recognition model. Step 5: Deploy the trained intent recognition model, dual classifier model, and vector database, and complete the intent recognition reasoning in online scenarios in a layered manner based on similarity thresholds and voting rules.

[0007] Furthermore, the text vector model used is BGE-M3 (BGE third-generation text embedding model), and the vector database is Milvus (Milvus open-source vector database).

[0008] Furthermore, the specific steps for step 1 are as follows: Step 11: Traverse the backend of the target business system, extract all menu names within the target business system, clean the extracted menu names, and obtain plain text menus. Step 12: Using the plain text menu name as input text, call the text vector model to perform vectorization calculation to obtain a semantic vector corresponding to each plain text menu name. Step 13: Create a data table in the vector database, and write each item into the data table according to the unique ID of the menu, the plain text menu name, and the corresponding semantic vector mapping relationship; at the same time, build a vector index for the semantic vector field, and store the data table and the vector index together in the vector database to obtain the basic vector library.

[0009] Furthermore, the multiple intent types include data query, form filling, opening menu, work hour entry, approval processing, system frequently asked questions and answers, and knowledge base questions and answers; The system uses FAQs (Common Standard Questions and Answers) for frequently asked questions. FAQs are standard question-and-answer pairs that store common system questions and answers. The knowledge base Q&A uses business knowledge base documents: scattered documents related to enterprise comprehensive business, which belong to the OOD data source.

[0010] Furthermore, the specific steps for step 2 are as follows: Step 21: Based on data query, form filling and menu opening intent, generate independent original training data in batches by concatenating keywords and menu name strings; Step 22: Manually simulate real user input, and annotate each item according to the work hour reporting and approval processing intention to generate raw training data; Step 23: Directly extract the question text from the FAQ as the raw training data for the system's common question and answer intent; Step 24: Randomly select a portion of text slices from the business knowledge base document, use Qwen3-32B to apply prompt words to the selected text slices, and Qwen3-32B generates possible user questions as the raw training data for the knowledge base question-answering intent; Step 25: Perform manual correction and supplementation on all raw training data. Step 26: Integrate, standardize, and quality check all optimized samples to finally output the standard initial dataset.

[0011] Furthermore, the specific steps for step 3 are as follows: Step 31: The standard initial dataset is randomly split into 70% training set, 20% validation set, and 10% test set. Step 32: Build independent Rag Classifier and Intent Classifier classification models, freeze all network parameters of the text vector model, and preset the feature dimension to 1024. Step 33: Extract 1024-dimensional text and keyword features sequentially, and call the basic vector library to generate 1024-dimensional menu features; Step 34: Concatenate the 1024-dimensional text features, keyword features, and menu features into vectors to obtain the fused features; Step 35: Formulate the iterative training rules for the dual classifier model; Step 36: Based on the evaluation results of the validation set, automatically generate new samples, remove duplicates and supplement the dataset to expand the diversity of training samples, perform closed-loop data augmentation, and update the training set, validation set and test set; Step 37: Iterate and the dual classifier model converges to complete the dual classifier model training.

[0012] Furthermore, step 33 is performed as follows: Step 331: Feed each user input text into the frozen text vector model, perform global semantic vectorization, and directly output 1024-dimensional text semantic features; Step 332: Detect the input text one by one according to the preset keyword library, identify only the first keyword that appears, obtain the keyword embedding feature by querying the keyword embedding table, calculate the relative position of the keyword, and fuse the keyword embedding feature and the relative position of the keyword through a fully connected layer to obtain a 1024-dimensional keyword feature. Step 333: Vectorize the user input text into a query vector using a text vector model. Calculate the cosine similarity between the query vector and each plain text menu vector in the vector database. Sort the queries by score from highest to lowest, select the top 3 plain text menus, and perform dimensionality upscaling through 3 fully connected layers to unify the dimensions to 1024, thus obtaining the standard menu features.

[0013] Furthermore, step 35 is performed as follows: Step 351: Determine the iterative logic of the dual classifier model; Step 352, Training Rules: Only use the partitioned training set to perform training rounds and iterative training, and monitor indicators such as loss value and accuracy in real time during the training process; Step 353: After a single training round is completed, immediately use the validation set to evaluate the performance of the current dual classifier model and record all samples that are classified incorrectly.

[0014] Furthermore, the specific steps for step 4 are as follows: Step 41: Integrate the standard initial dataset and the data-augmented dataset into a unified training dataset; Step 42: Train the Rag Classifier using the training dataset, and save the weights and configuration file after training. Step 43: Remove sample data other than knowledge base questions and answers, and organize and fine-tune the dataset according to a fixed template, standardizing the input and output formats; Step 44: Fine-tune the environment and core parameter configuration; Step 45: Start model fine-tuning and monitor the entire process; Step 46: After the fine-tuning process is completed, export the trained model file and name it Qwen3-4B-Intent Intent Recognition Model, which will be used as the dedicated intent recognition model for final online deployment.

[0015] Furthermore, step 5 involves the following specific steps: Step 51: Deploy the vector database, dual classifier model, and Qwen3-4B-Intent intent recognition model to the online runtime environment. Step 52: Receive natural language input text from online users, call the text vector model to vectorize the input content; use the generated vector as the query condition to retrieve the most semantically similar historical labeled sentences in the vector database, and at the same time obtain the corresponding recall intent and vector similarity score; Step 53: Determine if the similarity score is greater than or equal to 90%. If the determination is yes, directly output the recall intent and end the reasoning process. If the determination is no, proceed to step 54. Step 54: Call Intent Classifier to output the discrimination intent, call Rag Classifier to determine whether the discrimination intent is a knowledge base question answering intent. If the determination is yes, output the discrimination intent directly and end the reasoning; if the determination is no, proceed to step 55. Step 55: Call Qwen3-4B-Intent to perform recognition, obtain the recognition intent, and determine whether the similarity score is less than 70%. If the judgment is yes, directly output the recognition intent and end the reasoning. If the judgment is no, execute step 56. Step 56: Summarize the discrimination intent, identification intent, and recall intent. If all three are consistent, or any two results are the same, the intent with the most votes is taken as the final identification result and output, and the reasoning ends; if the three are completely inconsistent, the identification intent is output and the reasoning ends.

[0016] Compared with existing technologies, this invention adopts multi-path automatic generation of training samples, coupled with validation set-driven closed-loop data augmentation, which significantly reduces the workload of manual annotation and enables cold-start training of models in small sample scenarios. The training incorporates menu vector features, and the model is naturally bound to the semantics of business menus. When adding new functions to the system, only the menu vectorization needs to be entered into the database, without reconstructing the dataset or retraining the entire model, thus avoiding the problem of recognition errors when adding new functions. The dual-classifier architecture decouples out-of-order intent from regular business intent, and generates diverse OOD samples in batches to solve the recognition problem of scattered and difficult-to-fit knowledge base question-answering expressions. Multi-feature fusion and a three-party voting mechanism work together to improve recognition robustness. Layered threshold-based inference calculation, combined with LoRA lightweight fine-tuning of large models, effectively reduces inference time while ensuring high recognition accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 A flowchart of a method for training an intention recognition model for automatically synthesized data; Figure 2 The flowcharts for steps 1 and 2 are shown below; Figure 3 This is a schematic diagram of the dual classifier model structure; Figure 4 A schematic diagram illustrating the model training process using the data augmentation of this invention; Figure 5 This is a schematic diagram of the model training process without using the data augmentation of this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] Example 1: Please refer to the accompanying drawings in the instruction manual. Figure 1 and 5 A method for training an intention recognition model using automatically synthesized data, comprising the following steps: Step 1: Extract all menu names from the business system, vectorize them using a text vector model, and store them in a vector database to obtain the basic vector library; The text vector model chosen is BGE-M3; Vector database: Milvus; The specific steps for step 1 are as follows: Step 11: Traverse the backend of the target business system, extract all menu names within the target business system, clean the extracted menu names, and obtain plain text menus. The menu includes a main menu, sub-menus at levels two and multiple levels, and special function menus; The cleaning process involves removing invalid garbled characters, duplicate names, and blank content. Step 12: Using the plain text menu name as input text, call the text vector model to perform vectorization calculation to obtain a semantic vector corresponding to each plain text menu name. Step 13: Create a data table in the vector database, and write each item into the data table according to the unique ID of the menu, the plain text menu name and the corresponding semantic vector mapping relationship; at the same time, build a vector index for the semantic vector field, and store the data table and the vector index together in the vector database to obtain the basic vector library; The menu's unique ID is the primary key of the data table, used for data traceability, operation and maintenance management, and data addition, deletion, modification and query, but does not participate in feature calculation or online inference.

[0021] Step 2: Create original training data using differentiated methods for different types of intents, and integrate them after manual correction and supplementation to obtain a standard initial dataset.

[0022] The various intent types include data query, form filling, opening menu, work hour entry, approval processing, system frequently asked questions and answers, and knowledge base questions and answers; The system uses FAQs for frequently asked questions, which are standard question-and-answer pairs that store common system questions and answers. The knowledge base Q&A uses business knowledge base documents: scattered documents of comprehensive business categories of the enterprise (without a fixed Q&A format), which belong to the OOD data source.

[0023] The specific steps for step 2 are as follows: Step 21: Based on data query, form filling and menu opening intent, generate independent original training data in batches by concatenating keywords and menu name strings; The rules and examples for string concatenation are as follows: Data Query: The keywords are "search" and "query," paired with menu names, for example: Query my leave requests, find my leave requests; Form filling: The keywords are "add" or "new", paired with the menu name, for example: Add my leave, add my leave; Open Menu: The words used in this context are "open" and "enter," and are paired with the menu name. Examples: Open my leave request, Enter my leave request; Step 22: Manually simulate real user input, and annotate each item according to the work hour reporting and approval processing intention to generate raw training data; Example: Work Hours Reporting: Report 3 hours of work for the AI ​​Q&A project; Approval Processing: View my pending approval items.

[0024] Step 23: Directly extract the question text from the FAQ as the raw training data for the system's common question and answer intent; Step 24: Randomly select a portion of text slices from the business knowledge base document, use Qwen3-32B to apply prompt words to the selected text slices, and Qwen3-32B generates possible user questions as the raw training data for the knowledge base question-answering intent; Qwen3-32B: A large model with 32 billion parameters for 1000 general questions; Specific example: Prompt text: Based on the following content and the file content, ask some questions. Please return a JSON array format: ["Question 1", "Question 2"]. The number of questions is up to you, but each question must be an atomic question. The file name is: {filename}. The following is the document text content: {content}. Step 25: Perform manual correction and supplementation on all raw training data. Sentence expression correction: The sentences generated by rule concatenation are too mechanical and do not conform to the daily expression habits of users. They are manually rewritten into natural spoken language / formal office language.

[0025] For example, change "Add my leave request" to "I want to request leave".

[0026] Supplementing samples with business parameters: Artificially adding statements containing business parameters such as time, range, and number to obtain optimized samples.

[0027] For example, to supplement the sample: Query my leave records from last year.

[0028] Additional processing: The optimized samples are vectorized using a text vector model and simultaneously stored in a vector database to reserve data for subsequent online retrieval and feature matching.

[0029] Step 26: Integrate, standardize, and perform quality checks on all optimized samples to finally output the standard initial dataset; The specific steps for step 26 are as follows: Step 261, Classification and Tag Binding: Classify and summarize the optimized samples according to seven intent categories, accurately label each text sample with the corresponding intent tag, and form a standard format of user input text-intent tag; Step 262, Data Cleaning: Remove duplicate content, blank statements, garbled characters, and invalid samples from the optimized samples of labeled text-intent labels; Step 263, Sample Verification and Integration Output: Verify whether there are any missing or mislabeled contents and intent labels in the optimized sample content and intents after data cleaning and format unification. Integrate the optimized samples without missing or mislabeled contents from the seven intents to obtain the standard initial dataset. Step 3: Build a dual classifier model based on the standard initial dataset, extract text and keyword features in sequence, and call the basic vector library to generate menu features. Combine text, keyword and menu features, and with iterative training and data augmentation, complete the training of the dual classifier model. The dual classifier model includes: Rag Classifier (retrieval enhancement generative classifier) ​​and Intent Classifier (general intent classifier). Rag Classifier: A binary classification model with a 2-dimensional output. It only distinguishes between knowledge base question-and-answer intent and non-knowledge base question-and-answer intent. It is trained using data query, form filling, menu opening, work hour filling, approval processing, common system questions and answers, and knowledge base questions and answers. The labels are only used for binary classification. Intent Classifier: A six-class classification model with an output dimension of 6. It subdivides non-knowledge base inputs into the remaining six business intents, and only uses data query, form filling, menu opening, work hour filling, approval processing, and common system questions and answers for training. The specific steps for step 3 are as follows: Step 31: The standard initial dataset is randomly split into 70% training set, 20% validation set, and 10% test set. Training set: Core component used for iterative learning of parameters in the dual classifier model; Validation set: Used to evaluate the performance of the dual classifier model after each round of training, filter out misclassified samples, and drive subsequent data augmentation; Test set: Retained throughout the process, used to test the generalization ability of the final dual classifier model.

[0030] Step 32: Build independent Rag Classifier and Intent Classifier classification models, freeze all network parameters of the text vector model, and preset the feature dimension to 1024. The BGE-M3 model is used for text vectorization. All parameters of the BGE-M3 network are frozen and their weights are not updated during training. It is used only as a fixed feature extraction tool.

[0031] Step 33: Extract 1024-dimensional text and keyword features sequentially, and call the basic vector library to generate 1024-dimensional menu features; Step 33 is performed as follows: Step 331: Feed each user input text into the frozen text vector model, perform global semantic vectorization, and directly output 1024-dimensional text semantic features.

[0032] Step 332: Detect the input text one by one according to the preset keyword library, identify only the first keyword that appears, obtain the keyword embedding feature by querying the keyword embedding table, calculate the relative position of the keyword, and fuse the keyword embedding feature and the relative position of the keyword through a fully connected layer to obtain a 1024-dimensional keyword feature.

[0033] Preset keyword library: The fixed keyword list includes: open, enter, approval, pending approval, query, search, fill in, fill in, add, create, add new, POC, work hours, knowledge base.

[0034] The relative positions of keywords are calculated as follows:

[0035] pos: The absolute position of the keyword in the text; L: The total length of the input text; This represents rounding down; Step 333: Vectorize the user input text into a query vector using a text vector model, calculate the cosine similarity with each plain text menu vector in the vector database, sort them from high to low scores, select the top 3 plain text menus, and complete the dimensionality upgrade process through 3 fully connected layers to unify the dimensions to 1024 dimensions, thus obtaining the standard menu features. Step 34: Concatenate the 1024-dimensional text features, keyword features, and menu features into vectors to obtain the fused features; Step 34 is performed as follows: Step 341: The concatenated features are sequentially passed through two fully connected layers for feature interaction and fusion, with the ReLU function being used uniformly for activation.

[0036] Step 342, Regularization: After fusion, the layer is connected to the Dropout (random deactivation) layer, and the Dropout coefficient is set to 0.3; Step 343: Connect the classification head: Connect the corresponding output layer according to the dual classifier model: Rag Classifier: Connects to a 2D binary classification head and outputs the OOD (Out of Detail) discrimination result; Intent Classifier: Connects to a 6-dimensional six-classification head and outputs the general intent discrimination result.

[0037] Step 35: Formulate the iterative training rules for the dual classifier model; Step 35 is performed as follows: Step 351: Determine the iteration logic of the dual classifier model: Training epoch: The number of iterations within a single model cycle, controlling the number of times a single batch of data is learned; Iterative training round (data_iteration): A global large iteration round, each iteration training round = multiple training rounds + one round of data augmentation; Step 352, Training Rules: Only use the partitioned training set to perform training rounds and iterative training, and monitor indicators such as loss value and accuracy in real time during the training process.

[0038] Step 353: After a single training round is completed, immediately use the validation set to evaluate the performance of the current dual classifier model and record all samples that are misclassified. Step 36: Based on the evaluation results of the validation set, automatically generate new samples, remove duplicates and supplement the dataset to expand the diversity of training samples, perform closed-loop data augmentation, and update the training set, validation set and test set; Step 36 is performed as follows: Step 361: Traverse by intent category: Randomly sample 5 samples for each intent category; if there are no erroneous samples for a certain intent category in the validation set, then randomly sample 5 samples from the training set as reference samples; Step 362: Generate new data for the large model: Call the Qwen3 Plus (Tongyi Qianwen 3 Enhanced Edition) model, set the temperature coefficient to 1.4, and generate a set number of new user inputs that conform to real expression habits in batches, referring to the sample. The number of entries is set to 100. Step 363, Global Deduplication Verification: Compare the newly generated data with the existing full dataset, remove duplicate statements, and ensure the uniqueness of the dataset.

[0039] Step 364: Supplement the dataset in a tiered manner: Distribute new data proportionally: 40% into the training set, 30% into the validation set, and 30% into the test set, and update the training set, validation set, and test set.

[0040] Step 37: Iterate and the dual classifier model converges to complete the dual classifier model training; Step 37 is performed as follows: Step 371: After one round of iterative training is completed, the next round of the loop begins, and the entire training process is repeated continuously.

[0041] The entire training cycle includes training, validation and evaluation on the validation set, data augmentation, and updating the training set, validation set, and test set.

[0042] Step 372: Monitor the changes in the loss value and validation set accuracy of the dual classifier model throughout the process: Rely on dynamic data augmentation and update the training set, validation set and test set to allow the dual classifier model to continuously learn and avoid overfitting; Step 373: If, after multiple iterations, the validation set metrics only fluctuate slightly without a trend change, or if the maximum number of iterations is reached and the performance meets the target, immediately terminate the iteration, save the weights of the dual classifier model and archive the data to complete the dual classifier model training.

[0043] Step 4: Train Rag Classifier using the standard initial dataset and the data augmented dataset, and simultaneously use low-rank adaptation technique to perform supervised fine-tuning on the large language model to obtain a dedicated intent recognition model. The large language model selected is the Qwen3-4B basic large language model (Tongyi Qianwen 3-4 billion parameter large model). The specific steps for step 4 are as follows: Step 41: Integrate the standard initial dataset and the data-augmented dataset into a unified training dataset; Step 42: Train the Rag Classifier using the training dataset, and save the weights and configuration file after training. Step 43: Remove sample data other than knowledge base questions and answers, and organize and fine-tune the dataset according to a fixed template, standardizing the input and output formats: Input: Determine the intent of the user's input according to the following Faq Poc (Proof of Concept Business Scenario): POC related questions, Open Menu, Grid Data Query, Form Data Add, Process Data, Working Hours. Determine the user's intent based on the following: {current user input}. Output: The user's intent is: {User Intent} Step 44: Fine-tune the environment and core parameter configuration; Hardware environment: Two NVIDIA RTX A6000 GPUs (NVIDIA RTX A6000 professional graphics cards) are deployed to meet the computing power and memory requirements for parallel training; Training framework: The Hugging Face trl framework (a dedicated tool library for fine-tuning large platform models) is used to achieve model fine-tuning; Fine-tuning methods: LoRA (low-rank adaptation) and supervised fine-tuning (SFT) were selected. Core parameters: training epochs are set to 3, LoRA low-rank matrix dimension r=32, LoRA scaling factor alpha=32.

[0044] Step 45: Start model fine-tuning and monitor the entire process; Load the Qwen3-4B basic large language model and fine-tuning dataset, and start the LoRA supervised fine-tuning process; during training, monitor GPU utilization, model loss value, and validation set accuracy in real time to ensure stable training operation. The overall training time is about 1 hour.

[0045] Step 46: After the fine-tuning process is completed, export the trained model file and name it Qwen3-4B-Intent Intent Recognition Model, which will be used as the dedicated intent recognition model for final online deployment.

[0046] Qwen3-4B-Intent: A dedicated fine-tuning model for Qwen3-4 billion parameters; Step 5: Deploy the trained intent recognition model, dual classifier model, and vector database, and complete the intent recognition reasoning in online scenarios in a layered manner based on similarity thresholds and voting rules.

[0047] Step 5 is performed as follows: Step 51: Deploy the vector database, dual classifier model, and Qwen3-4B-Intent intent recognition model to the online runtime environment. Step 52: Receive natural language input text from online users, call the text vector model to vectorize the input content; use the generated vector as the query condition to retrieve the most semantically similar historical labeled sentences in the vector database, and at the same time obtain the corresponding recall intent and vector similarity score; Step 53: Determine if the similarity score is greater than or equal to 90%. If the determination is yes, directly output the recall intent and end the reasoning process. If the determination is no, proceed to step 54. Step 54: Call Intent Classifier to output the discrimination intent, call Rag Classifier to determine whether the discrimination intent is a knowledge base question answering intent. If the determination is yes, output the discrimination intent directly and end the reasoning; if the determination is no, proceed to step 55. Step 55: Call the Qwen3-4B-Intent intent recognition model to identify the intent, determine whether the similarity score is less than 70%. If the similarity score is yes, output the intent directly and end the reasoning. If the similarity score is no, proceed to step 56.

[0048] Step 56: Summarize the discrimination intent, identification intent, and recall intent. If all three are consistent, or any two results are the same, the intent with the most votes is taken as the final identification result and output, and the reasoning ends; if the three are completely inconsistent, the identification intent is output and the reasoning ends.

[0049] Figure 4 , Figure 5 The training curves for Acc (accuracy) and Loss (loss value) are shown separately with and without automatic data augmentation. It can be seen that when automatic data augmentation is used... Figure 4 The Acc curve decreases in each training iteration (reflecting the model's absorption of new data) and then slowly rises again (reflecting the model's learning of the new data). The Loss curve oscillates continuously, further confirming the model's continuous learning of new data. However, without automatic data augmentation ( Figure 5 After several iterations, the accuracy and loss values ​​almost reach a straight line, indicating that the current dataset is too simple for the model. At this point, the model's learning has stopped, and continuing to train may overfit the current data and fail to meet the needs of the actual online environment.

[0050] Specific examples: A user input: "Project implementation reimbursement menu location" is ambiguous in intent, as it could be categorized as either "open menu" or "Frequently Asked Questions (FAQs)". In the evaluation dataset, this type of data was labeled with both possible intents. The following are the recall (TP / (TP+FN)) scores for different models on each task. The Rag Classifier score only includes the intent for knowledge-based question answering, while the Intent Classifier and Qwen3 4B-Intent scores include all intents except knowledge-based question answering. The Voter (voting decision module / comprehensive voting result) score represents the results of the above voting methods, covering all intents. See Table 1 for specific results.

[0051] Table 1

[0052] As shown in the table above, after using Voter's voting decision-making, the recall rate for most intents (knowledge base question and answer, opening menu, data query form filling) is higher than that of Qwen3-32B. Since work hour reporting and approval processing are relatively simple intent judgments (generally containing keywords such as "work hours" or "approval"), both Voter and Qwen3-32B can achieve a recall rate of 100%. Voter's recall rate is slightly lower than that of Qwen3-32B only in the intent of common system questions. In summary, Voter's recall rate (83.4%) in the intelligent assistant intent recognition task is significantly higher than that of Qwen3-32B (70.9%), proving the effectiveness of the present invention.

[0053] This invention employs multi-path automatic generation of training samples, coupled with validation set-driven closed-loop data augmentation, significantly reducing manual annotation workload and enabling cold-start training of models in small-sample scenarios. Training incorporates menu vector features, naturally binding the model to business menu semantics. Adding new system functions only requires vectorizing the menu and storing it in the database, eliminating the need to reconstruct the dataset and retrain the entire model, thus avoiding errors in new function recognition. A dual-classifier architecture decouples out-of-order (OOD) intent from regular business intent, generating diverse OOD samples in targeted batches to solve the recognition challenges of scattered and difficult-to-fit questions and answers in the knowledge base. Multi-feature fusion and a three-party voting mechanism enhance recognition robustness. Layered threshold-based inference calculations, combined with LoRA lightweight fine-tuning of large models, effectively reduce inference time while maintaining high recognition accuracy.

[0054] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training an intention recognition model using automatically synthesized data, characterized in that: Includes the following steps: Step 1: Extract all menu names from the business system, vectorize them using a text vector model, and store them in a vector database to obtain the basic vector library; Step 2: Create original training data using differentiated methods for different types of intents, and integrate them after manual correction and supplementation to obtain a standard initial dataset; Step 3: Build a dual classifier model based on the standard initial dataset, extract text and keyword features in sequence, and call the basic vector library to generate menu features. Combine text, keyword and menu features, and with iterative training and data augmentation, complete the training of the dual classifier model. Step 4: Train Rag Classifier using the standard initial dataset and the data augmented dataset, and simultaneously use low-rank adaptation technique to perform supervised fine-tuning on the large language model to obtain a dedicated intent recognition model. Step 5: Deploy the trained intent recognition model, dual classifier model, and vector database, and complete the intent recognition reasoning in online scenarios in a layered manner based on similarity thresholds and voting rules.

2. The method for training an intention recognition model for automatically synthesized data according to claim 1, characterized in that, The text vector model used is BGE-M3, and the vector database is Milvus.

3. The method for training an intention recognition model for automatically synthesized data according to claim 2, characterized in that, The specific steps for step 1 are as follows: Step 11: Traverse the backend of the target business system, extract all menu names within the target business system, clean the extracted menu names, and obtain plain text menus. Step 12: Using the plain text menu name as input text, call the text vector model to perform vectorization calculation to obtain a semantic vector corresponding to each plain text menu name. Step 13: Create a data table in the vector database, and write each item into the data table according to the unique ID of the menu, the plain text menu name, and the corresponding semantic vector mapping relationship; at the same time, build a vector index for the semantic vector field, and store the data table and the vector index together in the vector database to obtain the basic vector library.

4. The method for training an intention recognition model for automatically synthesized data according to claim 3, characterized in that, The various intent types include data query, form filling, opening menu, work hour entry, approval processing, system frequently asked questions and answers, and knowledge base questions and answers; The system uses FAQs for frequently asked questions, which are standard question-and-answer pairs that store common system questions and answers. The knowledge base Q&A uses business knowledge base documents: scattered documents related to enterprise comprehensive business, which belong to the OOD data source.

5. The method for training an intention recognition model for automatically synthesized data according to claim 4, characterized in that, The specific steps for step 2 are as follows: Step 21: Based on data query, form filling and menu opening intent, generate independent original training data in batches by concatenating keywords and menu name strings; Step 22: Manually simulate real user input, and annotate each item according to the work hour reporting and approval processing intention to generate raw training data; Step 23: Directly extract the question text from the FAQ as the raw training data for the system's common question and answer intent; Step 24: Randomly select a portion of text slices from the business knowledge base document, use Qwen3-32B to apply prompt words to the selected text slices, and Qwen3-32B generates possible user questions as the raw training data for the knowledge base question-answering intent; Step 25: Perform manual correction and supplementation on all raw training data. Step 26: Integrate, standardize, and quality check all optimized samples to finally output the standard initial dataset.

6. The method for training an intention recognition model for automatically synthesized data according to claim 5, characterized in that, The specific steps for step 3 are as follows: Step 31: The standard initial dataset is randomly split into 70% training set, 20% validation set, and 10% test set. Step 32: Build independent Rag Classifier and Intent Classifier classification models, freeze all network parameters of the text vector model, and preset the feature dimension to 1024. Step 33: Extract 1024-dimensional text and keyword features sequentially, and call the basic vector library to generate 1024-dimensional menu features; Step 34: Concatenate the 1024-dimensional text features, keyword features, and menu features into vectors to obtain the fused features; Step 35: Formulate the iterative training rules for the dual classifier model; Step 36: Based on the evaluation results of the validation set, automatically generate new samples, remove duplicates and supplement the dataset to expand the diversity of training samples, perform closed-loop data augmentation, and update the training set, validation set and test set; Step 37: Iterate and the dual classifier model converges to complete the dual classifier model training.

7. The method for training an intention recognition model for automatically synthesized data according to claim 6, characterized in that, Step 33 is performed as follows: Step 331: Feed each user input text into the frozen text vector model, perform global semantic vectorization, and directly output 1024-dimensional text semantic features; Step 332: Detect the input text one by one according to the preset keyword library, identify only the first keyword that appears, obtain the keyword embedding feature by querying the keyword embedding table, calculate the relative position of the keyword, and fuse the keyword embedding feature and the relative position of the keyword through a fully connected layer to obtain a 1024-dimensional keyword feature. Step 333: Vectorize the user input text into a query vector using a text vector model. Calculate the cosine similarity between the query vector and each plain text menu vector in the vector database. Sort the queries by score from highest to lowest, select the top 3 plain text menus, and perform dimensionality upscaling through 3 fully connected layers to unify the dimensions to 1024, thus obtaining the standard menu features.

8. The method for training an intention recognition model for automatically synthesized data according to claim 7, characterized in that, Step 35 is performed as follows: Step 351: Determine the iterative logic of the dual classifier model; Step 352, Training Rules: Only use the partitioned training set to perform training rounds and iterative training, and monitor indicators such as loss value and accuracy in real time during the training process; Step 353: After a single training round is completed, immediately use the validation set to evaluate the performance of the current dual classifier model and record all samples that are classified incorrectly.

9. The method for training an intention recognition model for automatically synthesized data according to claim 8, characterized in that, The specific steps for step 4 are as follows: Step 41: Integrate the standard initial dataset and the data-augmented dataset into a unified training dataset; Step 42: Train the Rag Classifier using the training dataset, and save the weights and configuration file after training. Step 43: Remove sample data other than knowledge base questions and answers, and organize and fine-tune the dataset according to a fixed template, standardizing the input and output formats; Step 44: Fine-tune the environment and core parameter configuration; Step 45: Start model fine-tuning and monitor the entire process; Step 46: After the fine-tuning process is completed, export the trained model file and name it Qwen3-4B-Intent Intent Recognition Model, which will be used as the dedicated intent recognition model for final online deployment.

10. The method for training an intention recognition model for automatically synthesized data according to claim 9, characterized in that, Step 5 is performed as follows: Step 51: Deploy the vector database, dual classifier model, and Qwen3-4B-Intent intent recognition model to the online runtime environment. Step 52: Receive natural language input text from online users, call the text vector model to vectorize the input content; use the generated vector as the query condition to retrieve the most semantically similar historical labeled sentences in the vector database, and at the same time obtain the corresponding recall intent and vector similarity score; Step 53: Determine if the similarity score is greater than or equal to 90%. If the determination is yes, directly output the recall intent and end the reasoning process. If the determination is no, proceed to step 54. Step 54: Call Intent Classifier to output the discrimination intent, call Rag Classifier to determine whether the discrimination intent is a knowledge base question answering intent. If the determination is yes, output the discrimination intent directly and end the reasoning; if the determination is no, proceed to step 55. Step 55: Call Qwen3-4B-Intent to perform recognition, obtain the recognition intent, and determine whether the similarity score is less than 70%. If the judgment is yes, directly output the recognition intent and end the reasoning. If the judgment is no, execute step 56. Step 56: Summarize the discrimination intent, identification intent, and recall intent. If all three are consistent, or any two results are the same, the intent with the most votes is taken as the final identification result and output, and the reasoning ends; if the three are completely inconsistent, the identification intent is output and the reasoning ends.

Images