A method and system for intent recognition based on generative large language models

Abstract

This invention discloses an intent recognition method and system based on a generative large language model, belonging to the field of natural language processing. It solves the problems of insufficient generalization ability, high manual annotation cost, and small context window of the traditional BERT model for intent recognition. This invention retains the traditional NLP supervised classification idea, modifies the Qwen3-0.6B large language model, and adds a classification head to construct an intent classification model. The model is constructed through data processing and fine-tuning during training. In application, user input containing single-turn / multi-turn context is collected. The large model extracts semantic features, the classification head judges and outputs the intent confidence score, and outputs the valid intent after a 0.95 threshold validation. If the score is not met, historical dialogue is concatenated for re-identification. This invention achieves decoupling and fusion of deep semantic understanding and traditional classification structure, improving recognition accuracy, reducing annotation cost, and enhancing system controllability and scalability. It is suitable for multi-turn dialogue scenarios such as intelligent customer service.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing (NLP) technology, specifically relating to an intent recognition method and system based on a generative large language model. Background Technology

[0002] In the field of natural language processing, intent recognition is the core component of intelligent dialogue systems. Existing mainstream intent recognition technologies are all based on traditional natural language processing ideas, modeling intent recognition as a multi-classification problem. They use large-scale training data with manual annotation, combined with feature engineering or pre-trained language models (such as the BERT model) and classifiers to complete intent determination.

[0003] While the aforementioned traditional technologies can achieve basic intent recognition in simple, single-turn short text scenarios, they have significant shortcomings in practical business applications: First, their semantic generalization ability is insufficient. When users use unfamiliar expressions, colloquial expressions, or implicit expressions, the model is prone to misjudgment and cannot accurately capture the user's true intent. Second, manual annotation is costly. Due to the poor generalization effect of the BERT model, it is highly dependent on high-quality labeled data. In multi-business, multi-language, or cold-start scenarios, the cost of data acquisition and maintenance is high. Third, their contextual understanding ability is limited. The context window of the BERT model is only 512 tokens, and even most variant versions of the BERT series models are below 8k tokens. Furthermore, their ability to understand long texts is weak, and they can only adapt to simple scenarios of single-turn input + short text, unable to handle the extremely long inputs of multi-turn dialogues and the implicit contextual relationships within them.

[0004] Due to the aforementioned shortcomings, traditional intent recognition technology struggles to meet the demands of intelligent customer service, intelligent assistants, and other scenarios for high accuracy, low maintenance costs, and strong generalization capabilities. This results in models that are difficult to adapt to rapid business changes in practical applications, leading to high model maintenance costs and severely limited recognition accuracy and robustness. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, the present invention aims to provide an intent recognition method and system based on a generative large language model. While retaining the supervised classification approach of traditional natural language processing techniques, it introduces the generative large language model Qwen3-0.6B and adapts it for downstream intent recognition tasks. The original Qwen3-0.6B is deeply modified in terms of structure and function, reducing manual annotation costs, increasing the upper limit of the model's input context window, improving the understanding of the user's true intent, and achieving accurate recognition of complex, diverse, and unseen intents. Simultaneously, it enhances the system's versatility and scalability in multi-turn dialogues and cross-business scenarios.

[0006] An intent recognition method based on a generative large language model, comprising a model training phase and a model application phase;

[0007] The model training phase includes:

[0008] S11. Collect user text data in business scenarios and manually label intent categories to construct a supervised dataset of "text-intent" pairs;

[0009] S12. Load the supervised dataset and divide it into training and validation sets, and perform bidirectional encoding of the intent labels using label2id and id2label;

[0010] S13. Use a word segmenter that matches the pre-trained large language model to segment and encode the text, and unify the length of the input sequence;

[0011] S14. Load the pre-trained large language model as the feature extraction backbone network, add an intent discrimination and classification head to its output layer, construct an intent classification model, and align the padding configuration of the word segmenter and the model.

[0012] S15. Set the training hyperparameters and train the intent classification model using supervised fine-tuning. Update the model parameters using the cross-entropy loss function. Save the model weights after training is complete.

[0013] The pre-trained large language model is Qwen3-0.6B, and the model application stage includes:

[0014] S21. Collect natural language text input by the user, wherein the text contains single-turn input content or multi-turn dialogue context information;

[0015] S22. Input the text into the finely tuned Qwen3-0.6B large language model, perform contextual semantic modeling through a multi-layer Transformer self-attention mechanism, and generate high-dimensional semantic features that represent the semantics of the text. The large language model is only used for semantic feature extraction and does not directly output the text content.

[0016] S23. Input the high-dimensional semantic features into the intent discrimination and classification head for discriminative calculation, and output the classification results and confidence information of each preset intent category;

[0017] S24. Based on the confidence information, the validity of the recognition result is judged, and the intent tags that meet the conditions and their corresponding confidence scores are output to the downstream business system.

[0018] Preferably, in step S15, the training hyperparameters include batch size, gradient accumulation steps, learning rate, and number of training rounds. A larger effective batch size is simulated through the gradient accumulation mechanism, and model checkpoints are saved periodically during training.

[0019] Preferably, in step S22, the process of generating high-dimensional semantic features includes: inputting the discrete identifier sequence after word segmentation into the embedding layer of a large language model to obtain an initial embedding sequence, outputting a hidden state vector sequence after processing by a multi-layer Transformer network, selecting a preset position vector of the hidden state vector sequence or performing a pooling operation on it to obtain the high-dimensional semantic features.

[0020] Preferably, in step S23, the intent discrimination classification head is a fully connected neural network structure. It projects high-dimensional semantic features onto the discrimination space corresponding to the preset intent category through linear mapping to obtain a discrimination score. Then, it normalizes the discrimination score through a Softmax classifier to generate the probability distribution results of each intent category. The obtained probability value is the confidence level of the corresponding intent category.

[0021] Preferably, in step S24, the specific method for determining validity is as follows: output the top 5 intentions with the highest confidence level. If the value of the highest confidence level among the top 5 intentions exceeds a preset threshold, the recognition result is determined to be valid.

[0022] Preferably, the preset threshold is 0.95, and the sum of the confidence levels of each preset intent category is 1.

[0023] Preferably, if the highest confidence level does not exceed a preset threshold, the user's historical dialogue content is concatenated with the current input text and re-inputted into the intent classification model in the form of a multi-turn dialogue. The steps of the model application stage are repeated until the recognition result meets the validity conditions.

[0024] Preferably, the context window of the Qwen3-0.6B large language model is 40,000 tokens, which supports semantic modeling of long inputs in multi-turn dialogues.

[0025] Preferably, in step S13, excessively long texts are truncated and insufficiently long texts are padded to ensure that the dimensions of the input tensors are consistent during batch training, so that the input distribution of downstream tasks is consistent with that of the large language model pre-training stage.

[0026] An intent recognition system based on a generative large language model is used to implement the above-mentioned intent recognition method, including a model training module and a model application module;

[0027] The model training module includes a data construction and task definition unit, a semantic encoding and feature extraction unit, a model building unit, a model training unit, and a model storage unit connected in sequence. These units are used to complete the construction of supervised datasets, the encoding and processing of text and labels, the construction of an intent classification model, the supervised fine-tuning training of the model, and the storage of model weights, respectively.

[0028] The model application module includes an input unit, a semantic encoding unit, a feature extraction unit, an intent classification unit, and a confidence output unit connected in sequence. These units are used to collect user input text, generate multi-layer hidden layer semantic representations of the text, extract high-dimensional semantic features, perform intent discrimination and classification, and output confidence scores and intent labels that meet threshold conditions. The intent classification unit adopts a discriminative classification structure, which limits the model output space to a preset range of intent categories.

[0029] The core innovation of this invention lies in limiting the generative large language model to a semantic feature extraction component, without directly participating in the intent determination process. By adding an independent discriminative classification head to the output layer of the large language model, a deep fusion of the deep semantic understanding capability of the large language model and the traditional NLP discriminative classification modeling method is achieved. This design transforms the Qwen3-0.6B large language model, originally designed for generative tasks, into a discriminative classification task model, restricting the model output to a predefined fixed category space. This fully utilizes the large context window and strong semantic generalization capability of the large language model while retaining the advantages of the confidence output of the traditional BERT model series. It achieves decoupling and fusion of deep semantic understanding and traditional intent classification structure, reduces the uncertainty of model decision-making, and improves the interpretability and engineering feasibility of the system.

[0030] The advantages and effects of this application are as follows:

[0031] This application presents an intent recognition method based on a generative large language model. By introducing the Qwen3-0.6B large language model for semantic reasoning, it performs comprehensive analysis of user input at the sentence and document levels. This method can accurately capture implicit semantic relationships and deep semantic intents, effectively improving the accuracy and robustness of intent recognition in complex expressions and unseen expression scenarios. Ultimately, it achieves deep intent recognition and improves the accuracy of recognition in complex scenarios.

[0032] This application presents an intent recognition method based on a generative large language model. By leveraging the general language understanding capabilities of the large language model, it significantly reduces the reliance on large-scale manually labeled data, shortens the data preparation cycle, reduces system construction and subsequent maintenance costs, improves system deployment efficiency and scalability, and greatly reduces the costs of manual labeling and system maintenance.

[0033] This application presents an intent recognition method based on a generative large language model. The designed model supports a context window of 40,000 tokens and can uniformly model the context information of multi-turn dialogues. It integrates the current user input with the historical dialogue content to effectively eliminate semantic ambiguity and solves the problem that traditional models have small context windows and cannot handle multi-turn long inputs, thereby improving the recognition capability of multi-turn dialogue scenarios.

[0034] This application presents an intent recognition method based on a generative large language model. Combining the advantages of large language models and traditional classification models, it employs a structured output format and incorporates confidence quantification and threshold control mechanisms to assess the reliability of the recognition results, preventing the propagation of erroneous decisions. After adaptation to downstream business data, the model can achieve intent recognition across various business scenarios. Furthermore, the classification head can be replaced with different discriminative structures to adapt to the performance or efficiency requirements of different business scenarios. Simultaneously, the version control mechanism during model training supports continuous iteration and optimization, enhancing the stability, controllability, and security of the system in practical engineering applications.

[0035] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0036] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0038] Figure 1 A training flowchart for an intent recognition method based on a generative large language model designed for this application;

[0039] Figure 2 The flowchart illustrates the application of an intent recognition method based on a generative large language model designed for this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0041] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0042] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0043] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0044] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0045] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0046] Please refer to the attached document. Figure 1 and Figure 2 , Figure 1 This is a training flowchart for an intent recognition method based on a generative large language model, showing the entire process from starting the training system to saving and deploying the model. Each step is connected sequentially to complete the construction and fine-tuning of the intent classification model.

[0047] Please refer to the attached training procedure. Figure 1 ,include:

[0048] (1) The training system starts, predefines the intent set, and completes the basic configuration of the model, word segmenter, and data;

[0049] (2) Conduct data collection and task definition, collect business text and label the intent, and construct a supervised dataset;

[0050] (3) Perform data preprocessing and representation construction, divide the training / validation set, encode intent labels, and generate unified model input by text segmentation and encoding;

[0051] (4) Perform model loading and structure adaptation, load Qwen3-0.6B and add classification headers to build an intent classification model;

[0052] (5) Perform training configuration and supervised fine-tuning, set training hyperparameters, fine-tune the model and update parameters using labeled data;

[0053] (6) Prepare for model saving and deployment, save model checkpoints and final weights, and complete all preparations before model deployment.

[0054] Figure 2 The application flowchart of the intention recognition method based on the generative large language model designed for this application shows the entire process from the startup of the dialogue system to the output of the intention and the triggering of multiple rounds of dialogue. It clearly reflects the core logic of confidence threshold control and the loop mechanism of multiple rounds of dialogue. Specifically, it includes: (1) The dialogue system is started to build the actual running environment of the intention recognition model; (2) The user inputs text in the dialogue system, and the system records and transmits the original input to the intention recognition model; (3) The intention recognition model performs word segmentation and vectorization encoding on the input text and then infers, and outputs the original confidence score of each intention category through the discriminative classification structure; (4) The system normalizes the original score, calculates the probability value of each intention, outputs the TOP5 intentions and their corresponding confidence scores, and judges whether the maximum confidence score is greater than 0.95; (5) If the maximum confidence score is greater than 0.95, the system outputs the intention label / provides the URL jump, and the intention recognition process ends; if the maximum confidence score is not greater than 0.95, the system does not output content, splices the user's historical dialogue content and re-inputs it into the model in the form of multiple rounds of dialogue, and repeats the above recognition process until the termination condition is met.

[0055] Example 1: This example mainly introduces an intent recognition method and system based on a generative large language model. The core is to adapt the generative large language model Qwen3-0.6B to downstream intent recognition tasks, transforming the generative task model into a discriminative classification task model. Intent recognition is then achieved by combining the supervised classification idea of ​​traditional NLP. The overall implementation process is divided into two major stages: model training and model application. The corresponding system consists of a model training module and a model application module working together. The specific implementation methods of each stage and module are described in detail below.

[0056] I. Specific Steps for Model Training

[0057] The purpose of model training is to build and fine-tune an intent classification model that adapts to the business scenario, providing a model foundation for subsequent intent recognition and inference. This includes the following eight steps:

[0058] Training data acquisition: Collect natural language text data from real users in actual business scenarios, manually annotate each text based on the business semantic system to clarify its corresponding intent category, transform unstructured natural language into a supervised dataset of "text-intent" pairs, and finally store the dataset in structured formats such as CSV to ensure data readability and scalability.

[0059] Load the dataset and split it into training and validation sets: Load the structured "text-intent" pair dataset, randomly shuffle the dataset according to a preset ratio and divide it into training and validation sets. During the splitting process, a fixed random seed is used to ensure the representativeness of the data distribution and the reproducibility of the experiment. The validation set is used to evaluate the model's generalization ability on unseen samples and prevent overfitting during model training.

[0060] Intent label encoding: Maps the original string-based intent labels to consecutive integer IDs, constructs a bidirectional mapping relationship between labels and numerical spaces (label2id and id2label), transforms discrete semantic categories into numerical forms that the model can compute, clarifies the category space of the intent classification task, and provides a foundation for the subsequent calculation of the classification loss function.

[0061] Text segmentation and input encoding: A tokenizer matching the pre-trained large language model Qwen3-0.6B is used to segment, encode, and quantify the text. A uniform maximum sequence length is set, and text exceeding the length is truncated, while text below the length is padded to ensure that the dimensions of the input tensor are consistent during batch training, so that the input distribution of downstream tasks is consistent with that of the model pre-training stage.

[0062] Constructing an intent classification model: Load the pre-trained Qwen3-0.6B large language model as the feature extraction backbone network, and introduce a classification head for intent discrimination above its output layer. The classification head is a linear layer and can be implemented using a fully connected neural network structure. The main body of the large language model is responsible for capturing the deep semantic representation of the text, and the classification head is responsible for mapping the semantic representation to specific intent categories, thus completing the intent classification task modeling based on the large language model.

[0063] Padding Alignment with Model Configuration: Align the padding-related configurations of the tokenizer and the large language model, explicitly specify the padding symbols and their corresponding IDs, avoid inconsistencies in attention mask or padding processing during batch training, and ensure numerical stability and computational correctness during model training.

[0064] Training parameters and optimization strategy settings: Configure the key hyperparameters required for model training, including batch size, gradient accumulation steps, learning rate, number of training rounds, and log and model saving strategies; simulate a larger effective batch size under limited GPU memory conditions through gradient accumulation mechanism, thereby improving the utilization efficiency of hardware resources while ensuring training stability.

[0065] Model Training and Weight Saving: The intent classification model is supervised and fine-tuned based on labeled training data. The difference between the model's predictions and the actual intent labels is measured using the cross-entropy loss function, and the model parameters are updated using the backpropagation algorithm. This process, while preserving the general semantic understanding capabilities of the Qwen3-0.6B large language model, guides the model to learn the intent discrimination boundary in specific business scenarios, achieving targeted enhancement of task-level capabilities. During training, model checkpoints are saved periodically according to a preset strategy. After training, the final intent classification model weights are output, providing a foundation for model application.

[0066] II. Specific Steps for Model Application

[0067] The purpose of model application is to use the trained intent classification model to identify the intent of user-input natural language text, and output valid intent labels and corresponding confidence scores. Specifically, it includes the following four steps:

[0068] User input collection: Collect natural language text data entered by users in the dialogue system. The text may be only the content of a single round of input, or it may contain contextual information from multiple rounds of historical dialogue. The collected text serves as the raw input for subsequent semantic processing.

[0069] Semantic Feature Generation: User input text is fed into a finely tuned Qwen3-0.6B large language model. The large language model performs contextual semantic modeling on the text through a multi-layer Transformer self-attention mechanism, generating a high-dimensional semantic feature representation that represents the semantic information of the input text. In this process, the large language model is only used for semantic feature extraction and understanding, and does not directly output the text content; the specific generation process is as follows:

[0070] (1) The input text is preprocessed by a tokenizer that matches Qwen3-0.6B, which is then segmented into sub-word units and mapped to discrete identifier sequences (input_ids). At the same time, an attention mask is generated based on the text length to indicate the effective position for semantic modeling.

[0071] (2) Input input_ids into the embedding layer of the large language model, and map discrete identifiers into continuous vectors of fixed dimensions by looking up tables to obtain an initial embedding sequence containing word-level semantic information;

[0072] (3) The initial embedded sequence is processed by a multi-layer Transformer network. Each layer models the semantic relationship between sequence positions through a multi-head self-attention mechanism and performs nonlinear transformation through a feedforward neural network to achieve layer-by-layer abstraction from word-level semantics to sentence-level and intent-level semantics.

[0073] (4) After processing by all Transformer layers, the model outputs a sequence of hidden state vectors. The vector at the preset position of the sequence is selected or the pooling operation is performed to obtain a high-dimensional semantic feature representation for downstream intent discrimination tasks.

[0074] Intent discrimination calculation: The high-dimensional semantic features mentioned above are input into the intent classification head, which performs discriminative calculation on the semantic features and outputs the classification results and confidence information of each preset intent category. The specific calculation process is as follows: The classification head projects the high-dimensional semantic features onto the discrimination space corresponding to the number of preset intent categories through linear mapping to obtain the discrimination score of each intent category; then, the discrimination score is normalized by the Softmax classifier to generate the probability distribution results of each intent category. Each probability value is the confidence of the input text belonging to the corresponding intent category, and the sum of the confidence of all intent categories is 1. In this step, the classification head can be replaced with other discriminative intent classification structures to adapt to the performance or efficiency requirements of different business scenarios.

[0075] Result Verification and Output: Based on the confidence information output by the classification head, the validity of the intent recognition result is judged, and the intent tags that meet the conditions and their corresponding confidence scores are output to the downstream business system. The specific judgment rule is as follows: output the top 5 intents with the highest confidence scores. If the highest confidence score among the top 5 exceeds the preset threshold of 0.95, the intent is judged to be valid, and the corresponding intent tag and confidence score are output. If the highest confidence score does not exceed 0.95, no intent content is output, and a multi-turn dialogue mechanism is triggered.

[0076] III. Module Composition of an Intent Recognition System

[0077] The intent recognition system that implements the above method includes a model training module and a model application module. Each module has several functional units, which are connected in sequence and work together to complete the entire process from model building to intent recognition.

[0078] Model training module

[0079] (1) Data construction and task definition unit: used to construct a supervised dataset for intent classification tasks, collect user text from business scenarios and manually annotate it, transform unstructured text into structured data of “text-intent” pairs, and explicitly define intent labels to clarify the category space boundaries of classification tasks.

[0080] (2) Semantic encoding and feature extraction unit: It is used to complete the transformation from raw text to model input features, randomly shuffle the dataset and divide the training / validation set, encode the intent label into an integer ID, and at the same time, it uses the tokenizer to segment, truncate and fill the text to generate a token sequence in a unified format.

[0081] (3) Model building unit: used to build an intent classification model, load the pre-trained Qwen3-0.6B large language model as the semantic representation backbone network, introduce a classification head in its output layer, and adapt the general generative language model to a specific NLP discriminative task model by explicitly specifying the number of categories and the label mapping relationship.

[0082] (4) Model training unit: used for supervised fine-tuning of the model, defining the hyperparameters and optimization strategies required for training, calculating the classification loss using labeled data, updating the model parameters through the backpropagation algorithm, and enhancing the business intent discrimination ability while retaining the semantic understanding ability of the large language model.

[0083] (5) Model storage unit: used for saving and managing model training results, saving model checkpoints according to preset strategies, outputting the final model weights after training is completed, and supporting model deployment, rollback and continuous iteration through a clear directory structure and version control mechanism.

[0084] Model application module

[0085] (1) Input unit: used to receive natural language text input by the user, record and perform basic processing on the text, and then transmit the processed text data to the subsequent semantic coding unit.

[0086] (2) Semantic encoding unit: used to generate multi-layer hidden layer semantic representation of text. It encodes the input text through the trained intent recognition model, models the context dependency of the text using the self-attention mechanism of the multi-layer Transformer structure, and outputs a multi-layer hidden layer representation that integrates local and global semantics.

[0087] (3) Feature extraction unit: used to extract high-dimensional semantic feature vectors from the semantic representation of multiple hidden layers. By selecting specific position vectors of the preset layer / final layer or aggregating the hidden vectors of all tokens, a fixed-dimensional feature vector representing the overall semantics of the text is obtained and used as the input of the intent classification unit.

[0088] (4) Intent classification unit: used for intent discrimination and classification. It adopts a discriminative classification structure, restricts the model output space to the range of predefined intent categories, maps semantic feature vectors, and outputs the original classification scores of each preset intent category.

[0089] (5) Confidence output unit: used for confidence calculation and effective intent output. It normalizes the original classification score and converts it into a probability value of 0-1. It outputs the intent category and confidence corresponding to the highest probability and outputs the TOP5 intent categories as auxiliary reference. It compares the maximum confidence with the threshold of 0.95. If it exceeds the threshold, it outputs the intent name and confidence. If it is lower than the threshold, it does not output the content and triggers a multi-turn dialogue.

[0090] This embodiment is based on two core principles: model training principles and model application principles. Both revolve around deep semantic modeling, discriminative intent classification, and confidence control mechanisms, as detailed below:

[0091] Model training principle

[0092] (1) Principle of constructing task and supervision signal: Based on business scenario, construct intent classification task, assign clear intent labels to text samples through manual annotation, form a one-to-one correspondence between "text and intent", and provide clear optimization target and stable supervision signal for model training.

[0093] (2) Text representation and input distribution alignment principle: The text is segmented and encoded using the same tokenizer as the pre-trained large language model. By using a unified sequence length, truncation and padding strategy, the text is mapped to a standardized token sequence, ensuring that the input distribution of downstream tasks is highly consistent with the model pre-training stage, thus avoiding semantic information loss or performance degradation.

[0094] (3) Model structure adaptation and discriminant introduction principle: While keeping the main structure of the Qwen3-0.6B large language model unchanged, an intent discrimination classification head is introduced in the output layer to clarify the scale of the intent category space, transforming the generation task model into a discrimination task model, so that the model can directly output the probability distribution of intent categories.

[0095] (4) Supervised fine-tuning and parameter-oriented optimization principle: The classification loss between the model prediction result and the real intent is calculated using labeled data. The model parameters are updated through backpropagation to guide the model parameters to converge towards the direction of business intent discrimination, forming a stable and clear intent discrimination boundary, while retaining the general semantic understanding ability of the large language model.

[0096] Model application principle

[0097] (1) Semantic modeling principle: Utilize the multi-layer attention mechanism and context modeling capability of large language models to perform global semantic understanding of user input text, generate hidden layer semantic representations containing context information, and provide a high-quality semantic foundation for intent discrimination.

[0098] (2) Semantic feature construction principle: Select a predetermined position vector or obtain a fixed dimension semantic feature vector from the hidden layer semantic representation output by the large language model, and uniformly represent the overall semantic information of the text as the input feature for intent classification.

[0099] (3) Intent discrimination principle: Input the semantic feature vector into the discriminative intent classification model, map the semantic feature space to the intent category space through linear mapping, output the original classification score of each intent category, and complete the intent determination.

[0100] (4) Confidence quantification and threshold control principle: The original classification score is normalized to meet the probability distribution constraint. The maximum probability is used as the confidence of the recognition result. By comparing it with the preset threshold of 0.95, the output of the intent recognition result is controlled. At the same time, a multi-round dialogue or system termination mechanism is triggered to improve the reliability of the recognition result and the stability of system interaction.

[0101] The above description is merely a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter alterations to these embodiments within the spirit and principles of the present invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of the present invention, fall within the scope of protection of the present invention.

Claims

1. An intent recognition method based on a generative large language model, characterized in that, This includes the model training phase and the model application phase; The model training phase includes: S11. Collect user text data in business scenarios and manually label intent categories to construct a supervised dataset of "text-intent" pairs; S12. Load the supervised dataset and divide it into training and validation sets, and perform bidirectional encoding of the intent labels using label2id and id2label; S13. Use a word segmenter that matches the pre-trained large language model to segment and encode the text, and unify the length of the input sequence; S14. Load the pre-trained large language model as the feature extraction backbone network, add an intent discrimination and classification head to its output layer, construct an intent classification model, and align the padding configuration of the word segmenter and the model. S15. Set the training hyperparameters and train the intent classification model using supervised fine-tuning. Update the model parameters using the cross-entropy loss function. Save the model weights after training is complete. The pre-trained large language model is Qwen3-0.6B, and the model application stage includes: S21. Collect natural language text input by the user, wherein the text contains single-turn input content or multi-turn dialogue context information; S22. Input the text into the finely tuned Qwen3-0.6B large language model, perform contextual semantic modeling through a multi-layer Transformer self-attention mechanism, and generate high-dimensional semantic features that represent the semantics of the text. The large language model is only used for semantic feature extraction and does not directly output the text content. S23. Input the high-dimensional semantic features into the intent discrimination and classification head for discriminative calculation, and output the classification results and confidence information of each preset intent category; S24. Based on the confidence information, the validity of the recognition result is judged, and the intent tags that meet the conditions and their corresponding confidence scores are output to the downstream business system.

2. The intent recognition method based on a generative large language model according to claim 1, characterized in that, In step S15, the training hyperparameters include batch size, gradient accumulation steps, learning rate, and number of training rounds. A larger effective batch size is simulated through the gradient accumulation mechanism, and model checkpoints are saved periodically during training.

3. The intent recognition method based on a generative large language model according to claim 1, characterized in that, In step S22, the process of generating high-dimensional semantic features includes: inputting the discrete identifier sequence after word segmentation into the embedding layer of a large language model to obtain an initial embedding sequence, outputting a hidden state vector sequence after processing by a multi-layer Transformer network, selecting a preset position vector of the hidden state vector sequence or performing a pooling operation on it to obtain the high-dimensional semantic features.

4. The intent recognition method based on a generative large language model according to claim 1, characterized in that, In step S23, the intent discrimination classification head is a fully connected neural network structure. It projects high-dimensional semantic features onto the discrimination space corresponding to the preset intent category through linear mapping to obtain a discrimination score. Then, it normalizes the discrimination score through a Softmax classifier to generate the probability distribution results of each intent category. The obtained probability value is the confidence level of the corresponding intent category.

5. The intent recognition method based on a generative large language model according to claim 1, characterized in that, In step S24, the specific method for determining validity is as follows: output the top 5 intentions with the highest confidence. If the value of the highest confidence among the top 5 intentions exceeds a preset threshold, the recognition result is determined to be valid.

6. The intent recognition method based on a generative large language model according to claim 5, characterized in that, The preset threshold is 0.95, and the sum of the confidence levels of each preset intent category is 1.

7. The intent recognition method based on a generative large language model according to claim 5, characterized in that, If the highest confidence level does not exceed the preset threshold, the user's historical dialogue content is concatenated with the current input text and re-input into the intent classification model in the form of a multi-turn dialogue. The steps of the model application stage are repeated until the recognition result meets the validity conditions.

8. The intent recognition method based on a generative large language model according to claim 1, characterized in that, The Qwen3-0.6B large language model has a context window of 40,000 tokens, supporting semantic modeling of long inputs in multi-turn dialogues.

9. The intent recognition method based on a generative large language model according to claim 1, characterized in that, In step S13, excessively long texts are truncated and insufficiently long texts are padded to ensure that the dimensions of the input tensors are consistent during batch training, so that the input distribution of downstream tasks is consistent with that of the large language model pre-training stage.

10. An intent recognition system based on a generative large language model, characterized in that, The method for implementing the intent recognition method according to any one of claims 1-9 includes a model training module and a model application module; The model training module includes a data construction and task definition unit, a semantic encoding and feature extraction unit, a model building unit, a model training unit, and a model storage unit connected in sequence. These units are used to complete the construction of supervised datasets, the encoding and processing of text and labels, the construction of an intent classification model, the supervised fine-tuning training of the model, and the storage of model weights, respectively. The model application module includes an input unit, a semantic encoding unit, a feature extraction unit, an intent classification unit, and a confidence output unit connected in sequence. These units are used to collect user input text, generate multi-layer hidden layer semantic representations of the text, extract high-dimensional semantic features, perform intent discrimination and classification, and output confidence scores and intent labels that meet threshold conditions. The intent classification unit adopts a discriminative classification structure, which limits the model output space to a preset range of intent categories.

Images