Large model-based intention classification method and computer program product

Abstract

The embodiment of the invention provides an intention classification method based on a large model and a computer program product, and relates to the technical field of large language models. The method comprises the following steps: retrieving at least one first intention example matched with a query request from an intention example database, inputting the query request, each first intention example and a cue word template into an intention classification model to obtain a first-layer target intention of the query request, and taking the first-layer target intention as a retrieval condition, and searching at least one second intention example from the intention example database, inputting the query request, each second intention example and the cue word template into the intention classification model to obtain a second-layer target intention of the query request, and when the second-layer target intention meets the intention classification requirement of the query request, performing intention classification on the second-layer target intention. And taking the second-layer target intention as a final target intention of the query request. Therefore, efficient and accurate recognition of large-scale intentions can be realized on the premise of not exceeding a context window of a large model.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and more specifically, to an intent classification method and computer program product based on a large language model. Background Technology

[0002] In current intelligent interaction systems, intent recognition is a key step in achieving efficient human-computer interaction. Intent recognition based on large language models has become the mainstream solution. Its process typically involves combining query requests with intent examples to form prompt words, which are then input into a large model for classification.

[0003] However, as the number of intent examples surges, inputting all intent examples into a large model can lead to excessively long prompts that exceed the context window limit of the large model, resulting in some information being truncated or lost, thus affecting the recognition performance. Summary of the Invention

[0004] In view of this, the object of the present invention is to provide an intent classification method and computer program product based on a large model to at least partially improve the above-mentioned problems.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide an intent classification method based on a large model, comprising: In response to a query request, at least one first intent example matching the query request is retrieved from a pre-built intent example database; the intent example database includes multiple intent examples, including a query request example and multiple intents with a hierarchical structure; the first intent example includes the query request example and a first-level intent. The query request, each first intent example, and the prompt word template are input into the intent classification model to obtain the first-layer target intent of the query request; the first-layer target intent belongs to the first-layer intent of each first intent example. Using the first-level target intent as a retrieval condition, at least one second intent example is retrieved from the intent example database according to the query request; the second intent example includes the query request example and the second-level intent; the second-level intent belongs to the next level intent of the first-level intent; The query request, each second intent example, and the prompt word template are input into the intent classification model to obtain the second-layer target intent of the query request; the second-layer target intent belongs to the second-layer intent of each second intent example. If the second-layer target intent satisfies the intent classification requirement of the query request, the second-layer target intent shall be regarded as the final target intent of the query request.

[0006] Optionally, the method further includes: If the second-layer target intent does not meet the intent classification requirements of the query request, at least one Nth intent example is retrieved from the intent example database according to the query request. The Nth intent example includes the query request example and the Nth-layer intent. The Nth-layer intent belongs to the next level intent of the (N-1)th-layer intent, where N is an integer greater than or equal to 3. The query request, each of the Nth intent examples, and the prompt word template are input into the intent classification model to obtain the Nth layer target intent of the query request. If the target intent at the Nth layer satisfies the intent classification requirement of the query request, then the target intent at the Nth layer shall be taken as the final target intent of the query request. If the target intent of the Nth layer does not meet the intent classification requirements of the query request, at least one intent example of the N+1th layer is retrieved from the intent example database according to the query request. The intent example of the N+1th layer includes the query request example and the intent of the N+1th layer. The query request, each of the N+1th intent examples, and the prompt word template are input into the intent classification model to obtain the N+1th layer target intent of the query request. Until the target intent at layer N+M satisfies the intent classification requirement of the query request, the target intent at layer N+M is taken as the final target intent of the query request, where M is an integer greater than or equal to 1.

[0007] Optionally, retrieving at least one first intent example matching the query request from a pre-built intent example database in response to the query request includes: Based on the query request, a first preset number of first candidate intent examples are retrieved from the intent example database through sparse retrieval and dense retrieval. The first candidate intent examples are sorted by a sorting optimization model, and the first candidate intent examples in the top two columns are taken as the first intent examples.

[0008] Optionally, the step of using the first-layer target intent as a retrieval condition and retrieving at least one second intent example from the intent example database according to the query request includes: Based on the first-layer target intent, a second candidate intent example that is the first-layer target intent is retrieved from the intent example database; Based on the query request, a first preset number of third candidate intent examples are retrieved from the second candidate intent examples through sparse retrieval and dense retrieval. The third candidate intent examples are sorted using a sorting optimization model, and the top-ranked second preset number of candidate intent examples are taken as the second intent examples.

[0009] Optionally, the prompt template includes step analysis, a list of optional intents, a reference example, output requirements, example output, and a query request.

[0010] Optionally, after the step of taking the second-level target intent as the final target intent of the query request, where the second-level target intent satisfies the intent classification requirement of the query request, the method further includes: Obtain user feedback information in response to the query request; the user feedback information is used to indicate whether the final target intent is accurate. If the user feedback information is inaccurate, then the final target intent of the query request will be used to generate negative training samples. The training samples are added to the training dataset for incremental training or periodic retraining of the intent classification model.

[0011] Optionally, the method further includes the step of constructing the intent example database, which includes: Obtain multiple intent example samples; the intent example samples include query request examples and Lth layer intents; Each of the aforementioned intent example samples is input into the semantic vector generation model to generate intent vectors for each of the aforementioned intent example samples; Cluster analysis is performed on each of the intent vectors to obtain the (L-1)th layer intent of each intent example sample; the (L-1)th layer intent is the next level intent of the Lth layer intent; Each intent example sample and its corresponding L-1 layer intent are stored in the intent example database.

[0012] Optionally, before the step of storing each intent example sample and its corresponding L-1 layer intent into the intent example database, the method further includes: The sample intents of a preset proportion and their corresponding L-1 layer intents are input into multiple general large models to determine whether the classification is accurate, and the determination results given by each of the general large models are obtained. For each intent example sample, when the number of cases where the determination result indicates inaccurate classification is greater than a preset number, the intent example sample is marked as an uncertain category intent example sample; Calculate the central semantic vector for each of the (L-1)th layer intentions; the central semantic vector is the average of the intention vectors of all the intention example samples corresponding to the (L-1)th layer intention. Calculate the cosine similarity between the intent vector of the uncertain category intent example sample and each of the central semantic vectors; Select the K central semantic vectors with the highest cosine similarity and use their corresponding (L-1)th layer intentions as the first candidate intention categories; The uncertain category intent example sample and each of the first candidate intent categories are input into the multiple general large models to select the intent category, thereby obtaining at least one second candidate intent category; The second candidate intent category with the largest number of entries is taken as the L-1 layer intent of the uncertain category intent example sample.

[0013] Optionally, before the step of storing each intent example sample and its corresponding L-1 layer intent into the intent example database, the method further includes: For each of the intent example samples, multiple expanded query request examples that maintain semantic consistency but have different expressions are generated using a large language model to expand the intent example samples; Data cleaning is performed on each of the extended query request examples to obtain cleaned query request examples; Each of the cleaned query request examples, along with its corresponding Lth-level intent and L-1th-level intent, is assembled into a new intent example sample.

[0014] Secondly, embodiments of the present invention provide a computer program product, the computer program product including instructions, which, when executed on an electronic device, cause the electronic device to perform the method described in any of the above-mentioned embodiments.

[0015] The present invention provides an intent classification method and computer program product based on a large model. Through a hierarchical intent recognition mechanism, it first performs coarse-grained classification based on the top-level intent, and then dynamically loads the corresponding sub-intents for fine-grained recognition. This effectively reduces the number of candidate intents in a single classification without exceeding the context limitations of the large model, thereby improving the recognition accuracy and system efficiency in large-scale intent scenarios.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic structural block diagram of an electronic device provided in an embodiment of the present invention; Figure 2 This is one of the flowcharts illustrating an intent classification method based on a large model, provided as an embodiment of the present invention. Figure 3 This is a second flowchart illustrating an intent classification method based on a large model, provided as an embodiment of the present invention. Figure 4 The third flowchart illustrates an intent classification method based on a large model, as provided in an embodiment of the present invention. Figure 5 The fourth flowchart illustrates an intent classification method based on a large model, as provided in an embodiment of the present invention. Figure 6 The fifth flowchart illustrates an intent classification method based on a large model, as provided in an embodiment of the present invention. Figure 7 A flowchart illustrating a large-model-based intent classification method provided in this embodiment of the invention is shown in Figure 6. Figure 8 The seventh flowchart illustrates an intent classification method based on a large model, as provided in an embodiment of the present invention. Figure 9 This is the eighth flowchart illustrating an intent classification method based on a large model, provided as an embodiment of the present invention.

[0019] Reference numerals: 100-Electronic device; 101-Memory; 102-Communication interface; 103-Processor; 104-Communication bus. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] In current intelligent interaction systems, intent recognition is a crucial step in achieving efficient human-computer interaction. Intent recognition based on large language models has become the mainstream solution. Its process typically involves combining user questions with intent examples into a prompt, which is then input into a large model for classification. However, with the surge in the number of intents and examples, inputting all examples into the large model leads to excessively long prompts, exceeding the model's context window limit. This results in truncated or lost information, impacting recognition accuracy. Conversely, reducing the number of example inputs leaves the model lacking reference information, potentially forcing it to rely solely on intent name prediction, leading to a decrease in accuracy.

[0025] Based on the above, this invention provides an intent classification method and computer program product based on a large model. Through a hierarchical intent recognition mechanism, it first performs coarse-grained classification based on the top-level intent, and then dynamically loads the corresponding sub-intents for fine-grained recognition. This effectively reduces the number of candidate intents in a single classification without exceeding the context limitations of the large model, thereby improving the recognition accuracy and system efficiency in large-scale intent scenarios.

[0026] To implement the process steps and functions of the various examples of this invention, please refer to [link / reference]. Figure 1 , Figure 1This is a schematic structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes a memory 101 and a processor 103, which are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses 104 or signal lines. The memory 101 can be used to store software programs and modules, and the processor 103 executes the software programs and modules stored in the memory 101, thereby performing various functional applications and data processing.

[0027] Electronic device 100 can be, but is not limited to, a personal computer (PC), a server, a distributed computer, etc. It is understood that electronic device 100 is not limited to a physical server, but can also be a virtual machine on a physical server, a virtual machine built on a cloud platform, or any other computer that can provide the same functionality as the server or virtual machine. The operating system of electronic device 100 can be, but is not limited to, Windows, Linux, etc.

[0028] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0029] The communication connection between the electronic device 100 and external devices is achieved through at least one communication interface 102 (which can be wired or wireless).

[0030] Processor 103 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of this embodiment can be completed by integrated logic circuits in the hardware of processor 103 or by instructions in software form. Processor 103 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0031] Understandable. Figure 1 The structure shown is for illustrative purposes only; the electronic device 100 may also include components that are more advanced than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0032] The following is an exemplary description of the intent classification method based on a large model provided by this invention. See also... Figure 2 The subject executing this method can be one of the above. Figure 1 The electronic device 100 shown, the method includes as follows Figure 2 The following steps are described: S210: In response to the query request, retrieve at least one first intent example that matches the query request from the pre-built intent example database.

[0033] The intent example database includes multiple intent examples, including query request examples and multiple intents with a hierarchical structure; the first intent example includes query request examples and first-level intents.

[0034] S220: Input the query request, examples of each first intent, and prompt word template into the intent classification model to obtain the first-level target intent of the query request.

[0035] Among them, the first-level target intent belongs to the first-level intent in each first intent example.

[0036] S230: Using the first-level target intent as the retrieval condition, retrieve at least one second intent example from the intent example database according to the query request.

[0037] The second intent examples include query request examples and second-level intents; the second-level intents are the next level intents below the first-level intents.

[0038] S240: Input the query request, examples of each second intent, and prompt word template into the intent classification model to obtain the second-layer target intent of the query request.

[0039] Among them, the second-level target intent belongs to the second-level intent in each second intent example.

[0040] S250: If the second-level target intent satisfies the intent classification requirements of the query request, the second-level target intent shall be taken as the final target intent of the query request.

[0041] When a natural language query request (e.g., "Please check flights from Beijing to Shanghai tomorrow") is received from a user, the initial retrieval process is initiated first. This involves accessing a pre-built database of intent examples stored in a vector database.

[0042] The intent example database contains multiple intent examples, each consisting of two parts: Example of a query request: A typical user expression representing a specific intent.

[0043] Multiple intent labels with a hierarchical structure: These indicate the multi-level intent category to which the query request belongs, including at least a first-level intent (L1) and a second-level intent (L2), where L2 is the next level subclass of L1. For example, in the travel services sector, the original intent "book a flight" might be labeled as: L1: travel services, L2: flight booking.

[0044] Based on the current user query request, using vectorized retrieval or hybrid retrieval techniques, the system quickly locates a set of semantically similar candidate intents in the entire intent example database, and then selects several of the most relevant first intent examples. These first intent examples include their corresponding first-level intents (L1) and their query request examples.

[0045] After obtaining a set of high-quality first intent examples, they are combined with the current user query request and a standardized prompt template to form a structured input, which is then fed into the intent classification model for processing.

[0046] Optionally, the prompt template may include step analysis, a list of optional intents, a reference example, output requirements, example output, and a query request. For example, Table 1: Table 1

[0047] This standardized input method guides the large model to focus on the L1 level classification task, making judgments among a small number of high-level categories to obtain the first-level target intent of the query request.

[0048] After determining the first-level target intent, this is used as the target condition to limit the scope of the next retrieval. Specifically, all records in the intent example database whose first-level intent is the target intent are filtered out, forming a local subset.

[0049] Subsequently, based on the user's original query request, semantic retrieval is performed again in this subset to find the set of second intent examples that best match it. Each second intent example also includes: a query request example; a second-level intent (L2), and the L2 intent belongs to the aforementioned first-level target intent.

[0050] For example, if the first-level target intent is "travel services", then the second intent examples will come from its subclasses: "flight booking", "hotel booking", "ride-hailing services", etc.

[0051] Since the retrieval space has been compressed to a set of sub-intents under a certain high-level intent, the retrieval accuracy can be significantly improved and the introduction of irrelevant interference items can be reduced.

[0052] Next, similar to step S220, the current query request, the newly retrieved second intent example, and the same or adapted prompt word template are combined into a new input Prompt and submitted to the intent classification model again.

[0053] At this point, the model's task shifts to making accurate distinctions within a finer-grained set of intents. Because the number of intents involved in the comparison is significantly reduced, the model can fully consider the diverse examples of each intent, making more accurate and stable judgments. The output is the second-layer target intent, which must belong to the L2 intent set covered by the aforementioned second-intent examples.

[0054] Once it is confirmed that the second-level target intent clearly reflects the user's true intent, it can be used as the final target intent of this query request to trigger subsequent actions (such as calling the flight query interface, redirecting to the rebooking page, etc.).

[0055] The judgment criteria can be flexibly set according to the actual application scenario. For example, if L2 is already the smallest unit of business operation, it can be directly adopted; or, semantic analysis confirms that the second-layer target intent can fully respond to the core operation instructions in the query request.

[0056] This method achieves effective management and efficient recognition of large-scale intent sets by constructing a hierarchical intent structure and a staged recognition mechanism. It avoids the risk of information truncation caused by the context length limitation of large models, and improves classification accuracy and stability by progressively reducing the decision space.

[0057] Furthermore, when the second-level target intent is still insufficient to accurately express the user's intent, it can automatically extend downwards to the third (L3), fourth (L4), or even deeper levels for continuous refinement of the identification, until the classification granularity required for practical applications is met. See also Figure 3 The method may also include: S210: If the second-level target intent does not meet the intent classification requirements of the query request, retrieve at least one Nth intent instance from the intent instance database according to the query request.

[0058] The Nth intent example includes a query request example and the Nth layer intent; the Nth layer intent is the next level intent of the (N-1)th layer intent, where N is an integer greater than or equal to 3.

[0059] S220': Input the query request, examples of each Nth intent, and prompt word template into the intent classification model to obtain the Nth layer target intent of the query request.

[0060] S230: If the target intent at level N satisfies the intent classification requirements of the query request, then the target intent at level N shall be taken as the final target intent of the query request.

[0061] S240: If the target intent at layer N does not meet the intent classification requirements of the query request, continue to retrieve at least one intent example of layer N+1 from the intent example database according to the query request. The intent example of layer N+1 includes the query request example and the intent of layer N+1.

[0062] S250': Input the query request, each N+1th intent example, and the prompt word template into the intent classification model to obtain the N+1th layer target intent of the query request.

[0063] S260`: Until the target intent at layer N+M satisfies the intent classification requirements of the query request, the target intent at layer N+M is taken as the final target intent of the query request, where M is an integer greater than or equal to 1.

[0064] After completing step S240 and obtaining the second-level target intent, it is determined whether the intent has reached the final operational level required by the business. If the classification requirements are not met (e.g., the current intent is still too broad and needs to be further subdivided into subtypes), a recursive identification process towards deeper-level intents is initiated.

[0065] At this point, let N = 3. Using the target intent of the previous layer (i.e., the N-1th layer, L2 in this case) as the filtering condition, and combining it with the current user's query request, retrieve a set of Nth intent examples that semantically match it from the intent example database. Each Nth intent example also contains: a query request example, an Nth-level intent tag, and belongs to the next level subclass of the previous layer intent (LN-1), forming a deep node in the tree-like hierarchical structure. For example, under the L2 intent "flight rescheduling", there may be the following L3 sub-intents: domestic flight rescheduling, international flight rescheduling, and free rescheduling policy consultation.

[0066] The current user query request, the newly retrieved Nth intent example, and the standardized prompt template are organized into a structured Prompt and input into the intent classification model. This prompt template maintains the same format as used in steps S220 and S240. The intent classification model performs inference based on this information and outputs a structured result containing the identified Nth-level target intent. This result must belong to the intent set covered by the input Nth intent example.

[0067] If the target intent of the Nth layer satisfies the intent classification requirements of the query request, then the target intent of the Nth layer will be taken as the final target intent of the query request.

[0068] If the target intent at layer N still cannot fully match the user's needs, the recognition layer will continue to be expanded downwards.

[0069] At this point, let N + 1, using the aforementioned Nth-level target intent as the retrieval condition, and again filter out the N+1th intent examples under it from the intent example database. These examples all belong to more specific areas under the current intent path. For example, under "international flight rebooking", it can be further divided into: first-class international rebooking rules, economy-class international rebooking fees, and rebooking restrictions for special passengers (infants / elderly).

[0070] Repeat step S220' to construct a new input Prompt and submit it to the intent classification model to obtain the recognition result of the N+1th layer.

[0071] The recursive recognition process described above will continue until the target intent at a certain level (denoted as the N+M level) accurately reflects the user's needs. At this point, the system terminates the recognition process and takes the output of that level as the final target intent.

[0072] In the aforementioned S210 step, to improve the relevance and quality of the recall results in the initial retrieval stage and avoid false positives or false negatives due to keyword matching bias or insufficient semantic understanding, this embodiment of the invention employs a hybrid retrieval mechanism combined with a re-ranking model to achieve high-precision candidate intent example screening. See also Figure 4Step S210 may include the following sub-steps: S211: Based on the query request, retrieve a first preset number of first candidate intent examples from the intent example database through sparse retrieval and dense retrieval.

[0073] S212: Sort each first candidate intent example by a sorting optimization model, and take the second preset number of first candidate intent examples in the top of the sorted list as the first intent examples.

[0074] Upon receiving a user's query request, a dual-channel parallel retrieval process is initiated, performing a hybrid retrieval across the entire intent example database. The sparse retrieval method employs traditional information retrieval algorithms based on term frequency statistics, such as BM25 (Best Matching 25), utilizing TF-IDF weighting to calculate the keyword matching degree between the query request and the "query request examples" in each intent example. This method excels at capturing explicit lexical overlap and is suitable for handling precise terms and proper nouns. The dense retrieval method encodes the query request into a semantic vector (e.g., using embedding models such as Sentence-BERT, BGE, and Cohere), and pre-vectorizes and stores all query request examples from all intent examples in a vector database. By calculating cosine similarity or Euclidean distance, the top-K semantically closest candidates are retrieved. This method can identify synonyms, contextual relevance, and implicit intent.

[0075] The two retrieval methods described above operate independently, each returning a certain number of candidate results. Subsequently, the two results are deduplicated and fused (using RRF inverse sorting fusion or other weighting strategies), ultimately obtaining a first preset number (e.g., 100) of first candidate intent example sets.

[0076] The first set of candidate intent examples is sorted using a ranking optimization model. The top 20 examples (a second predetermined number) are then selected as the final first intent examples for L1 level classification. This ranking optimization model can be the bge-reranker series, BCEmbedding, Cohere Rerank, etc.

[0077] After completing the first level of target intent recognition, the second level of intent recognition is entered. To ensure that key information is not lost while narrowing the search scope, step S230 of this embodiment includes a three-level progressive filtering process. See also Figure 5 Step S230 may include the following sub-steps: S231: Based on the first-level target intent, query the intent example database to find the second candidate intent example whose first-level intent is the first-level target intent.

[0078] S232: Based on the query request, retrieve a first preset number of third candidate intent examples from the second candidate intent examples through sparse retrieval and dense retrieval.

[0079] S233: Sort each third candidate intent example by a sorting optimization model, and take the second preset number of candidate intent examples in the top of the sorted list as the second intent examples.

[0080] Using the first-level target intent output from the previous stage as a hard filter, a global filtering operation is performed in the intent example database to extract all records whose "first-level intent" field equals the target intent. This forms a localized second candidate intent example set, limiting the subsequent search space to the current intent branch.

[0081] Based on the already defined second candidate intent example set, a hybrid retrieval process is initiated again, simultaneously performing sparse and dense retrieval to evaluate relevance based on the original query request. This process is similar to step S211, but because the search space has been compressed to a subset under a certain high-level intent, the retrieval speed is faster, the vector comparison is more focused, and the keyword matching is more business-specific. A first preset number (e.g., 50) of highly relevant examples are selected to form the third candidate intent example set.

[0082] Next, consistent with step S212, the same or a dedicated ranking optimization model is invoked to finely score and re-rank the third candidate intent examples. Based on the scoring results, a second preset number (e.g., the top 15) of the top-ranked candidates are selected as the final second intent examples used for L2 level classification.

[0083] To improve the long-term stability and adaptability of the intent classification system, this embodiment of the invention further introduces a dynamic optimization mechanism based on user feedback after the final target intent output is completed. This mechanism continuously collects actual interaction data, identifies error cases, and uses this data for incremental learning or periodic retraining of the model. In one optional implementation, see [link to implementation details]. Figure 6 After step S250, the method further includes: S260: Obtain user feedback information in response to the query request; the user feedback information is used to indicate whether the final target intent is accurate.

[0084] S270: If the user feedback information is inaccurate, then the final target intent of the query request will be used to generate negative training samples.

[0085] S280: Add training samples to the training dataset for incremental training or periodic retraining of the intent classification model.

[0086] Once a second-level (or deeper-level) target intent is identified as the user's final target intent and triggers the corresponding business action, the feedback collection process is initiated. This user feedback can be obtained through various methods, including but not limited to: explicit feedback: user proactively expressing approval or correction, such as clicking the "helpful / not helpful" button, manually selecting the correct intent category, or submitting error correction suggestions; implicit feedback: inferring the accuracy of intent recognition through behavioral analysis, such as: the user quickly returning or repeating the question after accepting the intent redirection; frequently changing keywords and re-initiating requests during the conversation; completing tasks with significantly higher time than average; and an increased probability of being transferred to human customer service.

[0087] The above feedback signals are normalized to generate structured feedback tag data, which includes at least the following fields: the original query request, the final target intent determined, and the user feedback result (such as "accurate", "inaccurate" or confidence level).

[0088] For feedback records marked as "inaccurate", negative training samples are automatically constructed, and then the model optimization process is initiated periodically (e.g., daily, weekly) or as needed (e.g., when a certain number of bad cases have been accumulated).

[0089] To support the efficient operation of the aforementioned hierarchical intent recognition method, this embodiment of the invention also provides an automated method for constructing an intent example database. This method aims to automatically generate a hierarchical intent system through semantic clustering. (See also...) Figure 7 The method also includes a step of constructing an intent example database, which may include the following sub-steps: S310: Obtain multiple intent example samples; the intent example samples include query request examples and L-level intents.

[0090] S320: Input each intent example sample into the semantic vector generation model to generate the intent vector for each intent example sample.

[0091] S330: Perform cluster analysis on each intent vector to obtain the L-1 layer intent of each intent example sample; the L-1 layer intent is the next level intent of the L layer intent.

[0092] S340: Store each intent example sample and its corresponding L-1 layer intent into the intent example database.

[0093] First, a large number of original intent example samples are collected as the basic data source for generating the hierarchical structure. These samples can be obtained through the following means: real question records from historical user dialogue logs; standard expressions manually annotated by business experts; third-party open-source datasets or industry corpora; and simulated typical scenario question-and-answer pairs.

[0094] Each intent example sample contains two core fields: Query request example: a natural language expression representing a specific user need, such as "Help me cancel my flight from Beijing to Shanghai tomorrow"; Level L intent: the intent of the current level to which this expression belongs, which can be a first-level intent, a second-level intent, etc.

[0095] Each query request example is fed into a pre-trained semantic vector generation model, which converts it into a dense vector representation of fixed dimensions, i.e., an intent vector. This semantic vector generation model can be a sentence embedding model based on the Transformer architecture, such as Sentence-BERT.

[0096] After obtaining the semantic vectors of all intent examples, an unsupervised clustering algorithm is performed to automatically group semantically similar intent samples into the same cluster. Optional clustering algorithms include: K-Means, Hierarchical Clustering, DBSCAN, and HDBSCAN.

[0097] During the clustering process, the target number of clusters K can be preset according to business needs, or the optimal number of clusters can be dynamically determined through indicators such as the elbow rule and the silhouette coefficient.

[0098] Each clustering result corresponds to a higher-level abstract category, denoted as the L-1 level intent. For example, the L-1 level intent is: "Book a flight", "Check flights", "Change a ticket", "Refund a ticket", and the clustering generates the L-1 level intent: "Travel services".

[0099] This enables automatic abstraction and induction from fine-grained intents to coarse-grained categories, constructing the first-level parent node of a tree-like intent hierarchy.

[0100] After clustering is completed, the system binds each intent example sample to its newly generated L-1 layer intent label and persists it to the intent example database.

[0101] While semantic vector-based clustering methods can effectively achieve intent abstraction, some samples may be misclassified due to vector calculation errors, semantic ambiguity, or unclear boundaries. Therefore, this invention further introduces a multi-general large-scale model cross-validation mechanism to verify and correct the rationality of the initial clustering results. See also... Figure 8 In an alternative implementation, prior to step S340, the method may further include: S333: Input the preset proportion of intent example samples and their corresponding L-1 layer intents into multiple general large models to determine whether the classification is accurate, and obtain the judgment results given by each general large model.

[0102] Randomly select a certain proportion (e.g., 10%~30%, or even all) of intent example samples from the intent example database, along with their clustered L-1 layer intent samples. Figure 1 The results were then compiled into a Prompt and submitted to several advanced general-purpose language models for independent evaluation. These general-purpose models included, but were not limited to, GPT-4, Claude, Gemini, and DeepSeek.

[0103] S334: For each intent example sample, when the number of inaccurate classification results exceeds a preset number, the intent example sample is marked as an uncertain category intent example sample.

[0104] S335: Calculate the central semantic vector for each L-1 layer intent; the central semantic vector is the average of the intent vectors of all intent example samples corresponding to the L-1 layer intent.

[0105] S336: Calculate the cosine similarity between the intent vector of an uncertain category intent example sample and the semantic vector of each center.

[0106] S337: Select the K central semantic vectors with the highest cosine similarity and use their corresponding L-1 layer intentions as the first candidate intention categories.

[0107] S338: Input the uncertain category intent example samples and each first candidate intent category into multiple general large models to select the intent category, and obtain at least one second candidate intent category.

[0108] S339: The second candidate intent category with the largest number of entries is used as the L-1 layer intent of the uncertain category intent example sample.

[0109] For each intent example sample, count the number of models judged as "inaccurate classification". If this number exceeds a preset threshold (e.g., ≥ 3), the sample is considered to have significant classification controversy and is marked as an "uncertain category intent example sample".

[0110] For each confirmed valid L-1 layer intent category, calculate its central semantic vector. For each sample marked as "uncertain," calculate the cosine similarity between its intent vector and the central semantic vectors of all known L-1 layer intents. Select the top K (e.g., K=3) L-1 intents with the highest similarity as candidate categories, called the first candidate intent category. Construct a new Prompt to guide multiple large models to vote within the limited options. For example: Which category is the following request more likely to belong to? A. Travel services; B. Customer service; C. Ticketing management?; Request: "What should I do if my flight is delayed?"

[0111] Each model returns its recommended category, resulting in at least one second candidate intent category. The selection results of each model are statistically analyzed, and the category with the most votes is selected as the final L-1 layer intent for that sample, using a majority voting principle. In case of a tie, the sample can be submitted for manual review or the original clustering results can be retained with a warning label added.

[0112] To further enhance the generalization ability of the intent recognition model to diverse expressions, participants will... Figure 9 Before step S340, the method may further include: S337: For each intent example sample, use a large language model to generate multiple expanded query request examples that maintain semantic consistency but have different expressions, in order to expand the intent example sample.

[0113] S338: Perform data cleaning on each extended query request example to obtain the cleaned query request example.

[0114] S339: Assemble each cleaned query request example with its corresponding L-th layer intent and L-1-th layer intent into a new intent example sample.

[0115] Iterate through the existing intent example samples, and for each original "query request example", call the large language model to perform a semantically preserving rewriting task. For example, the prompt is: "Please generate 5 sentences with different expressions, which have the same semantics as the following sentences, but the words, word order and style can be changed: 'I want to cancel tomorrow's flight'".

[0116] Model outputs might include: "Can you help me cancel my flight tomorrow morning?", "I don't plan to take that flight tomorrow, how do I cancel it?", "I booked the wrong time, can I cancel my flight now?", "I'm not going on my flight tomorrow, please process my refund.", "Something came up and I need to cancel my booked flight."

[0117] This type of generation effectively covers various forms of expression, including colloquial, written, interrogative, and declarative sentences, greatly enriching the diversity of data.

[0118] To ensure the quality of generated content, the data cleaning process can include: deduplication: removing variants with highly repetitive semantics or only minor variations; semantic consistency verification: using vector similarity or re-calling the large model to determine if there is any deviation from the original meaning. High-quality augmented samples are retained after cleaning. Each cleaned augmented query request is re-bound to its original intent tags to form a new intent example sample, which is then added to the intent example database.

[0119] Through this process, the originally sparse intent categories can obtain sufficient representative samples, significantly alleviating the long-tail problem.

[0120] In summary, the intent classification method and computer program product based on a large model provided by this invention effectively solves the problems of excessively long prompts, information truncation, and decreased classification accuracy caused by the context window limitation of large language models in large-scale intent scenarios by constructing a hierarchical intent example database and combining it with a hierarchical progressive retrieval and classification mechanism. Through hybrid retrieval (sparse + dense) and re-ranking techniques, the method efficiently retrieves first-level candidate intents that semantically match the query request from the full intent database, achieving coarse-grained intent localization. Subsequently, based on the first-level target intent, the retrieval scope is progressively narrowed, and lower-level sub-intent examples are dynamically loaded for fine-grained classification. This significantly reduces the number of candidate intents in a single inference and improves classification accuracy and stability without exceeding the model's context capacity.

[0121] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0123] If the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0125] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. An intent classification method based on a large model, characterized in that, include: In response to a query request, at least one first intent example matching the query request is retrieved from a pre-built intent example database; The intent example database includes multiple intent examples, which include query request examples and multiple intents with a hierarchical structure. The first intent example includes the query request example and the first-level intent; Input the query request, each first intent example, and the prompt word template into the intent classification model to obtain the first layer of target intent of the query request; The first layer target intent belongs to the first layer intent in each first intent example; Using the first-layer target intent as a retrieval condition, at least one second intent example is retrieved from the intent example database according to the query request; the second intent example includes the query request example and the second-layer intent; The second layer of intent is the next level of intent below the first layer of intent; The query request, each second intent example, and the prompt word template are input into the intent classification model to obtain the second-layer target intent of the query request. The second-level target intent belongs to the second-level intent in each of the second intent examples; If the second-layer target intent satisfies the intent classification requirement of the query request, the second-layer target intent shall be regarded as the final target intent of the query request.

2. The method according to claim 1, characterized in that, The method further includes: If the second-layer target intent does not meet the intent classification requirements of the query request, at least one Nth intent example is retrieved from the intent example database according to the query request. The Nth intent example includes the query request example and the Nth-layer intent. The Nth-layer intent belongs to the next level intent of the (N-1)th-layer intent, where N is an integer greater than or equal to 3. The query request, each of the Nth intent examples, and the prompt word template are input into the intent classification model to obtain the Nth layer target intent of the query request. If the target intent at the Nth layer satisfies the intent classification requirement of the query request, then the target intent at the Nth layer shall be taken as the final target intent of the query request. If the target intent of the Nth layer does not meet the intent classification requirements of the query request, at least one intent example of the N+1th layer is retrieved from the intent example database according to the query request. The intent example of the N+1th layer includes the query request example and the intent of the N+1th layer. The query request, each of the N+1th intent examples, and the prompt word template are input into the intent classification model to obtain the N+1th layer target intent of the query request. Until the target intent at layer N+M satisfies the intent classification requirement of the query request, the target intent at layer N+M is taken as the final target intent of the query request, where M is an integer greater than or equal to 1.

3. The method according to claim 1, characterized in that, The step of retrieving at least one first intent example matching the query request from a pre-built intent example database in response to the query request includes: Based on the query request, a first preset number of first candidate intent examples are retrieved from the intent example database through sparse retrieval and dense retrieval. The first candidate intent examples are sorted by a sorting optimization model, and the first candidate intent examples in the top two columns are taken as the first intent examples.

4. The method according to claim 1, characterized in that, The step of using the first-layer target intent as a retrieval condition and retrieving at least one second intent example from the intent example database according to the query request includes: Based on the first-layer target intent, a second candidate intent example that is the first-layer target intent is retrieved from the intent example database; Based on the query request, a first preset number of third candidate intent examples are retrieved from the second candidate intent examples through sparse retrieval and dense retrieval. The third candidate intent examples are sorted using a sorting optimization model, and the top-ranked second preset number of candidate intent examples are taken as the second intent examples.

5. The method according to claim 1, characterized in that, The prompt template includes step analysis, a list of optional intents, a reference example, output requirements, example output, and a query request.

6. The method according to claim 1, characterized in that, After the step of taking the second-level target intent as the final target intent of the query request, where the second-level target intent satisfies the intent classification requirement of the query request, the method further includes: Obtain user feedback information in response to the query request; the user feedback information is used to indicate whether the final target intent is accurate. If the user feedback information is inaccurate, then the final target intent of the query request will be used to generate negative training samples. The training samples are added to the training dataset for incremental training or periodic retraining of the intent classification model.

7. The method according to claim 1, characterized in that, The method further includes the step of constructing the intent example database, which includes: Obtain multiple intent example samples; the intent example samples include query request examples and Lth layer intents; Each of the aforementioned intent example samples is input into the semantic vector generation model to generate intent vectors for each of the aforementioned intent example samples; Cluster analysis is performed on each of the intent vectors to obtain the (L-1)th layer intent of each intent example sample; the (L-1)th layer intent is the next level intent of the Lth layer intent; Each intent example sample and its corresponding L-1 layer intent are stored in the intent example database.

8. The method according to claim 7, characterized in that, Before the step of storing each intent example sample and its corresponding L-1 layer intent into the intent example database, the method further includes: The sample intents of a preset proportion and their corresponding L-1 layer intents are input into multiple general large models to determine whether the classification is accurate, and the determination results given by each of the general large models are obtained. For each intent example sample, when the number of cases where the determination result indicates inaccurate classification is greater than a preset number, the intent example sample is marked as an uncertain category intent example sample; Calculate the central semantic vector for each of the (L-1)th layer intentions; the central semantic vector is the average of the intention vectors of all the intention example samples corresponding to the (L-1)th layer intention. Calculate the cosine similarity between the intent vector of the uncertain category intent example sample and each of the central semantic vectors; Select the K central semantic vectors with the highest cosine similarity and use their corresponding (L-1)th layer intentions as the first candidate intention categories; The uncertain category intent example sample and each of the first candidate intent categories are input into the multiple general large models to select the intent category, thereby obtaining at least one second candidate intent category; The second candidate intent category with the largest number of entries is taken as the L-1 layer intent of the uncertain category intent example sample.

9. The method according to claim 7, characterized in that, Before the step of storing each intent example sample and its corresponding L-1 layer intent into the intent example database, the method further includes: For each of the intent example samples, multiple expanded query request examples that maintain semantic consistency but have different expressions are generated using a large language model to expand the intent example samples; Data cleaning is performed on each of the extended query request examples to obtain cleaned query request examples; Each of the cleaned query request examples, along with its corresponding Lth-level intent and L-1th-level intent, is assembled into a new intent example sample.

10. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 9.

Images