Intention recognition method and system for semantic similarity judgment based on large model
The method leverages large models for semantic similarity judgment to optimize intent recognition by converting intent classification into a similarity task, addressing the limitations of traditional models in handling complex queries and dynamic intent scenarios, resulting in improved efficiency and robustness.
Patent Information
- Application Number
- CN202411691833.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-03-28
Smart Images
Figure CN119721043A_ABST
Abstract
Description
Background Art
[0002] There are two main technical solutions at present:
[0003] One is to rely on keyword matching and rule engines to identify user intent. This method works well when dealing with simple and clear queries, but it is often difficult to accurately capture the user's true needs when facing complex and multi-intent queries.
[0004] Another approach is to use pre-trained language models such as BERT. Although this approach is effective in processing complex queries, it usually requires re-preparing incremental training data and retraining the model when dealing with incremental intents, which consumes a lot of manpower and resources.
[0005] Intent recognition tasks play an important role in insurance sales, customer service and other scenarios. Generally, they face two problems:
[0006] 1. When there are many intent categories to be identified, the text classification effect is not good based on traditional models such as Textcnn and Bert (due to limited model capabilities);
[0007] 2. The number and categories of intents in real scenarios may increase or decrease dynamically, especially when OOD situations occur, which traditional text classification models cannot handle. Summary of the invention
[0008] The present application provides an intent recognition method and system for semantic similarity judgment based on a large model, which is used to utilize the powerful semantic understanding ability of the large model, design prompts to fine-tune the model instructions, and make the large model output whether the query text is similar to the candidate text, thereby converting the intent classification task into a semantic similarity judgment task.
[0009] In a first aspect, a method for intention recognition based on a large model for semantic similarity judgment is provided, comprising the following steps:
[0010] Build a large similarity model;
[0011] The large similarity model is used to perform intent recognition.
[0012] In the above technical scheme, a large similarity model is constructed; the large similarity model is used to perform intent recognition; the powerful semantic understanding ability of the large model is used to design prompts to fine-tune the model instructions, so that the large model outputs whether the query text is similar to the candidate text, and the intent classification task is converted into a semantic similarity judgment task, thereby improving the efficiency and effect of intent recognition.
[0013] In a specific implementation scheme, the steps of constructing a large similarity model specifically include:
[0014] Build an intent knowledge base;
[0015] Based on the intention knowledge base, the similarity large model is trained.
[0016] In a specific implementation scheme, the steps of constructing the intention knowledge base are as follows:
[0017] The intent categories are defined and the intent knowledge base is constructed.
[0018] In a specific implementation manner, the intent categories include standard questions, similar questions, and intent names.
[0019] In a specific implementation scheme, the step of using the large similarity model to perform intent recognition specifically includes:
[0020] Obtain query text and candidate text;
[0021] The similarity model is used to determine whether the query text is similar to the candidate text, and an intent recognition result is obtained.
[0022] In a specific implementation scheme, the step of obtaining the candidate text is specifically as follows:
[0023] The candidate text is obtained through multi-way recall.
[0024] In a specific embodiment, it also includes:
[0025] The intent recognition result is optimized.
[0026] In a specific implementation scheme, the step of optimizing the intention recognition result is specifically:
[0027] For all results where the model outputs the “yes” token, the mean probability of the model generating the “yes” token is calculated by category, and the category with the highest mean probability is taken as the final intent recognition result.
[0028] In a specific implementation scheme, the multi-way recall includes BM25 recall and semantic vector recall.
[0029] Secondly, an intention recognition system based on a large model for semantic similarity judgment is provided, including:
[0030] Large model module, used to build a large similarity model;
[0031] The intention recognition module is used to perform intention recognition using the large similarity model.
[0032] In the above technical scheme, a large similarity model is constructed; the large similarity model is used to perform intent recognition; the powerful semantic understanding ability of the large model is used to design prompts to fine-tune the model instructions, so that the large model outputs whether the query text is similar to the candidate text, and the intent classification task is converted into a semantic similarity judgment task, thereby improving the efficiency and effect of intent recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A flowchart of an intention recognition method for determining semantic similarity based on a large model provided in an embodiment of the present application;
[0034] Figure 2 A structural block diagram of an intention recognition system for making semantic similarity judgment based on a large model provided in an embodiment of the present application;
[0035] Figure 3 A schematic diagram of an intent knowledge base provided in an embodiment of the present application;
[0036] Figure 4 A schematic diagram of candidate texts provided in an embodiment of the present application;
[0037] Figure 5 A specific flow chart of the intent recognition method for semantic similarity judgment based on a large model provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.
[0039] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.
[0040] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.
[0041] To facilitate understanding of the intention recognition method and system for semantic similarity judgment based on a large model provided in the embodiment of the present application, its application scenario is first explained. The intention recognition method and system for semantic similarity judgment based on a large model provided in the embodiment of the present application are used to utilize the powerful semantic understanding ability of the large model, design prompts to fine-tune the model instructions, and make the large model output whether the query text is similar to the candidate text, and convert the intention classification task into a semantic similarity judgment task. There are two main technical solutions at present: one is to rely on keyword matching and rule engines to identify user intentions. This method works better when dealing with simple and clear queries, but when facing complex and multi-intent queries, it is often difficult to accurately capture the user's real needs. The other is to use pre-trained language models such as BERT. Although this method is more effective in dealing with complex queries, when dealing with incremental intents, it is usually necessary to re-prepare incremental training data and retrain the model, which consumes a lot of manpower and resources. The intent recognition task plays an important role in scenarios such as insurance sales and customer service. Generally, it faces two problems: 1. When there are many intent categories that need to be recognized, the text classification effect is not good based on traditional models such as Textcnn and Bert (due to limited model capabilities); 2. The number and categories of intents in real scenarios may increase or decrease dynamically, especially when OOD situations occur, traditional text classification models cannot handle it. For this reason, the embodiment of the present application provides an intent recognition method and system based on a large model for semantic similarity judgment, so as to utilize the powerful semantic understanding ability of the large model, design prompts to fine-tune the model, and make the large model output whether the query text is similar to the candidate text, and convert the intent classification task into a semantic similarity judgment task. The following is a detailed description of the embodiment in conjunction with specific drawings.
[0042] refer to Figures 1 to 5 , Figure 1 A flowchart of an intention recognition method for determining semantic similarity based on a large model provided in an embodiment of the present application; Figure 2 A structural block diagram of an intention recognition system for making semantic similarity judgment based on a large model provided in an embodiment of the present application; Figure 3 A schematic diagram of an intent knowledge base provided in an embodiment of the present application; Figure 4 A schematic diagram of candidate texts provided in an embodiment of the present application; Figure 5 A specific flow chart of the intent recognition method for semantic similarity judgment based on a large model provided in an embodiment of the present application.
[0043] exist Figure 1 In the embodiment of the present application, a method for identifying intentions based on a large model for semantic similarity judgment is provided, comprising the following steps:
[0044] Build a large similarity model;
[0045] The large similarity model is used to perform intent recognition.
[0046] In the above technical scheme, a large similarity model is constructed; the large similarity model is used to perform intent recognition; the powerful semantic understanding ability of the large model is used to design prompts to fine-tune the model instructions, so that the large model outputs whether the query text is similar to the candidate text, and the intent classification task is converted into a semantic similarity judgment task, thereby improving the efficiency and effect of intent recognition.
[0047] In a specific implementation scheme, the steps of constructing a large similarity model specifically include:
[0048] Build an intent knowledge base;
[0049] Based on the intention knowledge base, the similarity large model is trained.
[0050] In a specific implementation scheme, the steps of constructing the intention knowledge base are as follows:
[0051] The intent categories are defined and the intent knowledge base is constructed.
[0052] In a specific implementation manner, the intent categories include standard questions, similar questions, and intent names.
[0053] In a specific implementation scheme, the step of using the large similarity model to perform intent recognition specifically includes:
[0054] Obtain query text and candidate text;
[0055] The similarity model is used to determine whether the query text is similar to the candidate text, and an intent recognition result is obtained.
[0056] In a specific implementation scheme, the step of obtaining the candidate text is specifically as follows:
[0057] The candidate text is obtained through multi-way recall.
[0058] In a specific embodiment, it also includes:
[0059] The intent recognition result is optimized.
[0060] In a specific implementation scheme, the step of optimizing the intention recognition result is specifically:
[0061] For all results where the model outputs the “yes” token, the mean probability of the model generating the “yes” token is calculated by category, and the category with the highest mean probability is taken as the final intent recognition result.
[0062] In a specific implementation scheme, the multi-way recall includes BM25 recall and semantic vector recall.
[0063] Specifically, refer to Figures 3 to 5 The intention recognition method for judging semantic similarity based on a large model includes:
[0064] 1. Prepare the intent knowledge base and train the similarity model
[0065] (1) Define intent categories and prepare a knowledge base, including standard questions, similar questions, and intent names, such as Figure 3 shown.
[0066] (2) Prepare training data pairs and construct prompt words, with two situations in total, for example: (positive example, positive example) and (positive example, negative example).
[0067] (3) Based on the training data pairs and the large model, train the similarity model.
[0068] 2. Intent Prediction
[0069] (1) Pre-processing: word segmentation, query segmentation, synonym replacement
[0070] (2) k candidate texts obtained through multi-way recall such as BM25 recall and semantic vector recall, such as Figure 4 shown.
[0071] (3) Using the PointWise approach (i.e., 1v1), the large similarity model is used to batch determine whether the query text is similar to the candidate text, and output a "yes / no" token and the corresponding probability.
[0072] (4) In terms of post-processing of model recognition results, if the traditional PointWise method is used, that is, the intent corresponding to the candidate text that is most similar to the query text (i.e., the probability of the "yes" token is the highest) is taken as the final intent recognition result, the results will be independent of each other, lacking a comprehensive judgment of all results, and the results are easily dominated by a large number of categories in the knowledge base, with large variance and poor robustness. Therefore, the method adopted in this application is to calculate the probability mean of the model generating the "yes" token by category for all results where the model outputs the "yes" token, and take the category with the highest probability mean as the final intent recognition result, thereby improving the stability and robustness of intent recognition.
[0073] In the above technical solution, after obtaining k candidate texts through multi-way recall such as BM25 and semantic vector, the pointwise method is used to batch judge whether the query text is similar to the candidate text through the large similarity model, and output "yes / no" and its probability. In order to improve the stability and robustness of intent recognition, all the results of the model outputting "yes" are counted by category and the category with the highest probability mean is taken as the final recognition result to avoid the problems caused by independent results; using the powerful semantic understanding ability of the large model, the prompt is designed to fine-tune the model, so that the large model outputs whether the query text is similar to the candidate text, and the intent classification task is converted into a semantic similarity judgment task, which improves the efficiency and recognition effect of intent recognition.
[0074] exist Figure 2 In the embodiment of the present application, a system for identifying intentions based on a large model for determining semantic similarity is provided, including:
[0075] Large model module, used to build a large similarity model;
[0076] The intention recognition module is used to perform intention recognition using the large similarity model.
[0077] In the above technical scheme, a large similarity model is constructed; the large similarity model is used to perform intent recognition; the powerful semantic understanding ability of the large model is used to design prompts to fine-tune the model instructions, so that the large model outputs whether the query text is similar to the candidate text, and the intent classification task is converted into a semantic similarity judgment task, thereby improving the efficiency and effect of intent recognition.
[0078] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.
[0079] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.
[0080] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0081] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.
Claims
1. A method for intention recognition based on a large model for semantic similarity judgment, characterized in that: The following steps are involved: Build a large similarity model; The large similarity model is used to perform intent recognition.
2. The method for intention recognition based on a large model for semantic similarity judgment according to claim 1 is characterized in that: The steps to build a large similarity model include: Build an intent knowledge base; Based on the intention knowledge base, the similarity large model is trained.
3. The method for intention recognition based on semantic similarity judgment based on a large model according to claim 2 is characterized in that: The steps to build an intent knowledge base are as follows: The intent categories are defined and the intent knowledge base is constructed.
4. The method for intention recognition based on semantic similarity judgment based on a large model according to claim 3 is characterized in that: The intent categories include standard questions, similar questions, and intent names.
5. The method for intention recognition based on semantic similarity judgment based on a large model according to claim 4 is characterized in that: The steps of using the large similarity model to perform intent recognition specifically include: Obtain query text and candidate text; The similarity model is used to determine whether the query text is similar to the candidate text, and an intent recognition result is obtained.
6. The method for intention recognition based on a large model for semantic similarity judgment according to claim 5 is characterized in that: The steps of obtaining the candidate text are specifically as follows: The candidate text is obtained through multi-way recall.
7. The method for intention recognition based on a large model for semantic similarity judgment according to claim 6 is characterized in that: Also includes: The intent recognition result is optimized.
8. The method for intention recognition based on a large model for semantic similarity judgment according to claim 7 is characterized in that: The steps of optimizing the intention recognition result are specifically as follows: For all results where the model outputs the "yes" token, the mean probability of the model generating the "yes" token is calculated by category, and the category with the highest mean probability is taken as the final intent recognition result.
9. The method for intention recognition based on a large model for semantic similarity judgment according to claim 8, characterized in that: The multi-way recall includes BM25 recall and semantic vector recall.
10. An intention recognition system based on a large model for semantic similarity judgment, characterized in that: include: Large model module, used to build a large similarity model; The intention recognition module is used to perform intention recognition using the large similarity model.