Semantic routing implementation method based on natural language intention recognition

Abstract

The invention discloses a semantic routing implementation method based on natural language intention recognition. The semantic routing implementation method comprises the following steps: 1, intention collection; 2, constructing an intention recognition classifier, and recognizing a preset intention; 3, constructing a routing component, and timely responding to the identified user intention; and 4, system integration: integrating the intention recognition classifier, the routing component and the RAG system. According to the method, the process of natural language intention recognition is simplified, more efficient and more intelligent semantic routing is generated for an RAG system, the query intention of a user can be understood more accurately, and more accurate and efficient services are provided.

Description

Technical Field

[0001] The present invention relates to the field of computer natural language information processing and artificial intelligence technology, and in particular to a semantic routing implementation method based on natural language intent recognition. Background Art

[0002] With the continuous advancement of RAG (Retrieval-Augmented Generation) technology and its wide application in the engineering field, we gradually realize that RAG has certain limitations when dealing with certain complex problems, and it cannot meet the query needs of all users. In practical applications, RAG is more suitable to be executed as a sub-process. This is because we usually need to first identify the intent of the user's query, and then guide the query to the corresponding sub-process for processing based on this intent.

[0003] Semantic routing is an intelligent query distribution technology that can intelligently route query requests to appropriate processing components or data sources based on the semantic content of natural language input, thereby improving the relevance and efficiency of responses.

[0004] Traditional routing systems mostly rely on preset rules or simple logical judgments (such as if / else statements) to distribute requests to the corresponding parts based on keyword or pattern input. However, in the face of complex natural language queries, this traditional approach may seem inadequate because it cannot fully understand the subtle differences in context and semantics.

[0005] Therefore, how to simplify the process through natural language intent recognition and build smarter and more accurate semantic routing to efficiently understand user queries and provide better services has become a technical problem that needs to be solved urgently in this field. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a semantic routing implementation method based on natural language intent recognition, which aims to simplify the process of natural language intent recognition, generate more efficient and intelligent semantic routing for the RAG system, and more accurately understand user query intentions and provide more accurate and efficient services.

[0007] To achieve the above object, the present invention discloses a method for implementing semantic routing based on natural language intention recognition, comprising the following steps:

[0008] Step 1: Collection of intentions;

[0009] Step 2: construct an intent recognition classifier to recognize the preset intent;

[0010] Step 3: Build a routing component to respond promptly to the identified user intent;

[0011] Step 4: System integration: Integrate the intent recognition classifier, routing components, and RAG system.

[0012] Preferably, in step 1, the intentions to be identified are determined and collected based on the specific needs of the business scenario, system performance and response requirements.

[0013] More preferably, step 2 is as follows:

[0014] Step 2.1, data collection: specifically, collecting data for training and testing the intent recognition classifier;

[0015] Step 2.2, data processing; specifically: cleaning the collected data, removing noise and irrelevant information, and marking the processed data to determine the category of the intent to which each piece of data belongs;

[0016] Step 2.3, model selection: specifically: according to specific needs, select appropriate machine learning, deep learning or large language model to identify the category of the intent;

[0017] Step 2.4, model training; specifically: using the labeled data to train the intent recognition classifier, if in step 2.3, a large language model is selected, the large language model is trained for intent recognition using a fine-tuning method;

[0018] Step 2.5, model evaluation; specifically: using the test set to evaluate the performance of the trained intent recognition classifier;

[0019] Step 2.6: Integration and deployment; specifically: publish the evaluated intent recognition classifier as a service.

[0020] More preferably, in step 2.2, when labeling the data, each of the intentions is numbered, and then the number of each of the intentions is assigned to the corresponding data.

[0021] More preferably, in step 2.2, a large language model may be used to perform data augmentation on categories with a small sample size.

[0022] More preferably, in step 2.3, the model specifically includes Naive Bayes, Support Vector Machine, and BERT model.

[0023] More preferably, in step 2.5, the performance of the trained intent recognition classifier is evaluated using an accuracy index, a recall index or an F1 score index.

[0024] More preferably, step 3 is as follows:

[0025] Step 3.1, determine the component function; specifically: according to the preset type of the intent and the response the user expects, design the function of the routing component that can meet the user's needs to ensure that each intent can get a correct response;

[0026] Step 3.2, determine the component strategy; specifically: determine the appropriate strategy according to the function of the routing component to complete the response function;

[0027] Step 3.3: Define routing interfaces and publish component services. Specifically, define different routing interfaces based on the recognition results of the intent recognition classifier on user queries to ensure that users are directed to the correct components, perform corresponding operations, and return the correct responses.

[0028] More preferably, in step 3.2, a correct strategy is adopted to implement an exception handling mechanism to ensure that the system can accurately handle errors and exceptions for unclear or unrecognizable user intentions.

[0029] More preferably, in step 3.2, multiple strategies may be used in series or in parallel to complete a more complex functional design to respond to the identified user intent.

[0030] Beneficial effects of the present invention:

[0031] The semantic routing constructed by the present invention based on the method and process of natural language intent recognition significantly improves the retrieval accuracy of the RAG system and the relevance of the generated content. Through the precise classification of input intent, user queries can be accurately assigned to the most appropriate processing component or data source, greatly improving the efficiency and response accuracy of the RAG system. In addition, the present invention optimizes content generation, reduces redundant calculations, and further improves the overall efficiency of the system.

[0032] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The intent recognition process according to an embodiment of the present invention is shown.

[0034] Figure 2 The following is a diagram showing a process for constructing an intent recognition classifier in one embodiment of the present invention.

[0035] Figure 3 The following is a diagram showing a process of constructing a routing component in an embodiment of the present invention.

[0036] Figure 4 A schematic diagram showing the integration of a map recognition module and a RAG system in one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0037] The present invention involves a question-answering system based on RAG technology and a large language model, which are explained as follows:

[0038] 1. The question-answering system based on RAG technology, referred to as the RAG system, is a natural language processing technology framework that combines retrieval and generation. It first retrieves information related to the input question from a predefined knowledge base, and then inputs this information as additional context into the generative model to generate answers, thereby improving the accuracy of the answers, ensuring data privacy and security, and enhancing explainability.

[0039] 2. Large Language Model (LLM), abbreviated as LLM, is an artificial intelligence model that is pre-trained based on massive text data and uses complex architectures such as Transformer. It can learn language grammar, semantics and other features, thus having multiple capabilities, such as generating various types of text (such as articles, poems, dialogues, etc.), answering questions, and performing text translation.

[0040] Example

[0041] like Figures 1 to 4 As shown, the semantic routing implementation method based on natural language intent recognition includes the following steps:

[0042] Step 1: Collection of intentions;

[0043] Step 2: Build an intent recognition classifier to identify the preset intent;

[0044] Step 3: Build a routing component to respond promptly to the identified user intent;

[0045] Step 4: System integration: Integrate the intent recognition classifier, routing components, and RAG system.

[0046] The present invention constructs an efficient semantic routing for the RAG system, which can accurately distribute user queries to the most appropriate processing components or data sources. This mechanism significantly improves the efficiency of the RAG system and the accuracy of the response.

[0047] Finally, in step 4, after integration with the RAG system, RAG becomes a sub-process, and the RAG process is called according to the user's usage intention.

[0048] In some embodiments, in step 1, the intents that need to be identified are determined and collected based on the specific needs of the business scenario, system performance, and response requirements.

[0049] For example: asking about weather conditions, checking company personnel, checking company news, etc.

[0050] In some embodiments, step 2 is as follows:

[0051] Step 2.1: Data collection: Specifically, collect data for training and testing the intent recognition classifier;

[0052] Step 2.2: Data processing: Specifically, clean the collected data, remove noise and irrelevant information, and annotate the processed data to determine the category of intent to which each piece of data belongs;

[0053] In some embodiments, in order to facilitate model training, each intent can be numbered when data is annotated, and an intent number can be assigned to each piece of data. By assigning the same number to the corresponding intent and the corresponding data, it is ensured that the data accurately matches the corresponding intent.

[0054] In some embodiments, a large language model may be used to perform data augmentation on categories with a small sample size. The imbalance in sample size may be measured using the category imbalance ratio (Imbalance Ratio), and the formula is:

[0055] Imbalance ratio = number of samples in the most sample categories / number of samples in the least sample categories

[0056] For example, if the largest class has 500 samples and the smallest class has 10 samples, the imbalance ratio is 50:1, which means the data is very unbalanced.

[0057] Step 2.3: Model selection: Specifically, select the appropriate machine learning, deep learning, or large language model to identify the intent category according to specific needs;

[0058] In some embodiments, the optional models specifically include Naive Bayes, Support Vector Machine, BERT model (a deep learning model) or a large language model.

[0059] Step 2.4: Model training: Specifically, use the labeled data to train the intent recognition classifier;

[0060] The training sample format is as follows:

[0061] Category 1: Technology News

[0062] Sample 1: “Apple released a new iPhone with greatly improved performance.”

[0063] Sample 2: "Artificial intelligence has made significant breakthroughs in the medical field, assisting doctors in diagnosing diseases."

[0064] Sample 3: "5G networks are becoming increasingly popular, bringing new opportunities for smart transportation."

[0065] Category 2: Sports News

[0066] Sample 1: "Messi scored a hat trick in the game and led the team to victory."

[0067] Sample 2: "At the Olympics, Chinese athletes won gold medals and made history."

[0068] Sample 3: "The NBA finals were fiercely contested, with two strong teams fighting desperately for the championship."

[0069] Category 3: Entertainment News

[0070] Sample 1: "A certain star's new movie is a big hit at the box office, and his acting skills are well received by the audience."

[0071] Sample 2: "The ratings of the popular variety show hit a new high, and the interaction between the guests sparked heated discussions."

[0072] Sample 3: "A pop singer released a new album, and the title song quickly topped the music charts."

[0073] If a large language model is selected in step 2.3, the large language model is trained for intent recognition using the fine-tuning method, and the fine-tuned large language model is used in subsequent steps for intent recognition classification. The format of the training sample is as follows:

[0074] ]

[0076] Step 2.5, model evaluation; specifically: use the test set to evaluate the performance of the trained intent recognition classifier. This embodiment uses the accuracy index for evaluation. The formula for calculating the accuracy index is as follows:

[0077] Accuracy index = number of samples predicted correctly / total number of samples;

[0078] In the formula, the number of samples predicted correctly is the sum of true positive examples and true negative examples, and the total number of samples is the number of all samples.

[0079] In some embodiments, the performance of the trained intent recognition classifier may also be evaluated using a recall metric or an F1 score metric.

[0080] Step 2.6, integration and deployment: Specifically, publish the evaluated intent recognition classifier as a service. It is necessary to ensure that the intent recognition classifier can receive and process user queries. This embodiment is deployed based on the trained BERT model, performs intent recognition based on user questions, and returns recognition results.

[0081] In some embodiments, a fine-tuned large language model is deployed, and intent recognition classification can be performed, and a prompt template method can be used to guide the large model to perform intent recognition and return recognition results. An example of a prompt template is given below:

[0082] "The user's expression is: My phone's screen suddenly went black. Please determine which of the following is the user's main intention: Intent A: To understand the possible reasons for the phone's black screen; Intent B: To find a way to restore the phone to normal; Intent C: To find out whether it needs to be repaired and how much the repair cost is."

[0083] In some embodiments, step 3 is as follows:

[0084] Step 3.1: Determine component functions. Specifically, design routing component functions that meet user needs based on the types of preset intents and the responses that users expect, ensuring that each intent can receive a correct response.

[0085] Step 3.2: Determine the component strategy; specifically: determine the appropriate strategy based on the function of the routing component to complete the response function;

[0086] According to the function of the routing component, determining the appropriate strategy includes multiple strategies, as shown below:

[0087] Strategy 1: Applicable to the retrieval of structured data. This strategy is designed for processing structured data and aims to achieve fast retrieval of data through efficient indexing and query mechanisms. It is applicable to database and table data and can ensure the accuracy and consistency of data.

[0088] Strategy 2: API interface of third-party system, we can use the data of third-party system, such as search engine API, to expand our information acquisition capabilities.

[0089] Strategy 3: Used to store and retrieve embedding vectors, suitable for fast similarity searches. These vectors are often used to represent text, images, or other types of data.

[0090] Step 3.3: Define routing interfaces and publish component services. Specifically, the user query first enters the intent recognition classifier. Based on the recognition results, different routing interfaces are defined to ensure that the user is directed to the correct component, the corresponding operation is performed, and the correct response is returned.

[0091] In certain embodiments, correct strategies are adopted to implement an exception handling mechanism to ensure that the system can accurately handle errors and exceptions when the user's intention is unclear or unrecognizable.

[0092] In some embodiments, multiple strategies may be used in series or in parallel in step 3.2 to complete a more complex functional design to cope with the identified user intent;

[0093] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A semantic routing implementation method based on natural language intent recognition; characterized in that: The steps include: Step 1: Collection of intentions; Step 2: construct an intent recognition classifier to recognize the preset intent; Step 3: Build a routing component to respond promptly to the identified user intent; Step 4: System integration: Integrate the intent recognition classifier, routing components, and RAG system.

2. The method for implementing semantic routing based on natural language intent recognition according to claim 1, characterized in that: In step 1, the intents that need to be identified are determined and collected based on the specific needs of the business scenario, system performance, and response requirements.

3. The method for implementing semantic routing based on natural language intent recognition according to claim 1, characterized in that: Step 2 is as follows: Step 2.1, data collection: specifically, collecting data for training and testing the intent recognition classifier; Step 2.2, data processing; specifically: cleaning the collected data, removing noise and irrelevant information, and marking the processed data to determine the category of the intent to which each piece of data belongs; Step 2.3, model selection: specifically: according to specific needs, select appropriate machine learning, deep learning or large language model to identify the category of the intent; Step 2.4, model training; specifically: using the labeled data to train the intent recognition classifier; Step 2.5, model evaluation; specifically: using the test set to evaluate the performance of the trained intent recognition classifier; Step 2.6: Integration and deployment; specifically: publish the evaluated intent recognition classifier as a service.

4. The method for implementing semantic routing based on natural language intent recognition according to claim 3 is characterized in that: In step 2.2, when labeling the data, each of the intentions is numbered, and then the number of each of the intentions is assigned to the corresponding data.

5. The method for implementing semantic routing based on natural language intent recognition according to claim 3 is characterized in that: In step 2.3, a large language model is used to perform data augmentation for categories with fewer samples.

6. The method for implementing semantic routing based on natural language intent recognition according to claim 3 is characterized in that: In step 2.3, the model specifically includes Naive Bayes, Support Vector Machine or BERT large language model.

7. The method for implementing semantic routing based on natural language intent recognition according to claim 3, characterized in that: In step 2.4, a fine-tuning method can be used to train the large language model for intent recognition.

8. The method for implementing semantic routing based on natural language intent recognition according to claim 3, characterized in that: In step 2.5, the performance of the trained intent recognition classifier is evaluated using an accuracy index, a recall index, or an F1 score index.

9. The method for implementing semantic routing based on natural language intent recognition according to claim 1, characterized in that: Step 3 is as follows: Step 3.1, determine the component function; specifically: according to the preset type of the intent and the response the user expects, design the function of the routing component that can meet the user's needs to ensure that each intent can get a correct response; Step 3.2: Determine component strategy; Specifically: according to the function of the routing component, determine the appropriate strategy to complete the response function; Step 3.3: Define routing interfaces and publish component services. Specifically, define different routing interfaces based on the recognition results of the intent recognition classifier on user queries to ensure that users are directed to the correct components, perform corresponding operations, and return the correct responses.

10. The method for implementing semantic routing based on natural language intent recognition according to claim 9, characterized in that: In step 3.2, the correct strategy should be adopted to implement the exception handling mechanism to ensure that the system can accurately handle errors and exceptions when the user intention is unclear or unrecognizable.

11. The method for implementing semantic routing based on natural language intent recognition according to claim 9, characterized in that: In step 3.2, multiple strategies can be used in series or in parallel to complete a more complex function design to respond to the identified user intent.

Images