Intention recognition method and device, electronic equipment, storage medium and computer program product

Abstract

The invention relates to the technical field of computers, and provides an intention recognition method and device, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: determining to-be-recognized information based on input information and historical information of a user corresponding to the input information; inputting the to-be-identified information into a preset small model to obtain a first intention identification result output by the preset small model; if the maximum score of each piece of business knowledge in the first intention recognition result is smaller than a preset threshold value of the corresponding business knowledge, inputting the to-be-recognized information and each piece of business knowledge into a preset large model to obtain a second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than that of the preset small model; the preset threshold value of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result comprises a matching result of the business knowledge and an intention type; and determining output information based on the second intention recognition result. The intention recognition efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an intent recognition method, device, electronic device, storage medium and computer program product. Background Art

[0002] Currently, the capabilities and accuracy of intelligent customer service systems can be improved through technologies such as intelligent customer service systems generated based on large language models and knowledge bases, intent recognition methods based on large models, and conversation intent recognition methods combined with large models.

[0003] However, the above technologies are insufficient in their ability to understand contextual intent, especially for complex and diverse user questions. It is difficult to quickly, comprehensively and accurately identify and answer intent, resulting in low efficiency in current intent identification. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes an intent recognition method, device, electronic device, storage medium and computer program product to solve the problem that it is currently difficult to quickly, comprehensively and accurately recognize and answer intent, and to improve the efficiency of intent recognition.

[0005] According to the first aspect of the present application, the intention recognition method includes: Determining information to be identified based on the input information and historical information of the user corresponding to the input information; Input the information to be identified into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; If the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and each business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result of the business knowledge and the intention type; Determine output information based on the second intention recognition result.

[0006] According to an embodiment of the present application, the preset small model is used to perform the following operations based on the information to be identified: Querying a preset number of business knowledge matching the information to be identified from a preset knowledge base; Determine the similarity between the preset number of business knowledge and the information to be identified as the score of the corresponding business knowledge; Determine at least one matching piece of business knowledge from the preset number of pieces of business knowledge; Determine a preset threshold value according to the information length of the target business knowledge; the target business knowledge is the business knowledge with the largest score among all the matched business knowledge; The matching business knowledge and single intent type information whose scores are greater than or equal to the corresponding preset threshold are taken as the first intent recognition result; or, if the score of the target business knowledge in the matching business knowledge is less than the corresponding preset threshold, each matching business knowledge is taken as the first intent recognition result.

[0007] According to one embodiment of the present application, the preset large model is obtained by the following operations: Obtain sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information; the intent type includes single intent, multiple intents, and rejected intent; Pre-training is performed based on the sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information to obtain the preset large model.

[0008] According to an embodiment of the present application, determining the information to be identified based on the input information and the historical information of the user corresponding to the input information includes: If the input information does not contain the preset words, and the historical information of the user corresponding to the input information is empty, the input information is determined as information to be identified; If the input information does not contain preset words, and the historical information of the user corresponding to the input information is not empty, a rewrite prompt word is determined according to the historical information and the input information; the rewrite prompt word is used to indicate that the input information is to be rewritten according to the historical information; information is rewritten according to the historical information, the rewrite prompt word and the input information to obtain information to be identified; the information rewriting includes information omission, information reference, information compression and no modification.

[0009] According to an embodiment of the present application, rewriting the information according to the historical information, the rewriting prompt word and the input information to obtain the information to be identified includes: splicing the historical information, the rewriting prompt word and the input information to obtain spliced ​​information; Input the splicing information into the preset large model to obtain the rewriting information output by the preset large model; The rewritten information is determined as information to be identified.

[0010] According to an embodiment of the present application, before determining the information to be identified based on the input information and the historical information of the user corresponding to the input information, the method further includes: If the input information includes at least one preset word, the output information is determined based on each of the preset words.

[0011] According to the second aspect of the present application, the intention recognition device includes: A first determination module, configured to determine information to be identified based on input information and historical information of a user corresponding to the input information; A first recognition module, used for inputting the information to be recognized into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; A second recognition module, for inputting the information to be recognized and each of the business knowledge into a preset large model to obtain a second intention recognition result output by the preset large model if the maximum score of each business knowledge in the first intention recognition result is less than a preset threshold value of the corresponding business knowledge; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold value of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes a matching result of the business knowledge and an intention type; The second determination module is used to determine output information based on the second intention recognition result.

[0012] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intention recognition method described in any one of the above methods is implemented.

[0013] According to the storage medium of the fourth aspect embodiment of the present application, the storage medium is a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the intention recognition methods described above.

[0014] According to the computer program product of the fifth aspect of the present application, the computer program includes a computer program, which, when executed by a processor, implements any one of the above-mentioned intention recognition methods.

[0015] The above one or more technical solutions in the embodiments of the present application have at least the following technical effects: The information to be identified is determined by combining the input information with the historical information of the user corresponding to the input information, and then the intent is identified by collaboratively using the preset small model and the preset large model based on the information to be identified. The dynamic threshold is used for fusion in the intent identification process to make judgments. Therefore, the contextual intent can be effectively understood and more reliable and rapid intent and intent type identification capabilities can be provided. In the face of complex and diverse user questions, intent identification can be performed quickly, comprehensively and accurately, and output information can be determined, thereby improving the efficiency of intent recognition.

[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the intent recognition method provided in an embodiment of the present application.

[0019] Figure 2 It is a functional image diagram of the threshold function in the intent recognition method provided in the embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of the overall process of the intent recognition method provided in the embodiment of the present application.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0022] The following is a further detailed description of the implementation of the present application in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but cannot be used to limit the scope of the present application.

[0023] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0024] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0025] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0026] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0027] The present application proposes an intent recognition method, device, electronic device, storage medium and computer program product.

[0028] Figure 1 is a flow chart of the intent recognition method provided in the embodiment of the present application, such as Figure 1 As shown, the intention recognition method includes: Step 110: determine the information to be identified based on the input information and the historical information of the user corresponding to the input information.

[0029] Step 120, input the information to be identified into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge.

[0030] Step 130, if the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and the business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result and intent type of the business knowledge.

[0031] Step 140: determine output information based on the second intention recognition result.

[0032] It should be noted that the intention recognition method provided in the embodiment of the present application can be applied to various scenarios where intelligent question and answer can be performed, such as in intelligent customer service systems, and more specifically in intelligent customer service robots. The execution subject of the intention recognition method of the present application can be a server, a computer device or a robot, such as a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA). It should be noted that the data involving user privacy that needs to be obtained in this application are all obtained through formal channels after authorization by the relevant users.

[0033] An intention recognition device may be provided or connected to the server or computer device of the present application, thereby controlling the intention recognition device to execute the intention recognition method of the present application.

[0034] In this application, when a user needs to interact with the intelligent customer service system, he can log in to the intelligent customer service system and input information in the designated input area of ​​the customer service system, such as inputting questions that need to be answered. The input information can be in the form of text, voice, etc. Thus, this application can obtain the user's input information.

[0035] In one embodiment, the application can obtain the input information of the current user, and use the mobile phone number of the current user as the unique identifier of the user, recorded as user_id, and use user_id as the key value.

[0036] At the same time, you can query whether the user has historical information, where the historical information may include information asked by the user and answered by the system. If there is historical information, it will be used as a value and stored in the Redis middleware.

[0037] Redis middleware is a Redis-based application or service that aims to provide functions such as caching, message queuing, data storage, and real-time data processing.

[0038] Redis is an open source in-memory data storage system. Historical information can include users' past queries and conversations. This historical information helps provide users with multiple rounds of answers.

[0039] The user's historical information can be recorded by using roles, which are divided into two types: user_role and system_role. The former records the user's original input, and the latter records the system's answer. After reading the historical information, this application intercepts the interaction content of the last three rounds as the actual historical information.

[0040] After the current round of interaction is completed, the current user's input information and the corresponding system answer are updated to the historical information, and the updated historical information is rewritten into Redis according to the user_id.

[0041] If the current user has no conversation for a long time, delete the key value corresponding to the current user_id from Redis to ensure timely cleanup and release of data. The session time can be set to 10 minutes, for example.

[0042] Furthermore, if the input information does not contain preset words, and the historical information of the user corresponding to the input information is empty (for example, a new user or a user who has not used the service for a long time), the input information is determined as information to be identified. The preset words of the present application may include preset sensitive words and preset keywords.

[0043] If the input information does not contain the preset words, and the historical information of the user corresponding to the input information is not empty, the historical information and the input information can be used to rewrite the information to provide appropriate user input as the information to be identified.

[0044] Furthermore, the intent recognition can be performed based on the information to be recognized by using a preset small model (hereinafter referred to as the small model), so that the intent recognition can be performed quickly, and the intent recognition result output by the preset small model is obtained and defined as the first intent recognition result. It should be noted that the first intent recognition result of the present application at least includes the matching result of the business knowledge.

[0045] If the maximum score of each business knowledge matched by the matching result of the business knowledge in the first intent recognition result is greater than or equal to a preset threshold determined according to the information length of the business knowledge, it is determined that the accuracy of the business knowledge is high enough and can be directly used to answer the user's question. At this time, the first intent recognition result may include the business knowledge and the single intent type information.

[0046] Furthermore, information search can be performed based on the first intention recognition result and a single answer to the input information can be generated.

[0047] However, if the maximum score of each business knowledge matched by the matching result of the business knowledge in the first intention recognition result is less than the preset threshold determined according to the information length of the corresponding business knowledge, it is determined that the accuracy of the business knowledge is insufficient and cannot be directly used to answer the user's question. It is necessary to use a preset large model (hereinafter referred to as the large model) to perform intent recognition based on the information to be recognized and the various business knowledge in the first intent recognition result. Since the preset large model has stronger understanding and generation capabilities and can handle complex intent recognition tasks, it can provide more reliable and rapid intent and intent type recognition capabilities. Therefore, the final intent recognition result including the matching results of the business knowledge and the intent type can be output as the second intent recognition result.

[0048] It should be noted that the preset large model and the preset small model in this application are two models with different model parameter magnitudes, and the model parameter magnitude of the preset small model is smaller than the model parameter magnitude of the preset large model. For example: the preset large model has billions or even hundreds of billions of model parameters, while the model parameters of the preset small model are only millions or even less. Moreover, the preset small model can only provide a single service capability, while the preset large model can provide multiple service capabilities. For example, the preset small model can only be used for intent recognition, while the preset large model can perform information rewriting in addition to intent recognition.

[0049] Furthermore, a response to the input information may be generated based on the second intention recognition result.

[0050] Specifically, if it is determined that the user's input information contains only one clear intent (ie, a single intent), a corresponding single answer is generated.

[0051] If multiple intents (i.e., multi-intent) are detected, a response containing multiple answers is generated to meet the different needs of the user.

[0052] If the user's input cannot be understood or is considered to involve sensitive content (i.e., rejection intent), a rejection answer is generated to reject the user's request.

[0053] It should be noted that the process of how to return corresponding output information according to the user's specific intention is not specifically limited in this application, and traditional technology or new technology can be used.

[0054] Finally, the generated answer is returned to the user, completing an interactive process.

[0055] According to the intent recognition method of the embodiment of the present application, the information to be recognized is determined by combining the input information with the historical information of the user corresponding to the input information, and then the intent recognition is performed collaboratively through the preset small model and the preset large model based on the information to be recognized, and the judgment is integrated according to the dynamic threshold in the intent recognition process. Therefore, the contextual intent can be effectively understood and a more reliable and rapid intent and intent type recognition capability can be provided, so that when facing complex and diverse user problems, the intent recognition can be performed quickly, comprehensively and accurately and the output information can be determined, thereby improving the efficiency of intent recognition.

[0056] Based on the above embodiment, determining the information to be identified based on the input information and the historical information of the user corresponding to the input information includes: If the input information does not contain the preset words, and the historical information of the user corresponding to the input information is empty, the input information is determined as information to be identified; If the input information does not contain preset words, and the historical information of the user corresponding to the input information is not empty, a rewrite prompt word is determined based on the historical information and the input information; the rewrite prompt word is used to indicate that the input information is to be rewritten based on the historical information; information is rewritten based on the historical information, the rewrite prompt word and the input information to obtain information to be identified; information rewriting includes information omission, information reference, information compression and no modification.

[0057] Specifically, if it is determined that the input information does not contain a preset word, and the historical information of the user corresponding to the input information is empty, that is, there is no historical data, the input information is determined as information to be identified.

[0058] If it is determined that the input information does not contain preset vocabulary, and the historical information of the user corresponding to the input information is not empty, that is, there is historical data, the rewriting prompt word can be determined based on the historical information and the input information, and then the information can be rewritten by omitting, referring, compressing and not modifying (specifically, the original sentence remains unchanged) based on the historical information, the rewriting prompt word and the input information, and the information to be identified is obtained after the rewriting is completed.

[0059] Among them, information compression specifically refers to the compression of long and difficult sentences.

[0060] The present application does not specifically limit the process of determining the rewriting prompt word according to the historical information and the input information. The purpose is to obtain the prompt word that can indicate how to rewrite the input information according to the historical information.

[0061] Furthermore, the rewriting prompt word can be configured in the support system of the preset large model, and a unique identifier prompt_id can be generated for the rewriting prompt word. After obtaining the identifier, the corresponding rewriting prompt word can be obtained according to prompt_id.

[0062] When the user has historical information, in addition to general reference and omission issues, this application compresses and rewrites long and difficult sentences specifically for complex and diverse user input issues, thereby being able to handle user input in various situations, providing a flexible rewriting method to adapt to different contextual requirements. And rewriting is performed based on the characteristics of business knowledge to ensure that the system can accurately understand and process different types of user input.

[0063] Further, information is rewritten according to the historical information, the rewriting prompt word and the input information to obtain information to be identified, including: splicing the historical information, the rewriting prompt word and the input information to obtain spliced ​​information; Input the splicing information into the preset large model to obtain the rewritten information output by the preset large model; The rewritten information is determined as information to be identified.

[0064] Specifically, the present application can perform splicing in the order of "historical information + rewriting prompt word + input information" to ensure the integrity and logic of the input information and obtain spliced ​​information.

[0065] Furthermore, the stitching information can be input into a fine-tuned preset macro model.

[0066] The preset large model can be a model that is fine-tuned on a pre-trained large language model (LLMs) (such as Qwen, etc.), and is fine-tuned to make it more suitable for specific business scenarios.

[0067] After receiving the input information, the preset large model uses its powerful natural language processing capabilities to semantically understand and analyze the spliced ​​information.

[0068] Among them, Qwen is an artificial intelligence model based on a large language model.

[0069] The preset large model deeply understands the user's intention based on the input splicing information. The preset large model not only simply responds to the user's current input, but also combines the context history and rewrite prompts to generate more accurate and user-friendly answers.

[0070] In general, the preset large model can flexibly use techniques such as omission, reference, and compression of long and difficult sentences by understanding contextual information and semantic relationships, and keep the original sentence unchanged when necessary, so as to effectively rewrite the sentence according to specific needs and make it more suitable for matching with the knowledge in the business knowledge base.

[0071] Specifically, for information omission, we can identify non-critical information in user input, delete these parts, and retain the core content that can accurately convey the user's intention. This method can help simplify sentences, reduce the interference of irrelevant information, and improve the efficiency of information transmission.

[0072] For example, the original sentence: Is it correct to deduct the fee? I just paid my phone bill a few days ago, how come I owe more than 50 yuan again?

[0073] Rewritten: Why do I have an outstanding payment for my phone bill after just paying it?

[0074] By omitting “Is the deduction correct?” and simplifying “I just paid the phone bill a few days ago” to “I just paid the phone bill,” we retain key information and make the sentence more concise.

[0075] For information reference, during the rewriting process, identify the detailed description part of the sentence and replace it with concise reference words or summary phrases to simplify the expression while retaining the main information.

[0076] This approach can reduce lengthy narratives and make the language more refined.

[0077] For example, the original sentence: How should I handle the change of the location of my mobile phone number from one city to another?

[0078] After rewriting: How to handle the change of ownership of a mobile phone number?

[0079] Replacing "the location changed from one city to another" with "the ownership changed" simplifies the sentence by using the reference relationship but keeps the meaning intact.

[0080] For long and difficult sentence compression, the core information and key points in the long sentence are identified, and the complex syntactic structure and redundant information are compressed into concise expressions. This method helps reduce the cognitive burden of users when understanding sentences and improves the speed and accuracy of information transmission.

[0081] For example, original sentence: Hello, can you help me check? I just called you with the secondary card and he said I opened a five-dollar card. What is it?

[0082] After rewriting: Query the opened businesses.

[0083] By compressing a long sentence containing multiple details, "I just called you with that secondary card and he said I opened a five-dollar something or other," it can be simplified to "query the opened services," retaining the core meaning and eliminating redundant information.

[0084] For the original sentence that remains unchanged, when analyzing user input, if the original sentence already has clear and accurate expression without redundant or ambiguous parts, choose not to rewrite it.

[0085] Preserving the original sentence helps avoid unnecessary modifications and keeps the user's intent intact.

[0086] For example, the original sentence: How to apply for the National Family Network?

[0087] After rewriting: How to apply for the National Family Network?

[0088] In some cases, you may choose to keep the original sentence unchanged, especially if the original sentence is clear enough to express the meaning.

[0089] Furthermore, the rewritten information may be determined as information to be identified.

[0090] When the user has historical information, in addition to general reference and omission issues, this application compresses and rewrites long and difficult sentences specifically for complex and diverse user input issues, thereby being able to handle user input in various situations, providing a flexible rewriting method to adapt to different contextual requirements. And rewriting is performed based on the characteristics of business knowledge to ensure that the system can accurately understand and process different types of user input.

[0091] Furthermore, before determining the information to be identified based on the input information and the historical information of the user corresponding to the input information, the method further includes: If the input information includes at least one preset word, the output information is determined based on each of the preset words.

[0092] This application can perform preliminary screening of user input information through a sensitive word module and a keyword matching module to ensure the security and legality of the system output content.

[0093] Specifically, if it is determined through the sensitive word module that the user's input information includes a preset sensitive word, prompt information that prompts the user that an answer cannot be returned due to the presence of the sensitive word is directly returned as output information.

[0094] If it is determined through the sensitive word module that the user's input information does not include a preset sensitive word, keyword matching is further performed through the keyword matching module.

[0095] If it is determined through the keyword matching module that the user's input information includes preset keywords, a corresponding answer can be set for each preset keyword in the present application, so the answer matching the preset keyword can be directly returned as output information.

[0096] This application can filter out words that may involve sensitive content through sensitive word matching to ensure the security and legality of the output content. Through keyword matching, the core information in the input information can be extracted for rapid processing.

[0097] Based on the above embodiment, the preset small model is used to perform the following operations based on the information to be identified: Querying a preset number of business knowledge matching the information to be identified from a preset knowledge base; Determine the similarity between a preset number of business knowledge and the information to be identified as the score of the corresponding business knowledge; Determine at least one matching piece of business knowledge from a preset number of pieces of business knowledge; The preset threshold is determined according to the information length of the target business knowledge; the target business knowledge is the business knowledge with the largest score among all the matching business knowledge; The matching business knowledge and single intent type information whose scores are greater than or equal to the corresponding preset threshold are taken as the first intent recognition result; or, if the score of the target business knowledge in the matching business knowledge is less than the corresponding preset threshold, each matching business knowledge is taken as the first intent recognition result.

[0098] In this application, business knowledge can be designed in advance.

[0099] In order to better coordinate the processing of large and small models, each piece of data in the business knowledge contains 10 fields and is stored in a distributed search engine (ElasticSearch, ES).

[0100] In the Frequently Asked Questions (FAQ) system, similar questions are questions asked by users during actual use, which are similar but not exactly the same as standard questions. They are extensions of standard questions.

[0101] Specifically, the detailed description of each field is shown in Table 1 below: .

[0102] For example, suppose a standard question in a FAQ system is "How do I reset my password?".

[0103] Users may ask similar questions such as "I forgot my password, how can I retrieve it?", "What are the steps to reset my password?", "I cannot log in, what should I do if I need to reset my password?". The FAQ system will identify these similar questions and determine whether they match based on the scores of the user input and similar questions, and match them with the standard question "How do I reset my password?". The matching rule is to set a fixed threshold. When the score is greater than the threshold, the match is successful, otherwise it is not a match, and the corresponding answer is output through the corresponding standard question.

[0104] After the information to be identified is input into the preset small model, the preset small model obtains the business knowledge most similar to the current user from the knowledge base. Specifically, it can include the following five steps: 1. ES candidate knowledge recall: The first step is to use ES to query and retrieve knowledge similar to the current user input. In this process, the system will search the pre-established knowledge base index based on the user's input keywords or sentences. Using ES inverted index technology, documents containing related words can be quickly located and potential matching knowledge pairs can be efficiently recalled. After a large number of experimental comparisons, in order to balance the accuracy and speed of recall, the number of candidates to be recalled is set to 200.

[0105] 2. Calculate similarity scores using text embedding models: The second step uses the bge-large-zh-v1.5 text embedding model to calculate the similarity between texts. Among the 200 candidate results obtained by ES recall, the model will embed each candidate knowledge with the current user input to generate a high-dimensional vector representation. Then, the degree of similarity is determined by calculating the cosine similarity between the user input and the candidate knowledge vector. The main purpose is to refine the results of the recall to obtain more accurate knowledge. bge-large-zh-v1.5 is a text embedding model designed specifically for Chinese text processing. It is mainly used to convert text into a fixed-dimensional vector representation, which can be widely used in a variety of natural language processing tasks.

[0106] 3. Sort knowledge by score from high to low: In the third step, after calculating the similarity score between the user input and the candidate knowledge, the system will sort them according to the similarity value. Generally, a higher similarity value indicates a higher degree of similarity between the two texts, so the similarity will be sorted from high to low.

[0107] 4. Deduplication based on postId: In the fourth step, there may be duplicate knowledge pairs in the sorted results. These duplicate contents may be caused by different expressions of standard questions. According to the postId deduplication steps in Table 1, remove these duplicate knowledge to ensure that users see unique and valuable knowledge. PostId deduplication refers to removing duplicate IDs (Identification) from a set of data containing multiple postIds to ensure that each postId appears only once in the final dataset.

[0108] 5. Extract the top k most relevant business knowledge: The last step is to extract the top k most relevant knowledge from the sorted list after deduplication. k is a pre-set parameter, usually with a value of 3, 5, 10, etc. In this solution, k=5 is taken according to the needs of the specific application scenario. Only the most relevant information is displayed to avoid information overload and ensure that users can quickly obtain the most useful answers.

[0109] Furthermore, the threshold can be calculated based on the sentence length of business knowledge, and a dynamic algorithm judgment strategy can be made based on the threshold: Based on a large number of online user logs, it is found that the fixed threshold strategy has great defects. For example, for two sentences with the same semantics, the sentence length characteristics are very different. The score of the shorter sentence is higher, and the score of the longer sentence is lower. Corresponding to the fixed threshold, the match with the high score is successful, and the match with the low score is unsuccessful, resulting in low accuracy of intent recognition.

[0110] Dynamically adjust the threshold of similarity matching based on the characteristics of the input query, such as the number of characters, to improve the accuracy and flexibility of matching. Specifically, the dynamic threshold setting adjusts the strictness of the match based on the length of the user's question: for longer sentences, they contain more information and are more semantically clear. Because the information is rich, even if the similarity score is slightly lower, its meaning can be understood more accurately, so a lower threshold is set; for shorter sentences, there is less information and it is more likely to be ambiguous, so a higher similarity score is required to ensure the accuracy of the match, so a higher threshold is set.

[0111] The threshold is adjusted dynamically according to the sentence length, and a large number of logs are collected and manually annotated to form a data set T.

[0112] Where T={( ),( ),……,( )},

[0113] Among them, length represents the length of the input, and score represents the score that best matches the input with similar questions in the knowledge base.

[0114] y represents the label, , , .

[0115] After the input is confirmed, the length and score have been calculated and determined, but whether the label y is equal to -1 or 1 needs to be determined based on the threshold at this time. If score is greater than or equal to the threshold, y is equal to 1, otherwise it is equal to -1. The following describes how to calculate the current threshold size based on the length feature of the sentence.

[0116] First, design a function: ,

[0117] Where sign is the sign function, that is: ,

[0118] in, is the Sigmoid function, that is: ,

[0119] Where w and b are the parameters of the function, which are used to measure the relationship between the sentence length and the current threshold.

[0120] The second step is to design a loss function Loss(L) and find the parameters (w, b) in the above function. The algorithm is as follows: for a given data set T, minimize the loss value of the loss function on the given data set T, and convert it into a solution to the following loss function L minimization problem: ,

[0121] Among them, min represents the minimization operation, L represents the defined loss function, Represents the value of the input label y, f(w,b) is equal to the sign function.

[0122] The third step is to minimize L under the given data set T, and obtain w=-0.02, b=2.5. That is, the final threshold function is obtained: ,

[0123] Represents the relationship between the length of the input sentence and the threshold.

[0124] Figure 2 A schematic diagram of a function image of a threshold function in the intent recognition method provided in an embodiment of the present application, Figure 2 The x-axis is the length of business knowledge, which may include 0, 10, 20, 30, 40, 50, etc., and the y-axis is the preset threshold of business knowledge, which may include 0.82, 0.84, 0.86, 0.88, 0.90, 0.92, etc. Figure 2The function graph is shown when the length of the sentence on the horizontal axis is in the range of [0, 50]. The graph shows the change of the function within this range. From the graph, we can see that the threshold value corresponding to the function value gradually decreases as the length of the sentence increases.

[0125] Therefore, by inputting the information to be recognized into the preset small model, the first intention recognition result output by the preset small model can be obtained.

[0126] Specifically, the preset small model selects the most similar similar question, and determines whether the first score is greater than or equal to the length When the threshold is calculated, the corresponding answer is returned directly, otherwise it enters the large model intent recognition.

[0127] If the knowledge is greater than the threshold, the first intention recalled by the preset small model is directly returned. Otherwise, the input is understood based on the user's candidate knowledge through the preset large model.

[0128] For inputs below the dynamic threshold, the understanding ability of the preset large model is used to reselect one, multiple, or zero most relevant standard questions from a maximum of five similar questions recalled, corresponding to single intent, multiple intent, and rejected intent.

[0129] This application utilizes a dynamic strategy of collaboration between large and small models to improve processing efficiency and response speed. Specifically, when the value is higher than the dynamic threshold, the result of the preset small model is directly returned, which greatly improves processing efficiency and response speed.

[0130] It can also improve accuracy, specifically: when it is below the dynamic threshold, the understanding ability of the preset large model is used to identify single intent, multiple intents and rejected intent to ensure the accuracy of the output.

[0131] The dynamic threshold processing strategy of the present application enhances the flexibility and adaptability of the system and can select the optimal strategy for processing according to specific circumstances.

[0132] Based on the above embodiment, the preset large model is obtained by the following operations: Obtain sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information; the intent type includes single intent, multiple intents, and rejection intent; Pre-training is performed based on sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information to obtain a preset large model.

[0133] In order to realize the recognition function of single intent, multiple intents and rejected intent, the present application can collect a large amount of user input data as sample data, and the data format includes three sections: input information, candidate intents and labels of intent types. Specifically, sample input information, one or more sample business knowledge corresponding to the sample input information, and labels of intent types corresponding to the sample input information can be obtained as sample data.

[0134] The data should cover a wide range of intents and be categorized as single intent, multiple intents, and input that does not fall into any known intent.

[0135] Furthermore, the pre-trained large language model can be fine-tuned. It should be noted that the present application can also set intent recognition prompt words in the pre-trained large language model to assist the large language model in learning how to process input data. The intent prompt words are used to prompt the large language model's processing strategy for input data and can give processing examples. For example, a possible intent recognition prompt word is: based on the input information, select the most matching one, multiple or zero business knowledge from the given multiple business knowledge and determine the intent type.

[0136] This application explicitly introduces the option of "other-rejection" category in the design of intent recognition prompt words. This category is used to select when the model cannot match any known candidate intent.

[0137] Optimize the design of intent recognition prompt words, specifically placing the most similar intent at the end of the entire intent recognition prompt word, so that it can better guide the large language model to recognize single intent, multiple intents, and rejected intents.

[0138] Further, single intent, multiple intent, and rejection will be identified Figure 3 The categories are trained as three independent subtasks together with the main single-intent task.

[0139] Through multi-task learning, the large language model can learn these tasks simultaneously, so that it can perform better when faced with different types of user input. The resulting preset large model can not only accurately classify known intents, but also effectively identify unknown intents and multiple intents.

[0140] When performing fine-tuning training, the present application can utilize the collected and enhanced sample data to perform fine-tuning training on the pre-trained large language model.

[0141] During the training process, the multi-task learning method is used to enable the large language model to learn multiple tasks simultaneously within a unified framework, thereby improving the model's ability to recognize different types of intents.

[0142] Continuously iterate and optimize the performance of the model to ensure its accuracy and robustness in practical applications.

[0143] Through the above steps, the preset large model obtained can better respond to diverse user inputs, provide more accurate intent recognition capabilities, and improve user experience and satisfaction.

[0144] Therefore, by inputting the information to be identified and various business knowledge into the preset large model, the second intention recognition result output by the preset large model can be obtained.

[0145] This application uses the "Other-Rejected" category processing mechanism to handle both in-domain and out-of-domain issues, avoid misclassification, and ensure that the system can handle various types of input. In addition, through the "Other-Rejected" category processing mechanism, the reliability and accuracy of the system output are improved to ensure that it can handle input that cannot be classified.

[0146] Figure 3 The overall flow diagram of the intent recognition method provided in the embodiment of the present application is as follows: Figure 3 As shown, after obtaining user input, the present application can perform sensitive word matching and keyword matching on the input information. If the input information does not contain sensitive words and keywords, the historical information of the current user is obtained. If it is determined that the historical information of the current user is empty, based on the input information, the knowledge pair most similar to the current user (i.e., the business knowledge mentioned above) is obtained from the knowledge base through a small model, and further dynamic intent recognition is performed according to the threshold. If it is determined that the historical information of the current user is not empty, the large model is rewritten based on the user's contextual historical information, and based on the rewritten data, the knowledge pair most similar to the current user is obtained from the knowledge base through a small model, and further dynamic intent recognition is performed according to the threshold.

[0147] Furthermore, if it is determined that the score of the business knowledge obtained by intent recognition is greater than or equal to a threshold, the business knowledge is output as the user's single intent, and an answer is returned based on the user's single intent; if it is determined that the score of the business knowledge obtained by intent recognition is less than a threshold, intent recognition is further performed through a large model to obtain the output of a single intent, multiple intents or rejected intent, and an answer is further returned based on the output.

[0148] The intention recognition device provided in the present application is described below. The intention recognition device described below and the intention recognition method described above can be referenced to each other.

[0149] Furthermore, the present application also provides an intent recognition device.

[0150] The intention recognition device comprises: A first determination module, configured to determine information to be identified based on input information and historical information of a user corresponding to the input information; A first recognition module, used for inputting the information to be recognized into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; A second recognition module, for inputting the information to be recognized and each of the business knowledge into a preset large model to obtain a second intention recognition result output by the preset large model if the maximum score of each business knowledge in the first intention recognition result is less than a preset threshold value of the corresponding business knowledge; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold value of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes a matching result of the business knowledge and an intention type; The second determination module is used to determine output information based on the second intention recognition result.

[0151] The intention recognition device of the present application determines the information to be recognized by combining the input information with the historical information of the user corresponding to the input information, and then performs intent recognition in coordination with the preset small model and the preset large model based on the information to be recognized, and integrates the judgment based on the dynamic threshold in the intention recognition process. Therefore, it can effectively understand the contextual intention and provide more reliable and rapid intent and intention type recognition capabilities, so that when facing complex and diverse user problems, it can also quickly, comprehensively and accurately recognize the intent and determine the output information, thereby improving the efficiency of intent recognition.

[0152] In one embodiment, the first determining module is specifically configured to: If the input information does not contain the preset words, and the historical information of the user corresponding to the input information is empty, the input information is determined as information to be identified; If the input information does not contain preset words, and the historical information of the user corresponding to the input information is not empty, a rewrite prompt word is determined according to the historical information and the input information; the rewrite prompt word is used to indicate that the input information is to be rewritten according to the historical information; information is rewritten according to the historical information, the rewrite prompt word and the input information to obtain information to be identified; the information rewriting includes information omission, information reference, information compression and no modification.

[0153] In one embodiment, the first determining module includes a rewriting unit, and the rewriting unit is used to: splicing the historical information, the rewriting prompt word and the input information to obtain spliced ​​information; Input the splicing information into the preset large model to obtain the rewriting information output by the preset large model; The rewritten information is determined as information to be identified.

[0154] In one embodiment, the first determining module includes a preprocessing unit, and the preprocessing unit is used to: If the input information includes at least one preset word, the output information is determined based on each of the preset words.

[0155] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the following method: determine the information to be identified based on the input information and the historical information of the user corresponding to the input information; Input the information to be identified into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; If the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and each business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result of the business knowledge and the intention type; Determine output information based on the second intention recognition result.

[0156] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0157] In another aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the methods provided in the above embodiments, for example, including: determining information to be identified based on input information and historical information of a user corresponding to the input information; Input the information to be identified into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; If the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and each business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result of the business knowledge and the intention type; Determine output information based on the second intention recognition result.

[0158] In another aspect, an embodiment of the present application further provides a computer program product having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the methods provided in the above embodiments, for example, including: determining information to be identified based on input information and historical information of a user corresponding to the input information; Input the information to be identified into a preset small model to obtain a first intention recognition result output by the preset small model; the first intention recognition result at least includes a matching result of business knowledge; If the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and each business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result of the business knowledge and the intention type; Determine output information based on the second intention recognition result.

[0159] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0160] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0161] Finally, it should be noted that the above implementation modes are only used to illustrate the present application, rather than to limit the present application. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that various combinations, modifications or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for intention recognition, characterized in that: include: Determining information to be identified based on the input information and historical information of the user corresponding to the input information; Inputting the information to be recognized into a preset small model to obtain a first intention recognition result output by the preset small model; The first intention recognition result at least includes a matching result of business knowledge; If the maximum score of each business knowledge in the first intention recognition result is less than the preset threshold of the corresponding business knowledge, the information to be recognized and each business knowledge are input into the preset large model to obtain the second intention recognition result output by the preset large model; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold of the business knowledge is determined according to the information length of the corresponding business knowledge; the second intention recognition result includes the matching result of the business knowledge and the intention type; Determine output information based on the second intention recognition result.

2. The method for intention recognition according to claim 1, characterized in that: The preset small model is used to perform the following operations based on the information to be identified: Querying a preset number of business knowledge matching the information to be identified from a preset knowledge base; Determine the similarity between the preset number of business knowledge and the information to be identified as the score of the corresponding business knowledge; Determine at least one matching piece of business knowledge from the preset number of pieces of business knowledge; Determine a preset threshold value based on the information length of the target business knowledge; The target business knowledge is the business knowledge with the largest score among all the matching business knowledge; The matching business knowledge and single intent type information with a score greater than or equal to a corresponding preset threshold are taken as the first intent recognition result; Alternatively, if the score of the target business knowledge in the matched business knowledge is less than the corresponding preset threshold, each matched business knowledge is used as the first intention recognition result.

3. The method for identifying intention according to claim 1, characterized in that: The preset large model is obtained by the following operations: Obtain sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information; the intent type includes single intent, multiple intents, and rejected intent; Pre-training is performed based on the sample input information, one or more sample business knowledge corresponding to the sample input information, and a label of the intent type corresponding to the sample input information to obtain the preset large model.

4. The method for identifying intention according to claim 1, characterized in that: The determining the information to be identified based on the input information and the historical information of the user corresponding to the input information includes: If the input information does not contain the preset words, and the historical information of the user corresponding to the input information is empty, the input information is determined as information to be identified; If the input information does not contain preset words, and the historical information of the user corresponding to the input information is not empty, a rewrite prompt word is determined according to the historical information and the input information; the rewrite prompt word is used to indicate that the input information is to be rewritten according to the historical information; information is rewritten according to the historical information, the rewrite prompt word and the input information to obtain information to be identified; the information rewriting includes information omission, information reference, information compression and no modification.

5. The method for intention recognition according to claim 4, characterized in that: The step of rewriting information according to the historical information, the rewriting prompt word and the input information to obtain the information to be identified includes: splicing the historical information, the rewriting prompt word and the input information to obtain spliced ​​information; Input the splicing information into the preset large model to obtain the rewriting information output by the preset large model; The rewritten information is determined as information to be identified.

6. The method for identifying intention according to claim 4, characterized in that: Before determining the information to be identified based on the input information and the historical information of the user corresponding to the input information, the method further includes: If the input information includes at least one preset word, the output information is determined based on each of the preset words.

7. An intention recognition device, characterized in that: include: A first determination module, configured to determine information to be identified based on input information and historical information of a user corresponding to the input information; A first recognition module, used for inputting the information to be recognized into a preset small model to obtain a first intention recognition result output by the preset small model; The first intention recognition result at least includes a matching result of business knowledge; A second recognition module, for inputting the information to be recognized and each piece of business knowledge into a preset large model to obtain a second intention recognition result output by the preset large model if the maximum score of each piece of business knowledge in the first intention recognition result is less than a preset threshold value of the corresponding business knowledge; the model parameter magnitude of the preset large model is greater than the model parameter magnitude of the preset small model; the preset threshold value of the business knowledge is determined according to the information length of the corresponding business knowledge; The second intention recognition result includes a matching result of business knowledge and an intention type; The second determination module is used to determine output information based on the second intention recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the intention recognition method as described in any one of claims 1-6 is implemented.

9. A storage medium, the storage medium being a non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the intention recognition method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the intention recognition method described in any one of claims 1 to 6 is implemented.

Images