Information processing method, electronic device, storage medium, and computer program product

Abstract

The application discloses an information processing method, an electronic device, a storage medium and a computer program product. The method comprises the following steps: in the case that the number of first information in a first set is greater than or equal to a first threshold value, clustering all the first information in the first set to obtain one or more candidate new intents; a first model is used for intent recognition processing on input information to obtain a corresponding processing result; the processing result represents the probability that the input information belongs to each intent of one or more intents identifiable by the first model; the first information represents user input information whose intent cannot be identified through the processing result of the first model; the first information under each candidate new intent in the one or more candidate new intents is input into the first model to obtain a processing result corresponding to each candidate new intent; and one or more new intents are determined from the one or more candidate new intents based on the processing result corresponding to each candidate new intent.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to an information processing method, electronic device, storage medium, and computer program product. Background Technology

[0002] In related technologies, intent recognition is performed on user input to distinguish the intent of the input information. Then, based on different intents, different task flows are flexibly connected to respond to user input. However, current intent recognition methods require all possible intents to be predefined. When encountering long-tail problems where predefined intents are not covered, they cannot be effectively recognized, nor can they flexibly recognize intents for user input in different scenarios. Summary of the Invention

[0003] To address the related technical problems, embodiments of this application provide an information processing method, an electronic device, a storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides an information processing method, the method comprising:

[0006] If the number of first information items in the first set is greater than or equal to a first threshold, all first information items in the first set are clustered to obtain one or more candidate new intents; the first model is used to perform intent recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intents that the first model can recognize; the first information represents user input information whose intent cannot be recognized by the processing result of the first model;

[0007] Input the first information of each candidate new intention in one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention.

[0008] Based on the processing results corresponding to each type of candidate new intention, one or more new intentions are determined from the one or more types of candidate new intentions.

[0009] In the above scheme, determining one or more new intentions from the one or more candidate new intentions based on the processing results corresponding to each type of candidate new intention includes:

[0010] The candidate new intentions, excluding those whose processing results meet the set conditions, are identified as the one or more types of candidate new intentions. The set conditions indicate that the number of second information under the candidate new intentions is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intention belonging to the same type of intention among the one or more types of intentions is greater than or equal to a third threshold.

[0011] The method in the above scheme further includes:

[0012] The first model is incrementally trained based on one or more new intentions to update the first model.

[0013] The method in the above scheme further includes:

[0014] The first information under each candidate new intention is augmented based on the second model to obtain the augmented information under each candidate new intention; the augmented information under each candidate new intention includes positive examples and negative examples of the first information under each candidate new intention; the second model is used to generate positive examples and negative examples of the input information;

[0015] The step of inputting the first information of each candidate new intention from one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention includes:

[0016] The first information and amplification information under each type of candidate new intention are input into the first model to obtain the processing result corresponding to each type of candidate new intention.

[0017] The method in the above scheme further includes:

[0018] Based on the newly acquired user input information, a comparative test was conducted between the updated first model and the original first model to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the original first model.

[0019] If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the first model after the update is greater than or equal to a fourth threshold, the updated first model will be rolled back to the first model before the update.

[0020] The method in the above scheme further includes:

[0021] Based on the one or more new intents, generate multiple candidate task flows corresponding to each new intent in the one or more new intents;

[0022] For each new intent, based on one or more set test parameters of multiple candidate task flows corresponding to that new intent, calculate the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent.

[0023] Based on the score of each candidate task process among multiple candidate task processes corresponding to this type of new intent, the task process corresponding to this type of new intent is determined.

[0024] The method in the above scheme further includes:

[0025] The user input information is input into the first model, the intent type corresponding to the user input information is output, and the task flow corresponding to the intent type is executed based on the intent type corresponding to the user input information.

[0026] If the first model cannot identify the intent type of the user input information, the user input information is added to the first set.

[0027] This application also provides an electronic device, including a processor and a memory for storing a computer program capable of running on the processor.

[0028] When the processor runs the computer program, it executes the steps of any of the above methods.

[0029] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0030] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0031] In the information processing method, electronic device, storage medium, and computer program product provided in this application embodiment, when the number of first information in a first set is greater than or equal to a first threshold, all first information in the first set is clustered to obtain one or more candidate new intentions; the first information under each candidate new intention in the one or more candidate new intentions is input into the first model to obtain the processing result corresponding to each candidate new intention; based on the processing result corresponding to each candidate new intention, one or more new intentions are determined from the one or more candidate new intentions; the first model is used to perform intention recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intentions that the first model can recognize; the first information represents user input information whose intention cannot be recognized by the processing result of the first model. The above scheme re-inputs the first information of each candidate new intent from the clustering into the first model, and compares it with the intent types that the first model can currently recognize through cross-samples. This allows for the accurate determination of new intents from the candidate new intents, enabling adaptive capture of new intents and real-time expansion of the intent types that the first model can recognize. This makes the first model applicable to different application scenarios, and the real-time expansion of the recognizable intent types is more in line with the actual application scenarios. Attached Figure Description

[0032] Figure 1 This is a schematic flowchart of an information processing method according to an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of an information processing system framework according to an application embodiment of this application;

[0034] Figure 3 This is a schematic diagram of the information processing device structure according to an embodiment of this application;

[0035] Figure 4 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0036] With the development and enhanced capabilities of large language models, functions such as natural language processing and customer service responses have significantly improved the flexibility and comprehensiveness of responses by introducing and collaboratively applying large language surface models and semantic models. The specific application method is as follows: the user inputs information, which can be of various modalities, such as voice, text, images, video, or a combination of multiple modalities. This information is then mapped into data in the same spatial domain through a multimodal mapping model, and the mapped data is input into the processing framework for processing. Within the processing framework, intent recognition is first performed on the input information to distinguish the intent behind the input. Then, based on different intents, different task flows are accessed for processing and a response is generated. The response can be designed with different modalities according to specific scenario requirements, such as voice, text, images, video, or a combination of multiple modalities. It is evident that the intent recognition module occupies a very important position in the processing framework, involving whether it can accurately capture the user's intent and whether it can navigate to the normal task flow, playing a crucial role in connecting the preceding and following steps. However, current intent recognition still has the following problems:

[0037] Static intent libraries have hard boundaries. Traditional intent recognition methods, such as intent classifiers based on rule engines or recurrent neural networks (RNNs), require predefining all possible intents. When encountering long-tail problems not covered by the intent library, intent recognition cannot effectively identify the intent of the problem. For example, input messages like "I want to add staff" are informal and colloquial, but the actual intent is to add employees to the system and navigate to the employee registration page. However, intelligent front-end systems cannot recognize this kind of colloquial intent and will navigate to other task flows or human customer service windows, resulting in inefficient completion of the target task.

[0038] Incremental new intents have a long delay: Dynamic models for intent recognition include multi-level intent processing methods and methods that align with modality processing before performing intent recognition. Although these methods have greatly improved upon traditional methods, when expanding the intent database, new data still needs to be manually labeled, loaded, and trained. The average iteration cycle ranges from several hours to tens of hours, which cannot meet the real-time requirements.

[0039] Fragmented processing flow: Unidentified intent is often categorized into other casual conversation categories and then handed over to human processing, resulting in low efficiency in collaboration between business systems and artificial intelligence (AI) modules.

[0040] The intent libraries of different scenarios tend to be similar, making it difficult to be personalized and flexible: the same system applied to different scenarios is often maintained or handled by a single technical team. As a result, similar intent processing and intent samples in different scenarios tend to be similar, making it difficult to extract unique intents and samples for different scenarios. Even if different teams are used to maintain the system in different scenarios, it will lead to an increase in labor costs.

[0041] Based on this, in various embodiments of this application, when the number of first information items in the first set is greater than or equal to a first threshold, all first information items in the first set are clustered to obtain one or more candidate new intentions; the first information items under each candidate new intention item in the one or more candidate new intentions are input into the first model to obtain the processing result corresponding to each candidate new intention item; based on the processing result corresponding to each candidate new intention item, one or more new intentions are determined from the one or more candidate new intentions; the first model is used to perform intention recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intentions that the first model can recognize; the first information represents user input information whose intention cannot be recognized by the processing result of the first model. The above scheme re-inputs the first information of each candidate new intent from the clustering into the first model, and compares it with the intent types that the first model can currently recognize through cross-samples. This allows for the accurate determination of new intents from the candidate new intents, enabling adaptive capture of new intents and real-time expansion of the intent types that the first model can recognize. This makes the first model applicable to different application scenarios, and the real-time expansion of the recognizable intent types is more in line with the actual application scenarios.

[0042] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0043] This application provides an information processing method applied to electronic devices, such as... Figure 1 As shown, the method includes:

[0044] Step 101: If the number of first information items in the first set is greater than or equal to the first threshold, cluster all the first information items in the first set to obtain one or more candidate new intents.

[0045] The first model is used to perform intent recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of one or more intents that the first model can recognize; the first information represents user input information whose intent cannot be recognized by the processing result of the first model.

[0046] Here, since there may be different expressions of the same intent within the first information, to reduce computation and increase sample diversity, the first information can be stored first, and the number of first information in the first set can be monitored. When the number of first information is greater than or equal to a first threshold, the new intent capture process is triggered. That is, all first information in the first set is clustered to obtain one or more candidate new intents. The first threshold can be set according to the actual application scenario. Clustering can be achieved with the help of a basic information model and a professional knowledge model applied to the actual application scenario. For example, the first threshold can be greater than or equal to 20 and less than or equal to 100. Specifically, it can be set to 50. When the number of first information in the first set is greater than or equal to 50, cluster analysis is performed on all first information in the first set using the basic information model and the professional knowledge model. The first information can be encoded and mapped to the same semantic space first, and then the encoding of the first information mapped to the same semantic space is clustered. 50 pieces of first information can be clustered into 3 to 5 categories.

[0047] The first set contains user input information whose intent cannot be identified by the processing results of the first model; this is called the first information, also known as unidentified user input information or unidentified intent. Unidentified intents can also be stored in an unidentified intent information database, meaning the first set can also be in the form of a database. The first model can be an intent recognition model, and one or more types of intents that the first model can recognize can be intent types within the intent database of the first model. In other words, the processing results of the first model can characterize the probability that the input information belongs to each type of intent within the intent database of the first model.

[0048] Step 102: Input the first information of each candidate new intention in one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention.

[0049] Here, after clustering all the first information in the first set to obtain one or more candidate new intentions, taking one of the candidate new intentions as an example, all the first information under this candidate new intention is input into the first model to obtain the processing result corresponding to this candidate new intention. The processing result corresponding to this candidate new intention includes the processing result corresponding to each of the first information under this candidate new intention. It can be determined whether this candidate new intention is a new intention by comprehensively considering the processing results corresponding to all the first information under this candidate new intention. The above operation is performed on the first information under each of the candidate new intentions in one or more candidate new intentions to obtain the processing result corresponding to each candidate new intention.

[0050] Step 103: Based on the processing results corresponding to each type of candidate new intention, determine one or more new intentions from the one or more candidate new intentions.

[0051] Here, after obtaining the processing results corresponding to each type of candidate new intent, taking one type of candidate new intent from one or more types of candidate new intents as an example, based on the processing results of all the first information under that type of candidate new intent included in the processing results corresponding to that type of candidate new intent, it is determined whether that type of candidate new intent is a new intent. For example, based on the probability that each piece of first information under that type of candidate new intent belongs to each type of intent in the intent library of the first model, the confidence that that type of candidate new intent is a new intent is calculated. The confidence that that type of candidate new intent is a new intent is compared with a set threshold to determine whether that type of candidate new intent is a new intent. The above operation is performed on each type of candidate new intent from one or more types of candidate new intents, thereby determining one or more types of new intents from one or more types of candidate new intents. The determined one or more types of new intents are used to expand the intent library of the first model, or to perform incremental training on the first model to increase the types of intents that the first model can recognize.

[0052] To achieve cross-validation and accurately determine one or more new intents from one or more candidate new intents, while also reducing the training load of incremental training, in one embodiment, determining one or more new intents from the one or more candidate new intents based on the processing results corresponding to each type of candidate new intent includes:

[0053] The candidate new intentions, excluding those whose processing results meet the set conditions, are identified as the one or more types of candidate new intentions. The set conditions indicate that the number of second information under the candidate new intentions is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intention belonging to the same type of intention among the one or more types of intentions is greater than or equal to a third threshold.

[0054] Here, candidate new intentions whose processing results meet the set conditions can be understood as candidate new intentions similar to one of the one or more types of intentions that the first model can identify. The first information of candidate new intentions whose processing results meet the set conditions is merged into the sample of that one type of intention. Candidate new intentions whose processing results do not meet the set conditions, that is, candidate new intentions other than those whose processing results meet the set conditions among one or more types of candidate new intentions, can be understood as candidate new intentions that are not similar to one or more types of intentions that the first model can identify. These candidate new intentions can be identified as new intentions.

[0055] Specifically, after obtaining the processing results corresponding to each type of candidate new intent, taking one type of candidate new intent as an example, the processing results corresponding to each first piece of information under that type of candidate new intent can be used to determine whether the processing results of that type of candidate new intent meet the set conditions. That is, it can be determined whether the number of first pieces of information under that type of candidate new intent with a probability greater than or equal to the third threshold of belonging to one or more types of intents that can be identified by the first model is greater than or equal to the second threshold. The second threshold corresponding to that type of candidate new intent can be determined based on the number of first pieces of information under that type of candidate new intent and a set ratio. For example, the set ratio is set to 70%, and the second threshold is set to the number of first pieces of information under that type of candidate new intent multiplied by 70%. The third threshold is set according to the actual application. For example, it can be set to 75%. Based on the above method, candidate new intents that meet the set conditions are determined, and the first pieces of information under the candidate new intents that meet the set conditions are merged into the sample of the same type of intent corresponding to the second information. Other candidate new intents other than those whose processing results meet the set conditions are determined as new intents.

[0056] For example, based on the processing results of each first piece of information under a certain type of candidate new intent, which includes the processing results of all first pieces of information under that candidate new intent, it is determined that the probability that more than 70% of the first pieces of information under that candidate new intent belongs to a certain type of intent type that can be identified by the first model is greater than or equal to 75%. It is considered that the candidate new intent is similar to the intent type that can be identified by the first model. The first pieces of information under that candidate new intent are merged with the samples of the intent type that can be identified by the first model. Based on this, by inputting the first pieces of information under the candidate new intent into the first model, the candidate new intent is compared with the intent that can be identified by the first model through cross-sample intent comparison. Samples of similar intents are merged, and samples of dissimilar intents are confirmed as new intents, thereby accurately identifying new intents.

[0057] To achieve efficient real-time updates of the first model, in one embodiment, the method further includes:

[0058] The first model is incrementally trained based on one or more new intentions to update the first model.

[0059] Here, when one or more new intents are identified from one or more candidate new intents, the first model is incrementally trained based on the identified new intents. Specifically, the first model can be trained based on samples of each new intent within the one or more new intents, and the first model can be fine-tuned by adjusting a small number of parameters. This achieves efficient fine-tuning of the first model, reduces training time, and improves update efficiency, enabling the first model to recognize new intents. The samples of new intents must include at least all the first information under that new intent. For example, model fine-tuning techniques such as Low-Rank Adaptation (LoRA) and Adapter Tuning can be used to update the first model, adjusting only 0.1%-1% of the parameters in the first model, and the update time can be controlled within 15-20 minutes.

[0060] To increase the accuracy of identifying new intents, in one embodiment, the method further includes:

[0061] The first information under each candidate new intention is augmented based on the second model to obtain the augmented information under each candidate new intention; the augmented information under each candidate new intention includes positive examples and negative examples of the first information under each candidate new intention; the second model is used to generate positive examples and negative examples of the input information;

[0062] The step of inputting the first information of each candidate new intention from one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention includes:

[0063] The first information and amplification information under each type of candidate new intention are input into the first model to obtain the processing result corresponding to each type of candidate new intention.

[0064] Here, after clustering all the first information in the first set to obtain one or more candidate new intentions, data augmentation is performed on the first information under each candidate new intention based on the second model to obtain augmented information under each candidate new intention. The first information and augmented information under each candidate new intention can be understood as samples of each candidate new intention. Specifically, for each piece of first information under each candidate new intention, positive and negative examples of each piece of first information can be generated by inputting each piece of first information into the second model to achieve data augmentation of each candidate new intention and obtain augmented information. For example, for each piece of first information, 30-50 positive examples and 20 negative examples can be generated by the second model. Positive examples include sentence variations, similar expressions, etc., while negative examples include intentions that are easily confused with the first information or the candidate new intention to which the first information belongs, or irrelevant intentions, etc.

[0065] With data augmentation and amplified information, the first information and amplified information under each candidate new intention are input into the first model. That is, the samples of each candidate new intention are input into the first model to obtain the processing results corresponding to each candidate new intention. Based on the processing results, cross-validation can be performed to determine the new intention, making the cross-validation results more accurate. For example, it improves the accuracy of determining candidate new intentions that meet the set conditions.

[0066] To achieve adaptive model updates, in one embodiment, the method further includes:

[0067] Based on the newly acquired user input information, a comparative test was conducted between the updated first model and the original first model to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the original first model.

[0068] If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the first model after the update is greater than or equal to a fourth threshold, the updated first model will be rolled back to the first model before the update.

[0069] Here, after fine-tuning some parameters of the first model, the updated first model is obtained; by comparing the performance of the first model before and after the update (A / B test), it is ensured that the first model used in the final application is always the optimal model, thus realizing the adaptive update of the model. Specifically, new user input information can be acquired and input into an updated first model to obtain the processing result of the first model. Based on the processing result of the updated first model on the newly acquired user input information, the intent type of the newly acquired user input information is determined, and the intent recognition accuracy of the updated first model is calculated. The intent recognition accuracy of the updated first model is compared with the intent recognition accuracy of the first model before the update. If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the updated first model is greater than or equal to a fourth threshold, it indicates that the performance of the first model before the update is better, and the updated first model is rolled back to the first model before the update, that is, the first model before the update is still used for intent recognition of user input information in actual applications. If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the updated first model is less than the fourth threshold, the current first model update is valid, and the updated first model is used for intent recognition of user input information in actual applications.

[0070] Alternatively, the system can continue using the original, unupdated first model for user access, while acquiring some new user input. This new input is then fed into the updated first model to obtain its processing result, allowing the calculation of the updated first model's intent recognition accuracy. The difference between the original and updated first model's intent recognition accuracy is then calculated. If the difference is greater than or equal to a fourth threshold, the updated first model is rolled back to the original model, meaning that in practical applications, user input is fed into the original first model for intent recognition. If the difference is less than the fourth threshold, the currently used original first model is updated to the updated first model, thus providing intent recognition services using the updated first model.

[0071] To efficiently complete user tasks, in one embodiment, the method further includes:

[0072] Based on the one or more new intents, generate multiple candidate task flows corresponding to each new intent in the one or more new intents;

[0073] For each new intent, based on one or more set test parameters of multiple candidate task flows corresponding to that new intent, calculate the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent.

[0074] Based on the score of each candidate task process among multiple candidate task processes corresponding to this type of new intent, the task process corresponding to this type of new intent is determined.

[0075] Here, when encountering an intent that the first model cannot recognize, the only recourse is to navigate to manual processing, which cannot effectively and efficiently handle user tasks. Therefore, it is necessary to define a corresponding task flow for each newly identified intent. This would allow the first model to navigate to the corresponding task flow when it recognizes the user input as such, completing the task and saving labor costs and processing time. Furthermore, it would enable the customization of task flows for specific scenarios, providing users with a better user experience and service environment.

[0076] Specifically, for each type of new intent, multiple candidate task flows corresponding to that new intent can be generated. For example, multiple candidate task flows can be generated based on experience in dealing with that type of new intent. Alternatively, a task flow generation model can be called, and candidate task flows corresponding to the intent can be planned and generated by combining the LangChain framework, the React framework, and the API (Application Programming Interface) toolkit. For example, the task can be decomposed first to determine the API to be called, and prompts can be designed based on the goals to be achieved. Based on the prompts and the API to be called, the task flow corresponding to that type of new intent can be generated through the task flow generation model.

[0077] When generating multiple candidate task flows corresponding to a newly added intent, the multiple candidate task flows are tested using newly acquired user input information, and user feedback is obtained to obtain one or more set test parameters for each candidate task flow. Based on the one or more set test parameters for each candidate task flow, a score is calculated for each candidate task flow. The set test parameters may include task completion time, number of interactions, amount of information provided, completion satisfaction, etc. The score can be determined by the value of the set test parameters, or by the value of the set test parameters and their corresponding weights. For example, 5% of the user input information in actual applications can be used to test multiple candidate task flows. Test parameters such as the number of interactions (experienced interactions) and amount of information provided during the task completion process of each task flow can be recorded. After the task is completed, test parameters such as task completion time and user feedback completion satisfaction can be obtained. Based on the recorded and / or obtained one or more set test parameters, the score of each candidate task flow can be calculated, thereby enabling quantitative analysis and qualitative evaluation of the candidate task flows. For example, the score of any candidate task flow can be calculated using the following formula:

[0078]

[0079] Among them, Score i The score represents the i-th task flow corresponding to this new intent, with a value range of [0,1]. This new intent corresponds to 3 candidate task flows, i∈[1,2,3]; Ni represents that the i-th task flow has been tested N times, where ni is the n-th test; T ni Characterizes the time taken to complete the task; E ni Represents the number of interactions experienced; I ni The amount of information provided by the representation; G niThe system represents completion satisfaction; 0.1, 0.2, and 0.5 are the weights of the corresponding test parameters. For task flows that focus on responding to user questions, completion satisfaction is more important among one or more test parameters, so the weight of completion satisfaction can be set to 0.5. The weights of each test parameter can also be adjusted according to different scenarios and different focuses.

[0080] Given that each candidate task process among multiple candidate task processes corresponding to this type of new intent has been calculated, the candidate task process with the highest score among multiple candidate task processes is selected as the task process corresponding to this type of new intent based on the score of each candidate task process. In other words, the optimal candidate task process is selected as the task process to be executed for this type of new intent in actual application.

[0081] To accurately navigate to the corresponding task flow and efficiently complete the user task, in one embodiment, the method further includes:

[0082] Input the user input information into the first model and output the processing result corresponding to the user input information;

[0083] If the processing result corresponding to the user input information indicates the intent type of the user input information, the task flow corresponding to the intent type of the user input information is executed.

[0084] If the processing result corresponding to the user input information cannot identify the intent type of the user input information, the user input information is added to the first set.

[0085] In practical applications, upon receiving user input information, the user input information is input into a first model for intent recognition to obtain the processing result corresponding to the user input information. If the processing result indicates the intent type of the user input information, i.e., intent recognition is successful, navigation is initiated to the task flow corresponding to the user input information, and the task flow corresponding to the intent type of the user input information is executed to complete the task corresponding to the user input information. If the processing result indicates that the intent type of the user input information cannot be recognized, i.e., the user input information is first information, the user input information is added to a first set. The number of first information items in the first set is monitored, and if the number of first information items in the first set is greater than or equal to a first threshold, a new intent capture process is triggered.

[0086] Specifically, the model can determine whether it can identify the intent type of the user input based on the probability that the user input belongs to each of one or more intent types that the first model can recognize. If the probability that the user input belongs to a certain intent type that the first model can recognize is greater than or equal to a fifth threshold, the first model can be determined to be able to identify the intent type of the user input, and the processing result corresponding to the user input indicates that the intent type of the user input is that certain intent type, and the task flow corresponding to that intent type is executed. If the probability that the user input belongs to each intent type that the first model can recognize is less than the fifth threshold, the first model cannot be determined to be unable to identify the intent type of the user input, and the processing result corresponding to the user input indicates that the intent type of the user input cannot be identified, and the user input is added to the first set. The fifth threshold can be set according to the actual application, for example, it can be set to 75%.

[0087] The following section provides a more detailed description of this application with reference to application examples.

[0088] like Figure 2 As shown, the information processing system framework can include an information processing workflow framework and a service model. The modules within the information processing workflow framework can be divided into five modules according to the information processing flow: an intent recognition module, a new intent capture module, a task flow processing module, a task flow generation module, and a flow evaluation module. In practical applications, the information processing workflow framework should at least include an intent recognition module and a new intent capture module; the use of other modules is determined based on the actual application scenario. The service model can include one or more of the following: a first model, a second model, a task flow generation model, a basic information model, and a professional knowledge model. Each module in the information processing workflow framework can call one or more of the service models to complete the corresponding information processing. This information processing system framework can be applied to scenarios requiring real-time responses to complex user intents, such as intelligent customer service, automated office work, personalized recommendations, and medical consultations.

[0089] Information processing methods may include the following steps:

[0090] Step 1: Input the user input information into the first model and output the processing result corresponding to the user input information.

[0091] Here, upon obtaining user input information, the user input information is input into the first model, and the processing result corresponding to the user input information is output; for example, based on Figure 2The information processing system framework can execute step 1 in the intent recognition module. If the processing result corresponding to the user input information indicates the intent type of the user input information, step 2 is executed; if the processing result corresponding to the user input information indicates that the intent type of the user input information cannot be recognized, step 3 is executed.

[0092] Step 2: Execute the task flow corresponding to the intent type of the user input information.

[0093] Here, when the processing result corresponding to the user input information indicates the intent type of the user input information—for example, the processing result indicates that the probability of the user input information belonging to a certain intent type recognizable by the first model is greater than or equal to a fifth threshold—based on this intent type, the task flow corresponding to that intent type is determined and executed, and the task flow execution result, i.e., the information processing result, is output. The task flow execution result can be a response to the user input information or the execution status of the user input task. For example, based on... Figure 2 The information processing system framework can execute step 2 in the task flow processing module, that is, the intent recognition module sends the intent type of the user input information to the task flow processing module. The task flow processing module executes the corresponding task flow based on the intent type of the user input information and outputs the processing result.

[0094] Step 3: Add the user input information to the first set.

[0095] Here, if the processing result corresponding to the user input information cannot identify the intent type of the user input information—for example, if the processing result indicates that the probability of the user input information belonging to each intent type recognizable by the first model is less than the fifth threshold—it is considered that the first model cannot identify the intent type of the user input information, and the user input information is added to the first set to capture the new intent. For example, based on... Figure 2 The information processing system framework can use the intent recognition module to take user input information whose intent type cannot be recognized by the first model as the first information and send it to the new intent capture module. The new intent capture module stores the received first information into the first set.

[0096] Step 4: If the number of first information items in the first set is greater than or equal to the first threshold, cluster all the first information items in the first set to obtain one or more candidate new intents.

[0097] Here, the number of first pieces of information in the first set is monitored. If the number of first pieces of information in the first set is greater than or equal to a first threshold, a new intent capture process is triggered. The first pieces of information in the first set are clustered to initially determine one or more candidate new intents. For example, based on Figure 2 The information processing system framework can execute step 4 in the newly added intent capture module. For example, it can invoke the basic information model and professional knowledge model to assist in clustering.

[0098] Step 5: Based on the second model, perform data augmentation on the first information under each type of candidate new intention to obtain the augmented information under each type of candidate new intention.

[0099] For example, based on Figure 2 The information processing system framework can be accessed by adding an intent capture module, and step 5 can be executed. The specific implementation process of steps 5-6 is described above and will not be repeated here.

[0100] Step 6: Input the first information and amplification information of each candidate new intention in one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention.

[0101] For example, based on Figure 2 The information processing system framework can be used to execute step 6 in the newly added intent capture module.

[0102] Step 7: Based on the processing results corresponding to each type of candidate new intent, determine one or more new intents from one or more candidate new intents.

[0103] Here, based on the processing results corresponding to each type of candidate new intent, candidate new intents other than those whose processing results meet the set conditions are determined as one or more types of new intents. The set conditions indicate that the number of second information under the candidate new intent is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intent belonging to the same type of intent among one or more types of intents is greater than or equal to a third threshold. For example, based on Figure 2 The information processing system framework can execute step 7 in the newly added intent capture module.

[0104] Step 8: Incrementally train the first model based on one or more new intentions to update the first model.

[0105] Here, the first information and augmented information under each new intent from one or more new intents are used as samples for each new intent. The first model is incrementally trained based on these samples to obtain an updated first model. For example, based on... Figure 2 The information processing system framework can send one or more newly identified intents to the intent recognition module by adding an intent capture module. The intent recognition module then fine-tunes the first model based on the received one or more newly identified intents.

[0106] Step 9: Based on the newly acquired user input information, conduct a comparative test between the updated first model and the original first model to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the original first model.

[0107] For example, based on Figure 2 The information processing system framework can execute step 9 in the intent recognition module. For the specific implementation process of steps 9-10, please refer to the relevant description above, which will not be repeated here.

[0108] Step 10: If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the first model after the update is greater than or equal to the fourth threshold, roll back the updated first model to the first model before the update.

[0109] Here, based on the comparative test results of the updated first model and the original first model, the optimal first model for users in the current information processing flow is determined. For example, based on... Figure 2 The information processing system framework can execute step 10 in the intent recognition module.

[0110] Step 11: Based on one or more new intents, generate multiple candidate task flows corresponding to each new intent in one or more new intents.

[0111] For example, based on Figure 2 In the information processing system framework, the intent type sent by the intent recognition module to the task flow processing module can also be a new intent type. When the task flow processing module receives a new intent, it forwards the new intent to the task flow generation module. Based on the received new intent, the task flow generation module generates multiple candidate task flows corresponding to the new intent.

[0112] Step 12: For each new intent, based on one or more set test parameters of multiple candidate task flows corresponding to that new intent, calculate the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent.

[0113] Here, when generating multiple candidate task flows for each of one or more new intents, each candidate task flow is tested. For each new intent, each candidate task flow can be tested multiple times using newly acquired user input information, resulting in one or more predefined test parameters for each candidate task flow in each test. Based on these predefined test parameters, a score is calculated for each candidate task flow. For example, based on... Figure 2The information processing system framework allows the task flow generation module to send multiple candidate task flows corresponding to each newly generated intent to the process evaluation module. The process evaluation module tests the multiple candidate task flows corresponding to each newly generated intent and calculates the score of each candidate task flow corresponding to each newly generated intent based on the test results, so as to evaluate the candidate task flows and determine the optimal task flow corresponding to each newly generated intent.

[0114] Step 13: Based on the score of each candidate task process in the multiple candidate task processes corresponding to this type of new intent, determine the task process corresponding to this type of new intent.

[0115] Here, the candidate task process with the highest score among multiple candidate task processes corresponding to this type of new intent is determined as the task process corresponding to this type of new intent. For example, based on Figure 2 The information processing system framework can determine the task flow corresponding to each type of new intent in the process evaluation module, and send the determined task flow corresponding to each type of new intent to the task flow processing module, so that when the task flow processing module receives the type of new intent sent by the intent recognition module, it can navigate to the task flow corresponding to the type of new intent.

[0116] It should be noted that steps 8-10 and 11-13 can be executed in parallel or sequentially, provided that one or more new intents have been identified.

[0117] In this application embodiment, the rigid boundary problem of traditional intent recognition is solved by adding intent capture. By generating task flows corresponding to the added intent, the task flow corresponding to the added intent is executed directly when the added intent is recognized, solving the problem of the uniformity of manual handling of added intents, reducing manual costs, increasing processing efficiency and system scenario adaptability. Qualitative and quantitative analysis of candidate task flows is achieved by setting test parameters. By expanding the types of intents that can be identified by the first model through added intents, the first model can be updated in real time, enabling the information processing system to further meet the needs of actual application scenarios. Furthermore, the added intents can provide richer information for other service models in the information processing system, which can be used for fine-tuning of other service models. Real-time updates of the entire information processing system make it universally applicable to different application scenarios, realizing the personalization and adaptive optimization of the information processing system.

[0118] To implement the methods of the embodiments of this application, the embodiments of this application also provide an information processing apparatus, such as... Figure 3 As shown, the device includes:

[0119] Clustering unit 301 is used to cluster all the first information in the first set when the number of first information in the first set is greater than or equal to a first threshold, to obtain one or more candidate new intents; the first model is used to perform intent recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intents that the first model can recognize; the first information represents user input information whose intent cannot be recognized by the processing result of the first model;

[0120] The first processing unit 302 is used to input the first information of each candidate new intention in the one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention.

[0121] The first determining unit 303 is used to determine one or more new intentions from the one or more candidate new intentions based on the processing results corresponding to each type of candidate new intention.

[0122] In one embodiment, the first determining unit 303 is specifically used to determine the candidate new intentions other than the candidate new intentions whose processing results meet the set conditions as the one or more types of new intentions; the set conditions indicate that the number of second information under the candidate new intentions is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intention belonging to the same type of intention in the one or more types of intentions is greater than or equal to a third threshold.

[0123] In one embodiment, the device further includes:

[0124] The training unit is used to incrementally train the first model based on one or more new intentions to update the first model.

[0125] In one embodiment, the device further includes:

[0126] An enhancement unit is used to perform data enhancement on the first information under each type of candidate new intention based on a second model to obtain augmented information under each type of candidate new intention; the augmented information under each type of candidate new intention includes positive examples and negative examples of the first information under each type of candidate new intention; the second model is used to generate positive examples and negative examples of the input information;

[0127] The first processing unit 302 is specifically used to input the first information and amplification information under each type of candidate new intention into the first model to obtain the processing result corresponding to each type of candidate new intention.

[0128] In one embodiment, the device further includes:

[0129] The test unit is used to compare the updated first model with the first model before the update based on the newly acquired user input information, and to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the first model before the update.

[0130] The rollback unit is used to roll back the updated first model to the original first model if the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the updated first model is greater than or equal to a fourth threshold.

[0131] In one embodiment, the device further includes:

[0132] The generation unit is used to generate multiple candidate task flows corresponding to each type of new intent based on the one or more types of new intents.

[0133] The calculation unit is used to calculate the score of each candidate task process in the multiple candidate task processes corresponding to each new intent, based on one or more set test parameters of the multiple candidate task processes corresponding to the new intent.

[0134] The second determining unit is used to determine the task flow corresponding to the new intent based on the score of each candidate task flow among multiple candidate task flows corresponding to the new intent.

[0135] In one embodiment, the device further includes:

[0136] The second processing unit is used to input user input information into the first model and output the processing result corresponding to the user input information.

[0137] The execution unit is configured to execute the task flow corresponding to the intent type of the user input information when the processing result corresponding to the user input information indicates the intent type of the user input information.

[0138] The third processing unit is configured to add the user input information to the first set when the processing result corresponding to the user input information indicates that the intent type of the user input information cannot be identified.

[0139] In practical applications, the clustering unit 301, the first processing unit 302, the first determining unit 303, the training unit, the enhancement unit, the testing unit, the rollback unit, the generation unit, the calculation unit, the second determining unit, the second processing unit, the execution unit, and the third processing unit can be implemented by a processor in an information processing device.

[0140] It should be noted that the above embodiments of the information processing device are only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the information processing device and the information processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0141] Based on the hardware implementation of the above program modules, embodiments of this application also provide an electronic device, such as... Figure 4 As shown, the electronic device 400 includes:

[0142] Communication interface 401 allows for information exchange with other devices or network nodes.

[0143] The processor 402 is connected to the communication interface 401 to enable information interaction with other devices or network nodes, and to execute the methods provided by one or more technical solutions in the above embodiments when running computer programs.

[0144] Memory 403 is used to store computer programs that can run on processor 402.

[0145] Specifically, the processor 402 is configured to cluster all the first information in the first set to obtain one or more candidate new intents when the number of first information in the first set is greater than or equal to a first threshold; the first model is configured to perform intent recognition processing on the input information to obtain a corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intents that the first model can recognize; the first information represents user input information whose intent cannot be recognized by the processing result of the first model;

[0146] Input the first information of each candidate new intention in one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention.

[0147] Based on the processing results corresponding to each type of candidate new intention, one or more new intentions are determined from the one or more types of candidate new intentions.

[0148] In one embodiment, the processor 402 is specifically configured to determine the candidate new intentions (excluding those whose processing results meet the set conditions) from the one or more candidate new intentions as the one or more new intentions; the set conditions indicate that the number of second information under the candidate new intentions is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intention belonging to the same type of intention among the one or more intentions is greater than or equal to a third threshold.

[0149] In one embodiment, the processor 402 is further configured to incrementally train the first model based on one or more new intentions to update the first model.

[0150] In one embodiment, the processor 402 is further configured to perform data augmentation on the first information under each type of candidate addition intent based on the second model to obtain augmented information under each type of candidate addition intent; the augmented information under each type of candidate addition intent includes positive examples and negative examples of the first information under each type of candidate addition intent; the second model is configured to generate positive examples and negative examples of the input information;

[0151] The first information and amplification information under each type of candidate new intention are input into the first model to obtain the processing result corresponding to each type of candidate new intention.

[0152] In one embodiment, the processor 402 is further configured to perform a comparative test on the updated first model and the first model before the update based on the newly acquired user input information, to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the first model before the update; if the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the updated first model is greater than or equal to a fourth threshold, the updated first model is rolled back to the first model before the update.

[0153] In one embodiment, the processor 402 is further configured to generate multiple candidate task flows corresponding to each of the one or more new intents based on the one or more new intents.

[0154] For each new intent, based on one or more set test parameters of multiple candidate task flows corresponding to that new intent, calculate the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent; based on the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent, determine the task flow corresponding to that new intent.

[0155] In one embodiment, the processor 402 is further configured to input user input information into the first model and output the processing result corresponding to the user input information;

[0156] If the processing result corresponding to the user input information indicates the intent type of the user input information, the task flow corresponding to the intent type of the user input information is executed.

[0157] If the processing result corresponding to the user input information cannot identify the intent type of the user input information, the user input information is added to the first set.

[0158] It should be noted that the specific processing procedure of processor 402 can be understood by referring to the above method.

[0159] Of course, in practical applications, the various components in electronic device 400 are coupled together through bus system 404. It can be understood that bus system 404 is used to realize the connection and communication between these components. In addition to a data bus, bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 4 In this application, all buses are designated as bus system 404. The memory 403 in this embodiment is used to store various types of data to support the operation of the electronic device 400. Examples of this data include any computer program used to operate on the electronic device 400.

[0160] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the processor 402. The processor 402 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 402 or by instructions in software form. The processor 402 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 402 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically in memory 403. The processor 402 reads information from memory 403 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0161] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0162] It is understood that the memory (memory 403) in this embodiment of the application can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0163] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 403 storing a computer program, which can be executed by the processor 402 of the electronic device 400 to complete the steps described in any of the aforementioned methods. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0164] For example, embodiments of this application also provide a computer program product, including a computer program that can be executed by a processor 402 of an electronic device 400 to perform the steps described in any of the foregoing methods.

[0165] It should be noted that terms such as "first" and "second" are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The term "and / or" in this document merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, the technical solutions described in the embodiments of this application can be arbitrarily combined without conflict. The above descriptions are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application.

Claims

1. An information processing method, characterized in that, The method includes: If the number of first information items in the first set is greater than or equal to a first threshold, all first information items in the first set are clustered to obtain one or more candidate new intents; the first model is used to perform intent recognition processing on the input information to obtain the corresponding processing result; the processing result represents the probability that the input information belongs to each of the one or more intents that the first model can recognize; the first information represents user input information whose intent cannot be recognized by the processing result of the first model; Input the first information of each candidate new intention in one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention. The candidate new intentions, excluding those whose processing results meet the set conditions, are identified as one or more types of new intentions. The set conditions indicate that the number of second information under the candidate new intentions is greater than or equal to a second threshold, and the second information indicates that the probability of the candidate new intention belonging to the same type of intention among the one or more types of intentions is greater than or equal to a third threshold.

2. The method according to claim 1, characterized in that, The method further includes: The first model is incrementally trained based on one or more new intentions to update the first model.

3. The method according to claim 1, characterized in that, The method further includes: The first information under each candidate new intention is augmented based on the second model to obtain the augmented information under each candidate new intention; the augmented information under each candidate new intention includes positive examples and negative examples of the first information under each candidate new intention; the second model is used to generate positive examples and negative examples of the input information; The step of inputting the first information of each candidate new intention from one or more candidate new intentions into the first model to obtain the processing result corresponding to each candidate new intention includes: The first information and amplification information under each type of candidate new intention are input into the first model to obtain the processing result corresponding to each type of candidate new intention.

4. The method according to claim 2, characterized in that, The method further includes: Based on the newly acquired user input information, a comparative test was conducted between the updated first model and the original first model to obtain the intent recognition accuracy of the updated first model and the intent recognition accuracy of the original first model. If the difference between the intent recognition accuracy of the first model before the update and the intent recognition accuracy of the first model after the update is greater than or equal to a fourth threshold, the updated first model will be rolled back to the first model before the update.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the one or more new intents, generate multiple candidate task flows corresponding to each new intent in the one or more new intents; For each new intent, based on one or more set test parameters of multiple candidate task flows corresponding to that new intent, calculate the score of each candidate task flow in the multiple candidate task flows corresponding to that new intent. Based on the score of each candidate task process among multiple candidate task processes corresponding to this type of new intent, the task process corresponding to this type of new intent is determined.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Input the user input information into the first model and output the processing result corresponding to the user input information; If the processing result corresponding to the user input information indicates the intent type of the user input information, the task flow corresponding to the intent type of the user input information is executed. If the processing result corresponding to the user input information cannot identify the intent type of the user input information, the user input information is added to the first set.

7. An electronic device, characterized in that, This includes a processor and memory for storing computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images