Sample type identification method and device, storage medium and program product

Abstract

The embodiment of the invention provides a sample type identification method and device, a storage medium and a program product. In the embodiment of the invention, under the guidance of the configuration information such as the positive and negative sample recognition rule required for performing positive and negative sample recognition based on the rule and the target prompt word and the intention knowledge base required for performing positive and negative sample recognition based on the sample recognition model corresponding to each interaction intention index configured through the configuration operation; and in combination with rule-based sample identification processing and sample identification model-based identification processing, performing sample identification on multiple pieces of interaction information generated in at least one interaction process contained in the sample set to obtain a first identification result and a second identification result, so as to conveniently identify the multiple pieces of interaction information based on the first identification result and the second identification result. The positive and negative sample type identification accuracy of the system is improved, the use of manpower is reduced, and the balance between the performance and efficiency of the positive and negative sample type identification is ensured.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and in particular, to a method, device, storage medium, and program product for sample type recognition. Background Art

[0002] Currently, when many large models process business classification metrics (i.e., label positive and negative samples for the samples involved in business metrics), they rely on machine learning algorithms for autonomous learning and information extraction. However, this method usually lacks the ability to finely adjust classification criteria, resulting in low classification accuracy of the model in specific fields or complex scenarios. Since the learning process of the model is mainly based on the features of the sample dataset and fails to fully combine manual annotation knowledge, its understanding and application of industry characteristics are insufficient.

[0003] While some traditional manual classification systems have relatively high classification accuracy, their efficiency is low and it is difficult to scale up. These systems often rely on a large amount of manual intervention, affecting the timeliness and accuracy of classification. In addition, manual classification is easily affected by subjective factors, and the stability and consistency of the results are difficult to guarantee.

[0004] Therefore, there is still an imbalance problem between performance and efficiency in the above solutions. Summary of the Invention

[0005] Multiple aspects of this application provide a method, device, storage medium, and program product for sample type recognition, so as to improve the accuracy of the system in recognizing sample types, reduce the use of manual labor, and ensure the balance between the performance and efficiency of sample type recognition.

[0006] An embodiment of the present application provides a method for sample type recognition, including: responding to a configuration operation to configure at least one piece of configuration information adapted to a target service provided by a target application, each piece of configuration information including an interaction intention metric, and the interaction intention metric corresponding to a positive and negative sample recognition rule required for positive and negative sample recognition based on rules and a target prompt word and an intention knowledge base required for positive and negative sample recognition based on a sample recognition model; collecting a sample set, where the sample set contains a plurality of interaction information generated during at least one interaction process between a user and the target application, and at least one interaction process is initiated by the user for the target service; performing rule-based positive and negative sample recognition processing on each piece of interaction information according to the positive and negative sample recognition rule and the interaction intention metric to obtain a first recognition result of each piece of interaction information, and the first recognition result indicates that the corresponding interaction information is a positive sample adapted to the interaction intention metric or a negative sample not adapted to the interaction intention metric; inputting the target prompt word, the interaction intention metric, and each piece of interaction information into a sample type recognition model, and under the guidance of the target prompt word, combining the intention knowledge base to identify whether each piece of interaction information is adapted to the interaction intention metric to obtain a second recognition result of each piece of interaction information, and the second recognition result indicates that the corresponding interaction information is a positive sample adapted to the interaction intention metric or a negative sample not adapted to the interaction intention metric; for each piece of interaction information, when the first recognition result and the second recognition result of the interaction information are consistent, determining the interaction information as a positive sample or a negative sample indicated by any one of the first recognition result and the second recognition result.

[0007] An embodiment of the present application further provides an electronic device, including: a memory and a processor; the memory is used for storing a computer program; the processor is coupled to the memory and is used for executing the computer program to implement the steps in the above method.

[0008] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is caused to implement the steps in the above method.

[0009] An embodiment of the present application further provides a computer program product, and the computer program product includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the processor is caused to be able to implement the steps in the above method.

[0010] In the embodiments of the present application, through configuration operations, interaction intention metrics adapted to the target services provided by the target application, positive and negative sample recognition rules required for positive and negative sample recognition based on rules, and target prompt words and intention knowledge bases required for positive and negative sample recognition based on a sample recognition model can be configured to guide the subsequent sample type recognition process using the configuration information, thereby improving the accuracy of positive and negative sample recognition. Further, under the guidance of the configuration information, combining sample recognition processing based on rules and recognition processing based on a sample recognition model, sample recognition is performed on multiple interaction information generated during at least one interaction included in the sample set, obtaining a first recognition result and a second recognition result, so as to determine whether each interaction information is a positive sample or a negative sample based on the first recognition result and the second recognition result, further improving the accuracy of the system's recognition of sample types, reducing the use of manual labor, and ensuring the balance between the performance and efficiency of sample type recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0012] Figure 1 is a schematic flowchart of a sample type recognition method provided by an exemplary embodiment of the present application;

[0013] Figure 2a is a schematic diagram of an evaluation page provided by an exemplary embodiment of the present application;

[0014] Figure 2b is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0015] Figure 2c is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0016] Figure 2d is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0017] Figure 2e is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0018] Figure 2f is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0019] Figure 2g is a schematic diagram of another evaluation page provided by another exemplary embodiment of the present application;

[0020] Figure 2hSchematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0021] Figure 2i Schematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0022] Figure 2j Schematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0023] Figure 2k Schematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0024] Figure 2m Schematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0025] Figure 2l Schematic diagram of another evaluation page provided for another exemplary embodiment of the present application;

[0026] Figure 3 Schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application. Detailed implementation manners

[0027] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to select authorization or rejection.

[0029] In view of technical problems such as the imbalance between performance and efficiency in the existing sample type recognition process, in the embodiments of the present application, through configuration operations, interaction intention indicators adapted to the target services provided by the target application, positive and negative sample recognition rules required for positive and negative sample recognition based on rules, and target prompt words and intention knowledge bases required for positive and negative sample recognition based on the sample recognition model can be configured, so as to use the configuration information to guide the subsequent positive and negative sample type recognition process, thereby improving the accuracy of positive and negative sample recognition. Further, under the guidance of the configuration information, by combining the sample recognition processing based on rules and the recognition processing based on the sample recognition model, sample recognition is performed on multiple interaction information generated during at least one interaction included in the sample set, and a first recognition result and a second recognition result are obtained, so as to determine whether each interaction information is a positive sample or a negative sample based on the first recognition result and the second recognition result. Determining the positive and negative sample types through the recognition results of the two methods further improves the accuracy of the system's recognition of sample types, reduces the use of manual labor, and ensures the balance between the performance and efficiency of positive and negative sample type recognition.

[0030] The following will describe in detail a solution provided by the embodiments of the present application with reference to the accompanying drawings.

[0031] Figure 1 It is a schematic flowchart of the sample type recognition method provided by the exemplary embodiment of the present application. As Figure 1 shown, the method includes:

[0032] 101. Respond to the configuration operation, and configure at least one piece of configuration information adapted to the target service provided by the target application. Each piece of configuration information includes an interaction intention indicator, and the interaction intention indicator corresponds to positive and negative sample recognition rules required for positive and negative sample recognition based on rules, and target prompt words and an intention knowledge base required for positive and negative sample recognition based on the sample recognition model;

[0033] 102. Collect a sample set, where the sample set includes multiple interaction information generated during at least one interaction between the user and the target application, and at least one interaction is initiated by the user for the target service;

[0034] 103. Perform rule-based positive and negative sample recognition processing on each piece of interaction information according to the positive and negative sample recognition rules and the interaction intention indicator, so as to obtain a first recognition result of each piece of interaction information. The first recognition result indicates that the corresponding interaction information is a positive sample adapted to the interaction intention indicator or a negative sample not adapted to the interaction intention indicator;

[0035] 104. Input the target prompt, interaction intention metrics, and each interaction message into the sample type recognition model. Under the guidance of the target prompt, combined with the intention knowledge base, identify whether each interaction message matches the interaction intention metrics to obtain the second recognition result of each interaction message. The second recognition result indicates that the corresponding interaction message is a positive sample that matches the interaction intention metrics or a negative sample that does not match the interaction intention metrics.

[0036] 105. For each interaction message, when the first recognition result and the second recognition result of the interaction message are consistent, determine the interaction message as a positive sample or a negative sample represented by any one of the first recognition result and the second recognition result.

[0037] Generally, to improve the inference ability of the model, the initial model can be trained based on a sample set suitable for the model's inference ability and the model can be fine-tuned based on the evaluation data of the inference results. The sample set contains multiple samples. Or, to improve the online task processing ability based on rules, the task processing results of samples based on rules can be evaluated, and the corresponding rules can be continuously optimized and adjusted based on the evaluation results. Taking the sample recognition model for positive and negative sample type recognition (binary classification) as an example, when training the sample recognition model based on samples, the initial sample recognition model is trained based on positive samples and negative samples. Positive samples and negative samples are two types of samples. Among them, a positive sample refers to a sample with target attributes or features, that is, a sample that the sample recognition model is expected to recognize as an affirmative result such as "yes", "have", "belong to", etc. A negative sample refers to a sample without target attributes or features, that is, a sample that the sample recognition model is expected to recognize as a negative result such as "no", "none", "not belong to", etc. Further, to further improve the accuracy of the inference results of the sample recognition model, the trained sample recognition model can be further fine-tuned based on positive samples and negative samples. Positive and negative samples are obtained by identifying the positive and negative sample types of each sample included in the sample set and labeling based on the identified sample types. Therefore, the accuracy of positive and negative sample type recognition determines the accuracy of the classification inference ability of the sample recognition model, that is, the accurate recognition of positive and negative sample types is the key to ensuring the accuracy of the inference results of the sample recognition model.

[0038] The sample recognition model can be involved in the classification and recognition of samples in multiple fields. The classification and recognition of samples in different fields require samples in that field to train the sample recognition model so that the sample recognition model has the ability to classify and recognize samples in that field. Taking the interaction field as an example, at least one of the multiple interaction information generated during at least one interaction process can be used as positive samples and negative samples to train the sample recognition model so that the sample recognition model has the ability to recognize the positive samples and negative samples included in the interaction information. The interaction process can be, for example, the interaction process between a user and any application, and any application can be referred to as the target application. The target application can be, for example, a service application, a social application, a real estate transaction application, etc. More specifically, the target application can provide multiple services (also known as operations), and the interaction process can be initiated by the user for any one of the services provided by the target application, and any one of the services can be referred to as the target service. The target service can be, for example, a customer service, a keyword search service, etc. Taking the customer service as an example, the interaction process is the chat process between the user and the customer service. In this regard, one interaction process can be one round of question and answer, and one round of question and answer includes any question raised by the user and the reply of the customer service to this question; one interaction process can also be multiple rounds of question and answer included in the current conversation between the user and the customer service. In addition, one interaction process can be at least one historical interaction process between at least one user and the target application, or at least one interaction process can be at least one current interaction process between at least one user and the target application. The current interaction process refers to the interaction process that is ongoing during the current time period (such as the same day, within several hours), and the historical interaction process refers to the interaction process that was conducted before the current time period. In addition, the interaction process between the user and the customer service can be carried out through the chat page provided by the customer service, and the communication content between the user and the customer service is used as the interaction information. Taking the keyword search service as an example, the interaction process is the information search process of the user based on keywords, and the user can search based on keywords through the search page provided by the keyword search service, and the keywords are the interaction information.

[0039] Regardless of the target service, the interaction information generated during user interactions with the target service typically reflects a corresponding interaction intent, also known as an interaction intent indicator. Each service may contain multiple interaction intent indicators. For example, if the user is a merchant partnering with the target application, the merchant can purchase a membership in the target application. This allows the merchant to register as a member and promote their products or services through postings within the target application, thereby attracting new customers. During this process, the merchant may have questions or needs and communicate with customer service, who can then resolve their issues or provide the services they need. For example, if a merchant receives few or no calls regarding their products or services, users can report the low number of calls to customer service through the chat page. Another example is if a post receives few or no clicks, users can report the low number of clicks to customer service through the chat page. For another example, if a user needs to top up their membership, they can notify customer service through the chat page so that they can provide the user with a recharge channel. Another example is if a user needs to renew their membership, they can notify customer service through the chat page. For example, when a user needs to post something, they can ask customer service for help through the chat page. Correspondingly, the interaction intent indicators included in customer service include but are not limited to: "few / no phone calls", "few / no clicks on posts", "want to top up", "want to renew", and "help post".

[0040] Based on this, in order to further improve the accuracy of positive and negative sample type identification, positive samples and negative samples that are adapted to each interaction intention indicator can be identified from the dimension of interaction intention indicators, so as to train or fine-tune the sample recognition model based on the positive samples and negative samples that are adapted to each interaction intention indicator, or fine-tune the rules corresponding to rule-based online tasks based on the samples and negative samples that are adapted to each interaction intention indicator.

[0041] The following describes in detail the specific process of identifying positive and negative samples that are suitable for each interaction intention indicator based on the dimension of the interaction intention indicator.

[0042] In an embodiment of the present application, in response to a configuration operation, at least one configuration information adapted to a target service provided by a target application can be configured to facilitate identification of positive and negative samples based on the configuration information. One configuration information can be configuration information adapted to each interaction intent indicator included in the target service provided by the target application.

[0043] Among them, the configuration operation can be performed by the user based on the information provided by the target application to configure the page. The configuration operation can be manual configuration or voice control configuration, etc. Manual configuration means that the user manually fills in the configuration information in the information item or manually selects the appropriate configuration information from the drop-down list associated with each information item. Voice control operation means that the user uses voice control to fill in the configuration information in the information item or uses voice control to select the appropriate configuration information from the drop-down list associated with each information item.

[0044] Among them, each configuration information includes an interaction intention indicator, and the interaction intention indicator corresponds to positive and negative sample recognition rules required for positive and negative sample recognition based on rules. Usually, each configuration information contains one interaction intention indicator, and the target service may contain one or more interaction intention indicators. The positive and negative sample recognition rules required for positive and negative sample recognition based on rules include positive sample recognition rules and negative sample recognition rules. The positive sample recognition rules define the recognition criteria for positive samples to be used to guide the positive sample recognition process of positive and negative sample recognition based on rules, and the negative sample recognition rules define the recognition criteria for negative samples to be used to guide the negative sample recognition process of positive and negative sample recognition based on rules.

[0045] Optionally, the positive and negative sample recognition rules can be defined by regular expressions (Regular Expression, abbreviated as Regex) to accurately identify positive and negative samples. Regular expressions are a powerful tool for string search and manipulation. They match text that conforms to a certain pattern by defining a series of rules, that is, they are used to define recognition rules and identify and extract text that conforms to a specific pattern. Generally speaking, regular expressions are tools for describing search patterns with special character combinations. For example, "*" represents any number of characters, and a*b can match any string starting with a and ending with b. Taking the "less phone calls intention" as an example, the regular expression can be used to define and identify the string "less phone calls / no phone calls", and the sample containing this string is determined as a positive sample, and the sample not containing this string is determined as a negative sample.

[0046] In the embodiments of the present application, the interaction intention indicators also correspond to target prompt words and an intention knowledge base required for positive and negative sample recognition based on a sample recognition model. The intention knowledge base contains knowledge information related to each intention indicator, and the specific meaning of each intention indicator can be learned through the knowledge information related to the intention indicator. Taking the "few calls intention" as an example, the knowledge information related to the "few calls intention" can be the definition of "few or no calls" and examples of "few calls". The definition of "few or no calls" can be, for example, "few calls means that the number of calls received from the user is less than a set number, and no calls means that no calls from the user are received", and an example of "few calls" can be "the merchant receives less than N calls". The target prompt words include at least one or a combination of more than one of the following: the first prompt word, the second prompt word, the third prompt word, and the fourth prompt word. Among them, the first prompt word is the task prompt word, which is used to prompt the specific task content of the sample recognition model throughout the process of positive and negative sample recognition. The first prompt word can be, for example, "Combined with business knowledge and / or intention indicator knowledge, based on the interaction information between the merchant and the customer service, determine whether the interaction information contains the intention of 'few / no calls'". The second prompt word is the model role prompt word, which is used to prompt the role played by the sample recognition model in the process of positive and negative sample recognition and the specific task content of this role. The second prompt word can be, for example, "You are an intention recognition expert, and you need to combine the interaction information between the merchant and the customer service (the merchant's current problem and historical conversation information) to determine whether the interaction information contains the intention of 'few / or no calls'". The third prompt word is the positive sample recognition rule, which is used to guide the sample recognition model to determine that the interaction information is a positive sample when it recognizes that the interaction information contains the intention of 'few / no calls'. The third prompt word can be, for example, "When the interaction information contains the keyword 'few / no calls', the interaction information is a positive sample". The fourth prompt word is the negative sample prompt word, which is used to guide the sample recognition model to determine that the interaction information is a negative sample when it recognizes that the interaction information does not contain the intention of 'few / no calls'. The fourth prompt word can be, for example, "When the interaction information contains the keyword 'few / no calls', the interaction information is a negative sample".

[0047] In an alternative embodiment, in response to a configuration operation, at least one configuration information adapted to the target service provided by the target application is configured, including: for each configuration information, a configuration interface is displayed, and the configuration interface includes: an interaction intention metric corresponding to the configuration information, and an information configuration item corresponding to the interaction intention metric, where the information configuration item includes a positive and negative sample recognition rule configuration item, a sample recognition model configuration item, a prompt word configuration item, and an intention knowledge base configuration item; in response to an edit operation on the positive and negative sample recognition rule configuration item, the positive sample recognition rule and the negative sample recognition rule required for positive and negative sample recognition based on the rule are obtained; in response to a selection operation on the sample recognition model configuration item, the sample recognition model corresponding to the interaction intention metric is obtained; in response to an edit operation on the prompt word configuration item, the target prompt word required for positive and negative sample recognition based on the sample recognition model is obtained; in response to an edit operation on the intention knowledge base configuration item, the intention knowledge base required for positive and negative sample recognition based on the sample recognition model is obtained. In this embodiment, the configuration interface for each configuration information and the interaction intention metric corresponding to each configuration information are preset. When the configuration interface for each configuration information is displayed, the interaction intention metric included in the configuration information and the information configuration item corresponding to the interaction intention metric are displayed simultaneously.

[0048] In another alternative embodiment, in response to a configuration operation, at least one configuration information adapted to the target service provided by the target application is configured, including: for each configuration information, a configuration interface is displayed, and the configuration interface includes: an intention metric configuration item; in response to a configuration operation on the intention metric configuration item, the interaction intention metric included in the configuration information is determined; an information configuration item is displayed on the configuration interface, and the information configuration item includes a positive and negative sample recognition rule configuration item, a sample recognition model configuration item, a prompt word configuration item, and an intention knowledge base configuration item; in response to an edit operation on the positive and negative sample recognition rule configuration item, the positive sample recognition rule and the negative sample recognition rule required for positive and negative sample recognition based on the rule are obtained; in response to a selection operation on the sample recognition model configuration item, the sample recognition model corresponding to the interaction intention metric is obtained; in response to an edit operation on the prompt word configuration item, the target prompt word required for positive and negative sample recognition based on the sample recognition model is obtained; in response to an edit operation on the intention knowledge base configuration item, the intention knowledge base required for positive and negative sample recognition based on the sample recognition model is obtained. In this embodiment, the configuration interface for each configuration information is preset, and the interaction intention corresponding to the configuration information is not preset. The user needs to configure the intention metric information in real time on the configuration interface to determine the intention metric configuration item. Additionally, since the information configuration items included in the configuration page corresponding to each configuration information may be different, when the configuration interface is displayed for each configuration information, only the intention metric configuration item may be displayed on the configuration page. After configuring the intention metric through the intention metric configuration item, the information configuration item corresponding to the intention metric is displayed.

[0049] In the embodiments of the present application, it is also necessary to collect a sample set, and the sample set can be a sample set corresponding to an interaction intention index. Optionally, there is a sample set query command line corresponding to the interaction intention index, and through this query command line, a sample set related to the interaction intention can be searched for and collected from the data source. Taking the sample set query command line as a JSONPath expression as an example, JSONPath is a query language for searching for and collecting specific elements in JSON data. Suppose there is the following JSON data: {"name":"John","age":"30"}, then using the JSONPath expression $.extend.intentions, "John" can be extracted. Another example is in the field of interaction. The sample set includes multiple interaction information generated during at least one interaction process between a user and a target application. Suppose there is the following JSON data:

[0050]

[0051] Based on the above JSON data, using the JSONPath expression $.extend.intentions, "merchant": "Why do I have so few calls?", "customer service": "Hello, manager. I see that you just started promotion not long ago, and the number of calls is a bit less. It is recommended that you refresh the post to improve the ranking and add some keywords to increase the exposure. Currently, the background traffic exposure is still very good. You can continue to observe for a period of time." can be extracted. It should be noted that this question-and-answer pair can be regarded as a sample.

[0052] Furthermore, after collecting the sample set corresponding to the interaction intention index, the sample set of the interaction intention index can be subjected to positive and negative sample identification processing. The positive and negative sample identification processing of the sample set of the interaction intention index can be initiated through a control on the interaction page. For the specific process of the positive and negative sample identification processing initiated through the control on the interaction interface, reference can be made to the relevant descriptions in the following scenario embodiments, which will not be elaborated here for the time being.

[0053] In the embodiments of the present application, the sample set corresponding to the interaction intention index can be subjected to positive and negative sample identification through two positive and negative sample identification methods. The two positive and negative sample identification methods include: positive and negative sample identification based on rules and positive and negative sample identification based on a sample identification model. The following will elaborate on the two sample identification methods respectively.

[0054] Method 1: Positive and negative sample identification based on rules

[0055] In the embodiments of the present application, the positive sample recognition rule in the positive and negative sample recognition rules is used to describe the keywords in the interaction intention index, and the negative sample recognition rule in the positive and negative sample recognition rules is used to describe the keywords that do not include those in the interaction intention index. Then, based on the positive and negative sample recognition rules and the interaction intention index, rule-based positive and negative sample recognition processing can be performed on each interaction information to obtain the first recognition result of each interaction information.

[0056] In some embodiments, performing rule-based positive and negative sample recognition processing on each interaction information according to the positive and negative sample recognition rules and the interaction intention index to obtain the first recognition result of each interaction information includes: according to the positive sample recognition rule, determining whether each interaction information contains the keywords in the interaction intention index. If it does, determining that the interaction information is a positive sample; according to the negative sample recognition rule, determining whether each interaction information does not contain the keywords in the interaction intention index. If it does not, determining that the interaction information is a negative sample. Distinguishing between the positive sample recognition rule and the negative sample recognition rule and respectively recognizing each interaction information based on the positive sample recognition rule and the negative sample recognition rule to judge the category of each interaction information can improve the accuracy of recognizing the category of interaction information.

[0057] Optionally, determining whether each interaction information contains the keywords in the interaction intention index according to the positive sample recognition rule includes: according to the positive sample recognition rule, determining the keywords in the interaction intention index; performing a word segmentation operation on each interaction information to obtain multiple word segments included in the interaction information; under the guidance of the positive sample recognition rule, determining whether the multiple word segments contain the keywords in the interaction intention index. By combining the word segmentation operation with the positive sample recognition rule to determine whether the multiple word segments included in each interaction information contain the keywords in the interaction intention index, the accuracy of determining the keywords is improved.

[0058] In an optional embodiment, according to the positive sample recognition rule, determining the keywords in the interaction intention index includes: performing a word segmentation operation on each interaction intention index to obtain multiple word segments included in the interaction intention index; and performing a word segmentation operation on the positive sample recognition rule to obtain multiple word segments included in the positive sample recognition rule; matching the multiple word segments included in the interaction intention index with the multiple word segments included in the positive sample recognition rule to obtain at least one pair of successfully matched word segments; extracting the semantic features of the at least one pair of successfully matched word segments and extracting the semantic features of the interaction intention index, and taking any one of the word segments in the word segment pair whose semantic feature distance from the semantic feature of the interaction intention index is less than the preset distance as the keyword. By performing word segmentation operations on the positive sample recognition rule and the interaction intention index respectively, and selecting the word segment that is close to the semantic feature of the interaction intention index from the at least one pair of successfully matched word segments as the keyword in the interaction intention index, rather than comparing the semantic features of all word segments with the semantics of the interaction information index, it not only improves the accuracy of determining the keywords in the interaction intention index, but also improves the efficiency of determining the keywords in the interaction intention index.

[0059] Among them, word segmentation is an important step in Natural Language Processing (NLP), and its purpose is to split continuous text strings into a sequence of meaningful words. The word segmentation operation can be implemented by rule-based methods. Rule-based word segmentation methods include the Forward Maximum Matching (MM) method and the Reverse Maximum Matching (RMM) method. Specifically, the implementation methods of the Forward Maximum Matching method and the Reverse Maximum Matching method are as follows: a dictionary is predefined, and the dictionary contains all possible words; further, the text is scanned from left to right by the Forward Maximum Matching method, and the longest matching word in the dictionary is used for word segmentation to obtain the word segmentation result; the text is scanned from right to left by the Reverse Maximum Matching method, and the longest matching word in the dictionary is used for word segmentation to obtain the word segmentation result. It should be noted that the Forward Maximum Matching method and the Reverse Maximum Matching method can be used alone or in combination, depending on the accuracy requirements of word segmentation and the application scenario. When the Forward Maximum Matching method and the Reverse Maximum Matching method are used alone, the specific implementation method of the Forward Maximum Matching method is as follows: scan the text of the interaction information from left to right and try to match the longest word each time. Its advantages are simple implementation and high speed. The specific implementation method of the Reverse Maximum Matching method is as follows: scan the text of the interaction information from right to left and try to match the longest word each time. Since there are many modifier-head structures in Chinese, the Reverse Maximum Matching method is more accurate than the Forward Maximum Matching method in some cases. For example, for "master postgraduate production", the Reverse Maximum Matching method can be correctly segmented into "master / research / production", while the Forward Maximum Matching method may be segmented into "master postgraduate / production". When the Forward Maximum Matching method and the Reverse Maximum Matching method are used in combination (Bidirectional Maximum Matching, Bi-MM), the Bidirectional Maximum Matching method uses both the Forward Maximum Matching method and the Reverse Maximum Matching method at the same time, and then compares the results of the two methods. The specific selection rule is as follows: if the number of words in the word segmentation results of the two methods is different, select the one with fewer words; if the number of words is the same, select the one with fewer single-character words; if it is still impossible to distinguish, choose any one. The Bidirectional Maximum Matching method can reduce the error of single-direction matching and improve the accuracy of word segmentation. [[ID=||1]] [[ID=||2]]

[0060] In addition, the word segmentation operation also includes statistical-based methods, deep learning-based methods, and other word segmentation methods. Among them, the rule-based word segmentation method relies on predefined dictionaries and word segmentation rules for word segmentation. The statistical-based word segmentation method uses statistical information in a large-scale corpus to determine the boundaries of words. Common models include Hidden Markov Model (HMM), Conditional Random Field (CRF), etc. The deep learning-based word segmentation method uses neural network models, such as Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Transformer, etc. Among them, the implementation method of the statistical-based word segmentation method includes: a trained statistical model, such as HMM or CRF, is integrated in the sample recognition model, and the trained statistical model is used to segment the interaction information contained in the sample set to obtain the word segmentation result. The training process of the statistical model is as follows: preprocess the interaction information, label the sample as the word segmentation result for training the model; extract features in the text of the sample, such as the context information of characters; use the labeled sample to train the statistical model, such as HMM or CRF. The implementation process of the deep learning-based word segmentation method includes: a trained neural network model, such as LSTM or Transformer, is integrated in the sample recognition model; the neural network model is used to segment the interaction information to obtain the word segmentation result. The training process of the neural network model includes: constructing a large-scale labeled sample; selecting a suitable neural network architecture; using the labeled sample to train the model and optimize the model parameters.

[0061] In another alternative embodiment, the sample recognition rule includes keywords in the interaction intention information index, and the sample recognition rule also has a preset data structure, and the keywords are located at a specified position in the data structure. Then, according to the positive sample recognition rule, determining the keywords in the interaction intention index includes: obtaining the keywords from the specified position in the data structure according to the data structure of the positive sample recognition rule.

[0062] In yet another alternative embodiment, the keywords can also be directly determined from the interaction intention index. Specifically, the interaction intention index has a preset data structure, and the keywords are stored at a specified position in the data structure, then the keywords can be directly obtained from the specified position corresponding to the data structure of the interaction intention index. For example, the interaction intention index is "less phone calls intention", and the data structure of "less phone calls intention" is "'less phone calls' (located at the specified position) + 'intention'", then the keyword "less phone calls" can be directly obtained from the specified position.

[0063] Further optionally, under the guidance of the positive sample recognition rule, it is determined whether the multiple word segments contain the keywords in the interaction intention index, including: calculating the semantic feature distance between the semantic features of the keywords in the interaction intention index described according to the positive sample recognition rule and the semantic features corresponding to the multiple word segments included in each interaction intention information. The semantic feature distance represents the semantic similarity. The smaller the semantic feature distance, the higher the semantic similarity. On the contrary, the smaller the semantic similarity. The word segments with a semantic feature distance smaller than the preset distance are used as the word segments similar to or the same as the keywords included in the interaction intention index, then it is determined that the multiple word segments contain the keywords in the interaction intention index. On the contrary, it is determined whether the multiple word segments contain the keywords in the interaction intention index.

[0064] Optionally, according to the negative sample recognition rule, it is determined whether each interaction information does not contain the keywords in the interaction intention index, including: determining the keywords in the interaction intention index according to the negative sample recognition rule; performing a word segmentation operation on each interaction information to obtain multiple word segments included in the interaction information; under the guidance of the negative sample recognition rule, it is determined whether the multiple word segments do not contain the keywords in the interaction intention index. By combining the word segmentation operation with the negative sample recognition rule, it is determined whether the multiple word segments included in each interaction information do not contain the keywords in the interaction intention index, which improves the accuracy of determining the keywords. For the specific implementation manners of the steps in this embodiment, reference may be made to the relevant descriptions of determining whether each interaction information contains the keywords in the interaction intention index according to the positive sample recognition rule, which will not be elaborated here.

[0065] Method 2: Positive and negative sample recognition based on a sample recognition model

[0066] In the embodiments of the present application, the target prompt word, the interaction intention index, and each interaction information may be input into the sample type recognition model. Under the guidance of the target prompt word, in combination with the intention knowledge base, it is recognized whether each interaction information is adapted to the interaction intention index to obtain the second recognition result of each interaction information.

[0067] In some embodiments, the sample type recognition model includes a knowledge learning network layer and a classification recognition network layer. Then, the target prompt word, the interaction intention index, and each interaction information are input into the sample type recognition model. Under the guidance of the target prompt word, in combination with the intention knowledge base, it is recognized whether each interaction information is adapted to the interaction intention index to obtain the second classification recognition result of each interaction information, including: using the knowledge learning network layer to learn the knowledge meanings related to each intention index included in the intention knowledge base; inputting the learned knowledge meanings related to each intention index, each interaction information, and the target prompt word into the classification recognition network layer. Under the guidance of the knowledge meanings related to each intention index and the target prompt word, it is recognized whether each interaction information is adapted to the interaction intention index to obtain the second classification recognition result of each interaction information.

[0068] Optionally, the knowledge learning network layer is used to learn the knowledge meanings related to each interaction intention index included in the intention knowledge base, including: extracting the semantic features of each knowledge in the knowledge related to each interaction intention index, and the semantic features can be in the form of vectors or matrices. Further optionally, encoding operations can be performed on each knowledge in the knowledge related to each interaction intention index to obtain the semantic features of each knowledge. For example, the word embedding technology is used to convert each knowledge related to each interaction intention index into a vector representation, and a knowledge graph of each interaction intention index is constructed, that is, the entities, attributes, and relationships of each knowledge in the knowledge related to each interaction intention index are determined, and the entities, attributes, and relationships are encoded as fact triples (v, r, v'), where v and v' are entities and r is a relationship. Among them, an entity refers to a thing with an independent meaning of existence, such as a person, a place, an organization, etc.; entity encoding can adopt methods such as vector representation and matrix representation. For example, in natural language processing, an entity can be converted into a word embedding vector, and a high-dimensional vector can be generated to represent the entity. An attribute is a data item used to describe the characteristics of an entity; in addition, each entity has one or more attributes. For example, the attributes of a student include student ID, name, gender, etc. Attribute encoding can convert the attribute value into a numerical or vector form. Gender can be encoded as 0 and 1 (0 for male and 1 for female), age can be directly represented by a numerical value, and text attributes (such as name) can be converted into vectors using the word embedding technology. The mutual relationship between entities refers to various connections and interactions existing between different entities. For example, the "elective" relationship between "students" and "courses". In an entity-relationship (ER) diagram, the relationship is usually represented by a diamond and connects the relevant entities; relationship encoding can adopt matrix representation or graph representation. For example, an adjacency matrix can be used to represent the relationship between entities, and the elements in the matrix indicate whether there is a relationship between entities and the strength of the relationship. In the graph representation, entities are used as nodes and relationships are used as edges, and the weight of the edge can represent the strength of the relationship. The knowledge graph provides rich semantic features for the model to help the model understand the knowledge meanings related to the intention index.

[0069] Further optionally, the sample recognition model further includes: a semantic feature extraction network layer, and the semantic feature extraction network layer is located between the knowledge learning network layer and the classification recognition network layer. Then, each interaction information and the target prompt word are input into the semantic feature extraction network layer, and the first prompt word (task prompt word), the second prompt word (model role), the third prompt word (positive sample recognition rule), the fourth prompt word (negative sample recognition rule) are encoded and each interaction information is encoded to obtain the semantic features (semantic feature vectors) of each interaction information and each prompt word in the same vector space as the semantic features of the knowledge related to each intention index.

[0070] Further optionally, the semantic feature extraction network layer can be a BERT (Bidirectional Encoder Representations from Transformers) model. Since the BERT model (BERT layer) can only process a limited text length at a time, in order to improve the semantic feature extraction efficiency of the semantic feature extraction layer, the model structure of the exemplary sample recognition model is as follows Figure 2a shown. The sample recognition model contains multiple BERT layers. When each interaction information and each prompt word are input into the semantic feature extraction network layer, if the length of the text content corresponding to each interaction information and each prompt word is greater than the text length threshold supported by each BERT layer (the text length thresholds supported by each BERT layer can be the same or different), each text content can be split into at least two text segments, and the length of each text segment is not greater than the text length threshold; further, the at least two text segments are input into at least two semantic feature extraction layers for encoding, that is, for semantic feature extraction, to obtain at least two semantic features corresponding to each text content. The at least two semantic feature extraction layers are the semantic feature extraction layers selected from multiple semantic feature extraction layers and having the same number as the at least two text segments; further, according to the at least two semantic features, a target semantic feature corresponding to each text content is generated to obtain the semantic feature corresponding to each interaction information and the semantic feature corresponding to each prompt word. In this embodiment, after obtaining the at least two text segments, video memory is allocated to the same number of semantic feature extraction layers according to the number of text segments to support their operation, which can not only improve the semantic feature extraction efficiency but also save video memory resources. It should be noted that when the text lengths of each text segment are different, it is necessary to adaptively select the semantic feature extraction layer that supports the text length of each text segment and allocate video memory to it to support its operation.

[0071] Optionally, each text segment contains multiple word segments; inputting the at least two text segments into at least two semantic feature extraction layers for semantic feature extraction to obtain at least two semantic features includes: performing a special word segment addition operation on each text segment to obtain at least two text segments containing special word segments; inputting each text segment into a semantic feature extraction layer with allocated video memory respectively, and based on the self-attention learning mechanism, learning the semantic features of each word segment in the text segment and the semantic association relationship between each word segment to obtain the target semantic feature corresponding to the special character as the semantic feature of the text segment, so as to obtain at least two semantic features.

[0072] Among them, a special word segmentation addition operation is performed on each text segment to obtain at least two text segments containing special word segmentation. Optional implementation methods include: selecting special characters, and the selection of special characters depends on the specific task and model architecture. Common special characters include <cls>(Classification) and <sep>(Separation), for example, in the BERT model, <cls>The tag is added to the beginning of the sentence, <sep>The tag is added to the end of the sentence; Tokenization: The text content is segmented into multiple tokens, which can be words, sub-words or characters; The tokenizer converts the text content into a sequence of tokens according to predefined rules, and the sequence of tokens contains multiple tokens in the text content; Adding special characters: Adding special characters to the sequence of tokens. For example, if processing a single sentence, <cls>will be added to the beginning of the sequence, <sep>will be added to the end of the sequence; Encoding converts multiple word segments (including special characters) into a numerical representation (in digital form) that the model can understand. Specifically, it can find the numerical representation of each word segment in a vocabulary (containing each word segment and its corresponding numerical mapping relationship) to input the numerical representation corresponding to each word segment into the model. The numerical representation corresponding to each word segment can characterize the initial semantic information of the word segment. The numerical representation corresponding to each word segment can be implemented in the form of a matrix, and the matrix contains the numerical representation corresponding to each word segment. Additionally, to enable the model to understand the positions of multiple word segments in the word segment sequence, positional encoding can also be added to each word segment in the word segment sequence, which is beneficial for the model to capture the sequential information in the sequence data.

[0073] Among them, each text segment containing special characters is respectively input into a semantic feature extraction layer. Based on the self-attention learning mechanism, it learns the semantic features of each word segment and the semantic association relationship between each word segment to obtain the semantic features of the text segment corresponding to each special character. Optional implementation methods include: performing vector mapping on each word segment in each text segment to obtain the query vector, key vector, and value vector corresponding to each word segment in the text segment; for each word segment in the text segment, calculating the matching degree between the query vector of the word segment and the key vectors of each word segment to obtain the attention weight of each word segment in the text segment; performing a fully connected calculation operation on the value vectors of each word segment in the text segment according to the attention weight of each word segment in the text segment to obtain at least two semantic features.

[0074] Optionally, input the numerical representation corresponding to the word segment sequence containing special characters into the semantic feature extraction network layer, and perform vector mapping on the numerical representation corresponding to each word segment in the target training sample to obtain the query vector, key vector, and value vector corresponding to each word segment, including: mapping the numerical representation of each word segment in the word segment sequence to the query vector, key vector, and value vector respectively through different weight matrices, such as mapping to the query vector, key vector, and value vector through three different weight matrices W^Q, W^K, and W^V respectively.

[0075] Optionally, for each word segment, calculate the matching degree between the query vector of the word segment and the key vectors of each word segment to obtain the attention weight of each word segment, including: for the current word segment (each word segment), perform a dot product calculation between the query vector of the word segment and the key vectors of each word segment (including the current word segment) to obtain multiple score values, and each score value represents the attention weight of the word segment providing the key vector for the current word segment. The attention weight reflects the relative importance of one word segment to another in the sequence. The higher the weight, the more important the corresponding word segment is in the current context.

[0076] Further optionally, the multiple fractional values may include positive fractions and negative fractions. To facilitate subsequent fully connected calculations, each fractional value can be normalized to obtain multiple normalized values, and all the normalized values are positive. The normalization process here can be an exponential calculation operation (with the fractional value as the parameter for the exponential calculation), but is not limited thereto.

[0077] Further optionally, after obtaining the attention weights of each token, before performing the pooling process, a weighted sum calculation can also be performed on the value vectors of each token according to the attention weights of each token to obtain a weighted sum vector (updated representation). The weighted sum vector combines the tokens at all positions in the input sequence, but the contribution degree of tokens at different positions is determined by their attention to the query. This means that for each token in the token sequence, its weighted sum vector is a weighted combination of the information of other tokens associated with it in the entire token sequence, and the weight size is determined by the attention scores (normalized). Through this process, the weighted sum vector of each token not only represents the semantic information of the token itself, but represents the comprehensive context semantic information of the entire token sequence, which enables the model to capture long-range dependencies within the sequence and consider the semantic information of the entire sequence when processing the current token.

[0078] Further optionally, the sample recognition model further includes a fully connected layer, and the fully connected layer is located after the feature extraction network layer. Then, after obtaining the weighted sum vector of each token, before performing the pooling process, a fully connected calculation operation can also be performed on the value vectors of each token according to the attention weights of each token to obtain the target semantic feature corresponding to the special character as the semantic feature of the corresponding text, including: inputting the value vector obtained by the weighted sum into the fully connected layer, and the fully connected layer includes two linear transformation functions and a non-linear activation function such as ReLU); the first linear transformation function maps the value vector to an intermediate dimension, then applies the non-linear activation function for non-linear calculation, and finally the second linear transformation function maps the non-linear calculation result back to the original dimension or the target dimension to output the semantic information of the text content corresponding to the special character. The semantic information of the text content integrates the context information of the entire input token sequence and can be used as the semantic information of the text content.

[0079] Further optionally, the sample recognition model further includes a pooling layer, and the pooling layer is located between the fully connected network layer and the classification recognition network layer. Then, after obtaining at least two semantic features, according to the at least two semantic features, a target semantic feature corresponding to the text content is generated, including: using the pooling layer to perform pooling processing on the at least two semantic features to obtain the target semantic feature corresponding to the text content.

[0080] In some embodiments, the pooling layer has a pooling window, and the window size of the pooling window is equal to the value of the sliding step of the pooling window; generating a target semantic feature corresponding to the text content according to at least two semantic features, including: inputting at least two semantic features into the pooling layer, performing semantic splicing on the at least two semantic features to obtain a spliced semantic feature, and the spliced semantic feature is a multi-dimensional parameter vector; based on the window size and the sliding step, for the multi-dimensional parameter vector, select the maximum parameter value as the output within the area covered by each pooling window to obtain the target semantic feature corresponding to the text content; or, based on the window size and the sliding step, for the multi-dimensional parameter vector, calculate the average value of all parameter values as the output within the area covered by each pooling window to obtain the target semantic feature corresponding to the text content; or, based on the window size and the sliding step, for the multi-dimensional parameter vector, calculate the sum of all parameter values as the output within the area covered by each pooling window to obtain the target semantic feature corresponding to each text content.

[0081] It should be noted that the extraction process of the semantic features in each embodiment of the present application can refer to the above implementation manner of extracting the target semantic features corresponding to each text content.

[0082] Optionally, input the learned knowledge meanings related to each intention index, each interaction information, and the target prompt word into the classification and recognition network layer. Under the guidance of the knowledge meanings related to each intention index and the target prompt word, identify whether each interaction information is adapted to the interaction intention index to obtain the second classification and recognition result of each interaction information, including: input the learned knowledge meanings related to each intention index, each interaction information, and the target semantic feature corresponding to the target prompt word into the classification and recognition network layer, splice or fuse the target semantic features corresponding to each interaction information and the target prompt word into a comprehensive semantic feature (semantic feature vector), such as using methods such as concatenation or weighted average to obtain a comprehensive vector; map the comprehensive vector to the number of output categories to obtain the second classification and recognition result of each interaction information.

[0083] For example, the linear function self.linear = nn.Linear(input_size, num_classes) is used to map the comprehensive input vector to the number of output result categories (in vector form). input_size is the dimension of the input feature vector, num_classes is the number of output categories, and nn.Linear refers to a linear transformation. Its mathematical expression is y = Wx + b, where W is the weight matrix and b is the bias vector. The softmax function is used to convert the number of output categories (in vector form) into corresponding probability values. Each probability value represents the probability that each interaction information belongs to each result category. Taking the classification recognition results of "yes (0)" and "no (1)" as an example, a probability threshold (50%) can be set. When the probability is greater than 50%, the classification recognition result is "yes (0)"; when the probability is less than 50%, the classification recognition result is "no (1)". The probability results can be expressed in the following way:

[0084] The first interaction information has a 70% probability of belonging to category 0 (yes) and a 30% probability of belonging to category 1 (no);

[0085] The second interaction information has a 20% probability of belonging to category 0 (yes) and an 80% probability of belonging to category 1 (no);

[0086] The third interaction information has a 60% probability of belonging to category 0 (yes) and a 40% probability of belonging to category 1 (no).

[0087] It should be noted that this embodiment does not limit the recognition order of positive and negative samples based on rules and based on a sample recognition model. For example, positive and negative samples can be recognized based on rules first and then based on the sample recognition model. Another example is that positive and negative samples can be recognized based on the sample recognition model first and then based on rules. Another example is that the recognition of positive and negative samples based on rules and the recognition of positive and negative samples based on the sample recognition model are carried out simultaneously. The simultaneous progress of the two recognition processes can improve the recognition efficiency of positive and negative samples.

[0088] Furthermore, after obtaining the first recognition result and the second recognition result of multiple interaction information, the interaction information with consistent first recognition result and second recognition result among the multiple interaction information is used as the first interaction information. The first interaction information is a true positive sample or a true negative sample. A true positive sample means that both the first recognition result and the second recognition result of the same interaction information are positive samples, and a true negative sample means that both the first recognition result and the second recognition result of the same interaction information are negative samples.

[0089] In this embodiment, there may be a situation where the first recognition result and the second recognition result of the same interaction information are inconsistent. In the case where the first recognition result and the second recognition result are inconsistent, the second recognition result of the sample recognition model can be used as the actual recognition result. That is to say, the second recognition result of the sample recognition model is used to verify the first recognition result obtained by sample recognition based on rules. Further, in order to avoid errors in the second recognition result based on the sample recognition model, the samples can also be manually recognized, so that the manual recognition result is used as the actual recognition result, which plays a role in manually verifying and rechecking the sample recognition result. Based on this, the interaction information in which the first recognition result and the second recognition result are inconsistent among multiple interaction information is used as the second interaction information. For the second interaction information, manual classification recognition is performed on the second interaction information to obtain the third recognition result of the second interaction information. The third recognition result indicates that the second interaction information is a positive sample that matches the interaction intention index or a negative sample that does not match the interaction intention index. In the case where the first recognition result and the second recognition result are inconsistent, manual classification recognition is performed on this interaction information to obtain the actual recognition result. The manual recognition accuracy is high, and the manual classification recognition result is used as the actual correct recognition result, which solves the problem of being unable to determine the correct recognition result corresponding to the interaction information when the first recognition result and the second recognition result are inconsistent.

[0090] Further, in order to improve the accuracy of the positive and negative sample recognition rules required for positive and negative sample recognition based on rules and the positive and negative sample recognition based on the sample recognition model, the positive and negative sample recognition rules or target prompt words in the configuration information can be optimized and adjusted. Specifically, based on the first recognition result, the second recognition result, and the third recognition result of the second interaction information, it can be determined whether the second interaction information is a false positive sample or a false negative sample; according to the false positive sample or the false negative sample, the positive and negative sample recognition rules or target prompt words in the configuration information are optimized and adjusted. Among them, a false positive sample refers to a sample whose second recognition result of the sample category recognition model is a positive sample while the third recognition result of manual recognition is a negative sample, and a false negative sample refers to a sample whose second recognition result of the sample category recognition model is a negative sample while the third recognition result of manual recognition is a positive sample.

[0091] Optionally, determining that the second interaction information is a false positive sample or a false negative sample based on the first recognition result, the second recognition result, and the third recognition result of the second interaction information includes: when the first recognition result of the second interaction information is a positive sample, and the second recognition result and the third recognition result of the second interaction information are the same and both are negative samples, then determine that the second interaction information is a false positive sample; when the first recognition result of the second interaction information is a negative sample, and the second recognition result and the third recognition result of the second interaction information are different and the third recognition result is a positive sample, then determine that the second interaction information is a false negative sample. That is to say, when the second recognition result and the third recognition result are the same, the second recognition result and the third recognition result are both the actual recognition results corresponding to the sample; when the second recognition result and the third recognition result are different, the third recognition result is the actual recognition result corresponding to the sample, and the accuracy of the first recognition result is determined based on the third recognition result.

[0092] Further optionally, optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information according to the false positive sample or the false negative sample, including: calculating the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate (i.e., the F1 score) for classifying and recognizing the sample set according to the respective quantities of true positive samples, true negative samples, false positive samples, and false negative samples included in the sample set; optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information based on the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and recognizing the sample set. Among them, the value range of the F1 score is [0,1]. The closer it is to 1, the better the performance of recognition based on the sample recognition model and recognition based on the rules. At the same time, considering the balance between the precision rate and the recall rate of recognition based on the sample recognition model and recognition based on the rules, the F1 score is very important for binary classification problems. Especially when it is desired to achieve a balance between precision and recall, a high F1 score means that a good balance has been achieved between the precision rate and the recall rate of recognition based on the sample recognition model and recognition based on the rules.

[0093] Further optionally, according to the respective quantities of true samples, true negative samples, false positive samples, and false negative samples included in the sample set, calculate the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and identifying the sample set, including: taking the sum of the number of true samples and the number of false positive samples as the numerator, and the sum of the number of true samples, the number of true negative samples, the number of false positive samples, and the number of false negative samples as the denominator, performing a quotient operation to obtain the accuracy rate; taking the number of true samples as the numerator and the sum of the number of true samples and the number of false negative samples as the denominator, performing a quotient operation to obtain the precision rate; taking the number of true samples as the numerator and the sum of the number of true samples and the number of true negative samples as the denominator, performing a quotient operation to obtain the recall rate; taking twice the product of the precision rate and the recall rate as the numerator and the sum of the precision rate and the recall rate as the denominator, performing a quotient operation to obtain the harmonic mean.

[0094] It can be understood that optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information according to false positive samples or false negative samples is actually optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information according to the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and identifying the sample set. That is, the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate can be used as the loss function for classification and identification. When the loss function does not meet the loss condition, optimize and adjust the positive and negative sample recognition rules or target prompt words in the configuration information until the loss function meets the loss condition.

[0095] It should be noted that if the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate are used as separate loss functions, then when optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information, each loss function needs to meet its respective corresponding loss condition. Or, a comprehensive function of the accuracy rate, precision rate, recall rate, and the harmonic mean between the precision rate and the recall rate can be calculated through calculation methods such as weighted summation, and the comprehensive function is used as the loss function. Then, when optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information, meet the loss condition of this loss function.

[0096] In the embodiment of the present application, through configuration operations, interaction intention metrics adapted to the target service provided by the target application, positive and negative sample recognition rules required for positive and negative sample recognition based on rules, and target prompt words and intention knowledge bases required for positive and negative sample recognition based on a sample recognition model can be configured, so as to use the configuration information to guide the subsequent positive and negative sample type recognition process, thereby improving the accuracy of positive and negative sample recognition. Further, under the guidance of the configuration information, combining rule-based sample recognition processing and sample recognition model-based recognition processing, sample recognition is performed on multiple interaction information generated during at least one interaction included in the sample set, and a first recognition result and a second recognition result are obtained, so as to determine whether each interaction information is a positive sample or a negative sample based on the first recognition result and the second recognition result. Determining the positive and negative sample types through the recognition results of the two methods further improves the accuracy of the system's sample type recognition, reduces the use of manual labor, and ensures the balance between the performance and efficiency of positive and negative sample type recognition.

[0097] The execution process of each step in the above embodiments of the present application can be page-controlled through corresponding pages. For this, in order to facilitate the understanding of the above technical solutions, the above technical solutions will be described below in conjunction with the exemplary schematic diagrams of each page.

[0098] First, in combination with Figure 2b and Figure 2c the overall functional structure and technical structure of the present application will be described. Figure 2b This is a schematic diagram of the exemplary functional structure of this application, including an offline task layer, a display layer, a control layer, and a data layer. Among them, the offline tasks include: service source data (business data source (Hive table)), SQL process for processing the Hive table, dp Hive2es scheduled task, and storing multi-service real samples in Elasticsearch (ES). That is, it involves extracting data from the service data source (Hive table), processing it through SQL, and then synchronizing the processed data to ES through a scheduled task, and finally using it for storing and querying real samples of multiple services (businesses). Among them, in the service (business) source data (Hive table), Hive is a data warehouse tool based on Hadoop, used for storing and processing large-scale structured data; business source data is usually stored in Hive in the form of tables; the Hive table structure defines the schema of the data, including field names, data types, etc.; the data is stored in a distributed file system (Hadoop Distributed File System, HDFS) in the form of files, usually saved in columnar storage formats such as Parquet and ORC to optimize the reading performance; business log data is loaded into the Hive table from the original data sources (such as log systems, databases, etc.) through the ETL (Extract, Transform, Load) process. The process of SQL processing the Hive table is to process the data in the Hive table, usually involving operations such as data cleaning, transformation, and aggregation. Specifically, Hive SQL: Use Hive SQL to query and process the data. Hive SQL is similar to the SQL of traditional relational databases but is optimized for large-scale data; data cleaning: Remove invalid data, duplicate data, and correct format errors, etc.; data transformation: Convert the data into a format suitable for subsequent processing, such as extracting specific fields, converting data types, etc.; data aggregation: Aggregate the data according to business requirements, such as statistical analysis by dimensions such as time and business type; store the result: Store the processed data in a new Hive table for subsequent tasks to use. The dp Hive2es scheduled task refers to triggering the data synchronization process through a scheduled task (such as a Cron Job) to synchronize the data in the Hive table to ES. Specifically, scheduled task scheduling: Use a Cron expression to define the execution frequency of the task (such as every hour, every day, etc.); data extraction: Extract the processed data from the Hive table. You can use the command-line tool of Hive (such as Hive-e) or through the JDBC interface of Hive; data transmission: Transmit the data to ES. You can use the Bulk API of ES for efficient data import; script implementation: Usually use Shell scripts or Python scripts to implement the entire synchronization process. Storing multi-business real samples in ES means storing the processed data in ES for real-time query and analysis.Specifically, for the ES cluster: Configure the ES cluster, including nodes, indexes, and shards; Index creation: Create indexes according to business requirements and define the mapping of the indexes (Mapping), including field names, data types, etc.; Data import: Use the Bulk API of ES to batch import data into the specified index; Real-time query: Utilize the query function of ES (such as the DSL query language) to query and analyze data in real time. The display layer includes: Configuration information management (metric management), configuration information editing (metric editing), evaluation home page, task list, task details, and result analysis display, etc. The control layer includes: Configuration information management (metric management), sample model identification (preliminary classification of large models), manual identification (manual annotation), and result analysis. The data layer includes: Service sample data (business sample data), multi-configuration information data (multi-metric data), sample identification model evaluation data (large model evaluation data), manual identification data (manual annotation data), and result data.

[0099] Figure 2c This is an exemplary technical architecture diagram of the present application, providing technical architecture support for each function in 2b. Among them, the front-end page corresponds to the display layer, and the front-end page realizes the content display of the display layer through technologies such as Vue, ElementUI, JavaScript, and Css. JavaWeb Service and RPC Service provide technical support for the execution process of the control layer. RPCService includes, but is not limited to, Base DAO, EntityInterface, and EntityService. The basic components provide data services for the data layer. The basic components include, but are not limited to: MySQL, JSON, multi-thread management, sample identification models, and MAVEN.

[0100] In this embodiment, an exemplary schematic diagram of the process of configuring at least one configuration information adapted to the target service provided by the target application in response to a configuration operation is as Figure 2d - 2e shown Figure 2d The selection page for the target service, which includes at least some of the services provided by the target application and the "edit metric" control corresponding to each service. Among them, at least some of the services can be, for example, intelligent customer service, intelligent customer service tag service, and enterprise WeChat order replacement recognition service, etc. In addition, the selection page of the target service also includes the interface name of the interface for each service to obtain data, the type to which each service belongs, the update time of each service, and other information. For each service, in response to the trigger operation on the "edit metric" control corresponding to the service, the configuration page of the service can be displayed. Taking the target service as the intelligent customer service (chat service) as an example, in response to the trigger operation on the "edit metric" control corresponding to the intelligent customer service, the configuration page corresponding to the intelligent service can be entered, and the configuration information of the service can be configured on the configuration page. An exemplary configuration page is as follows Figure 2e shown. The configuration page includes an interaction intention metric item, a "positive and negative sample recognition rule" item and a "sample set collection" item required for positive and negative sample recognition based on rules, which are used to configure the interaction intention metric, the positive and negative sample recognition rule, and the sample set collection rule respectively. The configuration page may also include a "new intention metric" control, a "related information corresponding to the new intention metric" control, and a "configuration save" control, etc. Based on the "new intention metric" control, a new intention metric can be added. Based on the "related information corresponding to the new intention metric" control, more information items related to the intention metric can be added. Based on the "configuration save" control, the configured configuration information can be saved to facilitate sample recognition based on the configured configuration information during the sample type recognition process.

[0101] In this embodiment, the sample set can be collected through the sample evaluation home page. Through this page, multiple samples related to the interaction intention metric to be evaluated can be queried and collected, that is, the sample set of this interaction intention can be collected through the sample evaluation home page of any interaction intention of the target service. An exemplary sample evaluation home page is as follows Figure 2f As shown, this page contains items such as "Evaluation Service", "Data Environment", "Start Date", "Start Time", "End Time", "Sample Screening (also known as Extended Query Screening)", "Query" control, and "Start Evaluation" control. Among them, the "Evaluation Service" item is used to configure the target service to be evaluated; the "Data Environment" item is used to configure the source of sample data (such as from ES - online data); the "Date" item is used to configure the date of evaluation; the "Start Time" item is used to configure the start time of evaluation; the "End Time" item is used to configure the end time of evaluation; the "Sample Screening" item is used to screen multiple samples associated with the interaction intent indicators to be evaluated from the database; the "Sample Screening" item corresponds to a sample query command line, such as the JSONPath expression in the above - mentioned embodiment; the sample query command line contains keywords related to the sample domain corresponding to the interaction intent indicator to query multiple samples containing the keyword, so as to identify positive and negative samples from multiple samples. In this figure, taking the interaction intent as "Less Phone Intent" and the target service as the intelligent customer service as an example, the keyword related to the sample domain corresponding to this interaction intent indicator is "phone". After the configuration of the evaluation service item, data environment item, start date item, start time item, end time item, and sample screening item is completed, in response to the triggering operation of the "Query" control, a sample set related to the intent indicator to be evaluated can be queried based on the above - configured information. The number of samples contained in the queried sample set related to "phone" is 2214. Among them, the sample set contains samples with the keyword "Less Phone / No Phone", and also contains samples with the keyword "phone" but without the keyword "Less Phone / No Phone".

[0102] Further, after collecting the sample set corresponding to the interaction intent indicator, in response to the triggering operation of the "Start Evaluation" control, the start evaluation page is displayed. An exemplary start evaluation page is as Figure 2g shown. In this figure, it contains: "Evaluation Task Name" item, "Intent Indicator Selection" item (also known as "Select Evaluation Indicator" item), "Requirement Association" item, and "OK" item. Among them, the "Evaluation Task Name" item is used to configure the name of the evaluation task, the "Intent Indicator Selection" item is used to configure the interaction intent indicator to be evaluated, and the "Requirement Association" item is used to configure whether this evaluation task is associated with other requirements. The configuration information required for the "Evaluation Task Name" item, "Intent Indicator Selection" item, and "Requirement Association" item can be set automatically in advance or set manually by the user in real - time. Further, in response to the triggering operation of the user on the "OK" control, the identification of positive and negative samples based on rules, the identification of positive and negative samples based on the sample identification model, and the evaluation of the identification results of the sample set are started. After the evaluation is completed, the evaluation task list page is displayed. A schematic diagram of an exemplary evaluation list page is as Figure 2h and Figure 2m As shown, the figure at least includes the "Query Details" control, "Query Statement" control, "Result Analysis" control, "View Analysis Results" control corresponding to the evaluation results of the sample set, and information details. The information details include, but are not limited to: sample set ID, task name, business name, associated requirements, total number of evaluations, number of completed evaluations, number of abnormal evaluations, and other information.

[0103] Further, in response to the trigger operation on the "View Details" control, a task details page is displayed. An exemplary task details page is as Figure 2i shown. The figure includes information such as the ID of each sample, automatic evaluation status, annotation status, etc., and the "View Evaluation Results" control corresponding to each sample. The automatic evaluation status at least includes two states: "Partial Interaction Intention Index Verification Passed" and "All Index Checks Failed". The samples with "Partial Interaction Intention Index Verification Passed" and "All Index Checks Failed" are those where the recognition results of the positive and negative sample recognition rules required for positive and negative sample recognition based on rules are inconsistent with the recognition results of positive and negative sample recognition based on the sample recognition model. Additionally, in response to the trigger operation on the "View Evaluation Results" control corresponding to any sample, the evaluation results of that sample are displayed. An exemplary sample evaluation result is as Figure 2j shown. The first recognition result of positive and negative sample recognition based on rules for the sample in the figure is a positive sample, and the second recognition result of positive and negative sample recognition based on the sample recognition model for the sample is a negative sample.

[0104] Further, as Figure 2j also includes an "Artificial Annotation" control. In the case where the first recognition result obtained from positive and negative sample recognition based on rules as shown in Figure 2j and the second recognition result of positive and negative sample recognition based on the sample recognition model are negative samples, artificial annotation can be performed. Then, in response to the trigger operation on the "Artificial Annotation" control, an artificial annotation page is displayed. An exemplary artificial annotation schematic diagram is as Figure 2k shown. The artificial annotation page includes interaction intention indicators and an "Artificial Annotation" item. In response to the selection operation based on the "Artificial Annotation" item, the artificial recognition result, that is, the third recognition result, is determined.

[0105] Further, as Figure 2h and Figure 2m shown, in response to the trigger operation on the Figure 2h "Result Analysis" control, the evaluation results can be analyzed. After the analysis is completed, in response to the trigger operation on the "View Analysis Results" control, an analysis results page can be displayed. An exemplary result analysis page is as Figure 2l As shown, the result analysis page includes the names of interaction intention metrics, the number of valid samples, the number of invalid samples, the number of true positives, the number of true negatives, the number of false positives, the number of false negatives, accuracy, precision, recall, and F1 score, so as to facilitate optimizing and adjusting the positive and negative sample recognition rules or target prompt words in the configuration information based on these data.

[0106] Figure 3 This is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application. As Figure 3 shown, it includes: a memory 30a and a processor 30b; the memory 30a is used to store a computer program; the processor 30b is coupled to the memory 30a and is used to execute the computer program to implement the steps in the above-mentioned sample type recognition method.

[0107] Furthermore, as Figure 3 shown, the electronic device further includes: other components such as a communication component 30c, a display 30d, a power supply component 30e, and an audio component 30f. Figure 3 Only some components are schematically shown herein, and it does not mean that the electronic device only includes Figure 3 the components shown.

[0108] The detailed implementation manners and beneficial effects provided by the embodiments of the present application have been described in detail in the foregoing embodiments, and will not be elaborated herein.

[0109] An exemplary embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the above method embodiments.

[0110] An exemplary embodiment of the present application further provides a computer program product, which includes a computer program / instructions, and when the computer program / instructions are executed by a processor, causes the processor to be able to implement the steps in the above method embodiments.

[0111] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0112] The above-mentioned communication component is configured to facilitate communication, in a wired or wireless manner, between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0113] The above-mentioned display includes a screen, and the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations.

[0114] The above-mentioned power supply component provides power for various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device where the power supply component is located.

[0115] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC). When the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals. The received audio signals can be further stored in a memory or transmitted via a communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0116] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, compact disc read-only memory (CD-ROM), optical memory, etc.) containing computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0118] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process Figure 1 one process or more processes and / or blocks Figure 1 steps for implementing the functions specified in one block or more blocks.

[0120] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), an input / output interface, a network interface, and memory.

[0121] The memory may include non-permanent memory in the form of computer-readable media, random access memory (Random Access Memory, RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

[0122] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change random access memory (Phase-change Random Access Memory, PRAM), static random access memory (SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (Digital Video Disc, DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0123] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0124] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.< / sep> < / cls> < / sep> < / cls> < / sep> < / cls>

Claims

1. A method for identifying a sample type, characterized in that, Including: In response to a configuration operation, at least one configuration information adapted to a target service provided by a target application is configured. Each configuration information includes an interaction intention metric, and the interaction intention metric corresponds to a positive and negative sample identification rule required for positive and negative sample identification based on rules, as well as a target prompt word and an intention knowledge base required for positive and negative sample identification based on a sample identification model; Collect a sample set, where the sample set contains a plurality of interaction information generated during at least one interaction between a user and the target application, and the at least one interaction is initiated by the user for the target service; Based on the positive and negative sample identification rules and the interaction intention metric, perform rule-based positive and negative sample identification processing on each interaction information to obtain a first identification result of each interaction information, where the first identification result indicates that the corresponding interaction information is a positive sample adapted to the interaction intention metric or a negative sample not adapted to the interaction intention metric; Input the target prompt word, the interaction intention metric, and each interaction information into the sample type identification model. Under the guidance of the target prompt word, in combination with the intention knowledge base, identify whether each interaction information is adapted to the interaction intention metric to obtain a second identification result of each interaction information, where the second identification result indicates that the corresponding interaction information is a positive sample adapted to the interaction intention metric or a negative sample not adapted to the interaction intention metric; For each interaction information, when the first identification result and the second identification result of the interaction information are consistent, determine that the interaction information is a positive sample or a negative sample indicated by any one of the first identification result and the second identification result.

2. The method according to claim 1, characterized in that In response to a configuration operation, configure at least one configuration information adapted to a target service provided by a target application, including: For each configuration information, display a configuration interface, and the configuration interface includes: the interaction intention metric corresponding to the configuration information, and information configuration items corresponding to the interaction intention metric, where the information configuration items include a positive and negative sample identification rule configuration item, a sample identification model configuration item, a prompt word configuration item, and an intention knowledge base configuration item; In response to an edit operation on the positive and negative sample identification rule configuration item, obtain a positive sample identification rule and a negative sample identification rule required for positive and negative sample identification based on rules; In response to a selection operation on the sample identification model configuration item, obtain the sample identification model corresponding to the interaction intention metric; In response to an edit operation on the prompt word configuration item, obtain a target prompt word required for positive and negative sample identification based on the sample identification model; In response to an edit operation on the intention knowledge base configuration item, obtain an intention knowledge base required for positive and negative sample identification based on the sample identification model.

3. The method according to claim 1, wherein The positive sample identification rule in the positive and negative sample identification rules is used to describe the keywords in the interaction intention metric, and the negative sample identification rule in the positive and negative sample identification rules is used to describe the keywords that do not include in the interaction intention metric; Perform rule-based positive and negative sample recognition processing on each interaction message according to the positive and negative sample recognition rules and the interaction intention index to obtain a first recognition result for each interaction message, including: According to the positive sample recognition rule, determine whether each interaction message contains the keywords in the interaction intention index. If it does, determine that the interaction message is a positive sample; According to the negative sample recognition rule, determine whether each interaction message does not contain the keywords in the interaction intention index. If it does not, determine that the interaction message is a negative sample.

4. The method according to claim 3, wherein According to the positive sample recognition rule, determine whether each interaction message contains the keywords in the interaction intention index, including: According to the positive sample recognition rule, determine the keywords in the interaction intention index; perform word segmentation on each interaction message to obtain multiple word segments included in the interaction message; under the guidance of the positive sample recognition rule, determine whether the multiple word segments contain the keywords in the interaction intention index; According to the negative sample recognition rule, determine whether each interaction message does not contain the keywords in the interaction intention index, including: According to the negative sample recognition rule, determine the keywords in the interaction intention index; perform word segmentation on each interaction message to obtain multiple word segments included in the interaction message; under the guidance of the negative sample recognition rule, determine whether the multiple word segments do not contain the keywords in the interaction intention index.

5. The method according to claim 1, wherein The sample type recognition model includes a knowledge learning network layer and a classification recognition network layer; input the target prompt word, the interaction intention index, and each interaction message into the sample type recognition model. Under the guidance of the target prompt word, in combination with the intention knowledge base, identify whether each interaction message is compatible with the interaction intention index to obtain a second classification recognition result for each interaction message, including: Use the knowledge learning network layer to learn the knowledge meanings related to each intention index included in the intention knowledge base; Input the learned knowledge meanings related to each intention index, each interaction message, and the target prompt word into the classification recognition network layer. Under the guidance of the knowledge meanings related to each intention index and the target prompt word, identify whether each interaction message is compatible with the interaction intention index to obtain a second classification recognition result for each interaction message.

6. The method according to any one of claims 1-5, characterized in that, It also includes: Use the interaction messages with consistent first and second recognition results among the multiple interaction messages as the first interaction messages. The first interaction messages are true positive samples or true negative samples; Use the interaction messages with inconsistent first and second recognition results among the multiple interaction messages as the second interaction messages. For the second interaction messages, perform manual classification recognition on the second interaction messages to obtain a third recognition result for the second interaction messages. The third recognition result indicates that the second interaction message is a positive sample compatible with the interaction intention index or a negative sample incompatible with the interaction intention index; Based on the first recognition result, the second recognition result, and the third recognition result of the second interaction information, determine whether the second interaction information is a false positive sample or a false negative sample; According to the false positive sample or the false negative sample, optimize and adjust the positive and negative sample recognition rules or the target prompt words in the configuration information.

7. The method according to claim 6, wherein Based on the first recognition result, the second recognition result, and the third recognition result of the second interaction information, determining whether the second interaction information is a false positive sample or a false negative sample includes: In the case where the first recognition result of the second interaction information is a positive sample, and the second recognition result and the third recognition result of the second interaction information are the same and both are negative samples, then determine that the second interaction information is a false positive sample; In the case where the first recognition result of the second interaction information is a negative sample, and the second recognition result and the third recognition result of the second interaction information are different and the third recognition result is a positive sample, then determine that the second interaction information is a false negative sample.

8. The method according to claim 6, characterized in that, According to the false positive sample or the false negative sample, optimizing and adjusting the positive and negative sample recognition rules or the target prompt words in the configuration information includes: According to the respective quantities of the true positive samples, the true negative samples, the false positive samples, and the false negative samples included in the sample set, calculate the accuracy rate, the precision rate, the recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and recognizing the sample set; Based on the accuracy rate, the precision rate, the recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and recognizing the sample set, optimize and adjust the positive and negative sample recognition rules or the target prompt words in the configuration information.

9. The method according to claim 8, wherein According to the respective quantities of the true positive samples, the true negative samples, the false positive samples, and the false negative samples included in the sample set, calculating the accuracy rate, the precision rate, the recall rate, and the harmonic mean between the precision rate and the recall rate for classifying and recognizing the sample set includes: Taking the sum of the true positive sample quantity and the false positive sample quantity as the numerator, and the sum of the true positive sample quantity, the true negative sample quantity, the false positive sample quantity, and the false negative sample quantity as the denominator, perform a division operation to obtain the accuracy rate; Taking the true positive sample quantity as the numerator, and the sum of the true positive sample quantity and the false negative sample quantity as the denominator, perform a division operation to obtain the precision rate; Taking the true positive sample quantity as the numerator, and the sum of the true positive sample quantity and the true negative sample quantity as the denominator, perform a division operation to obtain the recall rate; Taking twice the product of the precision rate and the recall rate as the numerator, and the sum of the precision rate and the recall rate as the denominator, perform a division operation to obtain the harmonic mean.

10. An electronic device, characterized in that, including: a memory and a processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method according to any one of claims 1-9.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor is caused to implement the steps in the method according to any one of claims 1-9.

12. A computer program product, characterized in that, The computer program product includes computer programs / instructions which, when executed by a processor, cause the processor to be able to implement the steps in the method according to any one of claims 1-9.

Images