Data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a data processing method and device, electronic equipment, a storage medium and a program product. The method comprises the steps of obtaining sample text data, wherein intention labels of different intention levels are labeled on each piece of sample text data; based on each piece of sample text data, respectively training models to obtain an intention recognition model of each intention level; performing intention classification processing on a first test sample through each intention recognition model to obtain a prediction intention sequence corresponding to each intention recognition model; based on an intention knowledge graph of the first test sample and a predicted intention sequence corresponding to each intention recognition model, determining an intention priority sequence formed by each intention level; and modifying an intention label of the sample text data based on the intention priority order to obtain label migrated sample text data. According to the invention, the intention recognition accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, electronic device, storage medium and program product. Background Art

[0002] In the human-computer interaction scenario, identifying conversation intent is the key to text understanding. Usually, a sentence or document contains multiple intents at the same time. Compared with the recognition of a single intent, multi-intent recognition is more difficult and requires distinguishing the relationship between multiple intents. Related technologies identify the number of valid intents in a conversation, search and splice in the vocabulary of different fields, and determine the complete text of the conversation. It is impossible to efficiently determine the conversation logic in a multi-intent scenario, and the accuracy of intent recognition is not high. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, device, electronic device, storage medium and program product, which can improve the accuracy of intent recognition.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a data processing method, the method comprising:

[0006] Acquire sample text data, each of the sample text data being annotated with intent labels of different intent levels;

[0007] Based on each of the sample text data, the model is trained respectively to obtain an intent recognition model for each of the intent levels;

[0008] Performing intent classification processing on the first test sample by each of the intent recognition models, obtaining a predicted intent sequence corresponding to each of the intent recognition models, wherein the predicted intent sequence is used to characterize an order of occurrence of intents corresponding to a first predicted intent label in the first test sample, and the first predicted intent label is an intent label predicted by the intent recognition model for the first test sample;

[0009] Determine the intent priority ranking of each of the intent level components based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each of the intent recognition models;

[0010] The intent labels of the sample text data are modified based on the intent priority ranking to obtain sample text data with label migration.

[0011] The present application provides a data processing device, including:

[0012] A model training module is used to obtain sample text data, each of which is annotated with an intent label of a different intent level; based on each of the sample text data, the model is trained separately to obtain an intent recognition model for each intent level;

[0013] A priority sorting module, used to perform intent classification processing on the first test sample through each of the intent recognition models to obtain a predicted intent sequence corresponding to each of the intent recognition models, wherein the predicted intent sequence is used to characterize the order of appearance of the intent corresponding to the first predicted intent label in the first test sample, and the first predicted intent label is the intent label predicted by the intent recognition model for the first test sample; based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each of the intent recognition models, determine the intent priority sorting of each of the intent level components;

[0014] The label migration module is used to modify the intent label of the sample text data based on the intent priority ranking to obtain the sample text data with the label migrated.

[0015] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A memory for storing computer executable instructions or computer programs;

[0017] The processor is used to implement the data processing method provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the data processing method provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the data processing method provided in the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] Based on the sample text data labeled with different intent levels, the models are trained separately to obtain the intent recognition model corresponding to each intent level, which helps the intent recognition model to understand different intents more accurately. By classifying the intent of each intent recognition model, the predicted intent sequence corresponding to the intent recognition model is obtained. Based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each intent recognition model, the intent priority ranking composed of each intent level is determined. The intent priority ranking can be used to identify the intents that need to be prioritized in multi-intent scenarios. Modify the intent label of the sample text data according to the intent priority ranking to realize the label migration of the sample text data and improve the accuracy and efficiency of intent recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of an application mode of the data processing method provided in an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the structure of the server provided in the embodiment of the present application;

[0024] Figure 3A It is a first flow chart of the data processing method provided in the embodiment of the present application;

[0025] Figure 3B is a second flow chart of the data processing method provided in an embodiment of the present application;

[0026] Figure 3C is a third flow chart of the data processing method provided in the embodiment of the present application;

[0027] Figure 3D is a fourth flow chart of the data processing method provided in an embodiment of the present application;

[0028] Figure 3E is a fifth flow chart of the data processing method provided in the embodiment of the present application;

[0029] Figure 4 This is the sixth flow chart of the data processing method provided in the embodiment of the present application.

[0030] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0032] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0035] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0036] The relevant data collection and processing in the embodiments of this application (for example, business conversations in human-computer interaction scenarios) should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0037] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0038] 1) Business conversation: A series of interactions between users and machines (such as intelligent assistants, virtual customer service, operating systems, etc.) to complete specific tasks or achieve specific goals. This process usually includes the user initiating a request, the system understanding the user's intention, the system executing the task to meet the user's needs, and feedback to the user.

[0039] 2) Natural Language Processing (NLP): It is an interdisciplinary subject in the fields of artificial intelligence and linguistics. It is dedicated to enabling computers to understand, interpret and generate human language. The goal is to enable computers to process and analyze natural language data to achieve effective communication between humans and machines.

[0040] 3) Intent: It is the purpose or goal of the user in a specific situation, usually reflected in the user's language or behavior. Intent can be a request, question or command in a conversation, or it can be a result that the user wants to achieve or a task that the user wants to complete. Intent is the external manifestation of user thinking and motivation, and is crucial to understanding and responding to the user's real needs. In artificial intelligence systems, intent is usually inferred and identified through text or voice signals input by the user. In the embodiment of the present application, intent is a request, question or command that the user wants to express in a business conversation.

[0041] 4) Intent recognition: refers to understanding and identifying the intentions expressed by users through language through natural language processing technology. By analyzing the user's language input (such as text or voice), the user's intention or purpose is understood and inferred. Intent recognition usually involves an automated text understanding process, including technologies such as part-of-speech tagging, syntactic analysis, semantic analysis, and context understanding to determine the actions the user wants to perform or the information he or she is seeking. In applications such as smart assistants, customer service, and recommendation systems, intent recognition is the basis for understanding user needs and providing corresponding services.

[0042] 5) Intent Recognition Level: In the multi-intent recognition task, multiple intents are sorted and graded according to their importance, priority, or impact on the user. This classification is usually used to help the machine better understand the user's intent, especially when the user expresses multiple intents in the same sentence.

[0043] 6) Intent Priority: It is to assign an order to different intents expressed by users in the multi-intent recognition task, and the system can prioritize those more important or urgent intents according to this order. The setting of intent priority can help the system distinguish which intents are the most critical when processing user input, thereby ensuring that the user's core needs are responded to quickly and accurately.

[0044] 7) Performance Metrics: Key indicators for measuring the effectiveness of a machine learning model or any other technology in achieving its intended goals, used to evaluate the performance of the model in a specific task or application, and to help researchers, developers, or decision makers understand the performance of the model. In the embodiments of the present application, performance indicators are a series of standards used to evaluate the performance of the intent recognition model when processing multiple intent recognition tasks, help understand the performance of the model in terms of accuracy and efficiency, and assist in guiding model optimization and iteration, including the accuracy, precision, and recall of the intent recognition model.

[0045] 8) Accuracy: It is the ratio of all intents correctly recognized by the model. In the embodiment of the present application, for the intent recognition model, the accuracy means the frequency with which the model can correctly recognize the intent in all input texts.

[0046] 9) Precision: refers to the proportion of actual positive samples among all samples with positive prediction results. In the embodiment of the present application, for the intent recognition model, the precision reflects the ratio of positive samples that the model can correctly predict when identifying a specific intent.

[0047] 10) Recall: refers to the proportion of samples that are correctly predicted as positive among all actual positive samples. In the embodiment of the present application, for the intent recognition model, the recall rate indicates the ratio of positive examples that the model can recognize to all actual positive examples.

[0048] 11) Priority determination principle: refers to the process of decision-making, in which the priority of each task is determined according to specific standards and rules to guide resource allocation and execution order, helping to ensure that important tasks are given priority, thereby improving efficiency and effectiveness.

[0049] 12) Logical analysis: It is a process in the field of natural language processing and artificial intelligence that aims to understand and parse the user's intentions, analyze the text or voice input by the user to determine the user's true purpose or request, and convert it into a form that the machine can understand.

[0050] 13) Logical score: It is a standard used to evaluate and measure the performance and accuracy of the model when processing language data. In the embodiment of the present application, the logical score is determined based on the overlap between the intent knowledge graph and the predicted intent sequence.

[0051] In the related technology, when there are multiple intentions in a business conversation during the human-computer interaction process, the solution for single intent recognition cannot accurately determine the business logic provided by the customer; to identify multiple valid intents, it is necessary to search and splice different field lexicons and determine the complete business conversation content; the solution for single intent recognition lacks consideration of the priority between multiple intents, ignores the analysis of the degree of impact of multiple intents on the results of the business conversation, and cannot accurately and efficiently identify user needs, resulting in confusing business conversation logic and low accuracy of intent recognition.

[0052] The embodiments of the present application provide a data processing method, a data processing device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of intent recognition.

[0053] The following describes an exemplary application of the electronic device provided in the embodiments of the present application. The device provided in the embodiments of the present application can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart TV, and a vehicle-mounted terminal, and can also be implemented as a server. The following describes an exemplary application when the device is implemented as a terminal or a server.

[0054] See also Figure 1 , Figure 1 It is a schematic diagram of the application mode of the data processing method provided in the embodiment of the present application, for supporting a data processing application, for example, Figure 1 The system involves a server 200, a network 300, a terminal device 400 and a database 500. The terminal device 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0055] In some embodiments, the user may be a client in a human-computer interaction scenario, the server 200 is a server for performing intent recognition processing on sample text data in a client's business session request, the terminal device 400 is a terminal operated by the user, an application capable of running a human-computer interaction scenario is installed in the terminal device 400, and the database 500 stores sample text data of the business session.

[0056] For example, the database 500 stores sample text data of a business conversation, the server 200 performs intent recognition on the sample text data, determines the intent priority ranking in the business conversation request, modifies the sample text data according to the determined intent priority ranking, and obtains the sample text data after label migration. The language model for the conversation business is trained based on the sample text data after label migration to obtain a trained language model. The trained language model can be stored in the server 200. The terminal device 400 is used to receive the user's business conversation request, and send the business conversation request to the server 200 through the network 300. The server 200 performs language processing on the business conversation request based on the trained language model, obtains the reply text of the business conversation, and feeds back the reply text of the business conversation to the terminal device 400. The application of the human-computer interaction scenario running in the terminal device 400 displays the reply text of the business conversation to the user, and interacts with the user in the business conversation.

[0057] In some embodiments, the data processing method of the embodiments of the present application can also be applied in the following application scenarios:

[0058] 1. In the human-computer interaction scenario of intelligent customer service, the data processing method provided in the embodiment of the present application can identify the intent when the user uses the intelligent customer service of the application. Customers may ask complex questions or demands to the intelligent customer service, including multiple intents. Through the data processing method provided in the embodiment of the present application, the intelligent customer service can more accurately identify and process the multi-intent requests raised by the customer, and provide more personalized and accurate responses.

[0059] 2. In the human-computer interaction scenario of voice assistants, the data processing method provided in the embodiments of the present application can identify the intention when the user uses the voice assistant. In smart home or car systems, voice assistant applications are usually installed. Users may issue multiple instructions through voice commands, and each instruction may contain multiple intentions of the user. Through the data processing method provided in the embodiments of the present application, the voice assistant can understand and process these instructions to ensure the execution of the correct intention and improve the user experience.

[0060] 3. In the human-computer interaction scenario of the online question-and-answer platform, the data processing method provided in the embodiment of the present application can identify the intention when the user uses the online question-and-answer platform. When the user uses the online question-and-answer platform, he or she will ask questions containing multiple intentions. Through the data processing method provided in the embodiment of the present application, the online question-and-answer platform can parse the questions more accurately and provide the most relevant answers or information.

[0061] The embodiments of the present application can be implemented through database technology. In short, a database can be regarded as an electronic file cabinet where electronic files are stored. Users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.

[0062] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0063] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the server provided in the embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 410, a memory 450 and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0064] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0065] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0066] The memory 450 includes a volatile memory or a nonvolatile memory, and may also include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0067] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0068] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0069] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Authentication (WiFi), and Universal Serial Bus (USB).

[0070] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The data processing device 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: model training module 4551, priority sorting module 4552 and label migration module 4553. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0071] In some embodiments, the terminal or server can implement the data processing method provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a voice assistant APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0072] The data processing method provided in the embodiments of the present application will be described in conjunction with the exemplary application and implementation of the electronic device provided in the embodiments of the present application.

[0073] The following describes the data processing method provided by the embodiment of the present application. As mentioned above, the electronic device implementing the device management method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0074] See also Figure 3A , Figure 3A is a first flow chart of the data processing method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained, Figure 3A The executive body is Figure 1 Server 200 in.

[0075] In step 301, sample text data is obtained.

[0076] Here, each sample text data is annotated with an intent label of a different intent level, and the intent level represents the priority level of the intent label of the sample text data.

[0077] For example, sample text data of business conversations in a human-computer interaction scenario is obtained, the sample text data includes different business conversations, intent recognition is performed on the sample text, multiple intents represented by the sample text are obtained, the multiple intents are used as intent labels, a priority level for each intent label is generated, and different intent levels are marked in the sample text data.

[0078] In a single sentence in the sample text data, there are usually multiple intentions of the user. A corresponding initial priority level is generated for each intention. The initial priority level of the multiple intentions contained in the single sentence is modified according to the principle of taking the intention with higher priority. When the priority level of the intent changes, the intent level of the sample text data containing multiple intents also changes adaptively. Intent is the purpose or goal of the user in a specific situation. It can be a request, question or command in a conversation, or it can be a result that the user wants to achieve or a task that the user wants to complete. Intent recognition is to understand and identify the intentions expressed by the user through language, and to understand and infer the user's intentions or purposes by analyzing the user's language input (such as text or voice).

[0079] In some embodiments, step 301 can be implemented in the following manner: obtaining sample text data, the sample data including multiple sample texts and an intent label for each sample text; encoding each type of intent label to obtain the intent level corresponding to each type of intent label; annotating the sample text data based on the intent level corresponding to each type of intent label to obtain sample text data annotated with different initial intent levels.

[0080] For example, the sample text data includes multiple sample texts and the intent label of each sample text. Each intent label is randomly encoded to obtain the corresponding intent level, and the sample text data is labeled according to the intent level. The sample text data is a text record of the communication between the user and the robot in the interactive scene, including the conversation content between the user and the robot. The content of the sample text data includes: the time of the conversation, the participants of the conversation, the topic of the conversation, the user's needs and questions, and the sample text content is usually marked with the corresponding data label. For example: in an interactive scenario of a job search application, a sample text data is "You can add my contact information", and the marked data label is "Agree to add contact information".

[0081] For example, the labels of intents are randomly generated, and all intents in the sample text data are randomly encoded as intent levels. The smaller the value of the intent priority, the higher the representation level. For example: Assuming that the current sample text contains seven intents, these seven intents are randomly encoded in the range of zero to six as the priority level of each intent. The intent with a priority of zero represents the highest priority, and the intent with an initial priority of six represents the lowest priority. The intent priority is to give an order to the different intents expressed by the user in the multi-intent recognition task. According to this order, more important or urgent intents are prioritized to ensure that the core needs of users are responded to quickly and accurately.

[0082] In an embodiment of the present application, by acquiring and labeling sample text data of different intent levels, complex user intentions can be identified, helping to ensure that high-priority intentions are given priority, and by determining the intent levels of multi-intent sample text data, the accuracy of the intent recognition task and user experience can be improved.

[0083] Continue to see Figure 3A In step 302, based on each sample text data, the model is trained separately to obtain the intent recognition model of each intent level.

[0084] For example, different intent levels are annotated for sample text data. In business conversations under different business scenarios, the same sample text data is annotated with different intent levels. Sample text data with different intent levels are used as training data, and the training model is called to train, so as to obtain intent recognition models corresponding to different intent levels. The model can be a pre-trained language model (Bidirectional Encoder Representation from Transformers, Bert), which can understand the meaning of words in context and capture the relationship between words.

[0085] In some embodiments, the following processing can be performed on the sample text data of each intent level: based on the sample text data of the intent level, classification processing is performed through the model to obtain a second predicted intent label corresponding to the sample text data of the intent level; based on the difference between the second predicted intent label and the sample intent label corresponding to the intent level, the loss function of the model is determined; based on the loss function, the parameters of the model are updated to obtain an intent recognition model corresponding to the intent level.

[0086] For example, based on the sample text data of the intent level, the model is called to perform classification processing. The model is a pre-trained language model. Intent recognition is currently a more common text classification task. The classification processing performed by the pre-trained language model is to classify the sample text data labels of different intent levels to obtain the second predicted intent label corresponding to the sample text data. After the model performs separate processing, the second predicted intent label of the sample text data is obtained, and the real sample intent label is compared with the predicted intent label to determine the difference between the second predicted intent label and the sample intent label corresponding to the intent level. The loss function in the form of cross entropy loss is obtained, and the model parameters are updated to obtain the intent recognition model corresponding to the trained intent level. The method for adjusting the parameters of the training model can be backpropagation processing. Backpropagation is the process of updating the model parameters by calculating the gradient of the loss function to the model parameters and backpropagating the gradient from the output layer to the input layer when using optimization algorithms such as gradient descent in neural networks to train the model.

[0087] For example, the model is trained based on sample text data with higher intent priorities marked with affirmative sentences. For the single sentence "It's me, who are you?" in the sample text, the priority intent is "It's me." The model is trained based on sample text data with higher intent priorities marked with interrogative sentences. The priority intent is the label "Unclear who's calling." That is, the model trained based on different intent recognition types is more sensitive to the corresponding initial intent level, and will prioritize identifying the intent label at that level.

[0088] In some embodiments, see Figure 3B , Figure 3B is a second flow chart of the data processing method provided in the embodiment of the present application. Figure 3A After step 302 in Figure 3B Steps 3021 to 3023 in the process determine the effectiveness indicators of the intent recognition model, as described in detail below.

[0089] In step 3021, based on the test sample set, classification processing is performed by each intent recognition model to obtain a predicted intent set corresponding to each intent recognition model.

[0090] Here, the test sample set includes multiple second test samples and an actual intent label corresponding to each second test sample, and the predicted intent set includes a second predicted intent label corresponding to each second test sample.

[0091] For example, after training to obtain the intent recognition models corresponding to different initial intent levels, classification processing is performed separately through each intent recognition model based on the test sample set. The test sample set is a representative test set that can well evaluate the model's capabilities, including multiple second test samples and the actual intent label corresponding to each second test sample. The result of the intention recognition model's reasoning on the test set is evaluated, and the degree of training of the current intent recognition model is analyzed based on the evaluation result. A set of predicted intents corresponding to each intent recognition model is obtained, including a second predicted intent label corresponding to each second test sample, which is used to compare with the actual intent label to determine the effectiveness index of the intent recognition model. In an embodiment of the present application, the effectiveness index is a series of standards used to evaluate the performance of the intent recognition model when processing multiple intent recognition tasks, help understand the model's performance in terms of accuracy and efficiency, and assist in guiding model optimization and iteration.

[0092] In step 3022, for each intent recognition model, based on the test sample set and the predicted intent set corresponding to the intent recognition model, the number of correct predictions and the number of incorrect predictions of the intent recognition model are determined.

[0093] For example, the test sample set includes the actual intent label corresponding to the second test sample, and the predicted set label in the predicted intent combination is compared to determine the number of correct predictions and the number of incorrect predictions of the intent recognition model, and the effect index is calculated based on the number of correct predictions and the number of incorrect predictions.

[0094] In step 3023, the effectiveness index of the intent recognition model is determined based on the number of correct predictions and the number of incorrect predictions.

[0095] Here, the effectiveness indicators of the intent recognition model include: accuracy, precision and recall.

[0096] For example, the accuracy rate is the ratio of the number of samples with correct predictions in the intent recognition model to the number of all prediction results, which can be expressed as the following formula (1), which is described in detail below.

[0097] (TP+TN) / (TP+TN+FP+FN)(1)

[0098] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, TN is the number of samples correctly predicted as negative by the intent recognition model, FP is the number of samples incorrectly predicted as positive by the intent recognition model, and FN is the number of samples incorrectly predicted as negative by the intent recognition model.

[0099] The precision rate is the ratio of the number of truly positive samples among all samples predicted to be correct in the intent recognition model, which can be expressed as the following formula (2), which is explained in detail below.

[0100] TP / (TP+FP)(2)

[0101] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, and FP is the number of samples incorrectly predicted as positive by the intent recognition model.

[0102] The recall rate is the ratio of the number of positive samples successfully predicted among all samples that are actually positive samples in the intent recognition model. It can be represented by the following formula (3), which is explained in detail below.

[0103] TP / (TP+FN)(3)

[0104] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, and FN is the number of samples incorrectly predicted as negative by the intent recognition model.

[0105] In some embodiments, after determining the effectiveness indicators of the intent recognition model in step 3023, the intent recognition model is also trained by the following method: when at least one effectiveness indicator of the intent recognition model is less than the corresponding effectiveness indicator threshold, the intent recognition model is re-trained based on sample text data of the corresponding intent level.

[0106] For example, different business sessions have different requirements for the effectiveness indicators of the intent recognition model. According to the actual interaction scenario, the thresholds corresponding to the accuracy, precision and recall are pre-configured. When at least one effectiveness indicator of the intent recognition model is less than the corresponding effectiveness indicator threshold, it indicates that the intent recognition model has not reached the training standard at this time, and the intent recognition model is retrained based on the sample text data of the corresponding intent level. When all effectiveness indicators are greater than the corresponding effectiveness indicator threshold, it indicates that the corresponding intent recognition model training has reached the predetermined training standard at this time, and subsequent logical analysis processing is performed based on the trained intent recognition model. The logical analysis understands and parses the user's intention, and analyzes the text or voice input by the user to determine the user's true purpose or request.

[0107] In the embodiment of the present application, sample text data of different intent levels are annotated and corresponding intent recognition models are trained respectively, so that the intent recognition model can more accurately recognize the user's intent. According to the calculation of the effect index, it is possible to more accurately evaluate whether the model meets the training standard, thereby effectively improving the performance and generalization ability of the intent recognition model.

[0108] In some embodiments, when performing Figure 3A After step 302 in , the intent recognition model is also trained by the following method: based on the third test sample, classification processing is performed by each intent recognition model respectively to obtain a third intent label and a prediction probability set corresponding to each intent recognition model, the prediction probability set including the prediction probability of the third test sample belonging to different types of intent labels; the following processing is performed for each intent recognition model: the difference between the prediction probabilities in the prediction probability set corresponding to the intent recognition model is determined; when the difference is less than the pre-configured difference, the intent recognition model is re-trained based on the sample text data of the corresponding intent level.

[0109] For example, during the training process of the intent recognition model, based on the third test sample, classification processing is performed by each intent recognition model respectively, and online test simulation is performed to generate online data to form a complete business session for the current interaction scenario. The online test evaluates the performance of the model in a real data environment, using actual user session data. These third test samples carry role information, such as: the conversation text between the user and the robot. Collect the complete conversation text record from the actual conversation, select the intent recognition model corresponding to a certain priority level that has been trained for inference, and obtain the third intent label and prediction probability set corresponding to each intent recognition model, including the prediction probability that the third test sample belongs to different types of intent labels.

[0110] For example, determine the difference between each predicted probability in the predicted probability set corresponding to the intent recognition model, pre-configure the probability difference according to the actual interaction scenario, call the pre-configured model performance evaluation module to perform performance indicator analysis, the pre-configured model performance evaluation module is the sklearn.metrics package, take the label list and the inference result list of the test data set as input, and the function provided in the sklearn.metrics package calculates the effect indicator. The pre-configured difference is a threshold set according to the specific application scenario, which is used to measure the model's ability to distinguish different intent classifications. When the difference is less than the pre-configured difference, it indicates that the model's recognition effect on the intent level is poor at this time, and it cannot distinguish the difference between different intents well. Re-train the intent recognition model based on the sample text data of the corresponding intent level.

[0111] In an embodiment of the present application, by conducting online testing based on the third test sample and simulating user interaction in a real environment, the performance of the intent recognition model can be evaluated in real time, the reliability and accuracy of the model in actual deployment can be improved, continuous optimization of the intent recognition model can be achieved, and intent recognition in different scenarios can be ensured.

[0112] Continue to see Figure 3A In step 303, each intention recognition model is used to perform intent classification processing on the first test sample to obtain a predicted intention sequence corresponding to each intention recognition model.

[0113] Here, the intent sequence is used to characterize the order in which the first predicted intent label appears in the first test sample. The first predicted intent label is the intent label predicted by the intent recognition model for the first test sample.

[0114] For example, based on the first test sample, each intent recognition model is called to perform intent classification processing, and a buried point test is performed for the intent classification process to obtain a predicted intent sequence corresponding to each intent recognition model. The buried point test is to output the actual output content of the key nodes during the model operation process, compare the actual output content with the expected content, and judge the model operation status.

[0115] In some embodiments, see Figure 3C , Figure 3C is a third flow chart of the data processing method provided in the embodiment of the present application, Figure 3A Step 303 can be achieved by Figure 3C Steps 3031 to 3034 in are implemented as described in detail below.

[0116] In step 3031, word segmentation is performed on the first test sample to obtain multiple text segments of the first test sample.

[0117] For example, the first test sample is a dialogue text with role information in a real data environment. The first test sample is segmented to obtain multiple text segments after the dialogue text segmentation, which are used to identify intent based on the text segments. The segmentation process is the process of dividing continuous text data into meaningful words or phrases. The segmentation process needs to consider context information to ensure that the text intent is correctly understood.

[0118] In step 3032, feature extraction processing is performed on multiple text segments in sequence to obtain text features of each text segment.

[0119] For example, feature extraction is performed on the text segments after word segmentation in turn to obtain text features corresponding to each text segment. Each text segment is regarded as a node in the first test sample, and the text features represent the node features of the node. Feature extraction is performed on the text segment through a feature extraction algorithm. The feature extraction algorithm can be a word embedding technology that converts words into vectors with semantic information. Word embedding technology is a method for learning low-dimensional vector representations of words or phrases. These vectors contain not only the semantic information of the text segment, but also the relationship between the text segments. Word embedding converts each text segment in the text data into a vector of fixed length, and each dimension represents a feature.

[0120] In step 3033, for each text feature, the probability that the text feature belongs to the corresponding candidate intent label is predicted, and the candidate intent label with the highest probability is used as the predicted intent label of the text feature. The order of appearance of each text fragment in the first test sample corresponds to the order of appearance of the predicted intent labels.

[0121] For example, the intent recognition model is called based on text features for prediction processing to obtain the probability of candidate intent labels corresponding to each text feature, that is, the probability of each intent level is recognized, and the intent label with the largest prediction probability is used as the final predicted intent label. The order of appearance of the text fragments in the first test sample corresponds to the order of appearance of the predicted intent labels.

[0122] In step 3034, each predicted intent label is combined according to the order of appearance of multiple text fragments to obtain a predicted intent sequence corresponding to the intent recognition model.

[0123] For example, according to the order of appearance of multiple text segments, each predicted intent label is combined to obtain a predicted intent sequence with the same order of appearance of the sample segments in the corresponding first test sample. For example, the sample segments in the first test sample are: robot: "introduce yourself, verify customer identity"; customer: "affirm / negative"; robot: "ask whether to upgrade to membership"; customer: "upgrade membership"; the identified predicted intent sequence: introduce yourself, verify customer identity, affirm, ask whether to upgrade membership, upgrade membership.

[0124] In the embodiment of the present application, by maintaining the order of corresponding text fragments and combining predicted intent labels to ensure the logical order between intents, the model can make more reasonable judgments. By recording the model's highest prediction probability as the key output through embedded point testing, the interpretability of the model is enhanced to assist in understanding the decision-making and adjustment process of the model.

[0125] Continue to see Figure 3A In step 304, based on the intention knowledge graph of the first test sample and the predicted intention sequence corresponding to each intention recognition model, the intention priority ranking of each intention level component is determined.

[0126] For example, extract the entities in the first test sample, construct the intent knowledge graph corresponding to the first test sample, compare the intents of nodes at the same position based on the intent knowledge graph and the predicted intent sequence, analyze the logic formed by the intent recognition model, and determine the logic score of each intent recognition model. Select the intent level with the highest logic score as the final intent level of the corresponding intent recognition model, sort each initial intent level, and determine the intent priority sequence.

[0127] In some embodiments, see Figure 3D , Figure 3D is a fourth flow chart of the data processing method provided in the embodiment of the present application, Figure 3A Step 304 can be accomplished by Figure 3D Steps 3041 to 3044 in are implemented as described in detail below.

[0128] In step 3041, the intent knowledge graph and the intent recognition model are compared with the intent tags at the same position in the predicted intent sequence, and the first number of intent tags at the same position and of the same type is counted.

[0129] For example, all entities in the first test sample are identified, the relationships between the entities are identified, an intent knowledge graph is constructed based on the entity relationships, the predicted intent sequence contains multiple sample text features, the sample text features correspond to different intent labels, the intent knowledge graph is compared with the intent labels at the same position in the predicted intent sequence, and the first number of intent labels with the same position and the same type is counted.

[0130] The following example illustrates that the first test sample is a human-computer interaction text in which a user consults a certain business; the intent labels of each node in the intent knowledge graph are [ Figure 1 ,meaning Figure 2 , intention 3...intention N], N is a positive integer, and the content of a certain predicted intention sequence is [intention Figure 1 , intention 3, intention Figure 2...Intent N]. Compare each intention with the same position in turn, assuming that only the second position intention Figure 2 And intent 3 is different, and the intents in other positions are the same, then the first number of intent tags with the same position and type is N-1.

[0131] In step 3042, the ratio between the first number and the total number of labels in the predicted intent sequence is taken as the overlap.

[0132] For example, the ratio between the first number and the total number of labels in the predicted intent sequence is taken as the overlap, and the overlap represents the similarity between the intent knowledge graph and the intent labels in the predicted intent sequence.

[0133] For example, if the first quantity is 4 and the total number of labels is 5, the overlap ratio is 0.8.

[0134] In step 3043, based on the degree of overlap, a logical score of the intent recognition model is determined.

[0135] Here, overlap is positively correlated with the logistic score.

[0136] For example, the overlap ranges from zero to one. The overlap is multiplied by a preconfigured coefficient (for example, 5) to calculate the value of the corresponding intent recognition model as the logical score. The higher the logical score, the higher the priority of the intent level corresponding to the corresponding intent recognition model. The intent levels can be sorted according to the logical score to obtain the intent priority ranking as the intent priority determination principle. The priority determination principle is to determine the priority order of each task according to specific standards and rules during the decision-making process, guide resource allocation and execution order, and help ensure that important tasks are given priority.

[0137] In step 3044, the intent priority ranking of each intent level component is determined based on the logical score corresponding to the intent recognition model of each initial intent level.

[0138] Here, in the intent prioritization, the order of intent levels is positively correlated with the priority.

[0139] For example, according to the logical score corresponding to the intention recognition model of each intent level, each intent level is sorted in descending order to obtain a priority sequence composed of the intent levels. The higher the corresponding logical score, the higher the order. The order of each intent level is positively correlated with the intent priority. The higher the intent priority, the higher the corresponding intent level order.

[0140] In an embodiment of the present application, by constructing an intent knowledge graph and comparing the predicted intent sequence with the intent of the same node in the knowledge graph, the logic formed by the intent recognition model can be analyzed, the accuracy and logic of intent recognition can be improved, the user intent can be prioritized, and the user's needs can be better understood and responded to.

[0141] Continue to see Figure 3A In step 305, the intent labels of the sample text data are modified based on the intent priority ranking to obtain sample text data with label migration.

[0142] For example, the intent label of the sample text data is modified based on the intent priority ranking. The intent priority ranking can be expressed by an operation formula, that is, high-priority intent + low-priority title image = high-priority intent = final intent, and the sample text data with label migration is obtained. Figure 1 Priority over intention Figure 2 The priority of Figure 1 The logic is better, that is, Figure 1 With intention Figure 2 When it appears in a single sentence in the same sample text, select Figure 1 as the final intention.

[0143] For example, an intent priority ranking is that the question priority is higher than the affirmative priority, and the negative priority is higher than the question priority. The corresponding calculation formula is: negative>question>affirmation. When affirmative, question and negative intentions appear in the same sentence, the intention of the sentence is determined to be negative. When question and affirmative intentions appear in the same sentence, the intention of the sentence is determined to be question. The intent label of the sample text is modified according to the calculation formula corresponding to the intent priority to obtain the sample text data with label migration.

[0144] In some embodiments, the types of intent tags include: preconfigured tags and non-preconfigured tags, see Figure 3E , Figure 3E is a fifth flow chart of the data processing method provided in the embodiment of the present application, Figure 3A Step 305 in can be performed by executing Figure 3E Steps 3051 to 3053 in are implemented as described in detail below.

[0145] In step 3051, the sample text data is subjected to intent classification processing to obtain a plurality of candidate intent labels for the sample text data.

[0146] For example, the sample text data is subjected to intent classification processing to obtain multiple candidate intent labels for the sample text data. The candidate intent label types include preconfigured labels and non-preconfigured labels. Preconfigured labels are intent labels pre-set in actual interaction scenarios. Preconfigured labels do not perform label migration according to intent priority sorting, while non-preconfigured labels perform label migration according to intent priority sorting.

[0147] Preconfigured tags can be set according to actual application scenarios. For example, in a scenario where users conduct business consultations, question-related intents can be set as preconfigured tags, so that question-related intents will not be migrated, ensuring that the human-computer interaction system gives priority to responding to users' question-related intents.

[0148] In step 3052, when the multiple candidate intent tags include only non-preconfigured tags, the intent tag is replaced with the first target intent tag with the highest intent level among the multiple candidate intent tags according to the intent priority sorting.

[0149] For example, when multiple candidate intent labels include only non-preconfigured labels, label migration is performed according to intent priority sorting, and the first target intent label with the highest intent level among the multiple candidate intent labels is selected to replace the corresponding intent label in the sample text.

[0150] In step 3053, when multiple candidate intent tags include at least one preconfigured tag, a second target intent tag is determined according to the intent priority ranking, and the intent tag is replaced with the second target intent tag.

[0151] Here, the second target intent tag is a candidate intent tag whose intent level is second only to at least one preconfigured tag.

[0152] For example, when multiple candidate intent labels include at least one preconfigured label, they are sorted according to intent priority, and the preconfigured labels are not replaced. Among the candidate intent labels other than the preconfigured labels, the second target intent label with the highest intent level is selected, and the corresponding intent label in the sample text is replaced according to the second intent label.

[0153] In the embodiment of the present application, by labeling the sample text data with different intent levels and training the model, the accuracy and logic of intent recognition can be improved to ensure that high-priority intents are processed first. Based on the intent priority ranking, the initial intent labels of the sample text data are modified to make the labels consistent with the intents with higher priority. Figure 1 By recording the key outputs of the model and the predicted intent sequence, the accuracy and logical rationality of intent recognition can be improved, the performance of the intent recognition model in complex interaction scenarios can be improved, user satisfaction can be enhanced, and the user interaction experience can be optimized.

[0154] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0155] In the human-computer interaction scenario, the intention recognition of the user's business conversation needs is a core task. It involves analyzing the text content of the user's input, accurately understanding and identifying the user's intention, and determines whether the human-computer interaction scenario can accurately respond to the user's needs and provide appropriate services and support. When conducting business conversations, the user's needs are often not single, but may contain multiple intentions. Therefore, robots in human-computer interaction scenarios need to have the ability to recognize multiple intentions, be able to identify multiple intentions from the user's input, and provide corresponding services based on these intentions, and perform in-depth semantic understanding and analysis.

[0156] Intent recognition is a common text classification task. The intent expressed in a single sentence is not single, but multiple in most cases, which is in line with the principle of high efficiency and labor saving in human communication. Therefore, when multiple intents appear in a single sentence, it is worth paying more attention to how to determine the more prioritized intent as the intent label.

[0157] Relevant technologies can extract key information from the text corresponding to the business conversation, identify and determine the user's single intention, and perform corresponding operations or provide corresponding services based on the determined specific intention. However, a single sentence expressed by a user in a business conversation may contain multiple intentions, or the expression may not be clear enough, resulting in the inability to accurately identify the specific intention.

[0158] The embodiment of the present application obtains sample text corresponding to the business conversation, and based on the different priorities of each intent in the sample text, trains multiple intent recognition models corresponding to different priorities. By testing and evaluating the intent recognition model, calling the logic analysis model to perform logical analysis on the trained intent recognition model, determining the determination principle of intent priority, modifying the intent priority label of the sample text according to the determination principle, making a reasonable determination of intent priority, improving the accuracy of intent recognition, assisting business conversations to more accurately understand user needs, and improving the efficiency of business completion.

[0159] The following is a description with reference to the accompanying drawings. Figure 4 , Figure 4 6 is a schematic diagram of a data processing method according to an embodiment of the present application. The execution subject may be a terminal device, a server, or a combination of the two. The present application embodiment takes the server as the execution subject as an example. Figure 4 The steps shown are explained in detail.

[0160] In step 401, a sample text set is obtained, and the priority of each intention of the sample text in the sample text set is determined.

[0161] For example, a sample text set is obtained, the sample text set includes multiple sample texts, the sample texts represent different business conversations and user intentions, each intention in the sample text is identified, and a corresponding intention priority is generated for each intention. The sample text is a text record of the communication between the user and the human-computer interaction scene in a business conversation in a specific human-computer interaction scene, including the user's conversation content. The content of the sample text includes: the time of the conversation, the participants of the conversation, the topic of the conversation, the user's needs and questions, and the sample text content is usually marked with corresponding data tags. For example: in an interactive scenario of a job search application, for a text "You can add my contact information", the marked data tag is "Agree to add contact information".

[0162] An intent is a user's purpose or goal in a specific situation. It can be a request, question or command in a conversation, or it can be a result that the user wants to achieve or a task that the user wants to complete. Intent recognition is to understand and identify the intentions expressed by the user through language, and to understand and infer the user's intentions or purposes by analyzing the user's language input (such as text or voice). Intent priority is to assign an order to different intentions expressed by the user in a multi-intent recognition task, so that the system can prioritize those more important or urgent intentions according to this order. In an embodiment of the present application, the initial priority of the intent is randomly generated, and all intents in the sample text are randomly encoded as the initial priority level. The smaller the value of the priority level, the higher the level representing the priority.

[0163] For example: assuming that the current sample text contains seven intents, these seven intents are randomly encoded in the range of zero to six as the priority level of each intent. The intent with a priority level of zero represents the highest priority level, and the intent with a priority level of six represents the lowest priority level.

[0164] For example, a single sentence text usually contains multiple user intents. For data containing multiple intents, multiple user intents in a single sentence text are identified, and the intent label of the sample text is modified according to the priority of the intent. The judgment principle is to take the intent with a higher priority as the intent label of the text. When the priority of the intent changes, the label of the multi-intent text data also changes adaptively, replacing the original intent label of the current text with the current intent with a higher priority.

[0165] In step 402, based on sample text data labeled with different priorities, a pre-trained language model is called for training to obtain intent recognition models corresponding to different intent priorities.

[0166] For example, different business scenarios or user needs will lead to different interpretations of the same text. The labels of the same data under different intent priorities are different. The data under different priorities are used as training data, and the pre-trained language model is called for training to obtain multiple intent recognition models corresponding to different intent priorities. That is, the intent priority corresponds to the intent recognition model one by one, and the intent recognition model corresponding to a certain priority is more sensitive to the intent of that priority and has higher recognition accuracy. The pre-trained language model classifies the labeled data to obtain the data corresponding to different intent priority categories, and uses the data under different priorities to train the model.

[0167] For example, a text content is "It's me, who are you?" The data label of the text content under the first priority is "It's me", and the data label under the second priority is "Unclear caller". The pre-trained language model classifies the text content based on the data labels corresponding to the text content under different priorities, and obtains the data corresponding to different intent priority categories. The pre-trained language model can understand the meaning of words in the context, capture the relationship between words, and use the data under different priorities to obtain the corresponding intent recognition models.

[0168] In step 403, different intent recognition models are called to perform classification processing based on the standard test set to obtain a predicted intent recognition result.

[0169] For example, after different intent recognition models are trained, the different intent recognition models are evaluated based on a standard test set (equivalent to the above-mentioned test sample set), the effectiveness indicators of the intent recognition model are determined, and whether the model meets the training standards is judged. An online test is performed on the trained intent recognition model based on the standard test set to obtain the predicted intent recognition result output by the intent recognition model. The standard test set is representative and can well evaluate the model's capabilities. The results of the model's reasoning on the test set are evaluated, and the degree of training of the current model is analyzed based on the evaluation results. In an embodiment of the present application, the effectiveness indicators are the accuracy, precision, and recall of the intent recognition model after evaluation.

[0170] For example, after obtaining the intent recognition result, the preconfigured model performance evaluation module is called to perform performance indicator analysis. The preconfigured model performance evaluation module is the sklearn.metrics package. The label list and inference result list of the test data set are taken as input. The function provided in the sklearn.metrics package calculates the effect indicators to obtain the accuracy, precision, and recall of the intent recognition model.

[0171] The accuracy rate is the proportion of all correctly predicted samples in the intent recognition model to all prediction results. The accuracy rate can be represented by formula (1), which is explained in detail below.

[0172] (TP+TN) / (TP+TN+FP+FN)(1)

[0173] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, TN is the number of samples correctly predicted as negative by the intent recognition model, FP is the number of samples incorrectly predicted as positive by the intent recognition model, and FN is the number of samples incorrectly predicted as negative by the intent recognition model.

[0174] The precision rate is the ratio of truly positive samples among all samples predicted to be correct in the intent recognition model. The precision rate can be represented by formula (2), which is described in detail below.

[0175] TP / (TP+FP)(2)

[0176] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, and FP is the number of samples incorrectly predicted as positive by the intent recognition model.

[0177] The recall rate is the ratio of positive samples successfully predicted among all samples that are actually positive samples in the intent recognition model. The recall rate can be represented by formula (3, which is described in detail below.

[0178] TP / (TP+FN)(3)

[0179] Among them, TP is the number of samples correctly predicted as positive by the intent recognition model, and FN is the number of samples incorrectly predicted as negative by the intent recognition model.

[0180] Different business sessions have different requirements for the model's effect indicators. The thresholds for accuracy, precision, and recall in the effect indicators are pre-configured according to the actual interaction scenarios. When the effect indicators meet the pre-configured thresholds, it indicates that the training effect of the corresponding intent recognition model has met the standards. Subsequent logical analysis and processing is performed based on the trained intent recognition model.

[0181] For example, during the training process of the intent recognition model, online tests are performed based on the standard test set to simulate and generate online data to form a complete business session for the current interaction scenario. Point-based tests are performed on the online tests to record the situations in which different intents are identified at different nodes in different contexts. After the intent recognition model training is completed, the key nodes are determined and the final labels of the key nodes are used as the prediction results of the intent recognition of the current intent recognition model.

[0182] Online testing evaluates the performance of the model in a real data environment, using actual user conversation data with role information, such as the conversation text between the user and the robot. Complete conversation text records are collected from actual conversations, and a trained intent recognition model corresponding to a certain priority is selected for inference to obtain the model inference label. The inference label is the final prediction result of the model output, for example, the inference label is "blacklist".

[0183] For example, the process of online testing is to perform tracking test. Tracking test is to output the actual output content of key nodes during the model operation, compare the actual output content with the expected content, and judge the operation of the model. Each step or stage after the text enters the model is regarded as a node. Each node has an impact on the final result. Different nodes have different impacts on the final result. The key node is the node that has the greatest impact on the final result in all processes, and is usually the node that enables the model to make key decisions. The current reasoning result of the system output of the model is stored at the key node, and the intent recognition model at the key node is obtained. The key node is the step that has an important impact on the result in all processes after the text is input from the model. For example: in the intent recognition process of the model, after the text enters the model, the steps of word segmentation, word vector acquisition, layer-by-layer encoding, etc. will be executed in sequence. Each step is regarded as a node, and finally the probability of recognition as each category is obtained. The category with the highest probability is output as the final label. For the intent recognition model, the key node is the step that infers the probability of different intents, that is, the step that has the greatest impact on the final result.

[0184] For example, for the intent recognition model, the probabilities of different intents are inferred when a call is completed. This process is used as the intermediate result of the previous node of the final intent. The probability difference of each intent label in the middle and the probability difference between the final intent and other label classifications are checked.

[0185] In step 404, the logic formed by the intention recognition models corresponding to different priorities is analyzed to determine the intention priority determination principle.

[0186] For example, a knowledge graph of a complete business conversation is constructed (equivalent to the above-mentioned intention knowledge graph), the intention objects appearing in the conversation text are sorted, the intentions are connected in series to form a flowchart (equivalent to the above-mentioned predicted intention order), and the logic analysis model is called to perform logical rationality analysis on the intentions in the flowchart. According to the overlap of nodes between the knowledge graph and the intention flowchart, the intent priority judgment principle is determined based on the overlap (equivalent to the above-mentioned intention priority sorting).

[0187] For example, the logic analysis model is a sequence labeling model of the intent flow, which labels the logical rationality scores between adjacent intents, assigns a logical rationality score to each intent, and finally adds up each score as the comprehensive score of a certain intent flow, which is used to evaluate the logical rationality of the entire intent flowchart. If the score is high, it means that the logic between the intents is reasonable; if the score is low, you may need to check whether there are logical jumps or unreasonable parts in the flowchart.

[0188] Based on business explanations and training documents, all business entities are obtained, the relationships between all entities are extracted, and a complete business knowledge graph is constructed. The intents in the conversation text are sorted in order to form a flow chart. For example, the interactive conversation between a robot and a user is:

[0189] Robot: "Introduce yourself and verify the customer's identity";

[0190] Customer: “Yes / No”;

[0191] Robot: "Ask if you want to upgrade to membership";

[0192] Customer: "Upgrade Membership";

[0193] The order of the identified corresponding intentions is: introducing oneself, verifying the customer's identity, affirmation, asking whether to upgrade membership, and upgrading membership. The corresponding flow chart is obtained according to the order.

[0194] Compare the nodes in the flowchart with the nodes in the knowledge graph respectively to determine the overlap between the flowchart and the knowledge graph. The overlap is a value between zero and one. Multiply the value by five to obtain the logical level value of the corresponding intent recognition model (equivalent to the above-mentioned logical score). The higher the overlap, the higher the logical level. The intent priority with the best logic is selected as the final intent priority determination principle.

[0195] In step 405, based on the intention priority determination principle, the intention priority of the original sample text is modified.

[0196] For example, based on the principle of intent priority determination, the intent priority of the original sample text is modified to the logically optimal intent, for example: Figure 1 Priority over intention Figure 2 The priority of Figure 1 The logic is better, that is, Figure 1 With intention Figure 2 When they appear in the same sentence, choose Figure 1 As the final intention. The principle of determining the priority of intention can be expressed by a calculation formula, that is: high-priority intention + low-priority title image = high-priority intention = final intention.

[0197] For example: The principle of determining intention priority is that question priority is higher than affirmative priority, and negative priority is higher than question priority. The corresponding calculation formula is: negative > question > affirmative. When affirmative, questionable and negative intentions appear in the same sentence, the intention of the sentence is determined to be negative. When questionable and affirmative intentions appear in the same sentence, the intention of the sentence is determined to be questionable.

[0198] The data processing method provided in the embodiment of the present application has the following beneficial effects:

[0199] Get all sample texts corresponding to the business conversation, determine the priority of texts containing multiple intents, and based on the sample text data labels of different priorities, call the intent recognition models corresponding to different intent priorities. By classifying different intent recognition models based on the standard test set, obtain the model's predicted intent recognition results, determine the key nodes in the execution steps, and determine the final output results of the model based on the key nodes. Analyze the logical rationality of the different intent recognition models based on the logical analysis model, determine the intent priority determination principle, and modify the intent priority of the original sample text to the logically optimal intent based on the determined intent priority determination principle. Through multi-priority training and logical analysis, the accuracy of intent recognition is improved, and it adapts to different business scenarios. According to accurate and reasonable intent recognition results, it effectively responds to care needs and optimizes user experience in different human-computer interaction scenarios.

[0200] The following is a description of an exemplary structure of the data processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the data processing device 455 of the memory 450 may include: a model training module 4551, which is used to obtain sample text data, each sample text data is annotated with an intent label of a different intent level; based on each sample text data, the model is trained separately to obtain an intent recognition model for each intent level; a priority sorting module 4552, which is used to perform intent classification processing on the first test sample through each intent recognition model to obtain a predicted intent sequence corresponding to each intent recognition model, the predicted intent sequence is used to characterize the order of appearance of the intent corresponding to the first predicted intent label in the first test sample, and the first predicted intent label is the intent label predicted by the intent recognition model for the first test sample; based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each intent recognition model, the intent priority sorting of each intent level composition is determined; a label migration module 4553, which is used to modify the intent label of the sample text data based on the intent priority sorting to obtain the sample text data with label migration.

[0201] In some embodiments, model training 4551 is also used to perform the following processing on the sample text data of each intent level: based on the sample text data of the intent level, classification processing is performed through the model to obtain a second predicted intent label corresponding to the sample text data of the intent level; based on the difference between the second predicted intent label and the sample intent label corresponding to the intent level, the loss function of the model is determined; based on the loss function, the parameters of the model are updated to obtain an intent recognition model corresponding to the intent level.

[0202] In some embodiments, after the model is trained based on each sample text data to obtain the intent recognition model of each intent level, model training 4551 is also used to perform classification processing based on the test sample set through each intent recognition model to obtain the predicted intent set corresponding to each intent recognition model, the test sample set includes multiple second test samples and the actual intent label corresponding to each second test sample, and the predicted intent set includes the second predicted intent label corresponding to each second test sample; for each intent recognition model, the number of correct predictions and the number of incorrect predictions of the intent recognition model are determined based on the test sample set and the predicted intent set corresponding to the intent recognition model; based on the number of correct predictions and the number of incorrect predictions, the effectiveness indicator of the intent recognition model is determined.

[0203] In some embodiments, the effect indicators include at least one of the following: accuracy, precision and recall; after determining the effect indicators of the intent recognition model based on the number of correct predictions and the number of incorrect predictions, model training 4551 is also used to re-train the intent recognition model based on sample text data of the corresponding intent level when at least one effect indicator of the intent recognition model is less than the corresponding effect indicator threshold.

[0204] In some embodiments, after the model is trained based on each sample text data to obtain the intent recognition model of each intent level, model training 4551 is also used to perform classification processing based on the third test sample through each intent recognition model to obtain the third intent label and prediction probability set corresponding to each intent recognition model, and the prediction probability set includes the prediction probability of the third test sample belonging to different types of intent labels; the following processing is performed for each intent recognition model: determine the difference between the prediction probabilities in the prediction probability set corresponding to the intent recognition model; when the difference is less than the pre-configured difference, re-train the intent recognition model based on the sample text data of the corresponding intent level.

[0205] In some embodiments, the priority sorting module 4552 is also used to perform the following processing respectively through each intention recognition model based on the first test sample: perform word segmentation processing on the first test sample to obtain multiple text fragments of the first test sample; perform feature extraction processing on the multiple text fragments in turn to obtain text features of each text fragment; for each text feature, predict the probability that the text feature belongs to the corresponding candidate intent label, and use the candidate intent label with the largest probability as the predicted intent label of the text feature, and the order of appearance of each text fragment in the first test sample corresponds to the order of appearance of the predicted intent labels; according to the order of appearance of multiple text fragments, each predicted intent label is combined to obtain a predicted intent sequence corresponding to the intent recognition model.

[0206] In some embodiments, the priority sorting module 4552 is also used to perform the following processing for each intent recognition model: compare the intent labels at the same position in the intent knowledge graph and the predicted intent sequence corresponding to the intent recognition model, and count the first number of intent labels with the same position and the same type; take the ratio between the first number and the total number of labels in the predicted intent sequence as the overlap; determine the logical score of the intent recognition model based on the overlap, wherein the overlap is positively correlated with the logical score; determine the intent priority sorting of each intent level according to the logical score corresponding to the intent recognition model of each intent level, and in the intent priority sorting, the order of the intent levels is positively correlated with the priority.

[0207] In some embodiments, the types of intent labels include: preconfigured labels and non-preconfigured labels. The label migration module 4553 is also used to perform intent classification processing on the sample text data to obtain multiple candidate intent labels for the sample text data; when the multiple candidate intent labels only include non-preconfigured labels, the intent label is replaced with the first target intent label with the highest intent level among the multiple candidate intent labels according to the intent priority ranking; when the multiple candidate intent labels include at least one preconfigured label, the second target intent label is determined according to the intent priority ranking, and the intent label is replaced with the second target intent label, and the second target intent label is a candidate intent label whose intent level is second only to at least one preconfigured label.

[0208] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the data processing method described above in the embodiment of the present application.

[0209] The present application embodiment provides a computer-readable storage medium, in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will execute the data processing method provided by the present application embodiment, for example, Figure 3A The data processing method is shown.

[0210] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0211] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0212] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0213] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0214] In summary, through the embodiments of the present application, different initial intent levels are labeled and model trained for sample text data to improve the accuracy and logic of intent recognition, ensure that high-priority intents are given priority, and ensure that the user interaction experience is optimized in complex interaction scenarios.

[0215] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire sample text data, each of the sample text data being annotated with intent labels of different intent levels; Based on each of the sample text data, the model is trained respectively to obtain an intent recognition model for each of the intent levels; Performing intent classification processing on the first test sample by each of the intent recognition models, obtaining a predicted intent sequence corresponding to each of the intent recognition models, wherein the predicted intent sequence is used to characterize an order of occurrence of intents corresponding to a first predicted intent label in the first test sample, wherein the first predicted intent label is an intent label predicted by the intent recognition model for the first test sample; Determine the intent priority ranking of each of the intent level components based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each of the intent recognition models; The intent labels of the sample text data are modified based on the intent priority ranking to obtain sample text data with label migration.

2. The method according to claim 1, characterized in that: The model is trained based on each of the sample text data to obtain an intention recognition model for each intention level, including: The following processing is performed on the sample text data of each intention level: Based on the sample text data of the intent level, performing classification processing through the model to obtain a second predicted intent label corresponding to the sample text data of the intent level; Determining a loss function of the model based on a difference between the second predicted intent label and the sample intent label corresponding to the intent level; The parameters of the model are updated based on the loss function to obtain an intent recognition model corresponding to the intent level.

3. The method according to claim 1, characterized in that After the model is trained based on each of the sample text data to obtain the intent recognition model of each intent level, the method further includes: Based on the test sample set, classification processing is performed by each of the intention recognition models to obtain a predicted intention set corresponding to each of the intention recognition models, the test sample set includes multiple second test samples and an actual intention label corresponding to each of the second test samples, and the predicted intention set includes a second predicted intention label corresponding to each of the second test samples; For each of the intent recognition models, based on the test sample set and the predicted intent set corresponding to the intent recognition model, determine the number of correct predictions and the number of incorrect predictions of the intent recognition model; Based on the number of correct predictions and the number of incorrect predictions, an effectiveness indicator of the intent recognition model is determined.

4. The method according to claim 3, characterized in that The performance indicators include at least one of the following: accuracy, precision and recall; After determining the effect index of the intent recognition model based on the number of correct predictions and the number of incorrect predictions, the method further includes: When at least one of the effect indicators of the intention recognition model is less than a corresponding effect indicator threshold, the intention recognition model is retrained based on the sample text data of the corresponding intention level.

5. The method according to claim 4, characterized in that After the model is trained based on each of the sample text data to obtain the intent recognition model of each intent level, the method further includes: Based on the third test sample, each of the intent recognition models performs classification processing respectively to obtain a third intent label and a prediction probability set respectively corresponding to each of the intent recognition models, wherein the prediction probability set includes prediction probabilities that the third test sample belongs to different types of intent labels; The following processing is performed for each of the intent recognition models: Determining the difference between the predicted probabilities in the predicted probability set corresponding to the intent recognition model; When the difference is less than a preconfigured difference, the intent recognition model is retrained based on the sample text data of the corresponding intent level.

6. The method according to claim 1, characterized in that The performing intent classification processing on the first test sample by each of the intent recognition models to obtain a predicted intent sequence corresponding to each of the intent recognition models includes: Based on the first test sample, the following processing is performed respectively by each of the intention recognition models: Performing word segmentation processing on the first test sample to obtain multiple text segments of the first test sample; Performing feature extraction processing on the multiple text segments in sequence to obtain text features of each of the text segments; For each of the text features, predict the probability that the text feature belongs to the corresponding candidate intent label, and use the candidate intent label with the greatest probability as the predicted intent label of the text feature, and the appearance order of each of the text segments in the first test sample corresponds to the appearance order of the predicted intent labels; According to the order of appearance of the multiple text fragments, each of the predicted intent labels is combined to obtain a predicted intent sequence corresponding to the intent recognition model.

7. The method according to claim 1, characterized in that The step of determining the intent priority ranking of each of the intent level components based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each of the intent recognition models comprises: The following processing is performed for each of the intent recognition models: Compare the intent knowledge graph and the intent labels at the same position in the predicted intent sequence corresponding to the intent recognition model, and count a first number of intent labels at the same position and of the same type; The ratio between the first number and the total number of labels in the predicted intent sequence is used as the overlap; Determining a logic score of the intent recognition model based on the overlap, wherein the overlap is positively correlated with the logic score; According to the logical score corresponding to the intention recognition model of each intention level, the intention priority ranking of each intention level is determined, and in the intention priority ranking, the order of the intention levels is positively correlated with the priority.

8. The method according to claim 1, characterized in that The types of the intent tags include: preconfigured tags and non-preconfigured tags; The modifying the intent label of the sample text data based on the intent priority ranking to obtain the sample text data with label migration includes: Performing intent classification processing on the sample text data to obtain a plurality of candidate intent labels for the sample text data; In a case where the multiple candidate intent tags include only non-preconfigured tags, replacing the intent tag with a first target intent tag with the highest intent level among the multiple candidate intent tags according to the intent priority ranking; In the case where the multiple candidate intent labels include at least one preconfigured label, a second target intent label is determined according to the intent priority ranking, and the intent label is replaced with the second target intent label, where the second target intent label is the candidate intent label whose intent level is second only to the at least one preconfigured label.

9. A data processing device, characterized in that: The device comprises: A model training module is used to obtain sample text data, each of which is annotated with an intent label of a different intent level; based on each of the sample text data, the model is trained separately to obtain an intent recognition model for each intent level; A priority sorting module, used to perform intent classification processing on the first test sample through each of the intent recognition models to obtain a predicted intent sequence corresponding to each of the intent recognition models, wherein the predicted intent sequence is used to characterize the order of appearance of the intent corresponding to the first predicted intent label in the first test sample, and the first predicted intent label is the intent label predicted by the intent recognition model for the first test sample; based on the intent knowledge graph of the first test sample and the predicted intent sequence corresponding to each of the intent recognition models, determine the intent priority sorting of each of the intent level components; The label migration module is used to modify the intent label of the sample text data based on the intent priority ranking to obtain the sample text data with the label migrated.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the data processing method according to any one of claims 1 to 8 when executing the computer executable instructions or computer programs stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the data processing method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the data processing method according to any one of claims 1 to 8 is implemented.

Images