Data processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a data processing method and device, equipment and a storage medium, which are used for improving the quality of call data in a training data set and improving the training effect of a dialogue model trained by using the training data set. The method comprises the following steps: acquiring a first data set comprising a plurality of pieces of first call data; inputting the first data set into a preset dialogue model to obtain the confusion degree of each piece of first call data; obtaining second call data based on the confusion degree of each piece of first call data to form a second data set; and based on the client intention of each piece of second call data in the second data set, performing clustering processing on the second data set, screening the second call data of each type of client intention, obtaining screened second call data, forming a training data set, and training a preset dialogue model by using the training data set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data processing method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of economic business of various industries, enterprises in the securities, banking, trust and other industries have accumulated a large amount of customers through various channels such as online and offline. The customer service personnel of the enterprise communicates with the customers by telephone according to the needs of the customers, for example, the customer service personnel of the enterprise interacts with the customers through telephone calls to provide services such as financial products, account information and solutions to customer questions. However, the number of customer service personnel is limited, in order to ensure timely service to customers, the enterprise will use customer service robots to replace customer service personnel to automatically communicate with customers who need to communicate, wherein the telephone communication between the customer service robot and the customer is realized through a dialogue model in the customer service robot.

[0003] In the prior art, the multiple call data of the customer service personnel who performs well in a set time period is usually taken as a training data set, and the dialogue model in the customer service robot is trained using the training data set. However, the quality of the multiple call data included in the training data set is uneven, thereby affecting the training effect of the dialogue model. SUMMARY

[0004] The embodiments of the present application provide a data processing method, device, equipment and storage medium, which are used to improve the quality of call data in the training data set and improve the training effect of the dialogue model trained using the training data set.

[0005] In a first aspect, the present application provides a data processing method, which comprises:

[0006] obtaining a first data set comprising multiple first call data;

[0007] inputting the first data set into a preset dialogue model to obtain the perplexity of each first call data;

[0008] obtaining second call data based on the perplexity of each first call data to form a second data set;

[0009] performing clustering processing on the second data set based on the customer intent of each second call data in the second data set, and screening the second call data of each customer intent to obtain screened second call data to form a training data set, so as to train the preset dialogue model using the training data set.

[0010] In the embodiments of the present application, a first data set including a plurality of first call data is obtained, and the first data set is input into a preset dialogue model to obtain a perplexity of each first call data, so as to more accurately represent the quality of the first call data by using the perplexity of the first call data. If the perplexity of the first call data is too small, it indicates that the first call data has more repeated data, and if the perplexity of the first call data is too large, it indicates that the first call data has syntax, misspelling and other problems. Therefore, the second call data is obtained based on the perplexity of each first call data to form a second data set, so as to improve the representativeness and balance of the second call data in the second data set, and further improve the quality of the second call data in the second data set. The second data set is clustered based on the customer intent of each second call data in the second data set, and the second call data of each customer intent is screened to obtain screened second call data to form a training data set, so as to train the preset dialogue model by using the training data set, thereby reducing the data amount of the training data set, and because the training data set includes a plurality of customer intent call data, the demand in different scenarios is met. Therefore, the quality of the call data in the training data set can be improved, and the training effect of the dialogue model trained by using the training data set can be improved.

[0011] In a second aspect, the present application provides a data processing apparatus, comprising:

[0012] an obtaining module configured to obtain a first data set including a plurality of first call data;

[0013] a processing module configured to input the first data set into a preset dialogue model to obtain a perplexity of each first call data;

[0014] a forming module configured to obtain second call data based on the perplexity of each first call data to form a second data set;

[0015] a screening module configured to cluster the second data set based on a customer intent of each second call data in the second data set, and screen the second call data of each customer intent to obtain screened second call data to form a training data set, so as to train the preset dialogue model by using the training data set.

[0016] In a third aspect, the present application provides an electronic device, comprising:

[0017] a memory configured to store program instructions;

[0018] a processor configured to call the program instructions stored in the memory, and perform the steps included in the method of any one of the first aspect according to the obtained program instructions.

[0019] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, the computer program comprising program instructions, when executed by a computer, causing the computer to perform the method of any one of the first aspect.

[0020] In a fifth aspect, the present application provides a computer program product, which comprises computer program code, when the computer program code is run on a computer, causing the computer to perform the method of any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is an application scenario diagram according to an exemplary embodiment;

[0023] Figure 2 is a flowchart of a data processing method according to an exemplary embodiment;

[0024] Figure 3 is a flowchart of a method for obtaining a first data set comprising a plurality of first call data according to an exemplary embodiment;

[0025] Figure 4 is a flowchart of a method for determining the confusion degree of the first call data according to an exemplary embodiment;

[0026] Figure 5 is a flowchart of a method for reducing the first call data with higher similarity according to an exemplary embodiment;

[0027] Figure 6 is a flowchart of a method for determining key information according to an exemplary embodiment;

[0028] Figure 7 is a detailed flowchart of a data processing method in a financial scenario according to an exemplary embodiment;

[0029] Figure 8 is a schematic diagram of a data processing device according to an exemplary embodiment;

[0030] Figure 9 is an electronic device schematic diagram of a data processing method according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application. The embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily without conflict. Moreover, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from the order shown.

[0032] The terms “first” and “second” in the specification and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific sequence. In addition, the term “comprising” and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. “Multiple” in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0033] In the technical solutions of the present application, the collection, transmission and use of data comply with the requirements of relevant national laws and regulations.

[0034] Before introducing the data processing method provided by the embodiments of the present application, in order to facilitate understanding, first, the technical background of the embodiments of the present application is introduced in detail.

[0035] With the development of economic business of various industries, enterprises in securities, banks, trusts and other industries have accumulated a large number of customers through various channels such as online and offline. Customer service personnel of the enterprise communicate with customers by telephone according to the needs of customers, for example, customer service personnel of the enterprise interact with customers through telephone calls to provide services such as financial products, account information and solving customer questions. However, the number of customer service personnel is limited, in order to ensure timely service to customers, the enterprise will use customer service robots to replace customer service personnel to automatically communicate with customers who need to communicate, wherein the telephone communication between the customer service robot and the customer is realized through a dialogue model in the customer service robot.

[0036] In the prior art, usually, multiple pieces of call data of a customer service personnel who performs well in a set time period are taken as a training data set, and a dialogue model in a customer service robot is trained by using the training data set. However, the quality of the multiple pieces of call data included in the training data set is uneven, thereby affecting the training effect of the dialogue model.

[0037] To solve the above technical problem, the present application provides a data processing method, device, equipment and storage medium, which are used for improving the quality of call data in a training data set and improving the training effect of a dialogue model trained by using the training data set.

[0038] Firstly, refer to Figure 1 which is a schematic diagram of an application scenario of an embodiment, including a collector 10 and a server 11. The collector 10 and the server 11 are connected through a network, which can be a local area network, a wide area network, etc. The server 11 can be any device capable of providing Internet service. The collector 10 can be a device for collecting first call data, for example, the collector 10 can be a mobile device (such as a mobile phone, a tablet computer, a notebook computer, etc.), or a personal computer (PC, Personal Computer). The collector 10 is used to collect first call data generated by a customer service personnel and a customer in a call, so as to obtain a first data set, and send the first data set to the server 11. The server 11 determines a training data set according to the obtained first data set including multiple pieces of first call data, so as to train a preset dialogue model by using the training data set.

[0039] One possible form of communication between the collector 10 and the server 11 is that the server 11 obtains the first data set including multiple pieces of first call data collected by the collector 10; inputs the first data set into the preset dialogue model to obtain the perplexity of each piece of first call data; obtains second call data based on the perplexity of each piece of first call data, and forms a second data set; performs clustering processing on the second data set based on the customer intent of each piece of second call data in the second data set, and filters the second call data of each type of customer intent, to obtain filtered second call data, and forms a training data set, so as to train the preset dialogue model by using the training data set.

[0040] In some embodiments, the data processing method provided by the present application is described below through specific embodiments, as shown in Figure 2 , which includes

[0041] Step 201, obtaining a first data set including multiple pieces of first call data.

[0042] Specifically, the first data set including multiple pieces of first call data in a set time period is obtained.

[0043] The set time period can be set according to actual conditions, for example, the set time period can be 1 month or 3 months.

[0044] Since there are many initial call data in the initial data set within the set time period, and the quality of the initial call data in the initial data set is uneven, the application screens the initial call data in the initial data set according to the task success rate of the customer service to determine the first data set. Figure 3 is a flowchart of a method for obtaining a first data set including a plurality of first call data according to an exemplary embodiment, as shown in Figure 3 The step 201 at least includes the following steps:

[0045] Step 301, obtaining an initial data set including a plurality of initial call data;

[0046] Specifically, an initial data set including a plurality of initial call data in a set time period is obtained.

[0047] The set time period can be set according to actual conditions, for example, the set time period can be 1 week. Each customer service corresponds to at least one initial call data. The initial call data is text data generated by the call between the customer service and the customer.

[0048] Since the call between the customer service and the customer generates voice data, in order to obtain the initial call data in the form of text, it is necessary to process the voice data by using the speech recognition technology to obtain the initial call data. The speech recognition technology can be ASR (Automatic Speech Recognition) or a speech recognition model. The specific method of using the speech recognition technology to process the voice data to obtain the initial call data in the application is prior art, which will not be described in detail here.

[0049] For example, the customer service 1 calls the customer surnamed Wang, obtains voice data, and inputs the voice data into the speech recognition model to obtain the following initial call data:

[0050] {Hello, hello, hello, are you Mr. Wang? I am, hello Mr. Wang, I am a staff of company T, calling to inform you that your H account has been increased, and your total limit is now 17,000, you can withdraw as needed, thank you, no, Mr. Wang, did you withdraw 6,000 last month? There is still 1,300 left, did you encounter any problems when withdrawing? I have money now, I don't need it}

[0051] Step 302, determining the task success rate of each customer service based on each initial call data;

[0052] The task success rate of each customer service represents the task success of the customer service, that is, the task success of the customer service in a set time period. For each task, the task success rate of each customer service in the task needs to be determined. The task can be selected according to actual conditions, for example, the task can be a withdrawal task, a increase task, a balance task, etc. Since the task can be selected according to actual conditions, if there are M tasks, the task success rate corresponding to the M tasks needs to be determined for any one customer service, and M is an integer greater than 0. Since each customer service handles different tasks, if the target task is not handled by the customer service in the set time period, the task success rate corresponding to the target task is determined to be 0. For example, if customer service 2 does not talk to the customer due to the withdrawal loan task in the set time period, it is determined that the task success rate of customer service 2 for the withdrawal loan task is 0.

[0053] In this application, the task success rate of the i-th customer service in the j-th task P ij can be determined according to the following formula:

[0054]

[0055] where a ij is the number of initial call data of the i-th customer service successfully completing the j-th task in a set time period, b ij is the total number of initial call data of the i-th customer service completing the j-th task in a set time period, i is an integer greater than or equal to 1, and j is an integer greater than or equal to 1.

[0056] For example, customer service 1 talks to 100 customers for a withdrawal loan task in 1 month, gets 100 initial call data, and 10 customers have withdrawal loan behavior, which indicates that the number of initial call data of customer service 1 successfully completing the withdrawal loan task in 1 month is 10, and therefore the task success rate of customer service 1 for the withdrawal loan task is 10 / 100=0.1.

[0057] Step 303, determining a plurality of target customer services based on the task success rate of each customer service;

[0058] Optionally, determining a plurality of target customer services based on the task success rate of each customer service, comprising:

[0059] For any one task, the task success rate of each customer service in the task is sorted in descending order, the ranking of each customer service is determined, and the customer service ranked before a set ranking is determined as a target customer service.

[0060] Wherein, the set ranking can be set according to actual conditions, for example, the set ranking can be 16.

[0061] For example, for task A, the task success rate of customer service 1 is 0.3, the task success rate of customer service 2 is 0.5, the task success rate of customer service 3 is 0.1, the task success rate of customer service 4 is 0.8, and the task success rate of customer service 5 is 0.9. In descending order, the ranking of customer service 1 is 4, the ranking of customer service 2 is 3, the ranking of customer service 3 is 5, the ranking of customer service 4 is 2, and the ranking of customer service 5 is 1. If the set ranking is 3, the customer services whose rankings are before the set ranking are customer service 4 and customer service 5, and therefore, customer service 4 and customer service 5 are determined as target customer services.

[0062] In the embodiments of the present application, the target customer services determined for different tasks can be completely different or have the same target customer service. If the target customer services determined for different tasks are completely different, the target customer services corresponding to each task are processed for merging; if the target customer services determined for different tasks have the same target customer service, the target customer services corresponding to each task are processed for merging and de-duplication.

[0063] For example, if customer service 1 and customer service 2 are determined as target customer services for task A, and customer service 1 and customer service 3 are determined as target customer services for task B, the target customer services determined by the present application include customer service 1, customer service 2, and customer service 3. If customer service 1 and customer service 2 are determined as target customer services for task A, and customer service 1 and customer service 2 are determined as target customer services for task B, the target customer services determined by the present application include customer service 1 and customer service 2. If customer service 1 and customer service 2 are determined as target customer services for task A, and customer service 3 and customer service 4 are determined as target customer services for task B, the target customer services determined by the present application include customer service 1, customer service 2, customer service 3, and customer service 4.

[0064] In step 304, first call data is obtained based on the initial call data of the plurality of target customer services, and a first data set is formed.

[0065] Specifically, the initial call data of the plurality of target customer services in the initial data set is taken as the first call data, and the first data set is formed.

[0066] For example, the initial data set includes 500 pieces of initial call data of customer 1, 300 pieces of initial call data of customer 2, 400 pieces of initial call data of customer 3, and 100 pieces of initial call data of customer 4. If customer service 2 and customer service 4 are target customer services, the initial call data of customer service 2 and the initial call data of customer 4 in the initial data set are taken as the first call data, and the first data set is formed, that is, the first data set includes 400 pieces of first call data.

[0067] In the embodiment of the present application, based on the initial data set including the plurality of initial call data in the set time period, the task success rate of each customer service is determined, and based on the task success rate of each customer service, the plurality of target customer services, i.e., the target customer services with higher task success rate in the set time period, are determined. Moreover, the present application obtains the first call data based on the initial call data of the plurality of target customer services to form the first data set, thereby reducing the data amount of the first data set and efficiently screening out the first call data with potential commercial value, i.e., improving the quality of the first call data in the first data set.

[0068] In step 202, the first data set is input into a preset dialogue model to obtain the perplexity of each first call data.

[0069] The perplexity represents the quality of the first call data. Figure 4 FIG. 3 is a flowchart of a method for determining the perplexity of the first call data according to an example embodiment. Figure 4 As shown in FIG. 3, the step 202 of inputting the first data set into the preset dialogue model to obtain the perplexity of each first call data includes at least the following steps:

[0070] In step 401, for any first call data, the first call data is input into the preset dialogue model to obtain a predicted call result.

[0071] The first call data includes a plurality of rounds of dialogue data, and each round of dialogue data includes a customer dialogue data and a customer service dialogue data. The customer dialogue data is the data expressed by the customer in the process of calling the customer service, and the customer service dialogue data is the data expressed by the customer service in the process of calling the customer. The predicted call result is the customer service dialogue data determined by the preset dialogue model based on the customer dialogue data in the first call data. Since the first call data includes a plurality of customer dialogue data and customer service dialogue data, the present application inputs the first call data into the preset dialogue model to obtain a sub-predicted call result corresponding to each customer dialogue data in the first call data. Based on the sub-predicted call result corresponding to each customer dialogue data, the predicted call result is obtained.

[0072] For example, the first call data is {Hello, you are Mr. Li, right? Ah, hello, I am not, did you dial the wrong number? I am very sorry, I disturb you, wish you a happy life, goodbye}. The first call data includes two pieces of customer dialogue data and two pieces of customer service dialogue data, that is, the first piece of customer dialogue data is {Hello, you are Mr. Li, right?}, the second piece of customer dialogue data is {Ah, hello, I am not, did you dial the wrong number?}, the first piece of customer service dialogue data is {You are Mr. Li, right?}, and the second piece of customer service dialogue data is {I am very sorry, I disturb you, wish you a happy life, goodbye}. The first call data is input into the preset dialogue model, the preset dialogue model predicts the first piece of customer service dialogue data according to the first piece of customer dialogue data in the first call data to obtain a first sub-predicted call result, and predicts the second piece of customer service dialogue data according to the second piece of customer dialogue data in the first call data to obtain a second sub-predicted call result. According to the first sub-predicted call result and the second sub-predicted call result, a predicted call result of the first call data is obtained.

[0073] In step 402, a loss value of the first call data between the predicted call result and the first call data is calculated by using a loss function of the preset dialogue model.

[0074] The loss function of the preset dialogue model can be set according to actual conditions, for example, the loss function can be an experience calibration error loss function, a cross-entropy loss function, or the like.

[0075] Since the predicted call result of the first call data includes a sub-predicted result corresponding to each piece of customer dialogue data, for any piece of customer dialogue data, a sub-loss value of the customer dialogue data between the customer service dialogue data and the corresponding sub-predicted call result is calculated by using the loss function of the preset dialogue model; and based on the sub-loss value of each piece of customer dialogue data, the loss value of the first call data is determined.

[0076] The application can perform summation operation on the sub-loss value of each piece of customer dialogue data to obtain a sum value, and take the obtained sum value as the loss value of the first call data; or perform average operation on the sub-loss value of each piece of customer dialogue data to obtain an average value, and take the obtained average value as the loss value of the first call data.

[0077] For example, the first call data is {Hello, you are Mr. Li, right? Ah, hello, I am, Mr. Li, you are, I am a staff member of Company T, calling to inform you that your certificate will expire on March 10, 2024, please update it as soon as possible, I know, goodbye, wish you a happy life, goodbye}. The first call data includes three customer conversation data and three customer service conversation data, i.e., the first customer conversation data is {Hello, you are Mr. Li, right?}, the second customer conversation data is {Ah, hello, I am}, and the third customer conversation data is {Good, I know, goodbye}. The first customer service conversation data is {You are Mr. Li, right?}, the second customer service conversation data is {Ah, hello, I am}, and the third customer service conversation data is {Good, I know, goodbye}.

[0078] The first call data is input into the preset conversation model. The preset conversation model predicts the first customer service conversation data according to the first customer conversation data in the first call data to obtain a sub-predicted conversation result A, predicts the second customer service conversation data according to the second customer conversation data in the first call data to obtain a sub-predicted conversation result B, and predicts the third customer service conversation data according to the third customer conversation data in the first call data to obtain a sub-predicted conversation result C. Therefore, the predicted conversation result of the first call data includes the sub-predicted conversation result A, the sub-predicted conversation result B, and the sub-predicted conversation result C. The loss function of the preset conversation model is used to calculate a sub-loss value A between the sub-predicted conversation result A and the first customer service conversation data, the loss function of the preset conversation model is used to calculate a sub-loss value B between the sub-predicted conversation result B and the second customer service conversation data, and the loss function of the preset conversation model is used to calculate a sub-loss value C between the sub-predicted conversation result C and the third customer service conversation data. Based on the sub-loss value A, the sub-loss value B, and the sub-loss value C, the loss value of the first call data is determined.

[0079] In step 403, an exponential function value with a natural constant as a base and the loss value of the first call data as an index is determined, and the exponential function value is taken as the perplexity of the first call data.

[0080] Specifically, the perplexity of the first call data k can be determined by the following formula:

[0081] perplexity(k)=e loss(k) ;

[0082] wherein loss(k) is the loss value of the first call data k, k is a positive integer greater than or equal to 0, and e is a natural constant.

[0083] In the embodiment of the present application, the first call data is input into a preset dialogue model to obtain a predicted call result, and based on the predicted call result and the first call data, a loss value of the first call data is determined, and based on the loss value of the first call data, a perplexity of the first call data is determined, so as to more accurately represent the quality of the first call data.

[0084] In the process of determining the first data set, the voice recognition technology is used to convert the voice data into text data. Due to the defects of the voice recognition technology and the influence of the noise data in the voice data, the converted call data may be inaccurate. Therefore, in order to ensure the accuracy of each first call data in the first data set, before the first data set is input into the preset dialogue model, the present application further comprises:

[0085] The error correction model and / or the set error correction information set are used to correct each first call data.

[0086] The error correction model can be pycorrector (an error correction tool) or other error correction models. The first call data is input into the error correction model to obtain the first call data corrected by the error correction model.

[0087] The set error correction information set includes at least one error correction information, which can be error correction words, error correction sentences, error correction phrases, etc. The set error correction information set can be set according to actual conditions.

[0088] In the embodiment of the present application, the error correction model and / or the set error correction information set are used to correct each first call data, so as to improve the accuracy of each first call data in the first data set, so as to ensure the subsequent data processing process based on the accurate first call data.

[0089] Optionally, before the first data set is input into the preset dialogue model, the present application further comprises:

[0090] The first data set is filtered based on the set filtering rule.

[0091] In the embodiment of the present application, the first data set is filtered based on the set filtering rule, so as to improve the quality of the first call data in the first data set.

[0092] Optionally, the set filtering rule includes part or all of the following:

[0093] The first call data without ending language in the first data set is removed;

[0094] The first call data with dialogue rounds less than the set round threshold in the first data set is removed;

[0095] eliminate the first call data in the first data set including the key information of the set event;

[0096] eliminate the first call data in the first data set whose customer feature information is inconsistent with the pre-stored customer feature information.

[0097] The end word can be bye, goodbye, etc. The number of dialogue rounds of the first call data is the number of rounds of dialogue between the customer service and the customer in the first call data. Each round of dialogue data includes one customer dialogue data and one customer service dialogue data. The set round threshold can be set according to actual conditions, for example, the set round threshold is 5. For example, the first call data is: Customer: Hello, hello. Customer service: Hello, are you Mr. Wang? Customer: Yes, hello. Customer service: Hey, Mr. Wang, this is a staff from Company B. I'm calling to inform you that your account has been increased. Your total limit is now 300,000, and you can withdraw as needed. In the first call data, the customer speaks once and the customer service speaks once, so the number of dialogue rounds of the first call data is 2.

[0098] In the embodiments of the present application, by eliminating the first call data without an end word in the first data set and eliminating the first call data with a number of dialogue rounds less than the set round threshold in the first data set, the noise data is reduced, thereby reducing the influence on the training of the dialogue model and the prediction of the call result.

[0099] The key information of the set event in the present application can be set according to actual conditions, for example, the key information of the event is the key word of the holiday, and the key word of the holiday includes: keywords such as Happy New Year, Happy New Year, Happy New Year, Happy New Year, Happy New Year, Qingming, May Day holiday, May Day activities, May Day, May Day, Dragon Boat Festival, Brothers, Mid-Autumn Festival, National Day, and November holiday. In the embodiments of the present application, if the first call data includes at least one keyword, the first call data is eliminated from the first data set to ensure that the training of the dialogue model is consistent with the actual scene.

[0100] The customer feature information in the present application can include customer name, credit limit, available withdrawal limit, principal balance, and recent withdrawal time, etc. Since the customer service and the customer have a call in real time to obtain the customer feature information, but the customer service fails to timely query the pre-stored customer feature information of the day, only the pre-stored customer feature information of the previous day can be obtained, in addition, there may be some recognition errors in speech recognition technology, therefore, based on the first call data, the customer feature information of the first call data is extracted, the customer feature information of the first call data is detected for consistency with the pre-stored customer feature information, if at least one customer feature information is inconsistent with the pre-stored customer feature information, the first call data is eliminated from the first data set to ensure the accuracy and reliability of the first call data and avoid the illusion of the dialogue model.

[0101] In the embodiments of the present application, the first call data in the first data set is filtered based on the ending words, the number of dialogue rounds, the key information of the set event, and the customer feature information, so as to ensure the accuracy and reliability of the first call data in the filtered first data set.

[0102] Optionally, before the first data set is input into the preset dialogue model, the format of each first call data is adjusted respectively, and each first call data is converted into an input format required by the preset dialogue model, so as to facilitate subsequent data processing.

[0103] The input format required by the preset dialogue model can be a character format, a number format, etc.

[0104] For example, if the characters in the first call data are in a traditional Chinese format, and the input format required by the preset dialogue model is a simplified Chinese format, the character format of the first call data is converted into a simplified Chinese format.

[0105] In step 203, the second call data is obtained based on the perplexity of each first call data, and a second data set is formed.

[0106] Optionally, the second call data is obtained based on the perplexity of each first call data, and includes:

[0107] The first call data with a perplexity within a set range is obtained from the first data set, and the obtained first call data is used as the second call data.

[0108] The set range can be set according to actual conditions, for example, the set range is 0.25-0.75, that is, the perplexity of the first call data is greater than or equal to 0.25, or less than or equal to 0.75, and the first call data is used as the second call data. The perplexity of the first call data represents whether the syntax of the first call data is correct, whether there are typos, etc., which is a probability of text compliance. The smaller the perplexity of the first call data is, the better the quality of the first call data is, and the greater the perplexity of the first call data is, the more likely the first call data has syntax problems or typos. According to actual analysis, if the perplexity of the first call data is less than 0.25, it indicates that the first call data has a lot of repeated data, such as repeating the same sentence many times, and if the perplexity of the first call data is greater than 0.75, it indicates that the first call data has syntax, typo, etc.

[0109] For example, if the set range is 0.25-0.75, and the perplexity of the first call data 1 is 0.3, the perplexity of the first call data 2 is 0.2, the perplexity of the first call data 3 is 0.8, and the perplexity of the first call data 4 is 0.5, it is determined that the perplexity of the first call data 1 and the perplexity of the first call data 4 are within the set range, and the first call data 1 and the first call data 4 are determined as the second call data.

[0110] Since there are many first call data with high similarity in the first data set, in order to ensure the diversity and integrity of the first call data, the first call data with high similarity needs to be reduced before obtaining the second call data based on the perplexity of each first call data. Figure 5 is a flowchart of a method for reducing first call data with high similarity according to an exemplary embodiment, as shown in Figure 5 includes the following steps:

[0111] Step 501: input the first data set into a preset dialogue model to obtain the call feature information of each first call data;

[0112] The call feature information of the first call data can be the call vector of the first call data. The call vector of the first call data is obtained by inputting the first call data into the preset dialogue model. Specifically, the first call data is input into the preset dialogue model, and the preset dialogue model outputs a hidden layer vector (hidden_state). The dimension of the hidden layer vector is: 1*first call data length*K, where K can be set according to actual conditions, for example, K=4096. The hidden layer vector is reduced in dimension, i.e. the average value of the middle dimension is taken, so that the dimension is reduced to 1*1*K. The K-dimensional vector is taken as the call vector of the first call data, i.e. the call feature information of the first call data.

[0113] After inputting the first data set into the preset dialogue model, the perplexity and the call feature information of each first call data output by the preset dialogue model can be obtained at the same time.

[0114] Step 502: based on the call feature information of each first call data, clustering processing is performed on each first call data to obtain a plurality of data clusters;

[0115] Specifically, based on the call feature information of each first call data, clustering processing is performed on each first call data by using a clustering method to obtain a plurality of data clusters.

[0116] The clustering method can be a k-means clustering algorithm, a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm, a k-means++ method, a spectral clustering method, and a hierarchical clustering method.

[0117] Taking the k-means algorithm as an example, based on the call feature information of each first call data, the k-means algorithm is used to perform clustering processing on each first call data, and 100 data clusters, i.e., 100 categories, are obtained, and each data cluster includes multiple similar first call data.

[0118] In step 503, for any one data cluster, based on the similarity between any two first call data in the data cluster, the first call data with a similarity greater than a set similarity threshold in the data cluster is removed.

[0119] Specifically, based on the similarity between any two first call data in the data cluster, the first quantity of first call data with a similarity greater than a set similarity threshold in the data cluster is removed.

[0120] The first quantity is less than or equal to the number of first call data with a similarity greater than a set similarity threshold in the data cluster, and the first quantity can be set according to actual conditions. For example, the first quantity can be 10. The first quantity can also be determined according to the number of first call data with a similarity greater than a set similarity threshold in the data cluster. Specifically, the more the number of first call data with a similarity greater than a set similarity threshold in the data cluster, the greater the first quantity, and vice versa. The first quantity of different data clusters can be the same or different. The set similarity threshold can be set according to actual conditions. For example, the set similarity threshold can be 0.9.

[0121] Specifically, based on the call feature information of any two first call data in the data cluster, the similarity between the two first call data is determined.

[0122] Taking the call feature information as a call vector as an example, the call vectors of any two first call data are x and y, respectively. The cosine similarity cosθ between the two first call data is determined by the following formula:

[0123]

[0124] where x i is the i-th component of the call vector x, y i is the i-th component of the call vector y, n is the total number of components, and i takes a value in the range of [1, n].

[0125] For example, if the first quantity is 50, the set similarity threshold can be 0.9, and there are 100 first call records in the data cluster with a similarity greater than 0.9, then 50 first call data with a similarity greater than 0.9 in the data cluster are removed.

[0126] Because the training data set includes a large amount of similar call data, the dialog model obtained by training the preset dialog model using the training data set is difficult to capture the overall distribution of the call data and is easily affected by extreme values, that is, the dialog model performs too well in the training data set, but performs poorly in the validation data set and the test data set, that is, the generalization error is large and the generalization ability is poor, thereby causing the problem of overfitting of the dialog model. Therefore, in the embodiments of the present application, based on the similarity between any two first call data in the data cluster, the first call data with a similarity greater than the set similarity threshold in the data cluster is removed, that is, the number of first call data with a large similarity in each data cluster is reduced, to ensure the diversity and integrity of the first call data and reduce the occurrence of the problem of overfitting of the dialog model.

[0127] In addition to obtaining the second call data based on the perplexity of each first call data, the second call data can also be obtained based on the perplexity of each first call data and the call feature information after removing the first call data with a similarity greater than the set similarity threshold in the data cluster.

[0128] Optionally, the second call data is obtained based on the perplexity of each first call data and the call feature information, comprising:

[0129] For any one data cluster, if the number of first call data with a perplexity within the set range in the data cluster is greater than or equal to a second quantity, then the second quantity of first call data with a perplexity within the set range is obtained from the data cluster, and the obtained first call data is used as the second call data.

[0130] If the number of first call data with a perplexity within the set range in the data cluster is less than the second quantity, then all first call data with a perplexity within the set range is obtained from the data cluster, and the obtained first call data is used as the second call data.

[0131] The second quantity can be set according to actual conditions, for example, the second quantity can be 100, and the second quantities of different data clusters can be the same or different. The set range can be set according to actual conditions, for example, the set range is 0.25-0.75.

[0132] For example, if the set range is 0.25-0.75, the second quantity is 100, and the first data set includes 3 data clusters, there are 100 pieces of first call data with the perplexity in the set range in data cluster 1, 200 pieces of first call data with the perplexity in the set range in data cluster 2, and 80 pieces of first call data with the perplexity in the set range in data cluster 3, then 100 pieces of first call data with the perplexity in the set range are obtained from data cluster 1, 100 pieces of first call data with the perplexity in the set range are obtained from data cluster 2, and 80 pieces of first call data with the perplexity in the set range are obtained from data cluster 3, the obtained first call data is taken as the second call data, and a second data set is formed, that is, the second data set includes 280 pieces of second call data.

[0133] In step 204, the second data set is clustered based on the customer intention of each piece of second call data in the second data set, and the second call data of each type of customer intention is screened to obtain screened second call data, and a training data set is formed to train the preset dialogue model by using the training data set.

[0134] Optionally, the second call data of each type of customer intention is screened to obtain the screened second call data, including:

[0135] For any type of customer intention, if the number of second call data of the type of customer intention is greater than or equal to a third quantity, the third quantity of second call data of the type of customer intention is obtained, and the obtained second call data is taken as the screened second call data;

[0136] If the number of second call data of the type of customer intention is less than the third quantity, all second call data of the type of customer intention is obtained, and the obtained second call data is taken as the screened second call data.

[0137] The third quantity can be set according to actual conditions, for example, the third quantity can be 500, and the third quantity of each type of customer intention can be the same or different. The process of training the preset dialogue model by using the training data set is a prior art, which will not be described in detail here.

[0138] For example, the second data set includes 1000 pieces of second call data belonging to the customer intention of Figure 1 400 pieces of second call data belonging to the customer intention of Figure 2 600 pieces of second call data belonging to the customer intention of Figure 3 If the third quantity is 500, 500 pieces of second call data belonging to the customer intention of Figure 1 400 pieces of second call data belonging to the customer intention of Figure 2 and 600 pieces of second call data belonging to the customer intention of Figure 3500 pieces of second call data, forming a training data set, and thus the training data set includes 1400 pieces of call data.

[0139] Optionally, after the second data set is formed, the method further includes:

[0140] Inputting the second data set into the intent recognition model to obtain the customer intent of each piece of second call data.

[0141] The intent recognition model can be a language representation model (Bidirectional Encoder Representations from Transformers, BERT), or other intent recognition models. The training process of the intent recognition model is a prior art, and thus will not be described in detail here.

[0142] In the embodiments of the present application, when the preset dialogue model is used to process the historical call data, it is found that the preset dialogue model cannot well reply to some key information in the historical call data. Therefore, in order to further improve the quality of the call data in the training data set and improve the accuracy of the dialogue model trained by using the training data set to process the call data containing some key information, the present application further includes, after forming the training data set:

[0143] Adding the first call data in the first data set, which contains at least one key information in the set key information set, to the training data set.

[0144] The key information can be a keyword, a key sentence, or other information, and the key information can be manually summarized according to the historical predicted call result of each piece of historical call data by the preset dialogue model.

[0145] In the embodiments of the present application, the first call data in the first data set, which contains at least one key information in the set key information set, is added to the training data set to ensure the effectiveness and representativeness of the call data in the training data set, further improve the quality of the call data in the training data set, and improve the accuracy of the dialogue model trained by using the training data set to process the call data containing some key information.

[0146] In the embodiments of the present application, the key information can also be determined according to the historical predicted call result of the historical call data by the preset dialogue model and the historical call data. Figure 6 is a flowchart of a method for determining key information according to an example embodiment, as shown in Figure 6 includes:

[0147] Step 601, obtaining a third data set including a plurality of historical call data;

[0148] Specifically, a third data set including a plurality of historical call data in a historical time period is acquired.

[0149] The historical time period can be determined according to actual conditions, for example, the historical time period is 1 month or 1 year.

[0150] In step 602, the third data set is input into a preset dialogue model to obtain a historical predicted call result of each historical call data.

[0151] The historical call data includes a plurality of rounds of dialogue data, and each round of dialogue data includes a piece of customer dialogue data and a piece of customer service dialogue data. The historical predicted call result is the customer service dialogue data determined by the preset dialogue model according to the customer call data in the historical call data. Since the historical call data includes a plurality of customer dialogue data and customer service dialogue data, the historical call data is input into the preset dialogue model to obtain a sub-historical predicted call result corresponding to each piece of customer dialogue data in the historical call data; and the historical predicted call result is obtained based on the sub-historical predicted call result corresponding to each piece of customer dialogue data.

[0152] In step 603, for any one historical call data, a loss value of the historical call data between the historical predicted call result of the historical call data and the historical call data is calculated by using a loss function of the preset dialogue model.

[0153] The loss function of the preset dialogue model can be determined according to actual conditions, for example, the loss function can be a cross-entropy loss function.

[0154] Since the historical predicted call result of the historical call data includes a sub-historical predicted result corresponding to each piece of customer dialogue data, for any one piece of customer dialogue data, a sub-historical loss value of the customer dialogue data between the customer service dialogue data and the corresponding sub-historical predicted call result is calculated by using the loss function of the preset dialogue model; and the loss value of the historical call data is determined based on the sub-historical loss value of each piece of customer dialogue data.

[0155] For example, the historical call data is: {Hello, how are you? Hello, are you Mr. Wang? Yes, hello. Hey, Mr. Wang, hello, I am a staff of company T, calling to inform you that your H account has been increased, and now your total limit is 20,000 yuan, which can be withdrawn as needed}, which includes two pieces of customer dialogue data and two pieces of customer service dialogue data, i.e. the first piece of customer dialogue data is {Hello, how are you?}, the second piece of customer dialogue data is {Yes, hello}, the first piece of customer service dialogue data is {Hello, are you Mr. Wang?}, and the second piece of customer service dialogue data is {Hey, Mr. Wang, hello, I am a staff of company T, calling to inform you that your H account has been increased, and now your total limit is 20,000 yuan, which can be withdrawn as needed}.

[0156] The historical call data is input into the preset dialogue model, the preset dialogue model predicts the first customer service call data according to the first customer call data in the historical call data to obtain a sub-historical predicted call result 1, and predicts the second customer service call data according to the second customer call data in the historical call data to obtain a sub-historical predicted call result 2. The loss function of the preset dialogue model is used to calculate a sub-historical loss value 1 between the sub-predicted call result 1 and the first customer service call data, and the loss function of the preset dialogue model is used to calculate a sub-historical loss value 2 between the sub-historical predicted call result 2 and the second customer service call data. Based on the sub-historical loss value 1 and the sub-historical loss value 2, the loss value of the historical call data is determined.

[0157] In step 604, if the loss value of the historical call data exceeds the set loss value, the information with the customer call characteristics in the historical call data is extracted as the key information.

[0158] The set loss value can be determined according to actual conditions, for example, the set loss value can be 0.3.

[0159] For example, the historical call data is: {Hello, hello, hello, is Mr. Wang correct? Ah, yes, hello, Mr. Wang, I am a staff of company T, calling to inform you that your H account has been increased, and your total limit is 20,000 yuan, which can be withdrawn as needed. Thank you, no, Mr. Wang, I see that you have withdrawn 10,000 yuan last month, and there is still 10,000 yuan not withdrawn. Is there a problem when withdrawing? I have money now! Congratulations! I apologize for the disturbance. Have a nice life, goodbye}. The preset dialogue model is used to predict the historical call data, the historical predicted call result output by the preset dialogue model is obtained, and the loss function of the preset dialogue model is used to calculate the loss value of the historical call data and the historical predicted call result, which is 0.5. If the set loss value is 0.1, it is determined that the loss value of the historical call data exceeds 0.1, therefore, the information with the customer call characteristics in the historical call data, such as the information of having money, now having money, and I have money now, is extracted as the key information.

[0160] In the embodiment of the application, the third data set including a plurality of historical call data is input into the preset dialogue model to obtain the historical predicted call result of each historical call data. For any one historical call data, the loss value of the historical call data is determined based on the historical predicted call result of the historical call data and the historical call data. When the loss value of the historical call data exceeds the set loss value, the information with the customer call characteristics in the historical call data is extracted as the key information to ensure the accuracy and representativeness of the key information.

[0161] The present application can be applied to a large number of scenarios, such as financial scenarios, sales scenarios, and the like. The data processing method provided by the present application will be described in detail below taking the financial scenario as an example, as shown in Figure 7 The present application can be applied to a large number of scenarios, such as financial scenarios, sales scenarios, and the like. The data processing method provided by the present application will be described in detail below taking the financial scenario as an example, as shown in

[0162] Step 701, obtaining an initial data set including a plurality of initial call data within 3 months;

[0163] The initial data set includes initial call data corresponding to 60 customer services, and each customer service corresponds to at least one initial call data. The initial data set includes 1 million initial call data.

[0164] Step 702, determining the task success rate of each customer service based on each initial call data, and determining a plurality of target customer services based on the task success rate of each customer service;

[0165] Specifically, based on each initial call data, the task success rate of each customer service for each task is determined. The tasks include balance tasks, non-balance tasks, credit T0 (accounting on the same day) tasks, and first loan long tail tasks. For each task, based on the task success rate of each customer service for the task, the top 15 customer services are ranked in descending order of the task success rate, and the top 15 customer services are selected as target customer services.

[0166] Step 703, obtaining first call data based on the initial call data of the plurality of target customer services to form a first data set;

[0167] The present application forms a first data set by taking the initial call data of each target customer service as the first call data. At this time, it is determined that the first data set includes 600,000 first call data.

[0168] Step 704, correcting each first call data by using an error correction model and / or setting an error correction information set;

[0169] Step 705, removing the first call data without an ending phrase in the first data set;

[0170] The ending phrase is a goodbye, goodbye, and the like.

[0171] Step 706, removing the first call data with a conversation round number less than 5 in the first data set;

[0172] The present application removes the first call data without an ending phrase in the first data set, and removes the first call data with a conversation round number less than 5 in the first data set. At this time, it is determined that the first data set includes 400,000 first call data.

[0173] Step 707, removing the first call data including key information of a set event in the first data set;

[0174] The key information of the setting event is the key information of a festival, and the key information of the festival includes information of Spring Festival, Happy New Year, Spring Festival, New Year, Happy New Year, Tomb Sweeping Day, May Day holiday, May Day activities, May Day, May Day Labor Day, Dragon Boat Festival, Mid-Autumn Festival, National Day, National Day holiday, and National Day holiday.

[0175] The first call data in the first data set is removed, and it is determined that the first data set includes 350,000 first call data.

[0176] Step 708, remove the customer feature information of the first call data in the first data set, and the first call data whose customer feature information is inconsistent with the pre-stored customer feature information;

[0177] The customer feature information includes customer name, credit limit, cashable limit, principal balance, and recent cash time information. The first call data in the first data set is removed, and it is determined that the first data set includes 250,000 first call data.

[0178] Step 709, input the first data set into a preset dialogue model to obtain the perplexity and call feature information of each first call data;

[0179] Step 710, based on the call feature information of each first call data, the clustering processing is performed on each first call data to obtain a plurality of data clusters;

[0180] Based on the call feature information of each first call data, the k-means algorithm is used to perform clustering processing on each first call data to obtain a plurality of data clusters.

[0181] Step 711, for any one data cluster, based on the similarity between any two first call data in the data cluster, the first number of first call data with similarity greater than 0.9 is removed from the data cluster;

[0182] Step 712, obtain the second number of first call data with perplexity in a set range from the data cluster, and take the obtained first call data as second call data to form a second data set;

[0183] The set range is 0.25-0.75. For example, the number of first call data with perplexity greater than 0.25 and less than 0.75 in each data cluster is greater than or equal to the second number, and the second number of first call data with perplexity greater than 0.25 and less than 0.75 in each data cluster is taken as second call data. At this time, it is determined that the second data set includes 180,000 second call data.

[0184] Step 713, input the second data set into the intention recognition model to obtain the customer intention of each second call data;

[0185] The application inputs each second call data into the intention recognition model respectively to obtain the customer intention of each second call data, thereby determining 108 types of customer intentions, including the intentions of unable to cash, too low credit, app uninstall, loan problem, etc.

[0186] Step 714, based on the customer intention of each second call data, the second data set is clustered and processed, and the second call data of each type of customer intention is screened to obtain the screened second call data, thereby forming a training data set;

[0187] For each type of customer intention, if the number of second call data of the customer intention is greater than or equal to a third number, the third number of second call data of the customer intention is obtained, and if the number of second call data of the customer intention is less than the third number, all second call data of the customer intention is obtained, thereby forming a training data set, and at this time, it is determined that the training data set includes 50,000 call data.

[0188] Step 715, in the first data set, the first call data existing at least one key information in the set key information set is added to the training data set, so as to train the preset dialogue model by using the training data set.

[0189] The application adds 40,000 first call data existing at least one key information in the set key information set in the first data set to the training data set, and at this time, the training data set includes 90,000 call data.

[0190] Based on the same inventive concept, the application provides a data processing device, please refer to Figure 8 The device comprises:

[0191] The acquisition module 801 is configured to acquire a first data set comprising a plurality of first call data;

[0192] The processing module 802 is configured to input the first data set into a preset dialogue model to obtain the perplexity of each first call data;

[0193] The forming module 803 is configured to obtain second call data based on the perplexity of each first call data to form a second data set;

[0194] The screening module 804 is configured to perform clustering processing on the second data set based on the customer intention of each piece of second call data in the second data set, screen the second call data of each customer intention, obtain screened second call data, form a training data set, and train the preset dialogue model by using the training data set.

[0195] As an optional implementation, the forming module 803 is configured to:

[0196] obtain, from the first data set, first call data whose perplexity is within a set range, and take the obtained first call data as the second call data.

[0197] As an optional implementation, before the obtaining of the second call data based on the perplexity of each piece of first call data, the forming module 803 is further configured to:

[0198] input the first data set into the preset dialogue model to obtain call feature information of each piece of first call data;

[0199] perform clustering processing on each piece of first call data based on the call feature information of each piece of first call data to obtain a plurality of data clusters;

[0200] for any one data cluster, remove first call data with a similarity greater than a set similarity threshold from the data cluster based on the similarity between any two pieces of first call data in the data cluster.

[0201] As an optional implementation, the processing module 802 is configured to:

[0202] for any one piece of first call data, input the first call data into the preset dialogue model to obtain a predicted call result;

[0203] calculate a loss value of the first call data between the predicted call result and the first call data by using a loss function of the preset dialogue model;

[0204] determine an exponential function value with a natural constant as a base and the loss value of the first call data as an index, and take the exponential function value as the perplexity of the first call data.

[0205] As an optional implementation, after the forming of the training data set, the screening module 804 is further configured to:

[0206] add, from the first data set, first call data containing at least one key information in a set key information set to the training data set.

[0207] As an optional implementation, the screening module 804 is configured to determine the key information by the following steps:

[0208] obtaining a third data set including a plurality of historical call data;

[0209] inputting the third data set into the preset dialogue model to obtain a historical predicted call result of each historical call data;

[0210] for any one historical call data, calculating a loss value of the historical call data between the historical predicted call result of the historical call data and the historical call data by using a loss function of the preset dialogue model;

[0211] if the loss value of the historical call data exceeds a set loss value, extracting information with customer call characteristics in the historical call data as the key information.

[0212] As an optional implementation, the obtaining module 801 is configured to:

[0213] obtaining an initial data set including a plurality of initial call data; each customer service corresponds to at least one initial call data;

[0214] determining a task success rate of each customer service based on each initial call data, wherein the task success rate of each customer service is used to represent the task success of the customer service;

[0215] determining a plurality of target customer services based on the task success rate of each customer service;

[0216] obtaining first call data based on the initial call data of the plurality of target customer services to form the first data set.

[0217] As an optional implementation, before the first data set is input into the preset dialogue model, the processing module 802 is further configured to:

[0218] performing error correction on each first call data by using an error correction model and / or a set error correction information set.

[0219] As an optional implementation, before the first data set is input into the preset dialogue model, the processing module 802 is further configured to:

[0220] filtering the first data set based on a set filtering rule.

[0221] As an optional implementation, the processing module 802 is configured to perform part or all of the following:

[0222] eliminating first call data without a closing speech in the first data set;

[0223] eliminating the first call data in the first data set whose number of conversation turns is less than a set turn threshold;

[0224] eliminating the first call data in the first data set including key information of a set event;

[0225] eliminating the first call data in the first data set whose customer feature information is inconsistent with pre-stored customer feature information.

[0226] Based on the same inventive concept, the embodiments of the present application provide an electronic device which can realize the functions of the data processing method discussed above, please refer to Figure 9 The device comprises a processor 901 and a memory 902, wherein the memory 902 is used to store program instructions;

[0227] The processor 901 invokes the program instructions stored in the memory and runs the program instructions to realize the following steps:

[0228] obtain a first data set comprising a plurality of first call data;

[0229] input the first data set into a preset conversation model to obtain a perplexity of each first call data;

[0230] obtain second call data based on the perplexity of each first call data to form a second data set;

[0231] perform clustering processing on the second data set based on a customer intent of each second call data in the second data set, and perform screening on the second call data of each customer intent to obtain screened second call data to form a training data set, so as to train the preset conversation model by using the training data set.

[0232] As an optional implementation, the obtaining of the second call data based on the perplexity of each first call data comprises:

[0233] obtain the first call data whose perplexity is within a set range from the first data set, and take the obtained first call data as the second call data.

[0234] As an optional implementation, before the obtaining of the second call data based on the perplexity of each first call data, the processor 901 is further used to perform:

[0235] input the first data set into the preset conversation model to obtain call feature information of each first call data;

[0236] cluster each first call data based on the call feature information of each first call data, to obtain a plurality of data clusters;

[0237] For any one data cluster, based on the similarity between any two first call data in the data cluster, the first call data with similarity greater than a set similarity threshold is removed from the data cluster.

[0238] As an optional implementation, the first data set is input into a preset dialogue model to obtain the perplexity of each first call data, comprising:

[0239] For any one first call data, the first call data is input into the preset dialogue model to obtain a predicted call result;

[0240] Using the loss function of the preset dialogue model, the loss value of the first call data between the predicted call result and the first call data is calculated;

[0241] Determine the exponential function value with natural constant as base and the loss value of the first call data as exponent, and take the exponential function value as the perplexity of the first call data.

[0242] As an optional implementation, after forming the training data set, the processor 901 is further used to execute:

[0243] The first call data with at least one key information in the set key information set in the first data set is added to the training data set.

[0244] As an optional implementation, the processor 901 is used to execute the following steps to determine the key information:

[0245] Obtain a third data set including a plurality of historical call data;

[0246] Input the third data set into the preset dialogue model to obtain a historical predicted call result of each historical call data;

[0247] For any one historical call data, using the loss function of the preset dialogue model, the loss value of the historical call data between the historical predicted call result of the historical call data and the historical call data is calculated;

[0248] If the loss value of the historical call data exceeds a set loss value, the information with customer call features in the historical call data is extracted as the key information.

[0249] As an optional implementation, the first data set including a plurality of first call data comprises:

[0250] obtain an initial data set comprising a plurality of initial call data; each customer corresponds to at least one initial call data;

[0251] determine a task success rate of each customer based on each initial call data, wherein the task success rate of each customer is used to represent the task success of the customer;

[0252] determine a plurality of target customers based on the task success rate of each customer;

[0253] obtain first call data based on the initial call data of the plurality of target customers to form the first data set.

[0254] As an optional implementation, before the first data set is input into the preset dialogue model, the processor 901 is further configured to perform:

[0255] correct each first call data by using an error correction model and / or setting an error correction information set.

[0256] As an optional implementation, before the first data set is input into the preset dialogue model, the processor 901 is further configured to perform:

[0257] filter the first data set based on a set filtering rule.

[0258] As an optional implementation, the setting of the filtering rule includes part or all of the following:

[0259] eliminate first call data in the first data set without a closing phrase;

[0260] eliminate first call data in the first data set with a number of dialogue rounds less than a set round threshold;

[0261] eliminate first call data in the first data set including key information of a set event;

[0262] eliminate first call data in the first data set with customer feature information inconsistent with pre-stored customer feature information.

[0263] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium, and a computer program product, which comprises computer program code, when the computer program code runs on a computer, so that the computer executes the data processing method as discussed above. Since the above-mentioned computer readable storage medium solves the problem by the similar principle as the data processing method, the implementation of the above-mentioned computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described here.

[0264] Based on the same inventive concept, the embodiment of the present application further provides a computer program product, which comprises computer program codes, and when the computer program codes run on a computer, the computer program codes make the computer execute the data processing method as any one of the foregoing. Since the computer program product solves problems in the same principle as the data processing method, the implementation of the computer program product can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0265] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program codes.

[0266] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0267] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0268] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of user operation steps are executed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0269] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the first dataset, which includes multiple first call data entries; The first dataset is input into a preset dialogue model to obtain the perplexity of each first call data. Based on the confusion level of each first call data, second call data is obtained to form a second dataset; Based on the customer intent of each second call data in the second dataset, the second dataset is clustered, and the second call data of each customer intent are filtered to obtain the filtered second call data, forming a training dataset, which is then used to train the preset dialogue model.

2. The method as described in claim 1, characterized in that, Obtaining the second call data based on the confusion level of each first call data entry includes: Obtain first call data whose confusion level is within a set range from the first dataset, and use the obtained first call data as the second call data.

3. The method as described in claim 1, characterized in that, Before obtaining the second call data based on the confusion level of each first call data, the method further includes: The first dataset is input into the preset dialogue model to obtain the call feature information of each first call data; Based on the call feature information of each first call data, each first call data is clustered to obtain multiple data clusters; For any given data cluster, based on the similarity of any two first call data points in the data cluster, first call data points with a similarity greater than a set similarity threshold are removed from the data cluster.

4. The method as described in claim 1, characterized in that, The step of inputting the first dataset into a preset dialogue model to obtain the perplexity level of each first call data includes: For any first call data, the first call data is input into the preset dialogue model to obtain the predicted call result; Using the loss function of the preset dialogue model, calculate the loss value of the first call data between the predicted call result and the first call data; Determine the value of an exponential function with a natural constant as the base and the loss value of the first call data as the exponent, and use the exponential function value as the perplexity of the first call data.

5. The method as described in claim 1, characterized in that, After forming the training dataset, the process also includes: The first call data in the first dataset that contains at least one key piece of information from a set of key information is added to the training dataset.

6. The method as described in claim 5, characterized in that, The key information was determined through the following steps: Obtain a third dataset that includes multiple historical call records; Input the third dataset into the preset dialogue model to obtain the historical predicted call results for each historical call data; For any historical call data, the loss function of the preset dialogue model is used to calculate the loss value of the historical call data between the historical predicted call result and the historical call data. If the loss value of the historical call data exceeds the set loss value, then information with customer call characteristics in the historical call data is extracted as the key information.

7. The method as described in claim 1, characterized in that, The acquisition of the first dataset, which includes multiple first call data points, includes: Obtain an initial dataset containing multiple initial call data entries; each customer service representative corresponds to at least one initial call data entry. Based on each initial call data, the task success rate of each customer service representative is determined, wherein the task success rate of each customer service representative is used to characterize the task success status of the customer service representative; Based on the task success rate of each customer service representative, multiple target customer service representatives are identified. The first call data is obtained based on the initial call data of the multiple target customer service representatives, forming the first dataset.

8. The method as described in claim 1, characterized in that, Before inputting the first dataset into the preset dialogue model, the method further includes: Error correction is performed on each of the first call data using an error correction model and / or by setting an error correction information set.

9. The method as described in claim 1, characterized in that, Before inputting the first dataset into the preset dialogue model, the method further includes: The first dataset is filtered based on the set filtering rules.

10. The method as described in claim 9, characterized in that, The filtering rules include some or all of the following: Remove the first call data in the first dataset that has no closing remarks; Remove the first call data in the first dataset whose number of dialogue rounds is less than a set threshold. Remove the first call data from the first dataset that includes key information about the event setting; Remove customer feature information from the first call data in the first dataset that is inconsistent with the pre-stored customer feature information.

11. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first dataset that includes multiple first call data entries; The processing module is used to input the first dataset into a preset dialogue model to obtain the perplexity of each first call data. A forming module is used to obtain second call data based on the confusion level of each first call data, thereby forming a second dataset; The filtering module is used to cluster the second dataset based on the customer intent of each second call data in the second dataset, and filter the second call data of each customer intent to obtain the filtered second call data, forming a training dataset, so as to train the preset dialogue model using the training dataset.

12. An electronic device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method as described in any one of claims 1-10.

Images