Data processing method and device

Abstract

The embodiments of this specification provide a data processing method and device, wherein the data processing method includes: obtaining historical conversation data containing at least two conversation roles, dividing the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment, determining a first conversation segment in each conversation set that contains a target conversation turn, and clustering the conversation data of the target conversation role in the first conversation segment to generate a corresponding first clustering result, determining conversation intention information of the target conversation role based on the first clustering result, determining a second conversation segment in each conversation set that is associated with the first conversation segment, and determining the target conversation data corresponding to the conversation intention information based on the second conversation segment.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, and a computer-readable storage medium. Background Art

[0002] With the emergence of more and more online service platforms, users can consult online through the service platforms, which undoubtedly adds convenience to users' lives. At present, online service consultations by users, such as questions raised by users based on product details or questions raised based on the services provided by e-commerce platforms, are usually based on manual online replies, or manual replies through customer service robots based on pre-set reply templates, which consumes a lot of labor costs.

[0003] At present, most service platforms have made improvements in this regard. Most of them are that the project providers use a pre-built knowledge base to generate relevant scripts. However, the construction of the knowledge base and the generation of scripts currently rely entirely on manual labor or on manual labor assisted by mining high-frequency customer messages. The initial labor cost is very high, and because knowledge needs to be iteratively updated, the knowledge base needs to be continuously maintained by manual labor, resulting in high maintenance costs. Therefore, it is urgent to propose an effective method to solve this problem. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a data processing device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0006] Acquire historical dialogue data containing at least two dialogue roles, and divide the historical dialogue data into at least two dialogue sets, wherein each dialogue set contains at least one dialogue segment;

[0007] Determine a first dialogue segment containing a target dialogue turn in each dialogue set, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result;

[0008] Determining the dialogue intention information of the target dialogue role according to the first clustering result;

[0009] A second conversation segment associated with the first conversation segment in each conversation set is determined, and target conversation data corresponding to the conversation intention information is determined based on the second conversation segment.

[0010] Optionally, determining, according to the second conversation segment, target conversation data corresponding to the conversation intention information includes:

[0011] Performing semantic analysis on the second dialogue segment in each dialogue set, and marking the second dialogue segment according to the analysis result;

[0012] Determine the target dialogue path corresponding to the dialogue intention information according to the generated annotation results;

[0013] According to the second dialogue segment corresponding to the target dialogue path, target dialogue data corresponding to the dialogue intention information is generated.

[0014] Optionally, determining a target dialogue path corresponding to the dialogue intention information according to the generated annotation results includes:

[0015] Establish the dialogue path corresponding to each dialogue set according to the generated annotation results;

[0016] Determining the conversation data included in the target clustering result, and determining the target conversation set to which the conversation data belongs, wherein the target clustering result is one of the first clustering results;

[0017] The dialogue paths corresponding to the target dialogue set are integrated to generate a target dialogue path corresponding to the dialogue intention information.

[0018] Optionally, establishing a dialog path corresponding to each dialog set according to the generated annotation results includes:

[0019] Calculate the transition probability between different annotation results according to the jump relationship between the second dialogue segments corresponding to the annotation results in each dialogue set;

[0020] Constructing a transfer matrix corresponding to the labeling result according to the transfer probability;

[0021] A dialogue path corresponding to each dialogue set is determined according to the transfer matrix.

[0022] Optionally, the data processing method further includes:

[0023] Determining a conversation type of the historical conversation data according to conversation data of a first conversation round in the historical conversation data;

[0024] The target dialogue turn is determined according to the dialogue type.

[0025] Optionally, when the historical conversation data is of the first conversation type, determining the first conversation segment containing the target conversation turn in each conversation set, and clustering the conversation data of the target conversation role in the first conversation segment to generate a corresponding first clustering result includes:

[0026] Determine the first dialogue segment in each dialogue set that contains the target dialogue turn;

[0027] The conversation data of the target conversation role in the first conversation segment is extracted, and the conversation data is clustered using a first density-based clustering algorithm to generate a corresponding first clustering result.

[0028] Optionally, in a case where the historical conversation data is of the second conversation type, after dividing the historical conversation data into at least two conversation sets, the method further includes:

[0029] Identify the third dialogue segment in each dialogue set;

[0030] Extracting the dialogue data of the first dialogue character in the third dialogue segment, and clustering the dialogue data of the first dialogue character using a mean clustering algorithm to generate a corresponding second clustering result;

[0031] Determining the starting conversation data corresponding to the conversation intention information according to the second clustering result;

[0032] Filtering part of the historical conversation data from the historical conversation data according to the initial conversation data, and dividing the part of the historical conversation data into at least two conversation sets; and / or,

[0033] The starting conversation data is added to the target conversation data.

[0034] Optionally, dividing the historical conversation data into at least two conversation sets includes:

[0035] Cleaning the historical conversation data and generating corresponding cleaning results;

[0036] Performing vectorization processing on the historical conversation data contained in the cleaning result to generate corresponding vectorized data;

[0037] Semantic recognition is performed on the historical conversation data included in the cleaning result according to the vectorized data, and the historical conversation data included in the cleaning result is divided into at least two conversation sets according to the semantic recognition result.

[0038] Optionally, after determining the second conversation segment associated with the first conversation segment in each conversation set, the method further includes:

[0039] Extracting the dialogue data of the target dialogue role in the first dialogue segment, and clustering the dialogue data using a second density-based clustering algorithm to generate a corresponding third clustering result;

[0040] Determining the long-tail intention information of the target dialogue role according to the third clustering result;

[0041] Determine the conversation data corresponding to the long-tail intent information according to the first conversation segment, and add the conversation data corresponding to the long-tail intent information to the target conversation data.

[0042] Optionally, determining, according to the second conversation segment, target conversation data corresponding to the conversation intention information includes:

[0043] Clustering the conversation data of the first dialogue role in the second dialogue segment, and clustering the conversation data of the second dialogue role in the second dialogue segment, to generate a corresponding fourth clustering result, wherein the second dialogue role is the target dialogue role;

[0044] The target dialogue data corresponding to the dialogue intention information is determined according to the fourth clustering result.

[0045] According to a second aspect of an embodiment of this specification, there is provided a data processing device, including:

[0046] An acquisition module is configured to acquire historical conversation data containing at least two conversation roles, and divide the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment;

[0047] A clustering module is configured to determine a first dialogue segment containing a target dialogue turn in each dialogue set, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result;

[0048] A first determination module is configured to determine the dialogue intention information of the target dialogue role according to the first clustering result;

[0049] The second determination module is configured to determine a second conversation segment associated with the first conversation segment in each conversation set, and determine target conversation data corresponding to the conversation intention information based on the second conversation segment.

[0050] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0051] Memory and processor;

[0052] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the data processing method.

[0053] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data processing method are implemented.

[0054] An embodiment of the present specification obtains historical conversation data containing at least two conversation roles, divides the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment, determines a first conversation segment in each conversation set that contains a target conversation turn, clusters the conversation data of the target conversation role in the first conversation segment to generate a corresponding first clustering result, determines conversation intention information of the target conversation role based on the first clustering result, determines a second conversation segment associated with the first conversation segment in each conversation set, and determines target conversation data corresponding to the conversation intention information based on the second conversation segment.

[0055] The embodiments of the present specification cluster the dialogue intention information of the target dialogue role in the first dialogue segment, determine the dialogue intention based on the clustering result, and further determine the target dialogue data corresponding to the dialogue intention based on the second dialogue segment associated with the first dialogue segment. This is conducive to reducing the complexity of constructing the target dialogue data, i.e., the speech, thereby helping to reduce the construction cost of the target dialogue data and ensure the accuracy of the construction results. In addition, this method is also conducive to improving the construction efficiency of the target dialogue data. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0057] FIG. 2( a ) is a schematic diagram of a first data processing process provided by an embodiment of the present specification;

[0058] FIG2( b ) is a schematic diagram of a second data processing process provided by an embodiment of this specification;

[0059] FIG2( c ) is a schematic diagram of a third data processing process provided by an embodiment of this specification;

[0060] Figure 3 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0061] Figure 4 is a schematic diagram of a data processing device provided by an embodiment of this specification;

[0062] Figure 5 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0063] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0064] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0065] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0066] First, the terms involved in one or more embodiments of this specification are explained.

[0067] Intent: refers to the purpose of the user's speech during the conversation.

[0068] Dialogue flow: The English name is taskflow. The dialogue flow is composed of a series of ordered dialogue nodes. There are many types of dialogue nodes, such as the user's trigger node (expressing user intention) and the robot's reply node.

[0069] Semi-automatic: The process of dialog flow mining first mines the recommended user's intention according to the dialog process, and the user makes a few checks and corrections. Then the underlying dialog structure is mined according to the order of the dialog. The user needs to check the mining results at each step, so it is called semi-automatic.

[0070] In this specification, a data processing method is provided. This specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0071] At present, the industry's dialogue platforms are conducting research on how to build dialogue robots in low-resource scenarios. However, there are fewer studies on scenarios where customers can provide a large amount of data-rich resources. We hope to semi-automatically mine the dialogue flow (including opening intentions, dialogue structure, node dialogue, etc.) from the large number of person-to-person dialogue logs provided by customers, and directly build a dialogue robot on this basis, greatly reducing the cost of robot construction (user intentions, process combing, dialogue annotation, etc.).

[0072] Figure 1 A processing flow chart of a data processing method provided according to an embodiment of the present specification is shown, including steps 102 to 108.

[0073] Step 102, obtaining historical dialogue data containing at least two dialogue roles, and dividing the historical dialogue data into at least two dialogue sets, wherein each dialogue set contains at least one dialogue segment.

[0074] Specifically, the historical conversation data is conversation data generated based on the target project, including but not limited to video data or audio data, etc. The target project includes but is not limited to information query projects, claims projects, loan projects, investment projects, etc. The embodiments of this specification take the information query project as an example for explanation. The specific implementation of the claims project, loan project, and investment project is similar to the specific implementation of the information query project. You can refer to the specific implementation of the information query project and will not go into details here.

[0075] The dialogue set is a collection of dialogue segments, each of which contains at least one dialogue segment, and the dialogue segments contained in each dialogue set are generated by dividing the historical dialogue data corresponding to a complete project processing process into dialogue segments; the dialogue segment consists of at least one dialogue round; and a dialogue round is a round of dialogue between at least two dialogue characters.

[0076] In actual applications, if the target project is an information query project, the dialogue roles include but are not limited to users who need to conduct information query and customer service who provide information query services to users; the historical dialogue data is the complete dialogue data between the user and the customer service generated during the information query process, and the historical dialogue data contains multiple complete dialogue data.

[0077] In the embodiment of this specification, it is necessary to construct the target dialogue data corresponding to the user's dialogue intention information according to the historical dialogue data, that is, the dialogue flow, so that after receiving the user's project processing request and determining the user's new dialogue intention information, the user can be automatically replied according to the dialogue flow. Therefore, after the user sends the project processing request to the project provider online or offline, the project provider matches the corresponding customer service to provide the user with the corresponding service, and collects the dialogue data between the user and the customer service, and uses it as the historical dialogue data to determine the target dialogue data corresponding to the user's dialogue intention information. The historical dialogue data is the video data or audio data formed by the project provider recording the dialogue content in the form of video or audio during the user's dialogue with the customer service in an offline manner, or it can be the picture data or text data formed by the project provider recording the dialogue content in the form of pictures or text during the user's dialogue with the customer service in an online manner.

[0078] In addition, after the historical conversation data is obtained, the historical conversation data needs to be converted into text information, so as to divide the historical conversation data into conversation segments based on the text information, and to determine the target conversation data corresponding to the user's conversation intention information based on the result of dividing the conversation segments, wherein the process of dividing the historical conversation data into conversation segments can be specifically implemented in the following manner:

[0079] Cleaning the historical conversation data and generating corresponding cleaning results;

[0080] Performing vectorization processing on the historical conversation data contained in the cleaning result to generate corresponding vectorized data;

[0081] Semantic recognition is performed on the historical conversation data included in the cleaning result according to the vectorized data, and the historical conversation data included in the cleaning result is divided into at least two conversation sets according to the semantic recognition result.

[0082] Specifically, after the historical conversation data is acquired, the historical conversation data may be first converted into text information, specifically, speech recognition is performed on the historical conversation data to obtain text information generated by the recognition.

[0083] In practical applications, speech recognition technology (Automatic Speech Recognition, ASR) can be used to convert the sound information in the historical conversation data into text information.

[0084] After obtaining the text information of the historical conversation data, the historical conversation data can be cleaned based on the text information. Since the recognition results of some conversation data in the text information obtained by speech recognition may have a low accuracy rate, such as incorrect recognition of conversation characters, or when the historical conversation data is video data or audio data, due to the presence of noise in some video data or audio data, or the low conversation voices of conversation characters in some video data or audio data, their semantics cannot be successfully recognized, or there may be too little conversation content between conversation characters in some historical conversation data. In response to the above situations, the recognition results (text information) of this part of the historical conversation data can be deleted to achieve cleaning of the historical conversation data.

[0085] After the cleaning is completed, the text information of the remaining historical conversation data can be vectorized to generate corresponding vectorized data, wherein the vectorization processing can be specifically achieved by inputting the text information into a neural network and encoding it by the encoding layer in the neural network.

[0086] After the encoding is completed, semantic recognition can be performed on the remaining historical conversation data based on the vectorized data generated by the encoding, and based on the semantic recognition result, the historical conversation data contained in the cleaning result can be divided into conversation segments to generate at least two conversation sets, wherein each conversation set contains at least one conversation segment, and the conversation segments contained in each conversation set are generated by dividing a complete conversation data into conversation segments.

[0087] In practical applications, after the complete conversation data is divided into multiple conversation segments, the conversation turns contained in the conversation segments can also be merged. For example, a complete conversation data contains 30 conversation turns. During the conversation segment division process, it is divided into 10 conversation segments according to the semantic recognition results. Each conversation segment contains 3 conversation turns. Since each conversation segment may contain some conversation data that has no practical meaning, such as: modal particles such as "hmm" and "ah", or some repeated conversation data, or the importance of different conversation roles in different conversation segments is different; therefore, the conversation turns in each conversation segment can be merged according to the importance of the conversation roles in the conversation segment, or the conversation data that has no practical meaning in the conversation segment can be deleted, and then the conversation segment generated after merging or deleting the conversation data is used as the conversation segment in the conversation set.

[0088] By cleaning the historical conversation data and using the cleaned data to determine the user's conversation intention and the target conversation data corresponding to the conversation intention, it is helpful to ensure the accuracy of the determination results of both the conversation intention and the target conversation data.

[0089] Step 104, determining the first dialogue segment containing the target dialogue turn in each dialogue set, and clustering the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result.

[0090] Specifically, the target dialogue role is one of at least two dialogue roles in the historical dialogue data.

[0091] Since the purpose of the embodiments of this specification is to construct target conversation data, i.e., a conversation flow, for the conversation intention information of different users under a target project based on historical conversation data, so as to realize automatic reply to the user through the conversation flow; therefore, in the process of generating the target conversation data, it is necessary to divide the conversation segments into conversation roles so as to determine the user's intention information using the conversation data of the target conversation role, and the target conversation role is the user.

[0092] In specific implementation, after determining the first conversation segment containing the target conversation turn in each conversation set, the conversation data of the users in the first conversation segment can be clustered to generate a corresponding first clustering result. Since different historical conversation data may correspond to different conversation types, the embodiment of this specification can first determine the conversation type of the historical conversation data, and then determine the target conversation turn according to the conversation type, which can be specifically implemented in the following manner:

[0093] Determining a conversation type of the historical conversation data according to conversation data of a first conversation round in the historical conversation data;

[0094] The target dialogue turn is determined according to the dialogue type.

[0095] Further, in the case where the historical dialogue data is of the first dialogue type, determining the first dialogue segment containing the target dialogue turn in each dialogue set, and clustering the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result includes:

[0096] Determine the first dialogue segment in each dialogue set that contains the target dialogue turn;

[0097] The conversation data of the target conversation role in the first conversation segment is extracted, and the conversation data is clustered using a first density-based clustering algorithm to generate a corresponding first clustering result.

[0098] Specifically, the conversation type can be determined based on the initiator of the conversation in the historical conversation data. Since at least two conversation roles in the historical conversation data can be customer service and user, if the initiator of the conversation in the historical conversation data is the user, the historical conversation data is determined as the first conversation type; if the initiator of the conversation in the historical conversation data is customer service, the historical conversation data is determined as the second conversation type. After the conversation type is determined, the target conversation turn can be determined based on the conversation type.

[0099] In actual applications, when the historical conversation data is of the first conversation type, it is equivalent to that the historical conversation was actively initiated by the user. In this case, usually the conversation data of the first few conversation rounds in the historical conversation data may mainly be the user expressing his intention to initiate a conversation to the customer service, that is, the user's conversation intention information can be determined based on the conversation data of the first 1 to 3 conversation rounds or the first 1 to 5 conversation rounds in the historical conversation data. Therefore, in this case, the conversation data of the first 1 to 3 conversation rounds or the first 1 to 5 conversation rounds in the historical conversation data can be used as the target conversation round, and the conversation segment containing the target conversation round can be used as the first conversation segment.

[0100] In addition, when the historical conversation data is of the second conversation type, it is equivalent to that the historical conversation was actively initiated by the customer service. In this case, the data of the first few conversation rounds in the historical conversation data may mainly be the customer service expressing its intention to initiate a conversation to the user, that is, the user's conversation intention information cannot be determined based on the conversation data of the first 1 to 3 conversation rounds or the first 1 to 5 conversation rounds in the historical conversation data. Therefore, the customer service's conversation data in the first 1 to 3 conversation rounds or the first 1 to 5 conversation rounds can be clustered first to determine the customer service's opening words (opening language) according to the clustering results, and the user's conversation intention information can be determined based on the conversation data of 4 to 8 conversation rounds or 6 to 9 conversation rounds in the historical conversation data; therefore, in this case, the conversation data of 4 to 8 conversation rounds or 6 to 9 conversation rounds in the historical conversation data can be used as the target conversation round, and the conversation segment containing the target conversation round can be used as the first conversation segment.

[0101] After determining the first conversation segment, extract the conversation data of the user in the first conversation segment, and cluster the conversation data using a first density-based clustering algorithm to generate a corresponding clustering result; wherein the first density-based clustering algorithm may be a HDBSCAN algorithm.

[0102] The embodiments of this specification determine the target dialogue turn according to the dialogue type, and use the first density-based clustering algorithm to cluster the dialogue data of the target dialogue role in the target dialogue turn, so as to determine the dialogue intention of the target dialogue role according to the clustering result, which is conducive to ensuring the accuracy of the dialogue intention determination result.

[0103] Step 106: Determine the dialogue intention information of the target dialogue role according to the first clustering result.

[0104] Specifically, after clustering the conversation data of the target conversation role in the first conversation segment to obtain multiple clustering results, semantic recognition can be performed on the conversation data of the users contained in each clustering result to determine the conversation intention information of the users corresponding to each clustering result based on the semantic recognition results.

[0105] Step 108: determining a second conversation segment associated with the first conversation segment in each conversation set, and determining target conversation data corresponding to the conversation intention information based on the second conversation segment.

[0106] Specifically, in the case where the historical conversation data is of the first conversation type, the conversation data of the first 1 to 3 conversation turns or the first 1 to 5 conversation turns in the historical conversation data are used as target conversation turns, and the conversation segment containing the target conversation round is the first conversation segment; therefore, the second conversation segment is the other conversation segments in each conversation set except the first conversation segment.

[0107] After determining the conversation intention information of the user corresponding to each clustering result, the conversation data of the user contained in the clustering result can be determined, and based on the second conversation segment contained in the conversation set to which the conversation data belongs, the target conversation data (conversation flow, which is composed of words corresponding to different conversation nodes) corresponding to the conversation intention information can be determined.

[0108] For example, the embodiment of this specification obtains 10 complete historical conversation data, namely historical conversation data 1 to historical conversation data 10, wherein historical conversation data 1 to historical conversation data 8 are of the first conversation type. In this case, the conversation data of the user contained in the first conversation segment of the first historical conversation data 1 to historical conversation data 8 can be clustered first to generate two cluster clusters, namely cluster cluster 1 and cluster cluster 2, wherein cluster cluster 1 contains the conversation data of the user in the first conversation segment of historical conversation data 1 to historical conversation data 4; therefore, after determining the user's conversation intention information 1 according to cluster cluster 1, the target conversation data corresponding to the conversation intention information 1 can be further determined according to the second conversation segment in historical conversation data 1 to historical conversation data 4.

[0109] In a specific implementation, determining target dialogue data corresponding to the dialogue intention information according to the second dialogue segment includes:

[0110] Performing semantic analysis on the second dialogue segment in each dialogue set, and marking the second dialogue segment according to the analysis result;

[0111] Determine the target dialogue path corresponding to the dialogue intention information according to the generated annotation results;

[0112] According to the second dialogue segment corresponding to the target dialogue path, target dialogue data corresponding to the dialogue intention information is generated.

[0113] Furthermore, the target dialogue path corresponding to the dialogue intention information is determined according to the generated annotation results, which can be specifically achieved in the following ways:

[0114] Establish the dialogue path corresponding to each dialogue set according to the generated annotation results;

[0115] Determining the conversation data included in the target clustering result, and determining the target conversation set to which the conversation data belongs, wherein the target clustering result is one of the first clustering results;

[0116] The dialogue paths corresponding to the target dialogue set are integrated to generate a target dialogue path corresponding to the dialogue intention information.

[0117] Furthermore, the dialogue path corresponding to each dialogue set is established based on the generated annotation results, including:

[0118] Calculate the transition probability between different annotation results according to the jump relationship between the second dialogue segments corresponding to the annotation results in each dialogue set;

[0119] Constructing a transfer matrix corresponding to the labeling result according to the transfer probability;

[0120] A dialogue path corresponding to each dialogue set is determined according to the transfer matrix.

[0121] Specifically, after determining the dialogue intention information (opening intention) of the target dialogue role, the target dialogue data corresponding to the dialogue intention information can be determined based on the second dialogue segment associated with the first dialogue segment in the dialogue set. Specifically, a semantic analysis can be performed on the second dialogue segment in each dialogue set, and the second dialogue segment can be annotated (labeled) according to the analysis result. For second dialogue segments belonging to the same semantic category, they can be annotated with the same label; wherein, the process of determining the opening intention is to cluster the user expressions (user dialogue data) contained in different dialogue segments in the same dialogue set, and determine the opening intention based on the clustering results.

[0122] After the labeling is completed, the jump relationship between the second dialogue segments in each dialogue set can be determined, and the jump relationship between the labels in each data set can be determined based on the correspondence between the labels and the second dialogue segments. Then, based on the jump relationship between the labels, the transition probability between different labels (labeling results) is calculated; then, a transition matrix corresponding to the label is constructed according to the transition probability, and the dialogue path corresponding to each dialogue set is determined according to the transition matrix.

[0123] After generating the conversation path corresponding to each conversation set, the conversation data contained in each clustering result can be determined, and the target conversation set to which the conversation data belongs can be determined. Then, the conversation paths corresponding to the target conversation set can be integrated to generate a target conversation path corresponding to the conversation intention information, wherein the label can be used as a node in the target conversation path; in addition, the path integration method includes but is not limited to filtering incomplete paths and merging reply nodes.

[0124] After generating the target dialogue path corresponding to the dialogue intention information, the target dialogue path can be hung under the current intention node, so that when the user's project request is received and it is determined that the user's intention is consistent with the intention information of the intention node, the user can be automatically replied based on the target dialogue path.

[0125] In addition, after the target dialogue path is generated, target dialogue data corresponding to the dialogue intention information can be generated according to the second dialogue segment corresponding to the target dialogue path.

[0126] The embodiments of the present specification annotate the second dialogue segment according to the semantic information of the second dialogue segment, and construct a transfer matrix according to the jump relationship between the second dialogue segments corresponding to different annotation results, so as to determine the dialogue path corresponding to each dialogue intention based on the transfer matrix, which is beneficial to ensuring the accuracy of the construction result of the dialogue path; in addition, this method is also beneficial to improving the efficiency of dialogue path construction.

[0127] As mentioned above, the historical conversation data may be divided into two conversation types. When the historical conversation data is of the second conversation type, after dividing the historical conversation data into at least two conversation sets, the method further includes:

[0128] Identify the third dialogue segment in each dialogue set;

[0129] Extracting the dialogue data of the first dialogue character in the third dialogue segment, and clustering the dialogue data of the first dialogue character using a mean clustering algorithm to generate a corresponding second clustering result;

[0130] Determining the starting conversation data corresponding to the conversation intention information according to the second clustering result;

[0131] Filtering part of the historical conversation data from the historical conversation data according to the initial conversation data, and dividing the part of the historical conversation data into at least two conversation sets; and / or,

[0132] The starting conversation data is added to the target conversation data.

[0133] Specifically, a schematic diagram of the first data processing process provided by an embodiment of the present specification is shown in Figure 2(a). When the historical conversation data is of the second conversation type, the third conversation segment is composed of conversation data of the first 1 to 3 conversation turns or the first 1 to 5 conversation turns in the historical conversation data, and the first conversation role is customer service.

[0134] After determining the third conversation segment in each conversation set, the customer service conversation data (customer service speech) in the third conversation segment can be extracted, and the customer service conversation data can be clustered using the mean clustering algorithm (K-means clustering algorithm) to generate the corresponding second clustering results (cluster 1 and cluster 2).

[0135] In actual applications, semantic recognition and integration can be performed on the customer service conversation data contained in the second clustering result, and the target conversation data with higher frequency of use can be determined based on the integration result. The integration result corresponding to the target conversation data is used as the starting node of the conversation flow with the conversation intention information, and the target conversation data is used as the speech of the starting node (system welcome / notification).

[0136] This method helps to ensure the integrity of the construction results of the target dialogue data.

[0137] In addition, after determining the conversation of the starting node, part of the historical conversation data can be screened from the historical conversation data according to the semantic integration result, and the part of the historical conversation data can be divided into at least two conversation sets; so as to further determine the user's conversation intention information and the target conversation data corresponding to the conversation intention information based on the first conversation segment containing the target conversation turn and the second conversation segment associated with the first conversation segment in each conversation set.

[0138] In a specific implementation, determining the target dialogue data corresponding to the dialogue intention information according to the second dialogue segment may also be implemented in the following manner:

[0139] Clustering the conversation data of the first dialogue role in the second dialogue segment, and clustering the conversation data of the second dialogue role in the second dialogue segment, to generate a corresponding fourth clustering result, wherein the second dialogue role is the target dialogue role;

[0140] The target dialogue data corresponding to the dialogue intention information is determined according to the fourth clustering result.

[0141] Specifically, a schematic diagram of the second data processing process provided by an embodiment of the present specification is shown in FIG2(b). After determining the conversation intention information (opening intention) of the target conversation role, the target conversation data corresponding to the conversation intention information can be determined based on the second conversation segment associated with the first conversation segment in the conversation set. Specifically, the conversation data of the user (the second conversation role, i.e., the target conversation role) can be extracted from the second conversation segment, and the conversation data of the user can be clustered to obtain a plurality of clusters. Then, the conversation data of the customer service (the first conversation role) can be extracted from the second conversation segment, and the conversation data of the customer service can be clustered to obtain a plurality of clusters. Then, each cluster can be clustered into As a dialogue node, the node speech of each dialogue node is determined according to the dialogue data contained in each cluster, and then the jump relationship between the dialogue nodes is determined according to the sequence of dialogues between the dialogue data of the users in each cluster and the dialogue data of the customer service, and then the transition probability between different dialogue nodes is calculated according to the jump relationship, and then the transition matrix corresponding to the dialogue node is constructed according to the transition probability, and the dialogue path corresponding to each dialogue intention information is determined according to the transfer matrix. Specifically, the dialogue nodes can be screened according to the transition probability between the dialogue nodes, that is, the dialogue nodes with a transition probability greater than a preset probability threshold are screened, and the dialogue path corresponding to each dialogue intention information is constructed according to the screening result.

[0142] In addition, after determining the second conversation segment associated with the first conversation segment in each conversation set, the method further includes:

[0143] Extracting the dialogue data of the target dialogue role in the first dialogue segment, and clustering the dialogue data using a second density-based clustering algorithm to generate a corresponding third clustering result;

[0144] Determining the long-tail intention information of the target dialogue role according to the third clustering result;

[0145] Determine the conversation data corresponding to the long-tail intent information according to the first conversation segment, and add the conversation data corresponding to the long-tail intent information to the target conversation data.

[0146] Specifically, a schematic diagram of the third data processing process provided by an embodiment of the present specification is shown in FIG2(c). After obtaining the historical conversation data from the project log of the target project, the historical conversation data is preprocessed, including sentence encoding, data cleaning, and conversation set division. After preprocessing, the user's conversation intention is mined starting from the starting conversation round of the data. After the user's conversation intention is mined, it is also necessary to send it to the reviewer for review, so that the reviewer can integrate the more similar intentions, or delete some useless or erroneous intentions, etc., so as to improve the accuracy of the intention mining results.

[0147] Furthermore, after determining the user's conversation intention, the conversation structure can be further mined from subsequent rounds, specifically, a transition matrix is ​​constructed to determine the conversation path corresponding to the conversation intention according to the transition probability between each conversation node in the transition matrix. After determining the conversation path, the long-tail intention and speech can also be retrieved from the noise. Specifically, the conversation data of the first conversation segment can be extracted, and the conversation data can be clustered using the HDBSCAN algorithm (density-based second clustering algorithm) to generate a corresponding third clustering result, and the long-tail intention information of the target conversation role can be determined according to the third clustering result, and the conversation data corresponding to the long-tail intention information can be determined according to the first conversation segment, and the conversation data corresponding to the long-tail intention information is added to the target conversation data to generate a conversation flow corresponding to the conversation intention, wherein the clustering density of the first clustering algorithm and the second clustering algorithm are different.

[0148] In actual applications, when using a first density-based clustering algorithm to cluster the conversation data of the target conversation role in the first conversation segment, the first clustering result with a density greater than or equal to a preset threshold can be displayed to the user by setting a threshold, so as to determine the user's intention based on the first clustering result. However, since other clustering results with a density less than the preset threshold may also contain some useful intention information, in order to ensure the accuracy of the intention determination result, the embodiment of this specification can perform long-tail intent identification from other clustering results with a density less than the preset threshold, and supplement the target conversation data based on the conversation data corresponding to the first conversation segment and the long-tail intent to generate a conversation flow corresponding to the conversation intention.

[0149] An embodiment of the present specification obtains historical conversation data containing at least two conversation roles, divides the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment, determines a first conversation segment in each conversation set that contains a target conversation turn, clusters the conversation data of the target conversation role in the first conversation segment, generates a corresponding clustering result, determines conversation intention information of the target conversation role based on the clustering result, determines a second conversation segment associated with the first conversation segment in each conversation set, and determines target conversation data corresponding to the conversation intention information based on the second conversation segment.

[0150] The embodiments of the present specification cluster the dialogue intention information of the target dialogue role in the first dialogue segment, determine the dialogue intention based on the clustering result, and further determine the target dialogue data corresponding to the dialogue intention based on the second dialogue segment associated with the first dialogue segment. This is conducive to reducing the complexity of constructing the target dialogue data, i.e., the speech, thereby helping to reduce the construction cost of the target dialogue data and ensure the accuracy of the construction results. In addition, this method is also conducive to improving the construction efficiency of the target dialogue data.

[0151] The following combination Figure 3 , taking the application of the data processing method provided in this specification in the information query project as an example, the data processing method is further explained. Figure 3 A processing flow chart of a data processing method provided by an embodiment of the present specification is shown, and the specific steps include step 302 to step 326.

[0152] Step 302: Acquire historical dialogue data containing at least two dialogue roles, and divide the historical dialogue data into at least two dialogue sets.

[0153] Each conversation set contains at least one conversation segment.

[0154] Step 304: Determine the conversation type of the historical conversation data according to the conversation data of the first conversation round in the historical conversation data.

[0155] Step 306: Determine a target dialogue turn according to the dialogue type.

[0156] Step 308: Determine the first conversation segment in each conversation set that contains the target conversation turn.

[0157] Step 310: extracting the conversation data of the user in the first conversation segment, and clustering the conversation data using a first density-based clustering algorithm to generate a corresponding first clustering result.

[0158] Step 312: Determine the conversation intention information of the user according to the first clustering result.

[0159] Step 314, determine the second dialogue segment associated with the first dialogue segment in each dialogue set, cluster the dialogue data of the first dialogue role in the second dialogue segment, and cluster the dialogue data of the second dialogue role in the second dialogue segment to generate a corresponding second clustering result.

[0160] Step 316, annotate the second clustering results, determine the jump relationship between the annotated results according to the sequence of the conversation data between the user and the customer service in each second clustering result, and calculate the transition probability between different annotated results according to the jump relationship.

[0161] Step 318: construct a transfer matrix corresponding to the labeling result according to the transfer probability.

[0162] Step 320: Determine the dialog path corresponding to each dialog set according to the transfer matrix.

[0163] Step 322: determine the conversation data included in the target clustering result, and determine the target conversation set to which the conversation data belongs.

[0164] Among them, the target clustering result is one of the clustering results.

[0165] Step 324 , integrating the dialogue paths corresponding to the target dialogue set to generate a target dialogue path corresponding to the dialogue intention information.

[0166] Step 326: Generate target dialogue data corresponding to the dialogue intention information according to the second dialogue segment corresponding to the target dialogue path.

[0167] The embodiments of the present specification cluster the dialogue intention information of the target dialogue role in the first dialogue segment, determine the dialogue intention based on the clustering result, and further determine the target dialogue data corresponding to the dialogue intention based on the second dialogue segment associated with the first dialogue segment. This is conducive to reducing the complexity of constructing the target dialogue data, i.e., the speech, thereby helping to reduce the construction cost of the target dialogue data and ensure the accuracy of the construction results. In addition, this method is also conducive to improving the construction efficiency of the target dialogue data.

[0168] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 4 FIG. 1 is a schematic diagram of a data processing device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises:

[0169] The acquisition module 402 is configured to acquire historical conversation data containing at least two conversation roles, and divide the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment;

[0170] The clustering module 404 is configured to determine a first dialogue segment containing a target dialogue turn in each dialogue set, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result;

[0171] A first determination module 406 is configured to determine the dialogue intention information of the target dialogue role according to the first clustering result;

[0172] The second determination module 408 is configured to determine a second conversation segment associated with the first conversation segment in each conversation set, and determine target conversation data corresponding to the conversation intention information based on the second conversation segment.

[0173] Optionally, the second determining module 408 is further configured to:

[0174] Performing semantic analysis on the second dialogue segment in each dialogue set, and marking the second dialogue segment according to the analysis result;

[0175] Determine the target dialogue path corresponding to the dialogue intention information according to the generated annotation results;

[0176] According to the second dialogue segment corresponding to the target dialogue path, target dialogue data corresponding to the dialogue intention information is generated.

[0177] Optionally, the second determining module 408 is further configured to:

[0178] Establish the dialogue path corresponding to each dialogue set according to the generated annotation results;

[0179] Determining the conversation data included in the target clustering result, and determining the target conversation set to which the conversation data belongs, wherein the target clustering result is one of the first clustering results;

[0180] The dialogue paths corresponding to the target dialogue set are integrated to generate a target dialogue path corresponding to the dialogue intention information.

[0181] Optionally, the second determining module 408 is further configured to:

[0182] Calculate the transition probability between different annotation results according to the jump relationship between the second dialogue segments corresponding to the annotation results in each dialogue set;

[0183] Constructing a transfer matrix corresponding to the labeling result according to the transfer probability;

[0184] A dialogue path corresponding to each dialogue set is determined according to the transfer matrix.

[0185] Optionally, the data processing device further includes:

[0186] a dialogue type determination module, configured to determine the dialogue type of the historical dialogue data according to the dialogue data of the first dialogue round in the historical dialogue data;

[0187] The dialogue turn determination module is configured to determine the target dialogue turn according to the dialogue type.

[0188] Optionally, when the historical conversation data is of the first conversation type, the clustering module 404 is further configured to:

[0189] Determine the first dialogue segment in each dialogue set that contains the target dialogue turn;

[0190] The conversation data of the target conversation role in the first conversation segment is extracted, and the conversation data is clustered using a first density-based clustering algorithm to generate a corresponding first clustering result.

[0191] Optionally, when the historical conversation data is of the second conversation type, the data processing device further includes:

[0192] A conversation segment determination module, configured to determine a third conversation segment in each conversation set;

[0193] A first generating module is configured to extract the dialogue data of the first dialogue role in the third dialogue segment, and cluster the dialogue data of the first dialogue role using a mean clustering algorithm to generate a corresponding second clustering result;

[0194] a processing module, configured to determine the starting conversation data corresponding to the conversation intention information according to the second clustering result;

[0195] a division module, configured to filter part of the historical conversation data from the historical conversation data according to the initial conversation data, and divide the part of the historical conversation data into at least two conversation sets; and / or,

[0196] An adding module is configured to add the initial conversation data to the target conversation data.

[0197] Optionally, the acquisition module 402 is further configured to:

[0198] Cleaning the historical conversation data and generating corresponding cleaning results;

[0199] Performing vectorization processing on the historical conversation data contained in the cleaning result to generate corresponding vectorized data;

[0200] Semantic recognition is performed on the historical conversation data included in the cleaning result according to the vectorized data, and the historical conversation data included in the cleaning result is divided into at least two conversation sets according to the semantic recognition result.

[0201] Optionally, the data processing device further includes:

[0202] a second generating module configured to extract the dialogue data of the target dialogue role in the first dialogue segment, and cluster the dialogue data using a second density-based clustering algorithm to generate a corresponding third clustering result;

[0203] An information determination module, configured to determine the long-tail intention information of the target dialogue role according to the third clustering result;

[0204] The data determination module is configured to determine the conversation data corresponding to the long-tail intent information according to the first conversation segment, and add the conversation data corresponding to the long-tail intent information to the target conversation data.

[0205] Optionally, the second determining module 408 is further configured to:

[0206] Clustering the conversation data of the first dialogue role in the second dialogue segment, and clustering the conversation data of the second dialogue role in the second dialogue segment, to generate a corresponding fourth clustering result, wherein the second dialogue role is the target dialogue role;

[0207] The target dialogue data corresponding to the dialogue intention information is determined according to the fourth clustering result.

[0208] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0209] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0210] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0211] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0212] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 500 may also be a mobile or stationary server. The memory 510 is used to store computer executable instructions, and the processor 520 is used to execute the following computer executable instructions:

[0213] Acquire historical dialogue data containing at least two dialogue roles, and divide the historical dialogue data into at least two dialogue sets, wherein each dialogue set contains at least one dialogue segment;

[0214] Determine a first dialogue segment in each dialogue set that contains a target dialogue turn, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding clustering result;

[0215] Determining the dialogue intention information of the target dialogue role according to the clustering result;

[0216] A second conversation segment associated with the first conversation segment in each conversation set is determined, and target conversation data corresponding to the conversation intention information is determined based on the second conversation segment.

[0217] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.

[0218] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which are used to implement the steps of the data processing method when executed by a processor.

[0219] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.

[0220] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0221] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0222] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0223] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0224] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Acquire historical dialogue data containing at least two dialogue roles, and divide the historical dialogue data into at least two dialogue sets, wherein each dialogue set contains at least one dialogue segment; Determine a first dialogue segment containing a target dialogue turn in each dialogue set, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result; Determining the dialogue intention information of the target dialogue role according to the first clustering result; Determine a second conversation segment associated with the first conversation segment in each conversation set, and determine target conversation data corresponding to the conversation intention information according to the second conversation segment, wherein determining the target conversation data corresponding to the conversation intention information according to the second conversation segment includes: Perform semantic analysis on the second dialogue segment in each dialogue set, and annotate the second dialogue segment according to the analysis result; establish a dialogue path corresponding to each dialogue set according to the generated annotation result; determine the dialogue data included in the target clustering result, and determine the target dialogue set to which the dialogue data belongs, wherein the target clustering result is one of the first clustering results; integrate the dialogue paths corresponding to the target dialogue set to generate a target dialogue path corresponding to the dialogue intention information; and generate target dialogue data corresponding to the dialogue intention information according to the second dialogue segment corresponding to the target dialogue path.

2. The data processing method according to claim 1, wherein the step of establishing a dialog path corresponding to each dialog set according to the generated annotation results comprises: Calculate the transition probability between different annotation results according to the jump relationship between the second dialogue segments corresponding to the annotation results in each dialogue set; Constructing a transfer matrix corresponding to the labeling result according to the transfer probability; A dialogue path corresponding to each dialogue set is determined according to the transfer matrix.

3. The data processing method according to claim 1, further comprising: Determining a conversation type of the historical conversation data according to conversation data of a first conversation round in the historical conversation data; The target dialogue turn is determined according to the dialogue type.

4. The data processing method according to claim 3, wherein when the historical dialogue data is of the first dialogue type, determining the first dialogue segment containing the target dialogue turn in each dialogue set, and clustering the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result, comprises: Determine the first dialogue segment in each dialogue set that contains the target dialogue turn; The conversation data of the target conversation role in the first conversation segment is extracted, and the conversation data is clustered using a first density-based clustering algorithm to generate a corresponding first clustering result.

5. The data processing method according to claim 3 or 4, when the historical conversation data is of the second conversation type, after dividing the historical conversation data into at least two conversation sets, further comprising: Identify the third dialogue segment in each dialogue set; Extracting the dialogue data of the first dialogue character in the third dialogue segment, and clustering the dialogue data of the first dialogue character using a mean clustering algorithm to generate a corresponding second clustering result; Determining the starting conversation data corresponding to the conversation intention information according to the second clustering result; Filtering part of the historical conversation data from the historical conversation data according to the initial conversation data, and dividing the part of the historical conversation data into at least two conversation sets; and / or, The starting conversation data is added to the target conversation data.

6. The data processing method according to claim 1, wherein dividing the historical conversation data into at least two conversation sets comprises: Cleaning the historical conversation data and generating corresponding cleaning results; Performing vectorization processing on the historical conversation data contained in the cleaning result to generate corresponding vectorized data; Semantic recognition is performed on the historical conversation data included in the cleaning result according to the vectorized data, and the historical conversation data included in the cleaning result is divided into at least two conversation sets according to the semantic recognition result.

7. The data processing method according to claim 4, after determining the second conversation segment associated with the first conversation segment in each conversation set, further comprising: Extracting the dialogue data of the target dialogue role in the first dialogue segment, and clustering the dialogue data using a second density-based clustering algorithm to generate a corresponding third clustering result; Determining the long-tail intention information of the target dialogue role according to the third clustering result; Determine the conversation data corresponding to the long-tail intent information according to the first conversation segment, and add the conversation data corresponding to the long-tail intent information to the target conversation data.

8. The data processing method according to claim 1, wherein determining the target dialogue data corresponding to the dialogue intention information according to the second dialogue segment comprises: Clustering the conversation data of the first dialogue character in the second dialogue segment, and clustering the conversation data of the second dialogue character in the second dialogue segment, to generate a corresponding fourth clustering result, wherein the second dialogue character is the target dialogue character; The target dialogue data corresponding to the dialogue intention information is determined according to the fourth clustering result.

9. A data processing device, comprising: An acquisition module is configured to acquire historical conversation data containing at least two conversation roles, and divide the historical conversation data into at least two conversation sets, wherein each conversation set contains at least one conversation segment; A clustering module is configured to determine a first dialogue segment containing a target dialogue turn in each dialogue set, and cluster the dialogue data of the target dialogue role in the first dialogue segment to generate a corresponding first clustering result; A first determination module is configured to determine the dialogue intention information of the target dialogue role according to the first clustering result; The second determination module is configured to determine the second dialogue segment associated with the first dialogue segment in each dialogue set, and determine the target dialogue data corresponding to the dialogue intention information based on the second dialogue segment, wherein the determination of the target dialogue data corresponding to the dialogue intention information based on the second dialogue segment includes: performing semantic analysis on the second dialogue segment in each dialogue set, and annotating the second dialogue segment according to the analysis result; establishing a dialogue path corresponding to each dialogue set according to the generated annotation result; determining the dialogue data included in the target clustering result, and determining the target dialogue set to which the dialogue data belongs, wherein the target clustering result is one of the first clustering results; integrating the dialogue paths corresponding to the target dialogue set to generate a target dialogue path corresponding to the dialogue intention information; and generating the target dialogue data corresponding to the dialogue intention information according to the second dialogue segment corresponding to the target dialogue path.

10. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to implement the steps of the data processing method according to any one of claims 1 to 8 when executing the computer-executable instructions.

11. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 8.

Images