Business data processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a business data processing method and device, electronic equipment and storage medium, which can be applied to the technical field of intelligent customer service. The business data processing method comprises: obtaining to-be-processed business data; determining N recall cluster centers from M cluster centers obtained after clustering, wherein the cluster center represents at least one piece of business data in the same type of business data, and M is greater than N; determining a target cluster center that satisfies a first preset condition in terms of a first similarity between the to-be-processed business data and the recall cluster center from the N recall cluster centers according to the first similarity between the to-be-processed business data and the recall cluster center; and classifying the to-be-processed business data into a cluster corresponding to the target cluster center.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent customer service technology, specifically to a business data processing method and apparatus, electronic device and storage medium. Background Technology

[0002] In the process of customer service, customer service personnel need to answer customer questions. With the increasing number of customer service incidents, it takes a lot of time to handle customer service incidents manually. How to achieve efficient and accurate responses to customer questions is an urgent problem to be solved.

[0003] In the process of implementing this disclosure, it was found that as the data volume of customer service events increases, the current response and processing methods for customer service events require a lot of time and computing resources. When computing resources are limited, it is difficult to cope with the large number of customer service events and cannot guarantee the timeliness of customer service event responses. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a business data processing method and apparatus, electronic device, storage medium and program product.

[0005] One aspect of this disclosure provides a business data processing method, comprising:

[0006] Obtain pending business data;

[0007] From the M cluster centers obtained after clustering, N recall cluster centers are determined. Each cluster center represents at least one piece of business data in the same type of business data, and M≥N.

[0008] Based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers, the target cluster center whose first similarity meets the first preset condition is determined from the N recall cluster centers;

[0009] The business data to be processed is assigned to the cluster corresponding to the target cluster center.

[0010] According to embodiments of this disclosure, the above method further includes:

[0011] From the K business data associated with the cluster center, determine L recall data, where K≥L;

[0012] K business data points are associated with L recall data points to obtain K association scores, where each association score is associated with one of the K business data points.

[0013] The target data among the K business data associated with the target association score is taken as the new cluster center associated with the K business data, where the target association score is: the association score among the K association scores that meets the second preset condition.

[0014] According to embodiments of this disclosure, K business data points are correlated with L recall data points to obtain K correlation scores, including:

[0015] The data from K business data points are compared with the L recall data points to obtain K sets of similarity results. Each set of similarity results is associated with one business data point from the K business data points, and each set of similarity results includes L second similarities.

[0016] Calculate the mean of the L second similarities in each similarity result set to obtain K association scores.

[0017] According to an embodiment of this disclosure, the association score that satisfies the second preset condition is the association score with the largest score among the K association scores.

[0018] According to embodiments of this disclosure, determining a target cluster center whose first similarity satisfies a first preset condition from the N recall cluster centers based on a first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers includes:

[0019] Based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers, target cluster centers with a first similarity greater than or equal to a preset similarity threshold are determined from the N recall cluster centers.

[0020] According to embodiments of this disclosure, the above method further includes:

[0021] If none of the N recalled cluster centers contain a target cluster center whose first similarity satisfies the first preset condition, a new cluster is created.

[0022] The pending business data is assigned to a newly created cluster, with the pending business data serving as the cluster core of the newly created cluster.

[0023] According to embodiments of this disclosure, after the M cluster centers are indexed and stored in an index database, the method further includes classifying the business data to be processed into the newly created clusters:

[0024] Create a new index for the cluster core of the newly created cluster;

[0025] Add the cluster centers of the newly created clusters and the newly created indexes to the index library so that the index library can be used to determine the recall cluster centers from the cluster centers obtained after clustering.

[0026] Another aspect of this disclosure provides a business data processing apparatus, including an acquisition module, a first determination module, a second determination module, and a first classification module.

[0027] The acquisition module is used to acquire business data to be processed.

[0028] The first determining module is used to determine N recall cluster centers from the M cluster centers obtained after clustering. Each cluster center represents at least one piece of business data in the same type of business data, and M≥N.

[0029] The second determining module is used to determine the target cluster center whose first similarity satisfies the first preset condition from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers in the N recall cluster centers.

[0030] The first classification module is used to classify the business data to be processed into the cluster corresponding to the target cluster center.

[0031] According to embodiments of this disclosure, the above-described apparatus further includes a third determining module, an association module, and an execution module.

[0032] The third determining module is used to determine L recall data from K business data associated with the cluster center, where K≥L;

[0033] The association module is used to perform association calculations on K business data and L recall data respectively to obtain K association scores, where each association score is associated with one of the K business data.

[0034] The execution module is used to take the target data among the K business data associated with the target association score as the new cluster center associated with the K business data, where the target association score is: the association score among the K association scores that meets the second preset condition.

[0035] According to embodiments of this disclosure, the associated module includes a first computing unit and a second computing unit.

[0036] The first calculation unit is used to calculate the similarity between the data in the K business data and the L recall data respectively, so as to obtain K sets of similarity result sets. Each set of similarity result sets is associated with one business data in the K business data, and each set of similarity result sets includes L second similarities.

[0037] The second calculation unit is used to calculate the mean of the L second similarities in each similarity result set to obtain K association scores.

[0038] According to an embodiment of this disclosure, the association score that satisfies the second preset condition is: the association score with the largest score among the K association scores.

[0039] According to an embodiment of this disclosure, the second determining module includes a determining unit, configured to determine a target cluster center from the N recall cluster centers whose first similarity is greater than or equal to a preset similarity threshold, based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers.

[0040] According to embodiments of this disclosure, it also includes a first creation module and a second classification module.

[0041] The first new creation module is used to create a new cluster when there is no target cluster center whose first similarity satisfies the first preset condition among the N recalled cluster centers.

[0042] The second classification module is used to classify the business data to be processed into newly created clusters, with the business data to be processed serving as the cluster core of the newly created clusters.

[0043] According to an embodiment of this disclosure, the M cluster centers are indexed and stored in an index library, and the apparatus further includes a second creation module and an addition module.

[0044] The second newly created module is used to create an index for the cluster center of the newly created cluster after the business data to be processed is classified into the newly created cluster.

[0045] The add module is used to add the cluster centers of newly created clusters and the newly created indexes to the index library, so that the index library can be used to determine the recall cluster centers from the cluster centers obtained after clustering.

[0046] Another aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the above-described business data processing method.

[0047] Another aspect of this disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the above-described business data processing method.

[0048] Another aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the above-described business data processing method. Attached Figure Description

[0049] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0050] Figure 1The illustration shows an application scenario diagram of a business data processing method, apparatus, device, medium, and program product according to embodiments of the present disclosure;

[0051] Figure 2 A flowchart illustrating a business data processing method according to an embodiment of the present disclosure is shown schematically.

[0052] Figure 3 A flowchart illustrating a business data processing method according to another embodiment of this disclosure is shown schematically;

[0053] Figure 4 A flowchart illustrating a business data processing method according to another embodiment of this disclosure is shown schematically;

[0054] Figure 5 A flowchart illustrating a method for adjusting cluster centers according to an embodiment of the present disclosure is shown schematically.

[0055] Figure 6 A schematic block diagram of a business data processing apparatus according to embodiments of the present disclosure is shown; and

[0056] Figure 7 A block diagram of an electronic device suitable for implementing a business data processing method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0057] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0058] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0059] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0060] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0061] In customer service, customer service personnel need to answer customer questions. With the increasing number of customer service incidents, handling them manually consumes a significant amount of time. Therefore, how to achieve efficient and accurate responses to customer questions is a pressing issue. Intelligent customer service has emerged as a solution. Intelligent customer service typically utilizes big data processing technology to process customer service data in order to provide intelligent responses.

[0062] In the process of developing this disclosure, it was discovered that big data processing technology can be applied to classify large amounts of customer service event data first, and then provide targeted responses for each category. This can significantly reduce the workload of responses and improve work efficiency. For example, clustering can be used to cluster business data text (dialogues, forums, comments, etc.) to achieve the classification of business data.

[0063] However, as the volume of customer service event data grows larger, existing algorithms have some shortcomings. For example, when using clustering algorithms such as k-means that require specifying the number of clusters, the number of clusters is difficult to predict, and there are a large number of outliers that should not be forcibly assigned to certain categories. When using clustering algorithms such as DBScan that do not require specifying the number of clusters, the computational load is very large because each data point is calculated to have similarity with other data, which consumes a lot of time and computing resources. When computing resources are limited, it is difficult to handle a large number of customer service events and cannot guarantee the timeliness of customer service event responses.

[0064] In view of this, embodiments of this disclosure provide a business data processing method, including:

[0065] Obtain pending business data;

[0066] From the M cluster centers obtained after clustering, N recall cluster centers are determined. Each cluster center represents at least one piece of business data in the same type of business data, and M≥N.

[0067] Based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers, the target cluster center whose first similarity meets the first preset condition is determined from the N recall cluster centers;

[0068] The business data to be processed is assigned to the cluster corresponding to the target cluster center.

[0069] Figure 1 The illustration shows an application scenario diagram of a business data processing method, apparatus, device, medium, and program product according to embodiments of the present disclosure.

[0070] like Figure 1 As shown, application scenario 100 according to this embodiment may include terminal devices 101, 102, and 103, network 104, and server 105. Network 104 is used as a medium to provide a communication link between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0071] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0072] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0073] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0074] According to embodiments of this disclosure, in the application scenario of the intelligent customer service disclosed herein, customers can interact with server 105 through terminal devices 101, 102, and 103. For example, customers can ask questions about a certain business event (such as order issues, logistics issues, etc.), and server 105 can respond. Alternatively, customers and server 105 can have a dialogue and discussion about a certain event.

[0075] According to embodiments of this disclosure, when the amount of customer problem data is large, server 105 can process the business data used to represent a large amount of customer dialogue content by executing the business data processing method of embodiments of this disclosure. For example, these business data can be clustered to form multiple categories, each category can be used to represent a type of business problem, and then a unified targeted response can be made for each category, and the response content can be displayed to the customer through terminal devices 101, 102, and 103.

[0076] It should be noted that the business data processing method provided in this embodiment can generally be executed by server 105. Correspondingly, the business data processing apparatus provided in this embodiment can generally be located in server 105. The business data processing method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the business data processing apparatus provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0077] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0078] The following will be based on Figure 1 The described scene, through Figures 2 to 7 The business data processing method of the disclosed embodiments will be described in detail.

[0079] Figure 2 A flowchart illustrating a business data processing method according to an embodiment of the present disclosure is shown schematically.

[0080] like Figure 2 As shown, the business data processing method of this embodiment includes operations S201 to S204.

[0081] In operation S201, obtain the business data to be processed;

[0082] In operation S202, N recall cluster centers are determined from the M cluster centers obtained after clustering. Each cluster center represents at least one piece of business data in the same type of business data, and M≥N.

[0083] In operation S203, based on the first similarity between the business data to be processed and the recall cluster centers in N recall cluster centers, the target cluster center whose first similarity satisfies the first preset condition is determined from the N recall cluster centers.

[0084] In operation S204, the business data to be processed is assigned to the cluster corresponding to the target cluster center.

[0085] According to embodiments of this disclosure, the above-described business data processing method is applied to intelligent customer service scenarios to classify business data related to customer service events, enabling intelligent responses based on the classified data. For example, in this scenario, a customer can ask a question about a specific business event (such as an order issue, logistics issue, etc.), and the server can respond; alternatively, the customer and the server can engage in a dialogue or discussion regarding a specific business event.

[0086] According to embodiments of this disclosure, a server can process business data (such as when logistics will ship, how long it will take to receive the goods, how to compensate for damaged goods, etc.) that represents a large amount of customer dialogues, comments, etc. by executing the business data processing method of this disclosure. For example, these business data can be clustered to form multiple categories, each category can be used to represent a type of business question, so that the server can then provide a unified response for each category and display the response content to the customer through the terminal device, which can greatly reduce the workload of response and improve work efficiency.

[0087] According to the embodiments of this disclosure, in the process of clustering a large amount of dialogue text business data using the above-described business data processing method, these text data (dialogues, forums, comments, etc.) can be combined to form an original text dataset and stored in a database. Then, each data in the original text dataset is traversed, and the business data processing method described in the embodiments of this disclosure is executed on each business data to be processed to classify it. This can be done by classifying it into an existing cluster or by creating a new data cluster.

[0088] According to embodiments of this disclosure, the above-mentioned clustering of business data can be performed using a clustering method that does not specify the number of clusters, based on the similarity between text data. Related technologies employ clustering methods such as DBScan, which involve calculating the similarity of each data point with other data separately, resulting in a large computational load and consuming significant time and computing resources. Embodiments of this disclosure employ a method of recalling some cluster centers from existing cluster centers, calculating the similarity between the business data to be processed and the recalled cluster centers, and classifying the data according to the similarity calculation results.

[0089] Specifically, in the above clustering method, operation S201 obtains the business data to be processed. For example, it can be to obtain the current business data to be processed from the database, or it can be one of the data in the original text dataset that has not yet been processed and obtained randomly, or it can be one of the data obtained in the processing order.

[0090] According to the embodiments of this disclosure, since the above-mentioned clustering of business data can be performed using a clustering method that does not specify the number of clusters, if the business data to be processed is the first piece of data, then this piece of data is directly used as the first data cluster, and this piece of data is used as the cluster center of this cluster. Then, based on the similarity principle, the second piece of business data to be processed is clustered (the similarity between the second piece of data and the first cluster center is calculated). According to preset conditions, if the second piece of business data to be processed cannot be classified into the first data cluster, then a second data cluster is created, and this piece of data is used as the cluster center of this new cluster. Otherwise, the second piece of business data to be processed can be classified into the first data cluster, and so on, until all the data is traversed and the clustering process is completed.

[0091] According to embodiments of this disclosure, each cluster obtained through clustering includes a cluster center. The text data in each cluster belongs to the same business category, and each category can be used to represent a type of business problem. For example, the business problem represented by the first cluster data is about the logistics delivery cycle, the business problem represented by the second cluster data is about the logistics compensation amount, the business problem represented by the third cluster data is about the order confirmation cycle, and so on. Each cluster includes a cluster center, which represents at least one piece of business data in the same type of business data. For example, the cluster center can be one of the business data in this cluster, or it can be two pieces of data that are the same or very similar to each other.

[0092] According to an embodiment of this disclosure, the above-described operation S202 determines N recall cluster centers from the M cluster centers obtained after clustering. This operation can be performed during the clustering process for each piece of data (except for the first piece of data), or it can be performed after a preset number of data are processed. For example, if the original text dataset contains 10,000 pieces of data, the recall operation described in operation S202 can be performed when the 300th piece of data is processed.

[0093] According to embodiments of this disclosure, determining N recall cluster centers from M cluster centers obtained after clustering can be achieved by recalling the same fixed number of N recall cluster centers from the existing M cluster centers during the clustering process. For example, recall can be started when the 1000th data point is reached. Starting from the 1001st data point, 30 data points are recalled from the existing cluster centers as recall cluster centers during the clustering process for each data point. Before the 1001st data point, no recall is performed, and similarity calculation is directly performed based on all existing cluster centers.

[0094] According to embodiments of this disclosure, N recall cluster centers are determined from the M cluster centers obtained after clustering. Alternatively, during the clustering process, an indefinite number of N recall cluster centers can be recalled from the existing M cluster centers. For example, a certain preset proportion of data (e.g., 0.05M or 0.03M) can be recalled each time from the existing M cluster centers as recall cluster centers.

[0095] According to embodiments of this disclosure, N recall cluster centers are determined from M cluster centers obtained after clustering. This can be done by recalling some data from existing cluster centers, or by recalling all data when the number of existing cluster centers is relatively small, and recalling some data when the number of existing cluster centers is relatively large. For example, if the number of existing cluster centers is less than a certain preset threshold, all data is recalled, and if the number of existing cluster centers is greater than or equal to a certain preset threshold, some data is recalled.

[0096] According to the embodiments of this disclosure, the business data can be clustered by the above-described business data processing method to divide the original data into multiple data categories. Each category represents a type of business problem, and a unified response can be made for each category, which can greatly reduce the workload of response and improve work efficiency.

[0097] According to embodiments of this disclosure, in the process of clustering business data, embodiments of this disclosure employ a method of recalling some cluster centers from existing cluster centers and calculating the similarity between the business data to be processed and the recalled cluster centers to achieve data classification. Compared with the density clustering method in related technologies, this method only requires comparing the similarity between the data to be processed and some cluster centers, significantly reducing the computational load of clustering. The computational load is reduced from (number of data entries) * (number of all cluster centers - which can reach hundreds of thousands or millions when the data volume is large) to (number of data entries) * (recalled data volume), improving the processor's running speed, eliminating the need for consuming a large amount of time and computing resources, reducing the requirements for computer hardware, and better handling a large number of customer service events when computing resources are limited, while ensuring the timeliness of customer service event responses and improving customer experience.

[0098] According to embodiments of this disclosure, in the process of clustering business data to be processed, determining target cluster centers whose first similarity satisfies a first preset condition from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers may include: determining target cluster centers whose first similarity is greater than or equal to a preset similarity threshold from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers.

[0099] According to an embodiment of this disclosure, the above method may be as follows: calculate the first similarity between the business data to be processed and each of the N recall cluster centers to obtain N first similarity values, select the largest similarity value as the target similarity value, and if the target similarity value is greater than a preset similarity threshold, take the recall cluster center corresponding to the target similarity value as the target cluster center, and classify the business data to be processed into the cluster corresponding to the target cluster center. This case is the case where the N recall cluster centers include the target cluster center.

[0100] According to an embodiment of this disclosure, when the largest similarity value (target similarity value) among the N calculated first similarity values ​​is less than or equal to a preset similarity threshold, a new cluster is created, and the business data to be processed is assigned to the new cluster. The business data to be processed serves as the cluster center of the new cluster. This situation occurs when the N recall cluster centers do not include a target cluster center whose first similarity satisfies the first preset condition.

[0101] According to embodiments of this disclosure, the above method can also be as follows: Calculate the first similarity between the business data to be processed and each of the N recall cluster centers to obtain N first similarity values. Directly select the recall cluster centers with first similarity values ​​greater than a preset similarity threshold as target cluster centers. If there is only one target cluster center, the business data to be processed is assigned to the cluster corresponding to the target cluster center. If there are more than one target cluster center, the business data to be processed is randomly assigned to the cluster corresponding to one of the target cluster centers, or the business data to be processed is assigned to the cluster corresponding to the target cluster center with the largest similarity value. If all N first similarities are less than or equal to the preset similarity threshold (this case occurs when there is no target cluster center among the N recall cluster centers that meets the conditions), a new cluster is created, and the business data to be processed is assigned to the newly created cluster.

[0102] According to embodiments of this disclosure, when calculating the first similarity between the business data to be processed and the center of the recalled cluster, various semantic-based text similarity algorithms can be used. For example, a cosine similarity algorithm based on W2V (word 2 vec) can be used, or a trained model, such as the BERT model or other models, can be used to assist in the calculation. By using semantic-based text similarity algorithms, the accuracy of text clustering can be improved compared to other algorithms based on non-semantic metrics.

[0103] According to embodiments of this disclosure, N recall cluster centers are determined from M cluster centers obtained after clustering. This can be done by randomly recalling N recall cluster centers from the M cluster centers, or by selecting N data that are most similar to the current business data to be processed from the M cluster centers as recall cluster centers. For example, multiple data that have a similarity greater than a certain preset threshold with the current business data to be processed are recalled.

[0104] Select N data points (data with similarity greater than a preset threshold) that are most similar to the current business data to be processed from M cluster centers as recall cluster centers. For example, a retrieval tool can be used as an aid to achieve the function of recalling data with similarity greater than a certain preset threshold.

[0105] Furthermore, in order to achieve recall using retrieval tools, existing cluster centers are indexed and stored in the index library of the retrieval tools. After each new cluster is created, an index is created for the cluster center of the new cluster, and the cluster center of the new cluster and the newly created index are added to the index library so that the retrieval tools and the index library can be used to determine the recall cluster centers from the cluster centers obtained after clustering.

[0106] According to embodiments of this disclosure, the retrieval tool can be a word-based retrieval tool, such as Lucene, ES, or Solr. The retrieval tool can also be a vector-based retrieval tool, such as Faiss.

[0107] According to the embodiments of this disclosure, the embodiments of this disclosure adopt a method of recalling some cluster centers from existing cluster centers and calculating the similarity between the business data to be processed and the recalled cluster centers to achieve data classification. It only requires comparing the similarity between the data to be processed and some cluster centers, which greatly reduces the amount of clustering calculation. Furthermore, since the recalled data is data that is relatively similar to the business data to be processed, the loss of clustering accuracy is small, and the amount of clustering calculation is greatly reduced while ensuring the calculation accuracy.

[0108] Based on the above embodiments, Figure 3 A flowchart illustrating a business data processing method according to an embodiment of the present disclosure is shown schematically.

[0109] like Figure 3 As shown, the data processing methods performed on each piece of business data to be processed include operations S301 to S304, which iterate through the data in the text dataset to be processed.

[0110] In operation S301, based on a preset number of cluster centers to be recalled, a portion of the existing cluster centers obtained after clustering are recalled as recall cluster centers. Specifically, during the clustering process, a fixed number of N recall cluster centers are recalled from the existing M cluster centers, based on the preset number of recall cluster centers. For example, recall could begin when the 500th data item is processed, and starting from the 501st data item, 30 data items are recalled from the existing cluster centers during the clustering process for each data item. The recalled data can be achieved using the retrieval tool Lucene. The Lucene index contains indexes of all existing cluster centers, which can be used to select N data items from the M cluster centers that are most similar to the currently pending business data as recall cluster centers.

[0111] In operation S302, the similarity between the current business data to be processed and each of the N recall cluster centers is calculated to obtain N similarity values.

[0112] In operation S303, if the largest similarity value among N similarity values ​​is greater than a preset threshold, the current data is assigned to the corresponding cluster, that is, to the cluster corresponding to the recall cluster center (target cluster center) associated with the largest similarity value.

[0113] In operation S304, if all N first similarities are less than or equal to the preset similarity threshold (this case is when the N recall cluster centers do not include the target cluster center that meets the conditions), a new cluster is created, and the business data to be processed is assigned to the newly created cluster. The business data to be processed is used as the cluster center of the newly created cluster.

[0114] In operation S305, an index is added to the newly created cluster centers and added to the Lucene index library so that during the clustering of the next data, Lucene can be used to select N data that are most similar to the next data from the existing cluster centers as the recall cluster centers.

[0115] Figure 4 A flowchart illustrating a business data processing method according to another embodiment of the present disclosure is shown.

[0116] like Figure 4 As shown, according to the business data processing method of this disclosure embodiment, after each traversal and execution of the data in the text dataset to be processed, the cluster centers of the multiple clusters obtained after each traversal and clustering can be adjusted and updated.

[0117] Specifically, according to embodiments of this disclosure, the specific flow of the above-described business data processing method may include:

[0118] Before performing clustering, parameters are set, including preset similarity thresholds (first similarity threshold, second similarity threshold), the number of recalled cluster centers c1 (default 30), the number of recalled data points c2 (default 30), and the number of iterations. The first similarity threshold, associated with a first preset condition, is used to determine target cluster centers whose first similarity satisfies the first preset condition (first similarity greater than or equal to the first similarity threshold) from the existing recalled cluster centers during clustering of each data point based on the similarity principle. The number of recalled cluster centers c1 is used to recall a fixed number of c1 cluster centers from the existing cluster centers during clustering of each data point. The number of recalled data points c2 is used to recall a fixed number of c2 data points from each cluster during the process of adjusting and updating the cluster centers of multiple clusters obtained after each clustering iteration. The second similarity threshold is used in the data recall process: it can be used during the clustering process for each data point to recall c1 data points from the existing cluster centers that are most similar to the current data to be processed – data points whose similarity to the current data to be processed is greater than or equal to the second similarity threshold; the second similarity threshold is also used during the process of adjusting and updating the cluster centers of multiple clusters to recall c2 data points from each cluster that are most similar to the current data – data points whose similarity to the current data is greater than or equal to the second similarity threshold.

[0119] Next, the data in the text dataset to be processed is traversed and clustered. The cluster centers of the multiple clusters obtained after traversing the data are adjusted and updated. Then, the data is clustered again based on the updated cluster centers until the preset number of iterations is reached.

[0120] Furthermore, adjusting and updating the cluster centers of the multiple clusters obtained after each clustering traversal can include (the following method is illustrated by taking the cluster center of any one of the clusters as an example):

[0121] From the K business data associated with the cluster center, determine L recall data, where K≥L;

[0122] K business data points are associated with L recall data points to obtain K association scores, where each association score is associated with one of the K business data points.

[0123] The target data among the K business data associated with the target association score is taken as the new cluster center associated with the K business data, where the target association score is: the association score among the K association scores that meets the second preset condition.

[0124] According to embodiments of this disclosure, determining L recall data from K service data associated with the cluster center can be done by randomly recalling L recall data from the K service data, or by recalling L data from the K service data that are most similar to the current data as recall data, for example, recalling multiple data that have a similarity greater than a second similarity threshold with the current data.

[0125] According to embodiments of this disclosure, L data points most similar to the current data are retrieved from K business data points as retrieved data. For example, a retrieval tool can be used as an aid to retrieve data with high similarity. Furthermore, to utilize the retrieval tool for retrieval, each data point in the existing clusters is indexed and stored in the retrieval tool's index library. The method for using the retrieval tool for retrieval can refer to the description of retrieving multiple cluster centers from existing cluster centers during the clustering process described above, and will not be repeated here. The retrieval tool can also be Lucene, Elasticsearch, Solr, Faiss, etc.

[0126] According to embodiments of this disclosure, determining L recall data from K service data associated with the cluster center can be done by recalling the same fixed number of L recall data from the K service data for each cluster; for example, 30 data points can be recalled from each cluster as recall data.

[0127] According to the embodiments of this disclosure, determining L recall data from K service data associated with the cluster center can also be done by recalling different numbers of L recall data from the K service data for each cluster; for example, for each cluster, a certain preset proportion (e.g., 0.005K or 0.01K) of data can be recalled from the K service data as recall data.

[0128] According to embodiments of this disclosure, by adjusting and updating the cluster centers obtained from each clustering, the accuracy of clustering can be improved, the clustered data can be more concentrated, and the data of each cluster can be made to basically belong to the same type of business problem. Subsequently, each type of problem can be responded to in a targeted manner, avoiding incorrect responses and improving customer experience.

[0129] According to embodiments of this disclosure, during the process of adjusting and updating the cluster centers obtained from each clustering, the computational load of association calculation can be significantly reduced by recalling some data from the business data associated with the cluster centers for association calculation. Each data only needs to be associated with a small amount of data, without having to be associated with all data. The computational load is reduced from n (cluster data volume) * n - 1 to n * c2 (recalled data volume). This improves the processor's running speed without consuming a large amount of time and computing resources, reducing the requirements for computer hardware. When computing resources are limited, it can better handle a large number of customer service events and ensure the timeliness of customer service event responses, thus improving customer experience.

[0130] Figure 5 A flowchart illustrating a method for adjusting cluster centers according to an embodiment of the present disclosure is shown.

[0131] like Figure 5 As shown, the method for adjusting the cluster center of any cluster includes operations S501 to S504.

[0132] In operation S501, for each data in a cluster, based on a preset recall data quantity, a portion of the data (equal to the preset recall data quantity) is recalled from the cluster as recall data. That is, for each cluster, the same fixed number of L recall data are recalled from K business data.

[0133] Next, the K business data points are correlated with the L recall data points to obtain K correlation scores, specifically including operations S502 and S503.

[0134] In operation S502, the similarity between each data in the cluster and the recalled data is calculated. Each data in the K business data is compared with the L recalled data to obtain K sets of similarity results. Each set of similarity results is associated with one business data in the K business data. Each set of similarity results includes L second similarities.

[0135] In operation S503, the mean of the L second similarities in each similarity result set is calculated to obtain K association scores. The score that meets the second preset condition, namely the largest score among the K association scores, is taken as the target association score, and the data associated with the target association score is taken as the new cluster center.

[0136] After completing the S504 operation and traversing the clusters, update the cluster center in the Lucene index.

[0137] The following example illustrates the method for adjusting the cluster center of any given cluster:

[0138] For example, for cluster 1, which contains 10,000 data entries: data 1, data 2, data 3, data..., the cluster center of cluster 1 is data 10.

[0139] For Data 1, Lucene is used to recall the 30 most similar data points from these 10,000 data points as the recall data (note that Data 1 itself is excluded); then, the similarity between Data 1 and these 30 recalled data points is calculated to obtain the first set of similarity results, which includes 30 similarity values; then the mean of these 30 similarity values ​​is calculated to obtain the association score of Data 1.

[0140] For Data 2, Lucene is used to recall the 30 most similar data points from the 10,000 data points as the recall data (note that Data 2 itself is excluded); then, the similarity between Data 2 and these 30 recalled data points is calculated to obtain the second set of similarity results, which includes 30 similarity values; then the mean of these 30 similarity values ​​is calculated to obtain the association score of Data 2.

[0141] After iterating through 10,000 data points in cluster 1, we obtain the association scores for these 10,000 data points. For example, data 1: 0.8; data 2: 0.7; data 3: 0.9... Among them, the highest association score is the score of 0.9 corresponding to data 3. Then, data 3 is taken as the new cluster center of cluster 1.

[0142] According to the embodiments of this disclosure, during the process of adjusting and updating the cluster centers obtained from each clustering, the computational load of the association calculation can be greatly reduced by recalling some data from the business data associated with the cluster centers and performing association calculations. Furthermore, since the recalled data is data that is similar to the current data, the loss of similarity calculation accuracy is small, and the computational load of the association calculation is greatly reduced while ensuring the calculation accuracy.

[0143] Based on the above-described business data processing method, this disclosure also provides a business data processing apparatus. The following will be combined with... Figure 6 The device is described in detail.

[0144] Figure 6 A schematic block diagram of a business data processing apparatus according to an embodiment of the present disclosure is shown.

[0145] like Figure 6 As shown, the business data processing device 600 of this embodiment includes: an acquisition module 601, a first determination module 602, a second determination module 603, and a first classification module 604.

[0146] Among them, the acquisition module 601 is used to acquire business data to be processed;

[0147] The first determining module 602 is used to determine N recall cluster centers from the M cluster centers obtained after clustering. The cluster centers represent at least one piece of business data in the same type of business data, and M≥N.

[0148] The second determining module 603 is used to determine the target cluster center whose first similarity satisfies the first preset condition from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers in the N recall cluster centers.

[0149] The first classification module 604 is used to classify the business data to be processed into the cluster corresponding to the target cluster center.

[0150] According to the embodiments of this disclosure, the business data can be clustered by the above-mentioned business data processing device into multiple data categories. Each category represents a type of business problem. Subsequently, a unified response can be made for each category, which can greatly reduce the workload of response and improve work efficiency.

[0151] According to embodiments of this disclosure, during the clustering process, the first determining module 602 and the second determining module 603 recall some cluster centers from the existing cluster centers, calculate the similarity between the business data to be processed and the recalled cluster centers to achieve the purpose of data classification. Compared with the density clustering method used in related technologies, it is only necessary to compare the similarity between the data to be processed and some cluster centers, which greatly reduces the amount of clustering computation. The amount of computation is reduced from M (number of data entries) * N (number of all cluster centers, which can reach hundreds of thousands or millions when the data volume is large) to M (number of data entries) * c1 (number of recalled data). This improves the processor's running speed, eliminates the need to consume a lot of time and computing resources, reduces the requirements for computer hardware, and can better handle a large number of customer service events when computing resources are limited, while ensuring the timeliness of customer service event responses and improving customer experience.

[0152] According to embodiments of this disclosure, the above-described apparatus further includes a third determining module, an association module, and an execution module.

[0153] The third determining module is used to determine L recall data from K business data associated with the cluster center, where K≥L;

[0154] The association module is used to perform association calculations on K business data and L recall data respectively to obtain K association scores, where each association score is associated with one of the K business data.

[0155] The execution module is used to take the target data among the K business data associated with the target association score as the new cluster center associated with the K business data, where the target association score is: the association score among the K association scores that meets the second preset condition.

[0156] According to embodiments of this disclosure, the associated module includes a first computing unit and a second computing unit.

[0157] The first calculation unit is used to calculate the similarity between the data in the K business data and the L recall data respectively, so as to obtain K sets of similarity result sets. Each set of similarity result sets is associated with one business data in the K business data, and each set of similarity result sets includes L second similarities.

[0158] The second calculation unit is used to calculate the mean of the L second similarities in each similarity result set to obtain K association scores.

[0159] According to an embodiment of this disclosure, the association score that satisfies the second preset condition is: the association score with the largest score among the K association scores.

[0160] According to an embodiment of this disclosure, the second determining module includes a determining unit, configured to determine a target cluster center from the N recall cluster centers whose first similarity is greater than or equal to a preset similarity threshold, based on the first similarity between the business data to be processed and the recall cluster centers among the N recall cluster centers.

[0161] According to embodiments of this disclosure, it also includes a first creation module and a second classification module.

[0162] The first new creation module is used to create a new cluster when there is no target cluster center whose first similarity satisfies the first preset condition among the N recalled cluster centers.

[0163] The second classification module is used to classify the business data to be processed into newly created clusters, with the business data to be processed serving as the cluster core of the newly created clusters.

[0164] According to an embodiment of this disclosure, the M cluster centers are indexed and stored in an index library, and the apparatus further includes a second creation module and an addition module.

[0165] The second newly created module is used to create an index for the cluster center of the newly created cluster after the business data to be processed is classified into the newly created cluster.

[0166] The add module is used to add the cluster centers of newly created clusters and the newly created indexes to the index library, so that the index library can be used to determine the recall cluster centers from the cluster centers obtained after clustering.

[0167] According to embodiments of this disclosure, any plurality of modules among the acquisition module 601, the first determination module 602, the second determination module 603, and the first classification module 604 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 601, the first determination module 602, the second determination module 603, and the first classification module 604 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the acquisition module 601, the first determination module 602, the second determination module 603, and the first classification module 604 can be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0168] Figure 7 A block diagram of an electronic device suitable for implementing a business data processing method according to an embodiment of the present disclosure is shown schematically.

[0169] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0170] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0171] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0172] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0173] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0174] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the business data processing methods provided in the embodiments of this disclosure.

[0175] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0176] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0177] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0178] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0180] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0181] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A business data processing method, comprising: Obtain pending business data; From the M cluster centers obtained after clustering, N recall cluster centers are determined, where each cluster center represents at least one piece of business data in the same type of business data, and M ≥ N; Based on the first similarity between the business data to be processed and the N recall cluster centers, a target cluster center whose first similarity satisfies the first preset condition is determined from the N recall cluster centers; as well as The business data to be processed is assigned to the cluster corresponding to the target cluster center; From the K service data associated with the cluster center, determine L recall data, where K ≥ L; The K business data are compared with the L recall data to obtain K sets of similarity results, and K association scores are obtained based on the K sets of similarity results, wherein each association score is associated with one of the K business data. The target data among the K business data associated with the target association score is taken as the new cluster center associated with the K business data, wherein the target association score is: the association score among the K association scores that satisfies the second preset condition.

2. The method according to claim 1, wherein each set of similarity results is associated with one of the K sets of business data, and each set of similarity results includes L second similarities; The K association scores are obtained by calculating the mean of the L second similarities in each set of similarity results.

3. The method according to claim 1, wherein: The association score that satisfies the second preset condition is the association score with the largest score among the K association scores.

4. The method according to claim 1, wherein, The step of determining the target cluster center whose first similarity satisfies the first preset condition from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers in the N recall cluster centers includes: Based on the first similarity between the business data to be processed and the N recall cluster centers, a target cluster center with the first similarity greater than or equal to a preset similarity threshold is determined from the N recall cluster centers.

5. The method according to claim 1, further comprising: If none of the N recalled cluster centers contain a target cluster center whose first similarity satisfies the first preset condition, a new cluster is created. The pending business data is assigned to a newly created cluster, where the pending business data serves as the cluster center of the newly created cluster.

6. The method according to claim 5, wherein, After the M cluster centers are indexed, they are stored in the index library. The method also includes, after classifying the business data to be processed into the newly created clusters: Create a new index for the cluster center of the newly created cluster; The newly created cluster centers and the newly created indexes are added to the index library so that the index library can be used to determine the recall cluster centers from the cluster centers obtained after clustering.

7. A business data processing apparatus, comprising: The acquisition module is used to acquire business data to be processed. The first determining module is used to determine N recall cluster centers from M cluster centers obtained after clustering, wherein the cluster centers represent at least one piece of business data in the same type of business data, and M≥N; The second determining module is used to determine a target cluster center whose first similarity satisfies a first preset condition from the N recall cluster centers based on the first similarity between the business data to be processed and the recall cluster centers in the N recall cluster centers. The first classification module is used to classify the business data to be processed into a cluster corresponding to the target cluster center; The third determination module is used to determine L recall data from K business data associated with the cluster center, where K≥L; The association module is used to calculate the similarity between the K business data and the L recall data respectively to obtain K sets of similarity result sets, and to obtain K association scores based on the K sets of similarity result sets, wherein each association score is associated with one of the business data in the K business data; The execution module is used to take the target data among the K business data associated with the target association score as the new cluster center associated with the K business data, where the target association score is: the association score among the K association scores that meets the second preset condition.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images