Data processing method and device, computer device and storage medium

Abstract

The application relates to a data processing method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of big data. The method comprises the following steps: acquiring a target data table, wherein the target data table comprises a plurality of business processing data of a target time period; performing clustering processing on each business processing data in the target data table according to each clustering index based on a target clustering analysis strategy, to obtain a plurality of clustering results corresponding to each clustering index; determining an association relationship between each clustering result according to each business processing data in each clustering result; determining at least one target clustering result group from each clustering result according to the association relationship between each clustering result, and generating a data analysis result according to each target clustering result group. By using the method, a time node or a behavior node of a risk of a mode in which a single staff member is responsible for global business can be determined.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In business processing, user-initiated requests are often handled by a single employee. This model, where a single employee is responsible for the entire business process, leads to lower security for the business system.

[0003] Currently, the model where a single employee is responsible for all user business operations can be improved by having multiple employees handle different parts of the business during the business process, thus avoiding the security issues associated with a single employee being responsible for all business operations.

[0004] However, in practice, due to factors such as a large number of users and limited human resources, it is still unavoidable that a single employee is responsible for the overall business. Furthermore, it is impossible to determine whether there is a risk associated with a single employee being responsible for the overall business within the business system. Summary of the Invention

[0005] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can detect the risk of a pattern in a business system where a single employee is responsible for the overall business operations, in order to address the aforementioned technical problems.

[0006] Firstly, this application provides a data processing method. The method includes:

[0007] Obtain the target data table, which contains multiple business processing data for the target time period;

[0008] Based on the target clustering analysis strategy, the business processing data in the target data table are clustered according to each clustering index to obtain multiple clustering results corresponding to each clustering index.

[0009] Based on the business processing data in each of the clustering results, determine the correlation between the clustering results;

[0010] Based on the correlation between the clustering results, at least one target clustering result group is determined from the clustering results, and data analysis results are generated based on the target clustering result groups.

[0011] In one embodiment, the step of clustering data in the target data table based on the target clustering analysis strategy, performing clustering processing on each of the business processing data in the target data table according to each clustering index, and obtaining multiple clustering results corresponding to each clustering index includes:

[0012] For any of the clustering indicators, determine the number of target clustering results corresponding to the clustering indicators in the target data table;

[0013] Based on the target clustering result number and the target clustering analysis algorithm, the business processing data in the target data table are clustered to obtain the target clustering result number of clustering results corresponding to the clustering index.

[0014] In one embodiment, determining the number of target clustering results corresponding to any of the clustering indicators in the target data table includes:

[0015] For any of the clustering indicators, the number of multiple clustering results corresponding to the clustering indicator is determined based on the number of business processing data corresponding to the clustering indicator.

[0016] Determine the sum of squared errors corresponding to the number of clustering results for each clustering index, and based on the sum of squared errors corresponding to the number of clustering results, determine the difference between the sum of squared errors of samples corresponding to any two adjacent numbers of clustering results;

[0017] Based on the target difference among the differences, the number of target clustering results corresponding to the clustering index is determined, wherein the number of target clustering results is the larger number of clustering results among the two adjacent numbers of clustering results corresponding to the target difference.

[0018] In one embodiment, determining the correlation between the clustering results based on the business processing data in each clustering result includes:

[0019] Based on the data of each business processing data in the target data table for each clustering result, a transaction table corresponding to the target data table is determined, and the transaction table includes items of each business processing data for each clustering result;

[0020] Based on the items of each clustering result for each of the business processing data in the transaction table, determine the first itemset;

[0021] The second itemset is determined based on the preset minimum support and the first itemset;

[0022] Based on the second itemset, the association rules for each item in the second itemset are determined, and based on the association rules for each item in the second itemset, the association relationships between the clustering results are determined.

[0023] In one embodiment, determining the second itemset based on a preset minimum support and the first itemset includes:

[0024] During the k-th round of processing, based on the preset minimum support, the k-th frequent itemset that satisfies the preset minimum support is determined in the (k-1)-th frequent itemset. When the number of items in the k-th frequent itemset is less than or equal to the preset number of items, the process proceeds to the (k+1)-th round of processing until the number of items in the m-th frequent itemset is greater than the preset number of items.

[0025] The frequent itemsets in each round of processing are used as the second itemset; where k and m are both positive integers, and when k is 1, the (k-1)th frequent itemset is the first itemset containing one item.

[0026] In one embodiment, determining the association relationships between the clustering results based on the association rules of each item in the second itemset includes:

[0027] Based on the support of the second itemset and the association rules between each item in the second itemset, determine the confidence level of the association rules between each item in the second itemset;

[0028] In the association rules between the items in the second set, the association rules with a confidence level greater than or equal to the preset minimum confidence level are taken as the target association rules;

[0029] The association relationships between the clustering results are determined based on the clustering results corresponding to the target association rule and the confidence level of the target association rule.

[0030] In one embodiment, determining at least one target clustering result group from the clustering results based on the correlation between the clustering results includes:

[0031] For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are determined as a target clustering result group. The target clustering result is any one of the clustering results.

[0032] In one embodiment, before obtaining the target data table, the method further includes:

[0033] Acquire multiple initial business processing data;

[0034] The initial business processing data is cleaned to obtain multiple business processing data.

[0035] Based on the business processing data described above, construct the target data table.

[0036] Secondly, this application also provides a data processing apparatus. The apparatus includes:

[0037] The acquisition module is used to acquire a target data table, which contains multiple business processing data for a target time period;

[0038] The clustering module is used to perform clustering processing on each of the business processing data in the target data table based on the target clustering analysis strategy and for each clustering index, so as to obtain multiple clustering results corresponding to each clustering index.

[0039] The determining module is used to determine the correlation between the clustering results based on the business processing data in each of the clustering results;

[0040] The generation module is used to determine at least one target clustering result group from the clustering results based on the correlation between the clustering results, and to generate data analysis results based on the target clustering result groups.

[0041] In one embodiment, the clustering module is specifically used for:

[0042] For any of the clustering indicators, determine the number of target clustering results corresponding to the clustering indicators in the target data table;

[0043] Based on the target clustering result number and the target clustering analysis algorithm, the business processing data in the target data table are clustered to obtain the target clustering result number of clustering results corresponding to the clustering index.

[0044] In one embodiment, the clustering module is specifically used for:

[0045] For any of the clustering indicators, the number of multiple clustering results corresponding to the clustering indicator is determined based on the number of business processing data corresponding to the clustering indicator.

[0046] Determine the sum of squared errors corresponding to the number of clustering results for each clustering index, and based on the sum of squared errors corresponding to the number of clustering results, determine the difference between the sum of squared errors of samples corresponding to any two adjacent numbers of clustering results;

[0047] Based on the target difference among the differences, the number of target clustering results corresponding to the clustering index is determined, wherein the number of target clustering results is the larger number of clustering results among the two adjacent numbers of clustering results corresponding to the target difference.

[0048] In one embodiment, the determining module is specifically used for:

[0049] Based on the data of each business processing data in the target data table for each clustering result, a transaction table corresponding to the target data table is determined, and the transaction table includes items of each business processing data for each clustering result;

[0050] Based on the items of each clustering result for each of the business processing data in the transaction table, determine the first itemset;

[0051] The second itemset is determined based on the preset minimum support and the first itemset;

[0052] Based on the second itemset, the association rules for each item in the second itemset are determined, and based on the association rules for each item in the second itemset, the association relationships between the clustering results are determined.

[0053] In one embodiment, the determining module is specifically used for:

[0054] During the k-th round of processing, based on the preset minimum support, the k-th frequent itemset that satisfies the preset minimum support is determined in the (k-1)-th frequent itemset. When the number of items in the k-th frequent itemset is less than or equal to the preset number of items, the process proceeds to the (k+1)-th round of processing until the number of items in the m-th frequent itemset is greater than the preset number of items.

[0055] The frequent itemsets in each round of processing are used as the second itemset; where k and m are both positive integers, and when k is 1, the (k-1)th frequent itemset is the first itemset containing one item.

[0056] In one embodiment, the determining module is specifically used for:

[0057] Based on the support of the second itemset and the association rules between each item in the second itemset, determine the confidence level of the association rules between each item in the second itemset;

[0058] In the association rules between the items in the second set, the association rules with a confidence level greater than or equal to the preset minimum confidence level are taken as the target association rules;

[0059] The association relationships between the clustering results are determined based on the clustering results corresponding to the target association rule and the confidence level of the target association rule.

[0060] In one embodiment, the generation module is specifically used for:

[0061] For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are determined as a target clustering result group. The target clustering result is any one of the clustering results.

[0062] In one embodiment, the device further includes:

[0063] The acquisition module is used to acquire multiple initial business processing data.

[0064] The cleaning module is used to clean the initial business processing data to obtain multiple business processing data.

[0065] The construction module is used to build the target data table based on the business processing data described above.

[0066] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the data processing methods described in the first aspect above.

[0067] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the data processing methods described in the first aspect above.

[0068] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the data processing methods described in the first aspect.

[0069] The aforementioned data processing method, apparatus, computer equipment, storage medium, and computer program product acquire a target data table containing multiple business processing data points within a target time period. Based on a target clustering analysis strategy, the business processing data points in the target data table are clustered according to various clustering indicators, resulting in multiple clustering results corresponding to each clustering indicator. The relationships between the clustering results are determined based on the business processing data points within each clustering result. At least one target clustering result group is determined from the clustering results based on these relationships, and data analysis results are generated based on these target clustering result groups. Because it is possible to determine clustering indicators corresponding to multiple business processing data points and perform clustering analysis on each business processing data point based on these indicators to determine the relationships between the clustering results, and based on the data analysis results corresponding to multiple clustering results with existing relationships, it is possible to identify time points or behavioral points where a single employee is responsible for overall business operations, thus mitigating the risk. Attached Figure Description

[0070] Figure 1 This is a flowchart illustrating a data processing method in one embodiment;

[0071] Figure 2 This is a schematic diagram illustrating an example of data representation of clustering metrics in one embodiment;

[0072] Figure 3 This is a flowchart illustrating the process of obtaining the target number of clustering results corresponding to a clustering index in one embodiment.

[0073] Figure 4 This is a flowchart illustrating the process of determining the target number of clustering results corresponding to a clustering index in one embodiment.

[0074] Figure 5 This is a flowchart illustrating the process of determining the second itemset in one embodiment;

[0075] Figure 6 A flowchart illustrating the process of determining the second itemset in another embodiment;

[0076] Figure 7 This is a flowchart illustrating the process of determining the relationships between clustering results in one embodiment;

[0077] Figure 8 This is a flowchart illustrating the process of constructing a target data table in one embodiment;

[0078] Figure 9 This is a flowchart illustrating an example of the data processing method in one embodiment;

[0079] Figure 10This is a structural block diagram of a data processing device in one embodiment;

[0080] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0082] In one embodiment, such as Figure 1 As shown, a data processing method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0083] Step 102: Obtain the target data table.

[0084] The target data table contains multiple business processing data for the target time period. The business processing data is the data generated during the business processing. For example, the business processing data includes staff identification information, business processing time information, staff behavior information, etc. The specific content of the business processing data is not limited in this embodiment.

[0085] In this embodiment of the application, the terminal can query business processing data in the database of the business system and construct a target data table from multiple business processing data within a target time period.

[0086] For example, referring to Table 1, the terminal can collect business processing data from the database of the business system based on a database platform such as a data lake. The business processing data may include staff identification, business processing time information, business processing code, transaction quantity, organization identification, user identification, etc.

[0087] Table 1

[0088]

[0089]

[0090] Step 104: Based on the target clustering analysis strategy, cluster the business processing data in the target data table according to each clustering index to obtain multiple clustering results corresponding to each clustering index.

[0091] Clustering refers to the process of organizing data into different categories. It's important to distinguish clustering from classification. Classification rules are manually defined, while clustering is determined by the inherent characteristics of the data. The three most common clustering methods are: group-based clustering, neural network methods, and statistical methods. The two most fundamental clustering algorithms are k-means clustering and k-medoids clustering (a centroid-based clustering algorithm).

[0092] Among them, the clustering indicators are the indicators in the target data table that need to be clustered, such as the business processing time information and the organization identifier in Table 1.

[0093] In this embodiment, the terminal performs clustering processing on each business processing data in the target data table for each clustering index based on the target clustering analysis strategy, and obtains multiple clustering results corresponding to each clustering index.

[0094] For example, referring to Table 1, the terminal can use a k-means clustering analysis strategy, where the clustering index is the identifier of the institution, to refer to... Figure 2 As shown, the terminal can determine the data table for this clustering metric. The terminal inserts a pivot table into the target data table; selects all business processing data in the cell range, and then selects "Institution Identifier" as the "Row" in the field list and the sum of "Transaction Quantity" as the "Value". The terminal then performs a preliminary analysis of the pivot table to obtain the data table for this clustering metric.

[0095] Based on this clustering index, as shown in Table 2, the terminal performs clustering processing on the business processing data in the target data table to obtain multiple clustering results corresponding to the organization identifier. Each clustering result includes a first-class organization and a second-class organization, with the centroid of the first-class organization being 47 and the centroid of the second-class organization being 504.33.

[0096] Table 2

[0097]

[0098] The terminal can use a k-means clustering analysis strategy. Given that the clustering metric is business processing time information, it can cluster the business processing data in the target data table based on that metric, obtaining multiple clustering results corresponding to the business processing time information. Each clustering result includes a first-order time cluster, a second-order time cluster, and a third-order time cluster. The centroid of the first-order time cluster is 18:20, the centroid of the second-order time cluster is 20:00, and the centroid of the third-order time cluster is 22:30. This process continues, allowing the terminal to determine multiple clustering results corresponding to each clustering metric.

[0099] Specifically, the clustering process is as follows:

[0100] The terminal selects k initial cluster centers from all business processing data. Based on the initial cluster centers, the terminal calculates the similarity (i.e., distance) between the business processing data that are not in the initial cluster centers and the initial cluster centers. Then, based on the similarity between each business processing data and the initial cluster centers, the terminal places each business processing data into its corresponding cluster. Then, based on the mean of all business processing data in the cluster, the terminal determines k new centers. The terminal repeats the above steps until the standard measure function begins to converge, and then stops the clustering process.

[0101] In this application, the standard measurement function can be determined by selecting the root mean square error. The specific method for determining the standard measurement function is not limited in this embodiment. The standard measurement function can be defined using formula (I).

[0102]

[0103] Where E is used to characterize the standard measure function, k is used to characterize the number of clusters (i.e., the number of clustering results), and C is used to characterize the clusters. i Z is used to characterize the i-th cluster. i The mean of the i-th cluster is used to represent the business processing data, and d is used to represent the distance.

[0104] The similarity (i.e., distance) between two business processing data can be determined using Euclidean distance, referring to formula (II). The specific process is as follows:

[0105]

[0106] Where d represents distance, n represents data of business processing data, x represents business processing data x, y represents business processing data y, and i represents the i-th clustering index of business processing data.

[0107] Among them, the clustering analysis strategy can be based on the k-means clustering analysis algorithm, and this application does not specifically limit this.

[0108] Step 106: Determine the correlation between the clustering results based on the business processing data in each clustering result.

[0109] In this embodiment of the application, the terminal determines the association rules between each clustering result based on the business processing data in each clustering result, and then determines the association relationship between each clustering result based on the association rules between each clustering result.

[0110] Among them, the association rules between the clustering results are used to determine the associations between the clustering results. For example, the association rule is: It can characterize the correlation between the first type of institutions and the first type of time. The correlation between the clustering results is used to characterize the relationships between the clustering results.

[0111] Among them, association rules refer to the methods used to find useful relationships between various things in a complex database system that are not immediately apparent. These relationships are then presented as rules, and after scientific organization, the corresponding associations are derived, providing accurate references for decision-making.

[0112] Antecedent, Consequence: Then P is called the antecedent, and Q is called the consequent.

[0113] Itemset (DR): This refers to properties; the number of properties corresponds to the number of itemsets. An itemset is defined as Item1 = {Item1, Item2, ..., Itemm}. A DR is also a collection of events. Furthermore, DR is a set of {0,1} attributes.

[0114] Step 108: Based on the correlation between the clustering results, determine at least one target clustering result group from each clustering result, and generate data analysis results based on each target clustering result group.

[0115] The data analysis results are used to record the clustering results in each target clustering result group, as well as the relationships between the clustering results.

[0116] In this embodiment, the terminal determines multiple clustering results that are related based on the relationships between them, and treats each group of related clustering results as a target clustering result group. The terminal then generates data analysis results based on these multiple target clustering result groups.

[0117] After the terminal generates the data analysis results, technicians can identify nodes with risks where a single employee is responsible for the overall business based on the correlation between the cluster results in each target cluster result group.

[0118] For example, the data analysis results generated by the terminal include a target clustering result group. This target clustering result group contains three clustering results: institutions with large transaction volumes, business processing time of 20:00, and viewing of personal user information. The terminal determines that there is a strong correlation between the clustering results in this target clustering result group. Therefore, the terminal can determine that there is a strong correlation between institutions with large transaction volumes, business processing time of 20:00, and viewing of personal user information. Thus, the terminal determines that the business processing data in the target data table involving institutions with large transaction volumes, business processing time of 20:00, and viewing of personal user information is risky data. Furthermore, it is known to technical personnel that the pattern of a single employee being responsible for the overall business often occurs in institutions with large transaction volumes. Therefore, the business processing data with a strong correlation to institutions with large transaction volumes, business processing time of 20:00, and viewing of personal user information is risky data. Consequently, the terminal can determine that 20:00 is a risky time node, and viewing of personal user information is a risky behavior node.

[0119] The above data processing method involves obtaining a target data table containing multiple business processing data points within a target time period; based on a target clustering analysis strategy, clustering is performed on each business processing data point in the target data table according to various clustering indicators, resulting in multiple clustering results corresponding to each indicator; the relationships between the clustering results are determined based on the business processing data points within each clustering result; at least one target clustering result group is identified from the clustering results based on these relationships; and data analysis results are generated based on these target clustering result groups. Because it is possible to determine clustering indicators corresponding to multiple business processing data points and perform clustering analysis on each business processing data point based on these indicators to determine the relationships between the clustering results, the data analysis results corresponding to multiple clustering results with existing relationships can identify time points or behavioral points where a single employee is responsible for the overall business operations, thus identifying potential risks.

[0120] In one embodiment, such as Figure 3 As shown, step 204 includes:

[0121] Step 302: For any clustering index, determine the number of target clustering results corresponding to the clustering index in the target data table.

[0122] The existing k-means algorithm has fast convergence speed, strong interpretability, and excellent clustering effect, making it widely used. However, the selection of the K value is difficult to determine. In practical applications, the K value is usually predetermined manually, and the algorithm and process are executed to obtain the result. Technicians inputting different K values ​​multiple times yield different clustering results, making it difficult for them to decide on the optimal K value. As K increases, the dispersion within each cluster decreases, and the total sum of squared distances (i.e., the sum of squared errors) E also decreases, but the rate of decrease becomes less significant. In the extreme case where K equals the number of business processing data (i.e., the total amount of data), each cluster contains only one business processing data point, and the total sum of squared errors is 0. Therefore, when selecting the optimal K value, it is necessary to choose the point where the slope of the decreasing trend of the total sum of squared errors becomes no longer obvious as K continues to increase—the "inflection point." The K value corresponding to this point is the optimal number of clusters (i.e., the target number of clustering results).

[0123] If we establish a Cartesian coordinate system with K as the x-axis and the sum of squared distances E corresponding to K as the y-axis, we can obtain a decreasing zigzag line graph. Since it is not a standard curve, it is difficult to form a functional relationship, making it difficult to directly calculate the slope using the derivative formula. Existing methods often only allow us to find the "inflection point" of the curve by drawing a graph. After drawing the graph, we need to identify the "inflection point" by human eye. This method is time-consuming and laborious, and human eye recognition is also prone to misjudgment.

[0124] The number of target clustering results is the number of categories in the optimal cluster corresponding to the clustering index.

[0125] In this embodiment of the application, the terminal determines the number of target clustering results corresponding to any clustering index in the target data table.

[0126] For example, when the clustering metric is business processing time information, and the target clustering strategy is a k-means clustering analysis strategy, the terminal determines multiple K values ​​based on the amount of business processing data corresponding to the clustering metric. Then, for each K value, the terminal determines the sum of squared errors corresponding to each K value among the multiple clustering results corresponding to that K value.

[0127] The terminal establishes a two-dimensional Cartesian coordinate system based on the sum of squared errors corresponding to each K value and each K value itself. Based on the sum of squared errors corresponding to each K value in the two-dimensional Cartesian coordinate system and each K value, the terminal determines the number of target clustering results corresponding to the clustering index.

[0128] Step 304: Based on the target clustering result number and the target clustering analysis algorithm, perform clustering processing on each business processing data in the target data table to obtain the target clustering result number of clustering results corresponding to the clustering index.

[0129] The target clustering analysis algorithm is the k-means clustering analysis algorithm.

[0130] In this embodiment of the application, the terminal performs clustering processing on each business processing data corresponding to each clustering indicator in the target data table according to the number of target clustering results corresponding to each clustering indicator and the target clustering analysis algorithm, so as to obtain the number of target clustering results corresponding to each clustering indicator.

[0131] In this embodiment, for each clustering index, the optimal number of target clustering results corresponding to the clustering index can be determined. The terminal determines multiple clustering analysis results based on the number of target clustering results and the clustering analysis algorithm, which facilitates the subsequent determination of the correlation between the clustering analysis results.

[0132] In one embodiment, such as Figure 4 As shown, step 302 includes:

[0133] Step 402: For any clustering indicator, determine the number of multiple clustering results corresponding to the clustering indicator based on the number of business processing data corresponding to the clustering indicator.

[0134] In this embodiment, the terminal determines the number of different business processing data corresponding to any given clustering index. For example, when the clustering index is the organization identifier, referring to Table 1, the terminal determines that the different business processing data corresponding to the organization identifier are 1, 2, 3, 4, 5, and 6. Therefore, the terminal determines that the number of different business processing data corresponding to this clustering index is 6.

[0135] Then, the terminal determines the number of clustering results corresponding to the clustering index based on the number of business processing data corresponding to the clustering index.

[0136] For example, when the clustering index is the identifier of the organization, the terminal determines that the number of different business processing data corresponding to the clustering index is 6, and then determines that the number of multiple clustering results corresponding to the clustering index is 1, 2, 3, 4, 5 and 6 respectively.

[0137] Step 404: Determine the sum of squared errors corresponding to the number of clustering results for each clustering index, and based on the sum of squared errors corresponding to the number of clustering results, determine the difference between the sum of squared errors of samples corresponding to the number of any two adjacent clustering results.

[0138] Since two points in a Cartesian coordinate system can form a straight line, the slope between the two points is easy to calculate. Assuming the two points are (x1, y1) and (x2, y2), the slope between these two points is t = (y1 - y2) / (x1 - x2). The slope reflects the trend of change. Therefore, the terminal can determine the point of abrupt change in slope based on the slope between two adjacent points in the Cartesian coordinate system.

[0139] Since K is the number of clusters, and the range of K values ​​is an integer greater than 0 and not exceeding the population size, the points on the broken line are (1, E1), (2, E2), (3, E3), (4, E4), etc. When calculating the slope t between two adjacent points, t1 = (E1-E2) / (1-2), t2 = (E2-E3) / (2-3), t3 = (E3-E4) / (3-4), t4 = (E4-E5) / (4-5), with the same denominator of -1. At this time, the magnitude of the slope t is completely determined by the difference of the sum of the squares of the total distances between two adjacent points in the Cartesian coordinate system.

[0140] In a Cartesian coordinate system, the larger the difference f between the squares of the total distances E between two adjacent points, the larger the slope t. The differences f1 = (E1-E2), f2 = (E2-E3), f3 = (E3-E4), f4 = (E4-E5), etc., represent the perpendicular distance f between two adjacent points on the y-axis in the Cartesian coordinate system. The terminal determines the differences in the perpendicular distances f between two adjacent points in the Cartesian coordinate system as g1 = |f1-f2|, g2 = |f2-f3|, g3 = |f3-f4|, g4 = |f4-f5|, etc. The terminal compares g1, g2, g3, and g4, and determines the largest difference, which is the starting point of the mutation. The terminal uses the K value corresponding to the point with the largest difference as the target clustering result number.

[0141] In this embodiment, the terminal determines the sum of squared errors corresponding to each clustering result number based on the number of clustering results corresponding to the clustering index, and subtracts the sum of squared errors of samples corresponding to each pair of adjacent clustering result numbers based on the sum of squared errors of each clustering result number, thereby determining the difference between the sum of squared errors of samples corresponding to any two adjacent clustering result numbers.

[0142] Exemplarily, when the number of clustering results corresponding to the clustering metric is 3, the terminal respectively determines the sum of squared errors corresponding to the number of clustering results being 1, the sum of squared errors corresponding to the number of clustering results being 2, and the sum of squared errors corresponding to the number of clustering results being 3. The terminal performs a subtraction operation on the sum of squared errors corresponding to the number of clustering results being 1 and the sum of squared errors corresponding to the number of clustering results being 2 based on each sum of squared errors to obtain a first difference; and performs a subtraction operation on the sum of squared errors corresponding to the number of clustering results being 2 and the sum of squared errors corresponding to the number of clustering results being 3 to obtain a second difference.

[0143] Step 406: Determine the target number of clustering results corresponding to the clustering metric according to the target difference among the differences.

[0144] Among them, the target number of clustering results is the larger number of clustering results among two adjacent numbers of clustering results corresponding to the target difference. The target number of clustering results is used to represent the most optimal number of clustering results corresponding to the clustering metric.

[0145] In the embodiments of the present application, the terminal sorts multiple differences in descending order to obtain the largest difference among the multiple differences, and uses this largest difference as the target difference. The terminal uses the larger number of clustering results among the two numbers of clustering results corresponding to the target difference as the target number of clustering results corresponding to the clustering metric according to the target difference among the differences.

[0146] Among them, the target clustering analysis strategy may include the following:

[0147] Define the data dimension NA according to the actual data to be analyzed. If it is one-dimensional data, it is defined as 1. If it is two-dimensional data, it is defined as 2, and so on; define the population size Psize according to the actual data to be analyzed. For example, if there are 600 numbers in one-dimensional data that need to be clustered and analyzed, the population size is 600; define the maximum number of iterations T according to the user's needs. If multiple iterations are required, increase the number of iterations; define the end condition ED according to the user's needs. If higher precision is required, increase the number of decimal places; define the function n_data(), n > 1. Through the n++ loop until n < Psize, for each K, the following process is executed in sequence:

[0148] Define the cluster center clu_cent[K], the cluster array int cluster[K][Psize], the number of a set of data in the cluster cluster_num[K], the sample fitness value fitness for judging the end condition, the fitness value old_fitness of the previous iteration, and the sum of squared errors Je of all samples.

[0149] Define the function input_data() to import data from an external file. For example, if the data storage file is test.data, use IF == NULL to check if there is no data file and prompt an error. Otherwise, read the data content all_data[i].p[j] through the loops i < Psize and j < NA;

[0150] Define the function Is_equal(a[], b, c) to check if the data is equal. If equal, return 1. Loop i < b; i++, and return 1 when a[i] == c;

[0151] Define the function Init_center(). Randomly generate three numbers between 0 and Psize. If there are duplicate random numbers, skip and continue. num = 0, and when num < K, num++. Use the Is_equal(rand_num_tmp, num, rand_num) function to loop and judge the result of rand_num = rand() % Psize to obtain the randomly initialized clustering centroids all_data[rand_num_tmp[i]].p[j], where i < K and j < NA;

[0152] Define the Euclid(x, y) Euclidean distance formula function to calculate the Euclidean distance from a set of data to the corresponding cluster center. The value of the sample to the cluster center is its Euclidean distance. i < NA, calculate distance += pow((all_data[x].p[i] - pop.clu_cent[y].p[i]), 2) through the loop i++. Then use the square root formula sqrt(distance) to obtain the Euclidean distance;

[0153] Define the function calculate_distance() to calculate the Euclidean distance from Psize groups of data to K centroids. i < Psize, j < K, loop i++ and j++, and use the Euclid(i, j) function to calculate the Euclidean distance all_data[i].distance[j] from Psize groups of data to K centroids;

[0154] Define the function Make_new_cluster() to generate new clusters, cluster the data, initialize the loop where the number i is less than the number of clusters K, loop where it is less than the population size Psize, j < K, loop j++, start looping from all_data[i].distance[0] until the minimum value min of all_data[i].distance[j] is found. At this time, assign the value of j to index, divide the cluster by pop.cluster[index][pop.cluster_num[index]++] = i, then i < K, j < pop.cluster_num[i], loop i++ and j++, calculate the sum of squared errors of the samples through the Euclidean distance of all samples and pop.Je += pow(all_data[pop.cluster[i][j]].distance[i], 2). pop.fitness gets the fitness value pop.old_fitness of the previous iteration, and the sum of squared errors of all samples pop.Je is the fitness value pop.fitness;

[0155] Define the function Make_new_center() to update the cluster centers. i < K, j < NA, loop i++ and j++, m < pop.cluster_num[i], loop m++, calculate the sum of all data of the j-th dimension of the i-th cluster tmp_sum += all_data[pop.cluster[i][m]].p[j], and take the average to get the new cluster center pop.clu_cent[i].p[j] = tmp_sum / pop.cluster_num[i];

[0156] Define the main function main(). (i < T) && (differ > ED), loop i++, call calculate_distance() to calculate the Euclidean distance in sequence, call Make_new_cluster() to generate new clusters, and call Make_new_center() to generate new centroids for the new clusters;

[0157] Define the function output1_info() to display the result output information. n > 1, loop through n++ until n < Psize. For each K = n, output the initial centroids, final centroids, and clusters corresponding to each K in sequence.

[0158] In the above process, each K generates the corresponding squared error and Je for all samples. Using the loop statement k++, f[k] = Je[k] - Je[k+1] is calculated. The absolute value formula is used to calculate g[k] = abs(f[k] - f[k+1]). The maximum value formula max(g[k]) is used to obtain the maximum g[k]. The most suitable K is output as (k+1). At the same time, the initial centroid, final centroid, and cluster class corresponding to K = k+1 are output.

[0159] In this embodiment, clustering can be performed by enumerating K values ​​using a iterative algorithm, obtaining the clustering result corresponding to each K value. By calculating the sum of squared errors between clusters with different K values, the algorithm replaces the plotting method to determine the mutation start point corresponding to the sum of squared errors between clusters, thereby selecting the optimal K value and clustering result. This facilitates subsequent clustering processing based on the target number of clustering results, resulting in more accurate clustering results. Furthermore, it improves the efficiency of clustering analysis while saving human resources. This embodiment is also applicable to clustering for other data analyses.

[0160] In one embodiment, such as Figure 5 As shown, step 206 includes:

[0161] Step 502: Based on the data of each business processing data in the target data table for each clustering result, determine the transaction table corresponding to the target data table.

[0162] The transaction table includes items for each business processing data for each clustering result, and the items for the clustering results may include clustering result identifiers.

[0163] In this embodiment of the application, the terminal maps each clustering result in the target data table to a clustering result identifier. Then, the terminal obtains the data of each business processing data in the target data table for each clustering result identifier based on each clustering result identifier.

[0164] Table 3

[0165]

[0166] For example, referring to Table 3, when the target data table is Table 3, the clustering results corresponding to the organization identifier are the first type of organization and the second type of organization; the clustering results corresponding to the staff behavior information are the first type of behavior (querying card information), the second type of behavior (verifying user identity information) and the third type of behavior (personal user information); the clustering results corresponding to the business processing time are the first type of time (18:00), the second type of time (19:00) and the third type of time (20:00); the clustering results corresponding to the business processing quarter are the first type of quarter (first and second quarters) and the second type of quarter (third and fourth quarters).

[0167] The terminal determines the cluster result identifier for each cluster result based on the clustering results corresponding to each clustering index in the target data table. Specifically, the terminal sets the cluster result identifier for the first type of organization to K1, the cluster result identifier for the second type of organization to K2; the cluster result identifier for the first type of behavior to K3, the cluster result identifier for the second type of behavior to K4, and the cluster result identifier for the third type of behavior to K5; the cluster result identifier for the first type of time to K6, the cluster result identifier for the second type of time to K7, and the cluster result identifier for the third type of time to K8; and the cluster result identifier for the first type of quarter to K9 and the cluster result identifier for the second type of quarter to K10.

[0168] Based on the clustering result identifiers of each clustering result, the terminal maps the business processing data in the target data table to Boolean values ​​to obtain the data logical table corresponding to the target data table, as shown in Table 4.

[0169] Table 4

[0170]

[0171] The terminal determines the transaction table corresponding to the target data table based on the data logical table corresponding to the target data table and the identifiers of each clustering result. The transaction table corresponding to the target data table can be referred to in Table 5.

[0172] Table 5

[0173]

[0174] Step 504: Determine the first itemset based on the items of each business processing data for each clustering result in the transaction table.

[0175] In this embodiment of the application, the terminal determines multiple first itemsets based on the items of each clustering result in each business processing data in the transaction table, wherein each itemset contains at least one item (i.e., clustering result identifier).

[0176] Step 506: Determine the second itemset based on the preset minimum support and the first itemset.

[0177] In this embodiment of the application, the terminal may store a preset minimum support, and the terminal determines the support of each first itemset based on the number of times each first itemset appears in the transaction table.

[0178] Here, support, denoted as sup, is the number of supports for a rule's antecedent (or consequent) divided by the number of records. In other words, it's the number of records to consider. The sup count is the number of occurrences of each attribute. Let the support of P be sup(P). support n is the number of business processing data in the target data table. This indicates that P and Q appear simultaneously in the same record.

[0179] Then, the terminal can determine the first itemset with a support greater than or equal to the preset minimum support from multiple first itemsets, and use the first itemsets with a support greater than or equal to the preset minimum support from multiple first itemsets as the second itemsets.

[0180] The preset minimum support can be set to 20%. In practical applications, the preset minimum support can also be set by technicians according to the actual situation. This application embodiment does not make a specific limitation on the preset minimum support.

[0181] Step 508: Based on the second itemset, determine the association rules of each item in the second itemset, and based on the association rules of each item in the second itemset, determine the association relationship between each clustering result.

[0182] In this embodiment of the application, the terminal determines the association rules of each item in the second itemset based on the second itemset, and determines the association relationship between the clustering results corresponding to each item in the second itemset based on the association rules of each item in the second itemset.

[0183] In this embodiment, the terminal can determine the second itemset based on each first itemset and the preset minimum support, and determine the association between each clustering result based on each item in the second itemset. This facilitates the subsequent determination of the time point or behavior point where there is a risk of a single staff member being responsible for the overall business based on the association between each clustering result.

[0184] In one embodiment, such as Figure 6 As shown, step 506 includes:

[0185] Step 602: During the k-th round of processing, based on the preset minimum support, determine the k-th frequent itemset that satisfies the preset minimum support in the (k-1)-th frequent itemset. When the number of items in the k-th frequent itemset is less than or equal to the preset number of items, proceed to the k+1-th round of processing until the number of items in the m-th frequent itemset is greater than the preset number of items.

[0186] Frequent itemsets are those itemsets in the set that have a support greater than a preset minimum.

[0187] In this embodiment of the application, the terminal performs k rounds of processing on each item in the transaction table corresponding to the target data table according to a preset minimum support, and determines multiple first itemsets.

[0188] Specifically, in the first round of processing, the terminal identifies a first frequent itemset that satisfies a preset minimum support within the first itemset, and determines that the number of items in the first frequent itemset is lower than a preset number of items, then proceeds to the second round of processing. In the second round of processing, the terminal identifies a second frequent itemset that satisfies a preset minimum support within the first frequent itemset, and determines that the number of items in the second frequent itemset is lower than a preset number of items, then proceeds to the third round of processing, until the number of items in the m-th frequent itemset exceeds the preset number of items.

[0189] For example, the first itemset is an itemset that contains at least one item. There can be multiple first itemsets, and the first itemsets containing one item can be referred to Table 6.

[0190] Table 6

[0191]

[0192] Then, referring to Table 7, the terminal selects the first itemset with a support higher than or equal to the preset minimum support from among the multiple first itemsets containing one item, based on the support of the first itemsets.

[0193] Table 7

[0194]

[0195] Referring to Table 8, the terminal determines the first itemset containing two items based on the first frequent itemset.

[0196] Table 8

[0197]

[0198] Referring to Table 9, the terminal selects the first itemset with a support higher than or equal to a preset minimum support as the second frequent itemset based on the support of multiple first itemsets containing two items.

[0199] Table 9

[0200]

[0201] Then, referring to Table 10, the terminal determines the first itemset containing three items based on the second frequent itemset.

[0202] Table 10

[0203]

[0204] Then, referring to Table 11, the terminal selects the first itemset with a support higher than or equal to the preset minimum support as the third frequent itemset based on the support of multiple first itemsets containing three items.

[0205] Table 11

[0206]

[0207]

[0208] Until the terminal determines the first itemset containing four items, as shown in Table 12, since the number of items in each first itemset is 4, the terminal determines that the number of items in the fourth frequent itemset is higher than the preset number of items, and ends the k-round processing process.

[0209] Table 12

[0210]

[0211] Step 604: Use the frequent itemsets from each round of processing as the second itemset.

[0212] Where k and m are both positive integers, and when k is 1, the (k-1)th frequent itemset is the first itemset containing one item.

[0213] In this embodiment, the terminal uses each frequent itemset in each round of processing as a second itemset. Specifically, the terminal uses the first frequent itemset, the second frequent itemset, and the third frequent itemset as a second itemset containing one item, a second itemset containing two items, and a second itemset containing three items, respectively.

[0214] In this embodiment, the terminal can determine the second itemset based on each first itemset and a preset minimum support, which facilitates the subsequent determination of the association rules of each cluster analysis result corresponding to each item based on the second itemset, and thus the determination of the association relationship between each cluster analysis result.

[0215] In one embodiment, such as Figure 7 As shown, step 508 includes:

[0216] Step 702: Determine the confidence of the association rules between items in the second itemset based on the support of the second itemset and the association rules between items in each second itemset.

[0217] Regarding the method for determining the confidence level: the association rule can be... Association rules have credibility. This indicates the percentage of transactions in the transaction table that contain both event P and event Q. This represents the percentage of the support sup(P∪Q) of P∪Q to the support sup(P) of its predecessor P. For example... The confidence level is calculated as follows: 2 / 3 (i.e., the support of {K1, K3, K7} (2 / 9) / the support count of {K1, K3} (3 / 9)) = 66.7%. Here, "sup" represents support and "confidence" represents confidence.

[0218] In this embodiment of the application, the terminal can determine the association rules between the items based on the items in each of the second itemsets. Specifically, when the second itemset is {K1, K3, K7}, the association rules corresponding to the second itemset can be found in Table 13.

[0219] Table 13

[0220]

[0221] When the second itemset is {K1, K5, K8}, the association rules corresponding to the second itemset can be found in Table 14.

[0222] Table 14

[0223]

[0224] When the second itemset is {K1, K5, K9}, the association rules corresponding to the second itemset can be found in Table 15.

[0225] Table 15

[0226]

[0227] When the second itemset is {K1, K8, K9}, the association rules corresponding to the second itemset can be found in Table 16.

[0228] Table 16

[0229]

[0230] When the second itemset is {K5, K8, K9}, the association rules corresponding to the second itemset can be found in Table 17.

[0231] Table 17

[0232]

[0233] When the second itemset is the second most frequent itemset, the association rules corresponding to the second itemset can be found in Table 18.

[0234] Table 18

[0235]

[0236] In the case where the second itemset is a second itemset containing four items, the association rules corresponding to the second itemset can be found in Table 19.

[0237] Table 19

[0238]

[0239]

[0240] Then, the terminal determines the confidence level of the association rules between items in each second item set based on the support of each second item set and the association rules between items in each second item set.

[0241] Step 704: In the association rules between the items in the second itemset, the association rules with a confidence level greater than or equal to the preset minimum confidence level are taken as the target association rules.

[0242] In this embodiment of the application, the terminal may store a preset minimum confidence level. Based on the confidence level of the association rules between the items in each second item set, the terminal selects the association rules with a confidence level greater than the preset minimum confidence level as the target association rules.

[0243] For example, with a preset minimum confidence level of 80%, the target association rule corresponding to the second itemset including two items can be referred to Table 20.

[0244] Table 20

[0245]

[0246] The target association rules corresponding to the second itemset, which includes three items, can be found in Table 21.

[0247] Table 21

[0248]

[0249] The target association rules corresponding to the second itemset, which includes four items, can be found in Table 22.

[0250] Table 22

[0251]

[0252] The preset minimum confidence level can be 80%. The preset minimum confidence level can also be set by technicians according to the actual situation in the application. This application embodiment does not make specific limitations on this.

[0253] Step 706: Determine the association relationship between the clustering results based on the clustering results corresponding to the target association rule and the confidence level of the target association rule.

[0254] In this embodiment, the terminal determines the clustering result corresponding to each item in multiple target association rules. Then, the terminal determines the association relationship between the clustering results based on the clustering results corresponding to the multiple target association rules and the confidence level of the target association rules.

[0255] Optionally, the terminal can determine the strength of the association between items in the target association rule based on the confidence level corresponding to the target association rule. For example, when the confidence level is greater than the preset minimum confidence level of 80%, it indicates that there is an association between items in the target association rule; when the confidence level is greater than the preset average confidence level of 90%, it indicates that there is a strong association between items in the target association rule.

[0256] For example, the terminal determines the association rules between the clustering results as the first type of behavior. The second type of institution (i.e., those that query card information) (Large transaction volume institutions); Third type of time The third type of behavior (i.e., business processing time is 20:00) Personal user information); Category III behavior The third type of time (i.e., personal user information) Business processing time is 8 PM; Category II institutions and Category II time Category 1 behavior (i.e., large-volume institutions with business processing time at 19:00) (Query card information); Type I behavior and Type II time The second category of institutions (i.e., those inquiring about card information and whose business processing time is 19:00) Large-volume trading institutions); second-class institutions and third-class behaviors The third type of time (i.e., large-volume institutional and individual user information) Business processing time is 8 PM; for Category II institutions and Category III time... The third type of behavior (i.e., institutions with large transaction volumes and business processing time of 20:00) Personal user information); Category 3 time and Category 1 quarter Category II institutions (i.e., those whose business hours are 8 PM and whose business quarters are the first and second quarters) (Large transaction volume institutions); Third category time and first category quarter) The third type of behavior (i.e., business processing time is 20:00 and business processing quarter is 1 or 2) (Personal user information). Third type of behavior and third type of time and first type of quarter. The second category of organizations (i.e., personal user information, business processing time is 20:00, and business processing quarters are 1 and 2) Large transaction volume institutions); second category institutions and third category time and first category quarter The third category of behavior (i.e., institutions with large transaction volumes, business processing time of 20:00, and business processing quarters of 1 and 2) Personal user information); Category 3 behavior and Category 2 organization and Category 1 quarterly The third category of time (i.e., personal user information, large transaction volume, and business processing quarters of 1 and 2) Business processing time is 8 PM; Category III behavior and Category III time and Category II institution The first category (i.e., personal user information, business processing time is 20:00, and large transaction volume institutions) Business processing quarters are Q1 and Q2; the third time period is the first quarter. Category II institutions and Category III activities (business processing time is 20:00 and business processing quarters are 1 and 2) (Information on large-volume institutional and individual users).

[0257] The terminal determines the association between clustering results based on the association rules and confidence levels among them.

[0258] For example, the terminal can determine the correlation between various clustering results, such as: the first type of behavior (i.e., querying card information) occurs in the second type of institution (i.e., institutions with a large number of transactions); the third type of time (i.e., the business processing time is 20:00) is strongly associated with the occurrence of the second type of behavior (i.e., user identity information verification); and the first type of behavior (i.e., querying card information) occurs more frequently after the business processing time is 19:00 in the second type of institution (i.e., institutions with a large number of transactions).

[0259] In this embodiment, the terminal can determine the association relationships between clustering results based on the target association rule corresponding to the second itemset and the confidence level corresponding to the target association rule. This facilitates subsequent identification of similar organizations or personnel based on the association relationships between clustering results, obtaining potential associations among various factors through association rules, mining information to find risk points, and providing a scientific basis for decision-making in the next step of preventing potential risks. This allows for better implementation of risk avoidance measures, effective prevention of operational risks, enhanced security of the business system, improved user experience during business processing, and ensures the smooth operation of the business system.

[0260] In one embodiment, step 208 includes:

[0261] For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are identified as a target clustering result group.

[0262] The target clustering result is any clustering result among all the clustering results.

[0263] In this embodiment, the terminal selects any one clustering result from the various clustering results as the target clustering result. Then, for the target clustering result, the terminal determines the target clustering result and the clustering results that are related to the target clustering result as a target clustering result group.

[0264] In this embodiment, the terminal can identify target cluster result groups that are correlated with each other among multiple clustering results. This facilitates the generation of data analysis results based on the target cluster result groups. This allows technicians to determine the time points or behavioral points where a single employee is responsible for the overall business operations, thus mitigating the risk.

[0265] In one embodiment, such as Figure 8 As shown, before step 202, the following steps are also included:

[0266] Step 802: Obtain multiple initial business processing data.

[0267] In this embodiment, the terminal can collect initial business processing data from the database of the business system based on a database platform such as a data lake. This initial business processing data can be time-series data, multimedia data, Web (World Wide Web) data, spatial data, etc. Each piece of initial business processing data may include staff identification, business processing time information, business processing code, transaction quantity, organization identification, user identification, etc.

[0268] Step 804: Perform data cleaning on each initial business processing data to obtain multiple business processing data.

[0269] In this embodiment, the terminal performs data cleaning on the initial business processing data. For example, the terminal converts multiple initial business processing data sets into an Excel spreadsheet to obtain an initial data table. The terminal then sequentially iterates through the columns in the initial data table, including staff identification, business processing time information, business processing code, transaction quantity, institution identification, and user identification, checking for empty data in any column. If an empty column is found, the terminal deletes the corresponding initial business processing data from the initial data table.

[0270] Optionally, the terminal can further filter each initial business processing data according to preset data requirement rules, and delete the initial business processing data that does not meet the preset data requirement rules.

[0271] The preset data requirement rules can be set by technical personnel in actual applications, and this application does not impose specific restrictions on them.

[0272] Optionally, the terminal may pre-store an incomplete data table. Before deleting the initial business processing data corresponding to the empty data from the initial data table, the initial business processing data corresponding to the empty data is stored in the incomplete data table, which facilitates subsequent data review and verification based on the incomplete data table.

[0273] After cleaning the initial data table, the terminal can perform data preprocessing. Specifically, the terminal can extract the business processing time column from the initial data table to obtain the year, month, day, and hour data columns corresponding to each initial business processing data. The terminal can then filter the month data column to obtain the quarter data column corresponding to each initial business processing data. The terminal can also pre-store a mapping table between transaction codes and transaction lists. Based on this mapping table, the terminal can match the transaction codes in the initial data table to determine the transaction name for each initial business processing data.

[0274] For example, the terminal extracts the business processing time column according to the formula MID(text,start_num,num_chars), obtains the year, month, day, and hour data columns corresponding to each initial business processing data, filters the month data column to obtain the quarter data column corresponding to each initial business processing data; the terminal matches the transaction codes in the initial data table according to the formula VLOOKUP(lookup_value,table_array,col_index_num,range_lookup) and the mapping table between transaction codes and transaction lists to determine the transaction name of each initial business processing data.

[0275] After the terminal completes data cleaning and preprocessing of each initial business processing data, it uses each initial business processing data as business processing data.

[0276] Step 806: Construct the target data table based on the data from each business process.

[0277] In this embodiment of the application, the terminal constructs the business processing data into a target data table.

[0278] In this embodiment, the initial data table can be cleaned and preprocessed to obtain a target data table that is relevant to the requirements, thereby improving the accuracy and efficiency of subsequent data clustering analysis.

[0279] In one embodiment, such as Figure 9 As shown, an example of a data processing method is also provided, the specific content of which includes:

[0280] Step A1: Connect to the business system database through platforms such as data lakes.

[0281] Step A2: Collect data from the business system database to obtain an initial data table.

[0282] Step A3 involves cleaning and preprocessing the initial data table to obtain the target data table.

[0283] Step A4 involves performing clustering and correlation analysis on the business processing data in the target data table to obtain the data analysis results.

[0284] Step A5: Identify risk factors based on the data analysis results.

[0285] Step A6: Determine the basis for risk prevention.

[0286] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0287] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0288] In one embodiment, such as Figure 10 As shown, a data processing device 1000 is provided, including: an acquisition module 1002, a clustering module 1004, a determination module 1006, and a generation module 1008, wherein:

[0289] The acquisition module 1002 is used to acquire a target data table, which contains multiple business processing data for a target time period.

[0290] The clustering module 1004 is used to perform clustering processing on each of the business processing data in the target data table based on the target clustering analysis strategy and for each clustering index, so as to obtain multiple clustering results corresponding to each clustering index.

[0291] The determining module 1006 is used to determine the correlation between the clustering results based on the business processing data in each of the clustering results.

[0292] The generation module 1008 is used to determine at least one target clustering result group from the clustering results based on the correlation between the clustering results, and to generate data analysis results based on the target clustering result groups.

[0293] The data processing apparatus provided in this disclosure can determine clustering indicators corresponding to multiple business processing data based on these data, and perform clustering analysis on each business processing data according to these indicators to determine the correlation between the clustering results. Based on the data analysis results corresponding to multiple clustering results with correlation, it is possible to determine the time points or behavioral points where there is a risk of a single employee being responsible for the overall business.

[0294] In one embodiment, the clustering module 1004 is specifically used for:

[0295] For any of the clustering indicators, determine the number of target clustering results corresponding to the clustering indicators in the target data table;

[0296] Based on the target clustering result number and the target clustering analysis algorithm, the business processing data in the target data table are clustered to obtain the target clustering result number of clustering results corresponding to the clustering index.

[0297] In one embodiment, the clustering module 1004 is specifically used for:

[0298] For any of the clustering indicators, the number of multiple clustering results corresponding to the clustering indicator is determined based on the number of business processing data corresponding to the clustering indicator.

[0299] Determine the sum of squared errors corresponding to the number of clustering results for each clustering index, and based on the sum of squared errors corresponding to the number of clustering results, determine the difference between the sum of squared errors of samples corresponding to any two adjacent numbers of clustering results;

[0300] Based on the target difference among the differences, the number of target clustering results corresponding to the clustering index is determined, wherein the number of target clustering results is the larger number of clustering results among the two adjacent numbers of clustering results corresponding to the target difference.

[0301] In one embodiment, the determining module 1006 is specifically used for:

[0302] Based on the data of each business processing data in the target data table for each clustering result, a transaction table corresponding to the target data table is determined, and the transaction table includes items of each business processing data for each clustering result;

[0303] Based on the items of each clustering result for each of the business processing data in the transaction table, determine the first itemset;

[0304] The second itemset is determined based on the preset minimum support and the first itemset;

[0305] Based on the second itemset, the association rules for each item in the second itemset are determined, and based on the association rules for each item in the second itemset, the association relationships between the clustering results are determined.

[0306] In one embodiment, the determining module 1006 is specifically used for:

[0307] During the k-th round of processing, based on the preset minimum support, the k-th frequent itemset that satisfies the preset minimum support is determined in the (k-1)-th frequent itemset. When the number of items in the k-th frequent itemset is less than or equal to the preset number of items, the process proceeds to the (k+1)-th round of processing until the number of items in the m-th frequent itemset is greater than the preset number of items.

[0308] The frequent itemsets in each round of processing are used as the second itemset; where k and m are both positive integers, and when k is 1, the (k-1)th frequent itemset is the first itemset containing one item.

[0309] In one embodiment, the determining module 1006 is specifically used for:

[0310] Based on the support of the second itemset and the association rules between each item in the second itemset, determine the confidence level of the association rules between each item in the second itemset;

[0311] In the association rules between the items in the second set, the association rules with a confidence level greater than or equal to the preset minimum confidence level are taken as the target association rules;

[0312] The association relationships between the clustering results are determined based on the clustering results corresponding to the target association rule and the confidence level of the target association rule.

[0313] In one embodiment, the generation module 1008 is specifically used for:

[0314] For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are determined as a target clustering result group. The target clustering result is any one of the clustering results.

[0315] In one embodiment, the apparatus further includes:

[0316] The acquisition module is used to acquire multiple initial business processing data.

[0317] The cleaning module is used to clean the initial business processing data to obtain multiple business processing data.

[0318] The construction module is used to build the target data table based on the business processing data described above.

[0319] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0320] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0321] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0322] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0323] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0324] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0325] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0326] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0327] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0328] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the target data table, which contains multiple business processing data for a target time period. The business processing data includes staff identification, business processing time information, business processing code, transaction quantity, institution identification, and user identification. Based on the target clustering analysis strategy, the business processing data in the target data table are clustered according to each clustering index to obtain multiple clustering results corresponding to each clustering index. Based on the business processing data in each of the clustering results, determine the correlation between the clustering results; Based on the correlation between the clustering results, at least one target clustering result group is determined from the clustering results, and data analysis results are generated based on the target clustering result groups. The step of determining the correlation between the clustering results based on the business processing data in each clustering result includes: Based on the data of each business processing data in the target data table for each clustering result, a transaction table corresponding to the target data table is determined, and the transaction table includes items of each business processing data for each clustering result; Based on the items of each clustering result for each of the business processing data in the transaction table, determine the first itemset; The second itemset is determined based on the preset minimum support and the first itemset; Based on the second itemset, determine the association rules for each item in the second itemset, and based on the association rules for each item in the second itemset, determine the association relationships between each clustering result; The step of determining at least one target clustering result group from the clustering results based on the correlation between the clustering results includes: For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are determined as a target clustering result group. The target clustering result is any one of the clustering results.

2. The method according to claim 1, characterized in that, The target clustering analysis strategy involves clustering the business processing data in the target data table according to each clustering index, resulting in multiple clustering results corresponding to each clustering index, including: For any of the clustering indicators, determine the number of target clustering results corresponding to the clustering indicators in the target data table; Based on the target clustering result number and the target clustering analysis algorithm, the business processing data in the target data table are clustered to obtain the target clustering result number of clustering results corresponding to the clustering index.

3. The method according to claim 2, characterized in that, Determining the number of target clustering results corresponding to any of the clustering indicators in the target data table includes: For any of the clustering indicators, the number of multiple clustering results corresponding to the clustering indicator is determined based on the number of business processing data corresponding to the clustering indicator. Determine the sum of squared errors corresponding to the number of clustering results for each clustering index, and based on the sum of squared errors corresponding to the number of clustering results, determine the difference between the sum of squared errors of samples corresponding to any two adjacent numbers of clustering results; Based on the target difference among the differences, the number of target clustering results corresponding to the clustering index is determined, wherein the number of target clustering results is the larger number of clustering results among the two adjacent numbers of clustering results corresponding to the target difference.

4. The method according to claim 1, characterized in that, The step of determining the second itemset based on the preset minimum support and the first itemset includes: During the k-th round of processing, based on the preset minimum support, the k-th frequent itemset that satisfies the preset minimum support is determined in the (k-1)-th frequent itemset. When the number of items in the k-th frequent itemset is less than or equal to the preset number of items, the process proceeds to the (k+1)-th round of processing until the number of items in the m-th frequent itemset is greater than the preset number of items. The frequent itemsets in each round of processing are used as the second itemset; where k and m are both positive integers, and when k is 1, the (k-1)th frequent itemset is the first itemset containing one item.

5. The method according to claim 4, characterized in that, The determination of the association relationships between the clustering results based on the association rules of each item in the second itemset includes: Based on the support of the second itemset and the association rules between each item in the second itemset, determine the confidence level of the association rules between each item in the second itemset; In the association rules between the items in the second set, the association rules with a confidence level greater than or equal to the preset minimum confidence level are taken as the target association rules; The association relationships between the clustering results are determined based on the clustering results corresponding to the target association rule and the confidence level of the target association rule.

6. The method according to claim 1, characterized in that, Before obtaining the target data table, the process also includes: Acquire multiple initial business processing data; The initial business processing data is cleaned to obtain multiple business processing data. Based on the business processing data described above, construct the target data table.

7. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire a target data table, which contains multiple business processing data for a target time period. The business processing data includes staff identification, business processing time information, business processing code, transaction quantity, institution identification, and user identification. The clustering module is used to perform clustering processing on each of the business processing data in the target data table based on the target clustering analysis strategy and for each clustering index, so as to obtain multiple clustering results corresponding to each clustering index. The determining module is used to determine the correlation between the clustering results based on the business processing data in each of the clustering results; The generation module is used to determine at least one target clustering result group from the clustering results based on the correlation between the clustering results, and to generate data analysis results based on the target clustering result groups. The determining module is specifically used for: Based on the data of each business processing data in the target data table for each clustering result, a transaction table corresponding to the target data table is determined, and the transaction table includes items of each business processing data for each clustering result; Based on the items of each clustering result for each of the business processing data in the transaction table, determine the first itemset; The second itemset is determined based on the preset minimum support and the first itemset; Based on the second itemset, determine the association rules for each item in the second itemset, and based on the association rules for each item in the second itemset, determine the association relationships between each clustering result; Specifically, the generation module is used for: For each clustering result, the target clustering result and the clustering results that are related to the target clustering result are determined as a target clustering result group. The target clustering result is any one of the clustering results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images