A data mining method and device for a multi-source heterogeneous service database

Abstract

The present disclosure relates to the field of data processing, and provides a data mining method and device for a multi-source heterogeneous business database. The method comprises: determining a second service end; when receiving confirmation information fed back by the second service end, sending a data aggregation strategy to the second service end; using the data aggregation strategy to cluster local business data and obtaining a first clustering center; receiving a second clustering center obtained by the second service end based on the data aggregation strategy to cluster local data; aggregating the first clustering center and the second clustering center and obtaining an aggregation center; re-clustering the local business data according to the aggregation center until a preset iteration number or the aggregation center no longer changes, and obtaining a mining data set. The present disclosure can mine similar business data in each resource party (service end) on the premise of protecting the data security of each resource party, and alleviate the problem that the performance of the model obtained by joint training is poor due to data heterogeneity.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a data mining method and device for a multi-source heterogeneous business database. BACKGROUND

[0002] For a multi-system business platform composed of multiple business subsystems, or different resource parties in the same industry, their local business data is usually distributedly stored in their respective local databases, that is, the data sources are different. The data of different data sources usually has the problem of data heterogeneity, thereby forming the parallel situation of multi-source heterogeneous databases.

[0003] Through joint learning training, an effective way to obtain a business model with better performance is obtained. However, in order to protect the security of their own data, each resource party participating in joint learning usually does not want to share their own data. In addition, the data of different data sources itself has the problem of data heterogeneity. If each resource party directly uses its own local data to participate in joint learning training, the model established cannot well fit the data of each resource party, and the performance of the business model obtained is also not satisfactory.

[0004] Therefore, it is urgent to provide a method for mining similar business data in each resource party under the premise of protecting the security of the data of each resource party, and alleviating the problem of poor performance of the model obtained by joint training due to data heterogeneity. SUMMARY

[0005] Therefore, the embodiments of the present disclosure provide a data mining method and device for a multi-source heterogeneous business database, to mine similar business data in each resource party under the premise of protecting the security of the data of each resource party, and alleviate the problem of poor performance of the model obtained by joint training due to data heterogeneity.

[0006] In a first aspect, the embodiments of the present disclosure provide a data mining method for a multi-source heterogeneous business database, comprising:

[0007] determining a second business end;

[0008] determining a second business end;

[0009] when receiving the confirmation information fed back by the second business end, sending a data aggregation strategy to the second business end;

[0010] using the data aggregation strategy to cluster the local business data and obtain a first clustering center;

[0011] receiving a second clustering center obtained by the second business end based on the data aggregation strategy to cluster the local data;

[0012] aggregating the first clustering center and the second clustering center and obtaining an aggregated center;

[0013] re-cluster the local service data according to the aggregated center until a preset iteration number or the aggregated center no longer changes, to obtain a mining data set.

[0014] In a second aspect, the disclosure provides a data mining device for a multi-source heterogeneous service database, comprising:

[0015] The determining module is configured to determine the second service end.

[0016] The sending module is configured to send the data aggregation strategy to the second service end when receiving the confirmation information fed back by the second service end.

[0017] The clustering module is configured to cluster the local service data using the data aggregation strategy and obtain a first clustering center.

[0018] The receiving module is configured to receive a second clustering center obtained by clustering the local data based on the data aggregation strategy by the second service end.

[0019] The aggregating module is configured to aggregate the first clustering center and the second clustering center and obtain an aggregated center.

[0020] The re-clustering module is configured to re-cluster the local service data according to the aggregated center until a preset iteration number or the aggregated center no longer changes, to obtain a mining data set.

[0021] In a third aspect, the disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0022] In a fourth aspect, the disclosure provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the above method.

[0023] Compared with the prior art, the beneficial effects of the embodiments of the present disclosure at least include: the data mining of the multi-source heterogeneous business database provided by the embodiments of the present disclosure can be applied to a first business end, when the first business end wants to train a business model with better performance through a joint learning method, a second business end can be determined; when receiving confirmation information fed back by the second business end, a data aggregation strategy is sent to the second business end; the local business data is clustered using the data aggregation strategy and a first clustering center is obtained; the second business end is received based on the data aggregation strategy to cluster the local data and obtain a second clustering center; the first clustering center and the second clustering center are aggregated and an aggregated center is obtained; the local business data is re-clustered according to the aggregated center until a preset iteration number or the aggregated center no longer changes, and a mining data set is obtained, and then the business data in the mining data set obtained by the above method can be used to participate in joint learning training to obtain a business model. The method can mine similar business data in each resource party (business end) on the premise of protecting the data security of each resource party (business end), and alleviate the problem that the performance of the model obtained by joint training is poor due to data heterogeneity. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a scene schematic diagram of an application scenario of the embodiments of the present disclosure;

[0026] Figure 2 is a flowchart of a data mining method of a multi-source heterogeneous business database provided by the embodiments of the present disclosure;

[0027] Figure 3 is a business data topology structure diagram in the data mining method of the multi-source heterogeneous business database provided by the embodiments of the present disclosure;

[0028] Figure 4 is a structural schematic diagram of a data mining device of a multi-source heterogeneous business database provided by the embodiments of the present disclosure;

[0029] Figure 5 is a structural schematic diagram of an electronic device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] In the following description, for the purposes of explanation, specific details are set forth in order to thoroughly convey the embodiments of the present disclosure. It should be apparent to those skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and methods have not been described in detail in order to avoid obscuring the present disclosure.

[0031] A multi-source heterogeneous service database data mining method and device according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 FIG. 1 is a scenario diagram of an application scenario of an embodiment of the present disclosure. The application scenario can include a first service end 101, a second service end 102, a service end 103, and a network 104.

[0033] The first service end 101 and the second service end 102 can be different resource parties in the same industry, or can be different service sub-systems in a multi-system service platform. The first service end 101 and the second service end 102 each deploy a respective service database, which is used to exist and manage service data in the aspects of consumption, production and operation, etc.

[0034] The service end 103 can be a server providing various services. For example, the server can be a single server, a server cluster composed of several servers, or a cloud computing service center, and the present disclosure does not limit this. The service end 103 can collect and manage information about the service data distribution of each data source, including service data type, database type information, and service data quality information, etc.

[0035] The network 104 can be a wired network using coaxial cables, twisted-pair wires, and optical fibers, or can be a wireless network that can realize interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc., and the present disclosure does not limit this.

[0036] When the first service end wants to train a service model with better performance through joint learning, the second service end can be determined. When receiving the confirmation information fed back by the second service end, the data aggregation strategy is sent to the second service end. The local service data is clustered using the data aggregation strategy to obtain a first clustering center. The second clustering center obtained by the second service end based on the data aggregation strategy is received. The first clustering center and the second clustering center are aggregated to obtain an aggregated center. The local service data is re-clustered according to the aggregated center until the preset iteration number or the aggregated center no longer changes, and a mining data set is obtained. Then, the service data in the mining data set is used to participate in joint learning training, and finally the required service model is obtained. The method can mine similar service data in each resource party (service end) on the premise of protecting the data security of each resource party (service end), and alleviate the problem of poor performance of the model obtained by joint training due to data heterogeneity.

[0037] It should be noted that the specific types, quantities and combinations of the first service end 101, the second service end 102, the server 103 and the network 104 can be adjusted according to the actual needs of the application scene, and the embodiments of the present disclosure do not limit them.

[0038] Figure 2 is a flowchart of a multi-source heterogeneous service database data mining method provided by the embodiments of the present disclosure. Figure 2 The multi-source heterogeneous service database data mining method of Figure 1 may be executed by the first service end 101. As Figure 2 shown, the multi-source heterogeneous service database data mining method includes:

[0039] Step S201, determining a second service end.

[0040] In an embodiment, a data mining request is sent to the server 103, and the data mining request includes a service data type and a service data quality requirement. The service database topology structure diagram returned by the server based on the data mining request is received, and the service database topology structure diagram includes a plurality of distributed data sources, and database type information and service data quality information of the service database corresponding to each data source. The second service end is determined based on the service database topology structure diagram.

[0041] Business data type, which is usually used to describe what kind of business data the resource party has. For example, the first business end is a medical equipment seller, and the business data it has can include medical equipment sales data, inventory data, order data, etc. For another example, the first business end is an energy enterprise, and the business data it has can include production energy data, consumption energy data, etc. Specifically, it can also be subdivided into different energy-related business data, such as electricity, water, gas, and coal data.

[0042] Business data quality requirements, which are usually standards for the quantity, integrity, and correctness of business data. In an example embodiment, different types of business data can be divided according to different business data types, and the quantity and integrity information of each type of business data can be counted, and corresponding scoring rules can be developed to determine the quality level of business data. Business data quality requirements can be specific quality level requirements.

[0043] Business data integrity usually refers to whether each time point of some time series data corresponds to business data. Business data correctness usually refers to whether there are abnormal values in a series of business data (such as negative values in sales business data).

[0044] Business database topology diagram is a diagram similar to mind map mode formed by arranging multiple distributed data sources, and database type information and business data quality information of each data source corresponding to the business database according to a certain logical structure.

[0045] As an example, the business data topology diagram can be designed as shown in Figure 3 As shown in Figure 3 , the first level of the business data topology diagram can be different business data types, the second level can be different data sources based on the various business data types in the first level, and the third level can be database type information and business data quality information corresponding to each data source.

[0046] Among them, the database type information includes database types (including MySQL database, postgre database, oracle database, etc.).

[0047] In an example embodiment, it is assumed that the first business end 101 is a seller of product A, and wants to participate in joint learning training with other business ends having similar business data to improve its product sales service and obtain the required business model. However, the first business end does not know the business data of other business ends, and cannot determine which business ends may have similar business data. At this time, the first business end 101 can send a data mining request to the service end 103. It is assumed that the business data type in the data mining request is the sales business data of product A, the business data quality requirement is to have a series of sales business data of product A from January 20XX to December 20YY, and the data amount is at least W. When the service end 103 receives the data mining request sent by the first business end 101, it can filter out the data sources having sales business data of product A and their related database type information and business data quality information from the information about the business data distribution of each data source, and generate a business data topology structure diagram describing the sales business data of product A according to the logical structure of the business data topology structure diagram as shown in FIG. 11, and feed back the business data topology structure diagram to the first business end 101. Figure 3

[0048] In some embodiments, determining the second business end based on the business database topology structure diagram comprises:

[0049] determining the target data source according to the database type information and the business data quality information;

[0050] determining the business end corresponding to the target data source as the second business end.

[0051] In combination with the above example, when the first business end 101 receives the business data topology structure diagram fed back by the service end 103, it can determine the target data source by viewing the database type information and the business data quality information of each data source on the business data topology structure diagram, and determine the business end corresponding to the target data source as the second business end.

[0052] ​As an example, it is assumed that the received business data topology graph includes data source 01, data source 02 and data source 03, wherein the database type of data source 01 is a MySQL database, which has a series of sales business data of product A from January 20XX to December 20YY, and the business data amount is greater than W; the database type of data source 02 is a postgre database, which has a series of sales business data of product A from January 20XX to December 20YY, and the business data amount is greater than W; and the database type of data source 03 is a MySQL database, which has a series of sales business data of product A from May 20XX to December 20YY, and the business data amount is greater than W. If the database type adopted by the first business end is a MySQL database, considering that the language environment of the same database type is the same, it is relatively easier to mine similar business data, so the same database type can be preferably adopted, and the data source with business data quality meeting the requirements is selected as the target data source. In the above example, data source 01 can be selected as the target data source. Then, the business end corresponding to data source 01 is the second business end. Of course, considering that the diversification of data can improve the generalization ability of the business model, data sources 01 and 03 can also be selected as the target data sources, or data sources 01 and 02 can be selected as the target data sources. The data source that can be selected as the target data source at least needs to meet the condition of having the same or similar types of business data, because if there is no business data of the same or similar types, it is impossible to mine valuable similar business data.

[0053] Step S202, when receiving the confirmation information fed back by the second business end, sending the data aggregation strategy to the second business end.

[0054] When the first business end 101 selects data source 01 as the target data source, the business end corresponding to data source 01 is the second business end, and the joint learning training application (including business data type and business data quality requirement information) can be sent to the second business end. After receiving the joint learning training application, if it is determined to participate, the second business end can feed back information indicating confirmation of participation to the first business end through email, telephone, short message, WeChat and the like. When receiving the confirmation information of the second business end, the first business end can send the data aggregation strategy to the second business end.

[0055] The data aggregation strategy mainly includes a method for indicating how the second business end aggregates its business data. The specific implementation process of the method is as follows.

[0056] Step S203, clustering the local business data using the data aggregation strategy and obtaining a first clustering center.

[0057] In some embodiments, the above step S203 includes:

[0058] classifying the local business data to obtain at least one classified data set, each of the classified data sets including a series of business data;

[0059] decomposing the series of business data in each of the classified data sets to obtain a plurality of classified features;

[0060] screening out one of the classified features with the highest matching degree with the preset mining target as a clustering target feature;

[0061] clustering the classified data sets in the local business data based on the clustering target feature to obtain a plurality of clustering clusters, and extracting a first clustering center of each of the clustering clusters.

[0062] The local business data refers to the business data stored in the local database and owned by the first business end.

[0063] The series of business data can refer to a series of data collected at different times, i.e., time series data. For example, it can be a series of sales business data of the first business end collected in various time dimensions such as a month, a quarter, a year, etc.

[0064] The classification of the local business data can be based on the data mining target (for example, mining the sales trend of A product from January 20XX to December 20YY for N consecutive years; or mining the sales trend of A product in January of each year from January 20XX to December 20YY; or mining the sales trend of A product in the third quarter of each year from January 20XX to December 20YY, etc.) to classify the local business data set.

[0065] As an example, assuming that the data mining target is to mine the sales trend of A product in January of each year from January 20XX to December 20YY, then the A product sales business data in January of each year from January 20XX to December 20YY can be divided from the local business data, wherein the A product sales business data in January of each year is a classified data set, and thus N classified data sets can be obtained, each of which can include the sales business data of A product in each day of January.

[0066] Next, the series of business data in each of the classified data sets is decomposed to obtain a plurality of classified features. For example, the STL (Template Library) algorithm can be used to decompose the series of business data in each of the classified data sets to obtain three classified features corresponding to each of the classified data sets, i.e., a trend component, a periodic component and a residual component.

[0067] The trend component mainly reflects the overall trend of the time series data, such as monotone increasing, monotone decreasing or irregular fluctuation. The periodic component mainly reflects the periodicity of the time series data, for example, the peak of sales always appears in a certain month from March to July. The length of the periodic component is generally not fixed, but it appears cyclically on a long time scale. The residual component mainly reflects the irregular change of the time series data. The residual is the difference between the observed value and the predicted value (fitting value), that is, the difference between the actual observation value and the regression estimate value.

[0068] Then, the classification feature "trend component" with the highest matching degree with the data mining target "mining the sales trend of A product in January of each year from January 20XX to December 20YY" is taken as the clustering target feature. Then, based on the clustering target feature, each classification data set in the local business data is clustered to obtain a plurality of clustering clusters, and a first clustering center of each clustering cluster is extracted. In combination with the above example, based on the "trend component", a series of business data in the N classification data sets are clustered to obtain a plurality of clustering clusters, and the first clustering centers of the plurality of clustering clusters are extracted. For example, based on the "trend component", a series of business data in a certain classification data set are clustered to obtain 3 clustering clusters, and each clustering cluster usually represents a trend. For example, clustering cluster 01 represents the sales trend of A product in a certain 10 days in January in the classification data set, clustering cluster 02 represents the sales trend of A product in a certain 5 days in January in the classification data set, and clustering cluster 03 represents the sales trend of A product in a certain 8 days in January in the classification data set.

[0069] In step S204, the second business end clusters the local data based on the data aggregation strategy and obtains a second clustering center.

[0070] Specifically, the operation process of the second business end clustering the local data based on the data aggregation strategy and obtaining the second clustering center can refer to the steps of the first business end clustering the local data based on the data aggregation strategy and obtaining the first clustering center, which will not be repeated here.

[0071] The second business end sends the second clustering center obtained by clustering to the first business end.

[0072] In step S205, the first clustering center and the second clustering center are aggregated to obtain an aggregation center.

[0073] In some embodiments, the first clustering identifier carried by the first clustering center and the second clustering identifier carried by the second clustering center are extracted;

[0074] The first clustering center and the second clustering center matched with the first clustering identifier and the second clustering identifier, respectively, are aggregated to obtain the aggregation center.

[0075] In combination with the above example, the first business end clusters the A product sales business data of January of each year from January 20XX to December 20YY based on the "trend component" respectively, obtains three clustering clusters, and extracts three first clustering centers. Each first clustering center can carry its business end identifier and the identifier information of year and month, or only carry the business end identifier and the year identifier information.

[0076] Similarly, the second business end clusters the A product sales business data of January of each year from January 20XX to December 20YY based on the "trend component" respectively, obtains N clustering clusters, and extracts N second clustering centers. Each second clustering center can carry its business end identifier and the identifier information of year and month, or only carry the business end identifier and the year identifier information.

[0077] As an example, it is assumed that the first clustering center has three, and their first clustering identifiers are (1, 20XX1, Jan1), (1, 20XX2, Jan2) and (1, 20XX3, Jan3) respectively, wherein "1" represents the first business end, "20XX1", "20XX2", "20XX3" represent different years respectively, and "Jan1", "Jan2", "Jan3" represent clustering clusters 01, 02, 03 obtained by the first business end clustering respectively. The second clustering center has three, and their first clustering identifiers are (2, 20XX1, Jan1), (2, 20XX2, Jan2) and (2, 20XX3, Jan3) respectively, wherein "2" represents the second business end, "20XX1", "20XX2", "20XX3" represent different years respectively, and "Jan" represents clustering clusters 01', 02', 03' obtained by the second business end clustering respectively.

[0078] The first clustering identifier and the second clustering identifier can be matched, which means that except for their business end identifiers being different, the rest of the identifier information is the same. For example, the first clustering identifier (1, 20XX1, Jan1) is matched with the second clustering identifier (2, 20XX1, Jan1); the first clustering identifier (1, 20XX2, Jan2) is matched with the second clustering identifier (2, 20XX2, Jan2); and the first clustering identifier (1, 20XX3, Jan3) is matched with the second clustering identifier (2, 20XX3, Jan3). The above two matched first clustering centers and second clustering centers are aggregated respectively to obtain three aggregation centers, which are denoted as aggregation centers 01, 02, 03 respectively.

[0079] In step S206, the local business data is re-clustered according to the aggregation center until the preset iteration number or the aggregation center no longer changes, and a mining data set is obtained.

[0080] In some embodiments, the Euclidean distance value between each cluster and the aggregation center is calculated respectively; the local business data is re-clustered according to the Euclidean distance value until a preset iteration number or the aggregation center no longer changes, and a mining data set is obtained.

[0081] As an example, the first business end can calculate the Euclidean distance 01 between its cluster 01, 02, 03 and the aggregation center 01, respectively, calculate the Euclidean distance 02 between its cluster 01, 02, 03 and the aggregation center 02, respectively, and calculate the Euclidean distance 03 between its cluster 01, 02, 03 and the aggregation center 03, respectively. Then, according to the calculated Euclidean distances 01, 02, 03, the A commodity sales business data of January of each year in the period from January 20XX to December 20YY in the local business data is re-clustered. For example, according to the calculated Euclidean distances 01, 02, 03, the A commodity sales business data of January 20XX1 is re-clustered to obtain three new cluster centers, and the cluster centers of the three new clusters are extracted.

[0082] In some embodiments, the local business data is re-clustered according to the Euclidean distance value until a preset iteration number or the aggregation center no longer changes, and a mining data set is obtained, comprising:

[0083] It is determined whether the Euclidean distance value meets a preset threshold range; if the Euclidean distance value meets the preset threshold range, it is determined that the aggregation center no longer changes, and the business data set corresponding to the plurality of clusters is determined as the mining data set.

[0084] In combination with the above example, if the Euclidean distances 01, 02, 03 all meet the preset threshold range (which can be flexibly set according to actual conditions, and is not specifically limited here), it is determined that the aggregation centers 01, 02, 03 no longer change, and the business data set corresponding to the clusters 01, 02, 03 is determined as the mining data set. That is, the A commodity sales business data set of January 1, January 5, and January 8 in the classification data set is determined as the mining data set.

[0085] The technical solution provided in this disclosure allows a first business terminal to determine a second business terminal when it wants to improve the performance of a business model through joint learning training. Upon receiving confirmation from the second business terminal, a data aggregation strategy is sent to it. The data aggregation strategy is used to cluster local business data to obtain a first cluster center. The second business terminal is then received, clustered based on the data aggregation strategy, to obtain a second cluster center. The first and second cluster centers are aggregated to obtain an aggregation center. The local business data is then re-clustered according to the aggregation center until a preset number of iterations or the aggregation center no longer changes, resulting in a mined dataset. The business data in this mined dataset is then used for joint learning training to ultimately obtain the desired business model. This method can mine similar business data from various resource providers (business terminals) while protecting their data security, thus mitigating the problem of poor model performance caused by data heterogeneity during joint training.

[0086] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0087] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0088] Figure 4 This is a schematic diagram of a data mining device for a multi-source heterogeneous business database provided in an embodiment of this disclosure. Figure 4 As shown, the data mining device for the multi-source heterogeneous business database includes:

[0089] Module 401 is configured to determine the second business terminal.

[0090] The sending module 402 is configured to send a data aggregation strategy to the second service terminal when it receives confirmation information from the second service terminal.

[0091] Clustering module 403 is configured to use a data aggregation strategy to cluster local business data and obtain the first cluster center;

[0092] The receiving module 404 is configured to receive the second cluster center obtained by the second service terminal by clustering local data based on the data aggregation strategy;

[0093] The aggregation module 405 is configured to aggregate the first cluster center and the second cluster center to obtain an aggregation center;

[0094] The re-clustering module 406 is configured to re-cluster local business data based on the aggregation center until a preset number of iterations or the aggregation center no longer changes, thus obtaining the mining dataset.

[0095] In some embodiments, the determination module 401 comprises:

[0096] a sending unit configured to send a data mining request to the server, the data mining request comprising a service data type and a service data quality requirement;

[0097] a receiving unit configured to receive a service database topology graph returned by the server based on the data mining request, the service database topology graph comprising a plurality of distributed data sources, and database type information and service data quality information of a service database corresponding to each data source;

[0098] a determination unit configured to determine a second service end based on the service database topology graph.

[0099] In some embodiments, the determination unit can be specifically configured to:

[0100] determine a target data source according to the database type information and the service data quality information;

[0101] determine the service end corresponding to the target data source as the second service end.

[0102] In some embodiments, the clustering module 403 comprises:

[0103] a classification unit configured to classify local service data to obtain at least one classified data set;

[0104] a decomposition unit configured to decompose a series of service data in the classified data set to obtain a plurality of classified features;

[0105] a screening unit configured to screen out one classified feature with the highest matching degree with a preset mining target as a clustering target feature;

[0106] a clustering unit configured to cluster the classified data set in the local service data based on the clustering target feature to obtain a plurality of clustering clusters, and extract a first clustering center of each clustering cluster.

[0107] In some embodiments, the first clustering center and the second clustering center are aggregated to obtain an aggregated center, comprising:

[0108] extracting a first clustering identifier carried by the first clustering center, and a second clustering identifier carried by the second clustering center;

[0109] respectively aggregating the first clustering center and the second clustering center matched with the first clustering identifier and the second clustering identifier to obtain the aggregated center.

[0110] In some embodiments, the local business data is re-clustered according to the aggregation center until a preset iteration number or the aggregation center no longer changes, to obtain a mining data set, including:

[0111] The Euclidean distance value between each cluster and the aggregation center is calculated respectively;

[0112] The local business data is re-clustered according to the Euclidean distance value until a preset iteration number or the aggregation center no longer changes, to obtain a mining data set.

[0113] In some embodiments, the local business data is re-clustered according to the Euclidean distance value until a preset iteration number or the aggregation center no longer changes, to obtain a mining data set, including:

[0114] It is determined whether the Euclidean distance value meets a preset threshold range;

[0115] If the Euclidean distance value meets the preset threshold range, it is determined that the aggregation center no longer changes, and the business data set corresponding to the plurality of cluster centers is determined as the mining data set.

[0116] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0117] Figure 5 is a schematic diagram of an electronic device 5 provided by the embodiments of the present disclosure. As shown in the figure, the electronic device 5 of this embodiment includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. The processor 501 implements the steps in each of the above method embodiments when executing the computer program 503. Alternatively, the processor 501 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 503. Figure 5

[0118] The electronic device 5 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 5 can include but is not limited to the processor 501 and the memory 502. Those skilled in the art can understand that the electronic device 5 can include more or fewer components or different components than those shown in the figure. Figure 5 The electronic device 5 is only an example of the electronic device 5 and does not constitute a limitation on the electronic device 5, and can include more or fewer components or different components than those shown in the figure.

[0119] ​The processor 501 can be a central processing unit (CPU), or other general purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

[0120] The memory 502 can be an internal storage unit of the electronic device 5, for example, a hard disk or a memory of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. The memory 502 can also include both the internal storage unit and the external storage device of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above described functions. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0122] The integrated modules / units, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program can include computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier signal and telecommunication signal.

[0123] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

Claims

1. A data mining method for multi-source heterogeneous business databases, characterized in that, include: First business side: Determine the second business segment; When a confirmation message is received from the second business terminal, a data aggregation strategy is sent to the second business terminal. The data aggregation strategy is used to cluster local business data and obtain the first cluster center; Receive the second cluster center obtained by the second business terminal based on the data aggregation strategy to cluster local data; The first cluster center and the second cluster center are aggregated to obtain the aggregate center; The local business data is re-clustered based on the aggregation center until a preset number of iterations or the aggregation center no longer changes, resulting in a mined dataset. The data aggregation strategy is used to cluster local business data and obtain the first cluster center, including: Classify local business data to obtain at least one categorized data set; A series of business data in the classification dataset are decomposed to obtain multiple classification features; The classification feature with the highest matching degree with the preset mining target is selected as the clustering target feature; Based on the clustering target features, the categorized data set in the local business data is clustered to obtain multiple clusters, and the first cluster center of each cluster is extracted. Aggregating the first cluster center and the second cluster center to obtain the aggregate center includes: Extract the first cluster identifier carried by the first cluster center and the second cluster identifier carried by the second cluster center; The first cluster centers and second cluster centers that match the first cluster identifier and the second cluster identifier are aggregated to obtain aggregate centers; The local business data is re-clustered based on the aggregation center until a preset number of iterations or the aggregation center no longer changes, resulting in a mined dataset, including: Calculate the Euclidean distance between each cluster and the aggregation center; Determine whether the Euclidean distance value meets the preset threshold range; If the Euclidean distance value meets the preset threshold range, then the aggregation center is determined to no longer change, and the business data set corresponding to the multiple clusters is determined as the mining dataset.

2. The method according to claim 1, characterized in that, The second business segment is defined, including: Send a data mining request to the server, the data mining request including the business data type and business data quality requirements; The system receives a business database topology diagram returned by the server based on the data mining request. The business database topology diagram includes multiple distributed data sources, as well as database type information and business data quality information of the business database corresponding to each data source. The second business terminal is determined based on the business database topology diagram.

3. The method according to claim 2, characterized in that, The second business terminal is determined based on the aforementioned business database topology diagram, including: Based on the database type information and business data quality information, the target data source is determined; The business terminal corresponding to the target data source is identified as the second business terminal.

4. A data mining device for multi-source heterogeneous business databases, characterized in that, include: The module is configured to determine the second business terminal. The sending module is configured to send a data aggregation strategy to the second service terminal when it receives confirmation information from the second service terminal. The clustering module is configured to use the data aggregation strategy to cluster local business data and obtain the first cluster center. The receiving module is configured to receive the second cluster center obtained by the second service terminal from clustering local data based on the data aggregation strategy; The aggregation module is configured to aggregate the first cluster center and the second cluster center to obtain an aggregation center; The re-clustering module is configured to re-cluster the local business data based on the aggregation center until a preset number of iterations or the aggregation center no longer changes, thereby obtaining the mining dataset. The data aggregation strategy is used to cluster local business data and obtain the first cluster center, including: Classify local business data to obtain at least one categorized data set; A series of business data in the classification dataset are decomposed to obtain multiple classification features; The classification feature with the highest matching degree with the preset mining target is selected as the clustering target feature; Based on the clustering target features, the categorized data set in the local business data is clustered to obtain multiple clusters, and the first cluster center of each cluster is extracted. Aggregating the first cluster center and the second cluster center to obtain the aggregate center includes: Extract the first cluster identifier carried by the first cluster center and the second cluster identifier carried by the second cluster center; The first cluster centers and second cluster centers that match the first cluster identifier and the second cluster identifier are aggregated to obtain aggregate centers; The local business data is re-clustered based on the aggregation center until a preset number of iterations or the aggregation center no longer changes, resulting in a mined dataset, including: Calculate the Euclidean distance between each cluster and the aggregation center; Determine whether the Euclidean distance value meets the preset threshold range; If the Euclidean distance value meets the preset threshold range, then the aggregation center is determined to no longer change, and the business data set corresponding to the multiple clusters is determined as the mining dataset.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3.

Images