Main data identification method and device, equipment and storage medium

Abstract

The present application relates to data processing, and provides a main data identification method, device, equipment and storage medium. The method obtains field features in multiple scene objects; extracts target features in the field features; obtains feature data corresponding to the target features from multiple preset data sources for clustering processing, obtains multiple data groups and data categories corresponding to each data group; generates a main data identification model based on the multiple data groups and the data categories; when receiving a main data identification request, obtains to-be-identified data and a to-be-identified scene according to the main data identification request; inputs the to-be-identified data and the to-be-identified scene into the main data identification model to obtain scene main data of the to-be-identified scene, thereby improving identification efficiency and identification effect. In addition, the present application also relates to blockchain technology, and the scene main data can be stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a master data identification method, device, equipment and storage medium. Background Art

[0002] Master data refers to the fundamental data shared by all business departments within an enterprise, characterized by high value, high sharing, high stability, and high quality. With the continuous advancement of digitization, the identification of master data is essential. Existing master data identification solutions primarily rely on expert scoring of qualitative features to determine master data. However, this approach lacks objectivity, resulting in poor identification results. Furthermore, this approach consumes significant time and labor costs, leading to low identification efficiency. Summary of the Invention

[0003] In view of the above, it is necessary to provide a master data identification method, device, equipment and storage medium that can improve the recognition effect and efficiency of scene master data.

[0004] In one aspect, the present invention provides a master data identification method, comprising:

[0005] Get field features in multiple scene objects;

[0006] extracting target features from the field features;

[0007] Acquire feature data corresponding to the target feature from multiple preset data sources and perform clustering processing to obtain multiple data groups and data categories corresponding to each data group;

[0008] generating a master data recognition model based on the plurality of data groups and the data categories;

[0009] When receiving a master data identification request, obtaining the data to be identified and the scene to be identified according to the master data identification request;

[0010] The data to be identified and the scene to be identified are input into the master data identification model to obtain scene master data of the scene to be identified.

[0011] According to a preferred embodiment of the present invention, obtaining field features in multiple scene objects includes:

[0012] Acquire all scenes in a preset scene library as the multiple scene objects;

[0013] The field corresponding to each scene object is obtained from the field library as the scene field, and the feature corresponding to the scene object is obtained as the field feature.

[0014] According to a preferred embodiment of the present invention, the feature data corresponding to the target feature is obtained from multiple preset data sources and clustered to obtain multiple data groups and the data category corresponding to each data group includes:

[0015] Constructing a regular expression based on the target feature;

[0016] Searching the plurality of preset data sources in parallel based on the regular expression to obtain a plurality of initial data;

[0017] Cleaning the multiple initial data to obtain the feature data;

[0018] Acquire target clustering models corresponding to the multiple scene objects;

[0019] The characteristic data is clustered based on the target clustering model to obtain the multiple data groups and the data category corresponding to each data group.

[0020] According to a preferred embodiment of the present invention, before obtaining the target clustering models corresponding to the multiple scene objects, the master data identification method further includes:

[0021] Acquiring scene training data from the plurality of scene objects;

[0022] Performing clustering processing on the scene training data based on preset clustering models constructed by multiple preset clustering algorithms to obtain multiple data sets for each preset clustering model, the multiple data sets including a first data set and a second data set;

[0023] For each preset clustering model, a first covariance matrix is ​​constructed based on any two data in each data set;

[0024] Constructing a second covariance matrix based on any data in the first data set and any data in the second data set;

[0025] generating an algorithm score for each preset clustering algorithm according to the first covariance matrix and the second covariance matrix;

[0026] Determine the preset clustering model with the largest algorithm score as the initial clustering model;

[0027] The initial clustering model is adjusted according to the scenario training data until the loss value of the initial clustering model no longer decreases, thereby obtaining the target clustering model.

[0028] According to a preferred embodiment of the present invention, the calculation formula of the algorithm score is:

[0029]

[0030] Wherein, y refers to the algorithm score, k refers to the number of the multiple data sets, a k refers to the sum of the elements of the first covariance matrix of the kth data set, b k It refers to the sum of the elements of the second covariance matrix constructed by the first data set and the k-th data set.

[0031] According to a preferred embodiment of the present invention, generating a master data recognition model based on the multiple data groups and the data categories includes:

[0032] For each data group, identify the source system of the data in that data group;

[0033] Count the number of systems of the source system;

[0034] generating a standard quantity according to the source quantity and the preset ratio of the plurality of preset data sources;

[0035] Determine a data group corresponding to a number of systems greater than the standard number as a target group;

[0036] Determine the data category corresponding to the target group as the target category, and determine the scene object corresponding to the same field feature as the target category as the target scene;

[0037] The master data recognition model is constructed according to the target category and the target scenario.

[0038] According to a preferred embodiment of the present invention, obtaining the data to be identified and the scenario to be identified according to the master data identification request includes:

[0039] Parsing the master data identification request message to obtain data information carried in the message;

[0040] extracting information indicating a scene from the data information as the scene to be identified, and extracting information indicating a path from the data information as a storage path;

[0041] All data are acquired from the storage path as the data to be identified.

[0042] On the other hand, the present invention further provides a master data identification device, comprising:

[0043] An acquisition unit, used to acquire field features in multiple scene objects;

[0044] an extraction unit, configured to extract target features from the field features;

[0045] The acquisition unit is further configured to acquire feature data corresponding to the target feature from a plurality of preset data sources and perform clustering processing to obtain a plurality of data groups and a data category corresponding to each data group;

[0046] a generating unit, configured to generate a master data recognition model based on the plurality of data groups and the data categories;

[0047] The acquisition unit is further configured to, upon receiving a master data recognition request, acquire the data to be recognized and the scene to be recognized according to the master data recognition request;

[0048] An input unit is used to input the data to be identified and the scene to be identified into the master data recognition model to obtain the scene master data of the scene to be identified.

[0049] On the other hand, the present invention further provides an electronic device, comprising:

[0050] a memory storing computer-readable instructions; and

[0051] A processor executes the computer-readable instructions stored in the memory to implement the master data identification method.

[0052] On the other hand, the present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the master data identification method.

[0053] It can be seen from the above technical solution that the present invention can comprehensively obtain the feature data for clustering through the multiple preset data sources, which is beneficial to improving the cross-domain capability of the master data recognition model. At the same time, by clustering the feature data, the feature data can be accurately divided into multiple data groups, and then the master data recognition model is generated according to the multiple data groups and the data category training, which can improve the recognition accuracy of the master data recognition model, and then use the master data recognition model to identify the data to be identified, thereby improving the accuracy of the scene master data. In addition, since there is no need for manual analysis of the data to be identified, the recognition efficiency of the scene master data can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a preferred embodiment of the master data identification method of the present invention.

[0055] Figure 2 It is a functional module diagram of a preferred embodiment of the master data identification device of the present invention.

[0056] Figure 3 It is a structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing the master data identification method. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1 FIG. 1 is a flow chart of a preferred embodiment of the master data identification method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0059] The master data identification method can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0060] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0061] The master data identification method is applied to one or more electronic devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Their hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0062] The electronic device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0063] The electronic device may include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network electronic device, a group of electronic devices consisting of multiple network electronic devices, or a cloud based on cloud computing consisting of a large number of hosts or network electronic devices.

[0064] The network where the electronic device is located includes, but is not limited to: the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0065] S10, obtaining field features in multiple scene objects.

[0066] In at least one embodiment of the present invention, the multiple scenario objects may be all project scenarios in an enterprise. For example, the multiple scenario objects may be claim settlement projects, etc.

[0067] The field characteristics refer to the specific characteristics of any field in the multiple scene objects. For example, the field characteristics of the field "customer" are: the number of data bits is 10; the data starts with specific numbers and letters, etc.

[0068] In at least one embodiment of the present invention, the electronic device acquiring field features in a plurality of scene objects includes:

[0069] Acquire all scenes in a preset scene library as the multiple scene objects;

[0070] The field corresponding to each scene object is obtained from the field library as the scene field, and the feature corresponding to the scene object is obtained as the field feature.

[0071] Through the above implementation, the field features can be acquired to the maximum extent.

[0072] S11, extracting target features from the field features.

[0073] In at least one embodiment of the present invention, the electronic device randomly extracts features from the field features and determines the extracted features as the target features.

[0074] S12, acquiring feature data corresponding to the target feature from a plurality of preset data sources and performing clustering processing to obtain a plurality of data groups and a data category corresponding to each data group.

[0075] In at least one embodiment of the present invention, the multiple preset data sources may include databases corresponding to the multiple scene objects, and the multiple preset data sources may also include shared databases corresponding to multiple enterprise departments.

[0076] The data category refers to the category feature corresponding to each data group. For example, the data category is: data starting with specific numbers and letters, etc.

[0077] In at least one embodiment of the present invention, the electronic device obtains feature data corresponding to the target feature from multiple preset data sources and performs clustering processing to obtain multiple data groups and data categories corresponding to each data group, including:

[0078] Constructing a regular expression based on the target feature;

[0079] Searching the plurality of preset data sources in parallel based on the regular expression to obtain a plurality of initial data;

[0080] Cleaning the multiple initial data to obtain the feature data;

[0081] Acquire target clustering models corresponding to the multiple scene objects;

[0082] The characteristic data is clustered based on the target clustering model to obtain the multiple data groups and the data category corresponding to each data group.

[0083] The characteristic data refers to initial data that does not contain repeated data.

[0084] The target clustering model refers to a model that can be used to cluster the multiple scene objects at the same time.

[0085] By searching the multiple preset data sources in parallel through the regular expression, the multiple initial data can be quickly acquired. By cleaning the multiple initial data, duplicate data in the feature data can be avoided, thereby affecting the clustering efficiency. Then, by clustering the feature data through the target clustering model, the multiple data groups and the data categories can be quickly and accurately obtained.

[0086] Specifically, before obtaining the target clustering models corresponding to the multiple scene objects, the master data identification method further includes:

[0087] Acquiring scene training data from the plurality of scene objects;

[0088] Performing clustering processing on the scene training data based on preset clustering models constructed by multiple preset clustering algorithms to obtain multiple data sets for each preset clustering model, the multiple data sets including a first data set and a second data set;

[0089] For each preset clustering model, a first covariance matrix is ​​constructed based on any two data in each data set;

[0090] Constructing a second covariance matrix based on any data in the first data set and any data in the second data set;

[0091] generating an algorithm score for each preset clustering algorithm according to the first covariance matrix and the second covariance matrix;

[0092] Determine the preset clustering model with the largest algorithm score as the initial clustering model;

[0093] The initial clustering model is adjusted according to the scenario training data until the loss value of the initial clustering model no longer decreases, thereby obtaining the target clustering model.

[0094] The multiple preset clustering algorithms may include, but are not limited to: K-Means algorithm, etc.

[0095] There are multiple data sets, and the second data set is a data set different from the first data set.

[0096] The scene training data is clustered using a preset clustering model constructed by the multiple preset clustering algorithms. The multiple data sets obtained by the clustering processing can be used to screen the preset clustering model, thereby improving the applicability of the initial clustering model in the multiple scene objects, and then further adjusting the initial clustering model to improve the accuracy of the target clustering model.

[0097] Specifically, the calculation formula of the algorithm score is:

[0098]

[0099] Wherein, y refers to the algorithm score, k refers to the number of the multiple data sets, a k refers to the sum of the elements of the first covariance matrix of the kth data set, b k It refers to the sum of the elements of the second covariance matrix constructed by the first data set and the k-th data set.

[0100] By setting k-1 in the denominator, the situation where there is only one data set can be avoided, thereby improving the accuracy of the algorithm score.

[0101] S13, generating a master data recognition model based on the multiple data groups and the data categories.

[0102] In at least one embodiment of the present invention, the master data identification model can be used for master data in the plurality of scene objects.

[0103] In at least one embodiment of the present invention, the electronic device generating a master data recognition model based on the multiple data groups and the data categories includes:

[0104] For each data group, identify the source system of the data in that data group;

[0105] Count the number of systems of the source system;

[0106] generating a standard quantity according to the source quantity and the preset ratio of the plurality of preset data sources;

[0107] Determine a data group corresponding to a number of systems greater than the standard number as a target group;

[0108] Determine the data category corresponding to the target group as the target category, and determine the scene object corresponding to the same field feature as the target category as the target scene;

[0109] The master data recognition model is constructed according to the target category and the target scenario.

[0110] The preset ratio may be set according to actual needs, for example, the preset ratio may be 60%.

[0111] By determining the data group corresponding to the number of systems greater than the standard number as the target group, it is possible to screen the multiple data groups based on the shared characteristics of the master data, thereby improving the accuracy of the target group, and then accurately construct the master data recognition model based on the target category and the target scenario.

[0112] Specifically, the electronic device obtains training data corresponding to the target category and the target scene at the same time, and obtains a decision tree network based on a decision tree algorithm, inputs the training data and the target scene into the decision tree network, obtains the predicted category of the training data, calculates the network loss value of the decision tree network according to the predicted category and the target category, adjusts the breadth and depth of the decision tree network until the network loss value no longer decreases, and obtains the master data recognition model.

[0113] The decision tree algorithm may be an ID3 decision tree algorithm or a CART decision tree algorithm. The present invention does not limit the specific algorithm of the decision tree algorithm.

[0114] S14: When a master data identification request is received, the data to be identified and the scene to be identified are acquired according to the master data identification request.

[0115] In at least one embodiment of the present invention, the master data identification request may be generated by a user who has a master data identification requirement, or may be generated by a machine at a specific time point.

[0116] The data to be identified refers to data that needs to be identified as master data.

[0117] The scenario to be identified refers to a specific scenario that requires master data identification.

[0118] In at least one embodiment of the present invention, the electronic device obtaining the data to be identified and the scene to be identified according to the master data identification request includes:

[0119] Parsing the master data identification request message to obtain data information carried in the message;

[0120] extracting information indicating a scene from the data information as the scene to be identified, and extracting information indicating a path from the data information as a storage path;

[0121] All data are acquired from the storage path as the data to be identified.

[0122] By parsing the message, since there is no need to parse the entire request, the efficiency of obtaining the data information is improved, thereby improving the efficiency of obtaining the data to be identified and the scene to be identified.

[0123] S15: Input the data to be identified and the scene to be identified into the master data identification model to obtain scene master data of the scene to be identified.

[0124] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned scenario master data, the above-mentioned scenario master data can also be stored in a node of a blockchain.

[0125] In at least one embodiment of the present invention, the scene master data refers to the master data belonging to the scene to be identified in the data to be identified.

[0126] In at least one embodiment of the present invention, the electronic device inputs the data to be identified and the scene to be identified into the master data recognition model, and the manner in which the scene master data of the scene to be identified is obtained is an inverse process of the manner in which the electronic device constructs the master data recognition model according to the target category and the target scene, and the present invention will not elaborate on this.

[0127] It can be seen from the above technical solution that the present invention can comprehensively obtain the feature data for clustering through the multiple preset data sources, which is beneficial to improving the cross-domain capability of the master data recognition model. At the same time, by clustering the feature data, the feature data can be accurately divided into multiple data groups, and then the master data recognition model is generated according to the multiple data groups and the data category training, which can improve the recognition accuracy of the master data recognition model, and then use the master data recognition model to identify the data to be identified, thereby improving the accuracy of the scene master data. In addition, since there is no need for manual analysis of the data to be identified, the recognition efficiency of the scene master data can be improved.

[0128] like Figure 2 , which is a functional block diagram of a preferred embodiment of a master data identification device according to the present invention. The master data identification device 11 includes an acquisition unit 110, an extraction unit 111, a generation unit 112, an input unit 113, a clustering unit 114, a construction unit 115, a determination unit 116, and an adjustment unit 117. As used herein, a module / unit refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and is stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0129] The acquiring unit 110 acquires field features from a plurality of scene objects.

[0130] In at least one embodiment of the present invention, the multiple scenario objects may be all project scenarios in an enterprise. For example, the multiple scenario objects may be claim settlement projects, etc.

[0131] The field characteristics refer to the specific characteristics of any field in the multiple scene objects. For example, the field characteristics of the field "customer" are: the number of data bits is 10; the data starts with specific numbers and letters, etc.

[0132] In at least one embodiment of the present invention, the acquiring unit 110 acquires field features in a plurality of scene objects including:

[0133] Acquire all scenes in a preset scene library as the multiple scene objects;

[0134] The field corresponding to each scene object is obtained from the field library as the scene field, and the feature corresponding to the scene object is obtained as the field feature.

[0135] Through the above implementation, the field features can be acquired to the maximum extent.

[0136] The extraction unit 111 extracts target features from the field features.

[0137] In at least one embodiment of the present invention, the extraction unit 111 randomly extracts features from the field features, and determines the extracted features as the target features.

[0138] The acquisition unit 110 acquires feature data corresponding to the target feature from a plurality of preset data sources and performs clustering processing to obtain a plurality of data groups and a data category corresponding to each data group.

[0139] In at least one embodiment of the present invention, the multiple preset data sources may include databases corresponding to the multiple scene objects, and the multiple preset data sources may also include shared databases corresponding to multiple enterprise departments.

[0140] The data category refers to the category feature corresponding to each data group. For example, the data category is: data starting with specific numbers and letters, etc.

[0141] In at least one embodiment of the present invention, the acquisition unit 110 acquires feature data corresponding to the target feature from multiple preset data sources and performs clustering processing to obtain multiple data groups and data categories corresponding to each data group including:

[0142] Constructing a regular expression based on the target feature;

[0143] Searching the plurality of preset data sources in parallel based on the regular expression to obtain a plurality of initial data;

[0144] Cleaning the multiple initial data to obtain the feature data;

[0145] Acquire target clustering models corresponding to the multiple scene objects;

[0146] The characteristic data is clustered based on the target clustering model to obtain the multiple data groups and the data category corresponding to each data group.

[0147] The characteristic data refers to initial data that does not contain repeated data.

[0148] The target clustering model refers to a model that can be used to cluster the multiple scene objects at the same time.

[0149] By searching the multiple preset data sources in parallel through the regular expression, the multiple initial data can be quickly acquired. By cleaning the multiple initial data, duplicate data in the feature data can be avoided, thereby affecting the clustering efficiency. Then, by clustering the feature data through the target clustering model, the multiple data groups and the data categories can be quickly and accurately obtained.

[0150] Specifically, before acquiring the target clustering models corresponding to the plurality of scene objects, the acquiring unit 110 acquires scene training data from the plurality of scene objects;

[0151] The clustering unit 114 performs clustering processing on the scene training data based on preset clustering models constructed by multiple preset clustering algorithms to obtain multiple data sets for each preset clustering model, wherein the multiple data sets include a first data set and a second data set;

[0152] The construction unit 115 constructs a first covariance matrix for each preset clustering model according to any two data in each data set;

[0153] The construction unit 115 constructs a second covariance matrix according to any data in the first data set and any data in the second data set;

[0154] The generating unit 112 generates an algorithm score of each preset clustering algorithm according to the first covariance matrix and the second covariance matrix;

[0155] The determining unit 116 determines the preset clustering model with the largest algorithm score as the initial clustering model;

[0156] The adjusting unit 117 adjusts the initial clustering model according to the scene training data until the loss value of the initial clustering model no longer decreases, thereby obtaining the target clustering model.

[0157] The multiple preset clustering algorithms may include, but are not limited to: K-Means algorithm, etc.

[0158] There are multiple data sets, and the second data set is a data set different from the first data set.

[0159] The scene training data is clustered using a preset clustering model constructed by the multiple preset clustering algorithms. The multiple data sets obtained by the clustering processing can be used to screen the preset clustering model, thereby improving the applicability of the initial clustering model in the multiple scene objects, and then further adjusting the initial clustering model to improve the accuracy of the target clustering model.

[0160] Specifically, the calculation formula of the algorithm score is:

[0161]

[0162] Wherein, y refers to the algorithm score, k refers to the number of the multiple data sets, a k refers to the sum of the elements of the first covariance matrix of the kth data set, b kIt refers to the sum of the elements of the second covariance matrix constructed by the first data set and the k-th data set.

[0163] By setting k-1 in the denominator, the situation where there is only one data set can be avoided, thereby improving the accuracy of the algorithm score.

[0164] The generating unit 112 generates a master data recognition model based on the plurality of data groups and the data categories.

[0165] In at least one embodiment of the present invention, the master data identification model can be used for master data in the plurality of scene objects.

[0166] In at least one embodiment of the present invention, the generating unit 112 generates the master data recognition model based on the multiple data groups and the data categories, including:

[0167] For each data group, identify the source system of the data in that data group;

[0168] Count the number of systems of the source system;

[0169] generating a standard quantity according to the source quantity and the preset ratio of the plurality of preset data sources;

[0170] Determine a data group corresponding to a number of systems greater than the standard number as a target group;

[0171] Determine the data category corresponding to the target group as the target category, and determine the scene object corresponding to the same field feature as the target category as the target scene;

[0172] The master data recognition model is constructed according to the target category and the target scenario.

[0173] The preset ratio may be set according to actual needs, for example, the preset ratio may be 60%.

[0174] By determining the data group corresponding to the number of systems greater than the standard number as the target group, it is possible to screen the multiple data groups based on the shared characteristics of the master data, thereby improving the accuracy of the target group, and then accurately construct the master data recognition model based on the target category and the target scenario.

[0175] Specifically, the generation unit 112 obtains training data corresponding to both the target category and the target scenario, and obtains a decision tree network based on a decision tree algorithm, inputs the training data and the target scenario into the decision tree network, obtains the predicted category of the training data, calculates the network loss value of the decision tree network according to the predicted category and the target category, adjusts the breadth and depth of the decision tree network until the network loss value no longer decreases, and obtains the master data recognition model.

[0176] The decision tree algorithm may be an ID3 decision tree algorithm or a CART decision tree algorithm. The present invention does not limit the specific algorithm of the decision tree algorithm.

[0177] When receiving a master data recognition request, the acquisition unit 110 acquires the data to be recognized and the scene to be recognized according to the master data recognition request.

[0178] In at least one embodiment of the present invention, the master data identification request may be generated by a user who has a master data identification requirement, or may be generated by a machine at a specific time point.

[0179] The data to be identified refers to data that needs to be identified as master data.

[0180] The scenario to be identified refers to a specific scenario that requires master data identification.

[0181] In at least one embodiment of the present invention, the acquiring unit 110 acquires the data to be identified and the scene to be identified according to the master data identification request, including:

[0182] Parsing the master data identification request message to obtain data information carried in the message;

[0183] extracting information indicating a scene from the data information as the scene to be identified, and extracting information indicating a path from the data information as a storage path;

[0184] All data are acquired from the storage path as the data to be identified.

[0185] By parsing the message, since there is no need to parse the entire request, the efficiency of obtaining the data information is improved, thereby improving the efficiency of obtaining the data to be identified and the scene to be identified.

[0186] The input unit 113 inputs the data to be identified and the scene to be identified into the master data recognition model to obtain scene master data of the scene to be identified.

[0187] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned scenario master data, the above-mentioned scenario master data can also be stored in a node of a blockchain.

[0188] In at least one embodiment of the present invention, the scene master data refers to the master data belonging to the scene to be identified in the data to be identified.

[0189] In at least one embodiment of the present invention, the input unit 113 inputs the data to be identified and the scene to be identified into the master data recognition model to obtain the scene master data of the scene to be identified. The method of the generation unit 112 constructing the master data recognition model according to the target category and the target scene is an inverse process to each other, and the present invention will not elaborate on this.

[0190] It can be seen from the above technical solution that the present invention can comprehensively obtain the feature data for clustering through the multiple preset data sources, which is beneficial to improving the cross-domain capability of the master data recognition model. At the same time, by clustering the feature data, the feature data can be accurately divided into multiple data groups, and then the master data recognition model is generated according to the multiple data groups and the data category training, which can improve the recognition accuracy of the master data recognition model, and then use the master data recognition model to identify the data to be identified, thereby improving the accuracy of the scene master data. In addition, since there is no need for manual analysis of the data to be identified, the recognition efficiency of the scene master data can be improved.

[0191] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention for implementing the master data identification method.

[0192] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12 , a processor 13 , and computer-readable instructions stored in the memory 12 and executable on the processor 13 , such as a master data identification program.

[0193] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, buses, etc.

[0194] The processor 13 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the entire electronic device 1 using various interfaces and lines, and executing the operating system of the electronic device 1 as well as various installed applications, program codes, etc.

[0195] Exemplarily, the computer-readable instructions may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions may be divided into an acquisition unit 110, an extraction unit 111, a generation unit 112, an input unit 113, a clustering unit 114, a construction unit 115, a determination unit 116, and an adjustment unit 117.

[0196] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements the various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the electronic device. The memory 12 can include non-volatile and volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, at least one disk storage device, a flash memory device, or other storage devices.

[0197] The memory 12 may be an external memory and / or an internal memory of the electronic device 1. Furthermore, the memory 12 may be a physical memory, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0198] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned embodiment methods by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0199] The computer-readable instructions include computer-readable instruction codes, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer-readable instruction codes, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), or a random access memory (RAM).

[0200] The blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed master data identification, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0201] Combine Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement a master data identification method, and the processor 13 can execute the computer-readable instructions to implement:

[0202] Get field features in multiple scene objects;

[0203] extracting target features from the field features;

[0204] Acquire feature data corresponding to the target feature from multiple preset data sources and perform clustering processing to obtain multiple data groups and data categories corresponding to each data group;

[0205] generating a master data recognition model based on the plurality of data groups and the data categories;

[0206] When receiving a master data identification request, obtaining the data to be identified and the scene to be identified according to the master data identification request;

[0207] The data to be identified and the scene to be identified are input into the master data identification model to obtain scene master data of the scene to be identified.

[0208] Specifically, the specific implementation method of the processor 13 for the above computer readable instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0209] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0210] The computer-readable storage medium stores computer-readable instructions, wherein the computer-readable instructions, when executed by the processor 13, are used to implement the following steps:

[0211] Get field features in multiple scene objects;

[0212] extracting target features from the field features;

[0213] Acquire feature data corresponding to the target feature from multiple preset data sources and perform clustering processing to obtain multiple data groups and data categories corresponding to each data group;

[0214] generating a master data recognition model based on the plurality of data groups and the data categories;

[0215] When receiving a master data identification request, obtaining the data to be identified and the scene to be identified according to the master data identification request;

[0216] The data to be identified and the scene to be identified are input into the master data identification model to obtain scene master data of the scene to be identified.

[0217] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0218] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0219] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0220] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A master data identification method, characterized in that: The master data identification method comprises: Get field features in multiple scene objects; extracting target features from the field features; Acquire feature data corresponding to the target feature from a plurality of preset data sources and perform clustering processing to obtain a plurality of data groups and a data category corresponding to each data group, wherein the plurality of preset data sources include databases corresponding to the plurality of scene objects and a shared database corresponding to a plurality of enterprise departments; Generating a master data recognition model based on the multiple data groups and the data categories includes: for each data group, identifying a source system of data in the data group; counting the number of systems of the source system; generating a standard number based on the number of sources and a preset ratio of the multiple preset data sources; determining a data group corresponding to a number of systems greater than the standard number as a target group; determining a data category corresponding to the target group as a target category, and determining a scenario object corresponding to a field feature identical to the target category as a target scenario; and constructing the master data recognition model based on the target category and the target scenario; When receiving a master data identification request, obtaining the data to be identified and the scene to be identified according to the master data identification request; The data to be identified and the scene to be identified are input into the master data identification model to obtain scene master data of the scene to be identified.

2. The master data identification method according to claim 1, wherein: The acquiring of field features in multiple scene objects includes: Acquire all scenes in a preset scene library as the multiple scene objects; The field corresponding to each scene object is obtained from the field library as the scene field, and the feature corresponding to the scene object is obtained as the field feature.

3. The master data identification method according to claim 1, wherein: The feature data corresponding to the target feature is obtained from multiple preset data sources and clustered to obtain multiple data groups and data categories corresponding to each data group, including: Constructing a regular expression based on the target feature; Searching the plurality of preset data sources in parallel based on the regular expression to obtain a plurality of initial data; Cleaning the multiple initial data to obtain the feature data; Acquire target clustering models corresponding to the multiple scene objects; The characteristic data is clustered based on the target clustering model to obtain the multiple data groups and the data category corresponding to each data group.

4. The master data identification method according to claim 3, wherein: Before obtaining the target clustering models corresponding to the multiple scene objects, the master data identification method further includes: Acquiring scene training data from the plurality of scene objects; Performing clustering processing on the scene training data based on preset clustering models constructed by multiple preset clustering algorithms to obtain multiple data sets for each preset clustering model, the multiple data sets including a first data set and a second data set; For each preset clustering model, a first covariance matrix is ​​constructed based on any two data in each data set; Constructing a second covariance matrix based on any data in the first data set and any data in the second data set; generating an algorithm score for each preset clustering algorithm according to the first covariance matrix and the second covariance matrix; Determine the preset clustering model with the largest algorithm score as the initial clustering model; The initial clustering model is adjusted according to the scenario training data until the loss value of the initial clustering model no longer decreases, thereby obtaining the target clustering model.

5. The master data identification method according to claim 1, wherein: Acquiring the data to be identified and the scene to be identified according to the master data identification request includes: Parsing the master data identification request message to obtain data information carried in the message; extracting information indicating a scene from the data information as the scene to be identified, and extracting information indicating a path from the data information as a storage path; All data are acquired from the storage path as the data to be identified.

6. A master data identification device, characterized in that: The master data identification device comprises: An acquisition unit, used to acquire field features in multiple scene objects; an extraction unit, configured to extract target features from the field features; The acquisition unit is further configured to acquire feature data corresponding to the target feature from a plurality of preset data sources and perform clustering processing to obtain a plurality of data groups and a data category corresponding to each data group, wherein the plurality of preset data sources include databases corresponding to the plurality of scene objects and a shared database corresponding to a plurality of enterprise departments; A generating unit is configured to generate a master data recognition model based on the multiple data groups and the data categories, including: for each data group, identifying a source system of data in the data group; counting the number of systems of the source system; generating a standard number based on the number of sources and a preset ratio of the multiple preset data sources; determining a data group corresponding to a number of systems greater than the standard number as a target group; determining a data category corresponding to the target group as a target category, and determining a scenario object corresponding to a field feature identical to the target category as a target scenario; and constructing the master data recognition model based on the target category and the target scenario. The acquisition unit is further configured to, upon receiving a master data recognition request, acquire the data to be recognized and the scene to be recognized according to the master data recognition request; An input unit is used to input the data to be identified and the scene to be identified into the master data recognition model to obtain the scene master data of the scene to be identified.

7. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor is configured to execute computer-readable instructions stored in the memory to implement the master data identification method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the master data identification method according to any one of claims 1 to 5.

Images