Data processing method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: acquiring a plurality of data tags corresponding to data assets, and classifying the plurality of data tags; summarizing label attributes of the data labels under the label category in the data label category system, and determining a summarizing result as a display category of the data map; determining a target data label meeting an association rule with the display classification in the data label category system, and establishing an association relationship between the target data label and the display classification; obtaining a to-be-displayed first data asset, determining a first target data label to which the first data asset belongs, and determining a first display classification to which the first target data label belongs according to the association relationship; and displaying the first data asset in a first display classification in the data map. According to the method and the device, the technical problem that a related data map construction method cannot quickly and flexibly display data assets of multiple dimensions is solved.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and specifically, to a data processing method and device, a non-volatile storage medium, and an electronic device. Background Art

[0002] In recent years, with the continuous development of mobile Internet technology, data has gradually become one of the core competitiveness of enterprises and an important asset. Data maps, as a panoramic, multi-dimensional, visual method of displaying the status, classification, and distribution of data, can help enterprises understand the overall situation of their own data in a timely manner and assist in corporate decision-making.

[0003] At present, the method of building data maps is generally to display data maps based on the metadata of data assets, but pre-defining metadata takes a long time, is slow to deploy, is difficult for business personnel to understand, and is relatively inflexible. The method of using metadata to build data maps cannot quickly and flexibly display data types and relationships in multiple dimensions, and also has certain limitations on using data for data analysis and business scenario applications.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a data processing method and device, a non-volatile storage medium, and an electronic device to at least solve the technical problem that the method for constructing a related data map cannot quickly and flexibly display data assets of multiple dimensions.

[0006] According to one aspect of the present application, a data processing method is provided, including: obtaining multiple data tags corresponding to a data asset, classifying the multiple data tags to obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; summarizing the tag attributes of the data tags under the tag category in the data tag category system, and determining the summary results as display categories of a data map; determining a target data tag that meets an association rule with the display category in the data tag category system, and establishing an association relationship between the target data tag and the display category; obtaining a first data asset to be displayed, determining a first target data tag to which the first data asset belongs, and determining a first display category to which the first target data tag belongs based on the association relationship; and displaying the first data asset in the first display category in the data map.

[0007] Optionally, multiple data tags are classified to obtain a data tag category system, including: determining the preset target objects and the relationship between the preset target objects as the root directory, determining the classification dimension of the data tag as the branch directory, and determining the label identifier and label value of the data tag as the leaf directory; according to the root directory, the branch directory and the leaf directory, multiple data tags are classified to obtain a data tag category system.

[0008] Optionally, summarizing the tag attributes of data tags under the tag category in the data tag category system includes: summarizing tags with a first attribute as the tag attribute to obtain a display classification based on data standards, wherein the first attribute includes at least one of the following: data standard number, data standard name, and keywords corresponding to the data standard; summarizing tags with a second attribute as the tag attribute to obtain a display classification based on data management, wherein the second attribute includes at least one of the following: source system, system management department, data ownership department, business demand department, data demand department; summarizing tags with a third attribute as the tag attribute to obtain a display classification based on data security, wherein the third attribute includes at least one of the following: security classification, security level, and authorization status; summarizing tags with a fourth attribute as the tag attribute to obtain a display classification based on data business, wherein the fourth attribute includes at least one of the following: business type, business management department, business process stage, online management information, and offline management information; summarizing tags with a fifth attribute as the tag attribute to obtain a display classification based on data usage, wherein the fifth attribute includes at least one of the following: number of data queries and number of calls to the application where the data is located.

[0009] Optionally, determining and displaying target data tags that meet association rules in a data tag category system includes: obtaining a data tag set in the data tag category system; counting the number of occurrences of each data tag in the data tag set to obtain a first candidate item set; determining a first frequent item set in the first candidate item set based on the number of occurrences of each data tag in the data tag set, wherein the first frequent item set is a set of data tags whose frequency of occurrence in the data tag set is greater than or equal to a preset minimum support threshold; combining the first frequent item set to obtain a second candidate item set; counting the number of occurrences of each data tag in the second candidate item set; and determining a first frequent item set based on the number of occurrences of each data tag in the data tag set. According to the number of times the label appears in the second candidate item set, the second frequent item set is determined in the second candidate item set, wherein the second frequent item set is a set of data labels whose frequency of appearance in the second candidate item set is greater than or equal to a preset minimum support threshold; repeatedly perform the above statistical steps, determination steps and combination steps until the candidate item set is an empty set, and obtain the target frequent item set; for each target frequent item set, generate all possible initial association rules; calculate the confidence of each initial association rule; determine the association rules whose confidence is greater than or equal to the minimum confidence threshold in the initial association rules; determine and display the target data labels that meet the association rules in the data label category system.

[0010] Optionally, determining a target data tag that satisfies association rules with a display classification in a data tag category system includes: for each display classification, determining a plurality of first association rules that have the longest length and the highest support count and match the data tag, and for each first association rule, determining a classification count based on the length of the association rule and the count of a single item in the association rule to obtain a plurality of classification counts corresponding to each display classification; for each display classification, determining the first association rule with the largest classification count as the association rule corresponding to the display classification; and determining a target data tag that satisfies association rules with the display classification in the data tag category system.

[0011] Optionally, displaying the first data asset in a first display category in a data map includes: obtaining user usage information of the data asset, wherein the usage information includes at least one of the following: data query and access records, data download and export records, data tag modification and creation records, and browsing path information on the data map; dividing users into multiple different behavior groups based on the usage information; and displaying the first data asset in the first display category in the data map in different ways for different behavior groups.

[0012] According to another aspect of the present application, a data processing device is also provided, including: an acquisition module, used to acquire multiple data tags corresponding to a data asset, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; a first determination module, used to summarize the tag attributes of the data tags under the tag category in the data tag category system, and determine the summary results as the display category of the data map; an establishment module, used to determine the target data tag that meets the association rule with the display category in the data tag category system, and establish an association relationship between the target data tag and the display category; a second determination module, used to acquire a first data asset to be displayed, determine the first target data tag to which the first data asset belongs, and determine the first display category to which the first target data tag belongs based on the association relationship; a display module, used to display the first data asset in the first display category in the data map.

[0013] According to another aspect of the present application, a non-volatile storage medium is provided, the storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0014] According to another aspect of the present application, an electronic device is provided, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0015] According to yet another aspect of the present application, a computer program is provided, wherein the above data processing method is implemented when the computer program is executed by a processor.

[0016] According to another aspect of the present application, a computer program product is provided, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above data processing method is implemented.

[0017] In the present application, a plurality of data tags corresponding to a data asset are obtained, and the plurality of data tags are classified to obtain a data tag category system, wherein the data tag category system includes a plurality of different tag categories; the tag attributes of the data tags under the tag category are summarized in the data tag category system, and the summary results are determined as display categories of the data map; a target data tag that satisfies an association rule with the display category is determined in the data tag category system, and an association relationship between the target data tag and the display category is established; a first data asset to be displayed is obtained, a first target data tag to which the first data asset belongs is determined, and a first display category to which the first target data tag belongs is determined based on the association relationship; and the first data asset is displayed in the first display category in the data map, thereby achieving the purpose of quickly and flexibly displaying data assets of multiple dimensions, thereby realizing the technical effect of improving data governance efficiency, and further solving the technical problem that the construction method of the relevant data map cannot quickly and flexibly display data assets of multiple dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a data label category system according to an embodiment of the present application;

[0021] Figure 3 is a structural diagram of a data processing device according to an embodiment of the present application;

[0022] Figure 4 It is a hardware structure block diagram of a computer terminal according to a data processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present application, a method embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0027] Step S101, obtain multiple data tags corresponding to the data assets, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories.

[0028] In step S101, firstly, multiple data tags associated with data assets are obtained through data analysis and business needs sorting. Data tags can be attribute descriptions of data assets, such as data type, source, update frequency, etc. Then, these data tags are classified according to certain logic and business needs, and a data tag category system is constructed to form a hierarchical tag directory for easy management and query.

[0029] Step S102, summarizing the label attributes of the data labels under the label category in the data label category system, and determining the summary result as the display category of the data map.

[0030] In step S102, the data tags under each tag category are summarized and key tag attributes are extracted, such as tag definition, value range, application, etc. These attributes will become the basis for data map display classification. Display classification is a high-level category on the data map, which will be determined based on the summary results of tag attributes. The purpose is to present different dimensions and levels of data assets on the data map, so that users can filter and view according to their needs.

[0031] Step S103, determining a target data tag that satisfies an association rule with a display category in the data tag category system, and establishing an association relationship between the target data tag and the display category.

[0032] In step S103, by using the association rule analysis in data mining technology, the statistical correlation between the target data tag and the display category is found, that is, which data tags are frequently associated with a specific display category. These data tags that meet the association rules are called target data tags. Next, the association relationship between the target data tag and the display category is established. This association relationship will guide how the data assets are classified and displayed on the data map.

[0033] Step S104, obtaining a first data asset to be displayed, determining a first target data tag to which the first data asset belongs, and determining a first display category to which the first target data tag belongs based on the association relationship.

[0034] In step S104, a data asset to be displayed (e.g., a first data asset) is first obtained. Then, based on the characteristics of the data asset, its related data tags (e.g., a first target data tag) are determined, and then, based on the previously established association relationship, it is determined to which display category (e.g., the first display category) these data tags should belong, thereby determining the display position of the data asset on the data map.

[0035] Step S105: display the first data asset in a first display category in the data map.

[0036] In step S105, the first data asset is placed at a corresponding position on the data map according to the determined display category, completing the visual display of the data asset. Each category on the data map will centrally display the data assets associated with it, so that users can easily view and analyze data by category, improving the accessibility and usability of the data.

[0037] According to the above steps, multiple data tags corresponding to the data assets are obtained, and the multiple data tags are classified to obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; the tag attributes of the data tags under the tag categories are summarized in the data tag category system, and the summary results are determined as the display categories of the data map; the target data tags that meet the association rules with the display categories are determined in the data tag category system, and the association relationship between the target data tags and the display categories is established; the first data asset to be displayed is obtained, the first target data tag to which the first data asset belongs is determined, and according to the association relationship, the first display category to which the first target data tag belongs is determined; the first data asset is displayed in the first display category in the data map, so as to achieve the purpose of quickly and flexibly displaying data assets of multiple dimensions, thereby realizing the technical effect of improving data governance efficiency.

[0038] The following Figure 1 The steps shown are exemplary and explanatory.

[0039] Figure 2 is a schematic diagram of a data label category system according to an embodiment of the present application, such as Figure 2 As shown, multiple data tags are classified to obtain a data tag category system, which can be achieved by the following method: the preset target objects and the relationship between the preset target objects are determined as the root directory, the classification dimension of the data tag is determined as the branch directory, and the label identifier and label value of the data tag are determined as the leaf directory; according to the root directory, the branch directory and the leaf directory, multiple data tags are classified to obtain a data tag category system.

[0040] In the above embodiment, first, data labels are designed according to the characteristics of data content and the needs of business scenarios. Identify which attributes are critical and valuable and can be used as labels. When designing labels, the uniqueness, descriptiveness and relevance of labels to the business should be considered to ensure that labels can accurately reflect the important characteristics of data assets. Secondly, the data label system is divided into three categories: "people", "things" and "relationships". This classification is based on the object type described by the data, making the label system more intuitive and easy to understand. Among them, "people" labels are used to describe information related to personnel, "things" labels are used to describe the characteristics of entities or non-personnel objects, and "relationship" labels are used to represent the connection between different data assets. Finally, a tree structure is used as the modeling basis of the label category system. This structure has clear hierarchical levels and is easy to manage and expand. The tree structure consists of a root directory, a branch directory and a leaf / flower directory, which correspond to objects, label categories and labels and attributes respectively. This classification method can clearly show the hierarchical relationship between data labels, so that the label system is systematic. Specifically, in the tree structure, the object is the root directory and is the highest level of the label system. Further refine the three categories of "people", "things" and "relationships" to determine specific objects. For example, "people" can be subdivided into "customers", "employees", etc., "things" can be subdivided into "products", "equipment", etc., and "relationships" can be subdivided into "purchase relationships", "usage relationships", etc. The branch directory represents the tag category, which is a classification of tags. After the object classification is determined, create corresponding tag categories according to business needs and data characteristics. For example, under the object of "people", you can create tag categories such as "basic information", "social relations", and "economic status", while under the object of "things", you can create categories such as "physical properties", "usage history", and "maintenance records". The leaf / flower directory corresponds to specific tags and attributes. Under each tag category, define specific tags and their attributes. For example, under the "basic information" category, there can be tags such as "name", "age", and "gender". Each tag will also have specific attribute values. For example, the attribute value of the "age" tag can be "25 years old", "30 years old", etc.

[0041] Through the above hierarchical structure, multiple data tags can be systematically classified and organized to form a data tag category system. This system can help business personnel and data managers understand and manage data more intuitively, and also provide a clear framework and path for subsequent data map construction, data query and data analysis. For example, when you need to view all data assets related to "data security", you can quickly locate the "data security" section in the branch directory, and then view all leaf directories (specific data tags) and their associated data assets, to achieve efficient data management and analysis.

[0042] According to some other optional embodiments of the present application, summarizing the tag attributes of data tags under the tag category in the data tag category system can be achieved by the following method: summarizing the tags whose tag attribute is the first attribute to obtain a display classification based on data standards, wherein the first attribute includes at least one of the following: data standard number, data standard name, and keywords corresponding to the data standard; summarizing the tags whose tag attribute is the second attribute to obtain a display classification based on data management, wherein the second attribute includes at least one of the following: source system, system management department, data ownership department, business demand department, data demand department; summarizing the tags whose tag attribute is the third attribute to obtain a display classification based on data security, wherein the third attribute includes at least one of the following: security classification, security level, and authorization status; summarizing the tags whose tag attribute is the fourth attribute to obtain a display classification based on data business, wherein the fourth attribute includes at least one of the following: business type, business management department, business process stage, online management information, and offline management information; summarizing the tags whose tag attribute is the fifth attribute to obtain a display classification based on data usage, wherein the fifth attribute includes at least one of the following: number of data queries and number of calls to the application where the data is located.

[0043] In the above embodiment, labels of different attributes are summarized to obtain five types of display classifications, namely: data standards, data management, data security, data services, and data usage.

[0044] Display classification based on data standards: Summarize all tags with the first attribute (such as data standard number, data standard name, keyword, etc.), which usually describe the standard definition, classification and identification of data. The summary results will form a display classification "based on data standards" on the data map. This classification helps users quickly locate all data assets that follow the same or similar data standards, facilitating standardized management and consistency checks.

[0045] Display classification based on data management: Summarize all tags with the second attribute (source system, system management department, data ownership department, etc.). These attributes mainly describe the source of data and management responsibility. The display classification "based on data management" obtained after summary can let users understand the production environment and management structure of data assets, which is of great significance for data governance, permission control and process optimization.

[0046] Display classification based on data security: Summarize all tags with third attributes (such as security classification, security level, and authorization status). These attributes are closely related to data security, sensitivity, and access control. Through summarization, a display classification based on data security can be constructed, which helps users understand the security status of data assets, conduct risk assessments, and formulate security policies.

[0047] Display classification based on data business: Summarize all tags with the fourth attribute (such as business type, business management department, business process stage, etc.). These attributes reflect the role and value of data assets in the business process. The display classification "based on data business" obtained after summary allows users to understand data assets from a business perspective, which is crucial for business decision-making and process optimization.

[0048] Display classification based on data usage: Summarize all tags with the fifth attribute (such as the number of data queries, the number of application calls, etc.). These attributes quantify the frequency and popularity of data usage. The display classification "based on data usage" formed after the summary helps to identify key data and hot data, which is directly helpful for resource optimization, performance monitoring and user experience improvement.

[0049] In some optional embodiments of the present application, determining and displaying target data tags that meet association rules in a data tag category system can be achieved by the following method: obtaining a data tag set in the data tag category system; counting the number of occurrences of each data tag in the data tag set to obtain a first candidate item set; determining a first frequent item set in the first candidate item set based on the number of occurrences of each data tag in the data tag set, wherein the first frequent item set is a set of data tags whose frequency of occurrence in the data tag set is greater than or equal to a preset minimum support threshold; combining the first frequent item set to obtain a second candidate item set; counting the number of occurrences of each data tag in the second candidate item set The method comprises the following steps: determining a second frequent item set in the second candidate item set according to the number of occurrences of each data tag in the second candidate item set, wherein the second frequent item set is a set of data tags whose occurrence frequency in the second candidate item set is greater than or equal to a preset minimum support threshold; repeatedly performing the above statistical steps, determination steps and combination steps until the candidate item set is an empty set, and obtaining a target frequent item set; for each target frequent item set, generating all possible initial association rules; calculating the confidence of each initial association rule; determining association rules whose confidence is greater than or equal to the minimum confidence threshold in the initial association rules; determining and displaying target data tags that meet the association rules in the data tag category system.

[0050] In the above embodiment, first, all data tags are extracted from the constructed data tag category system to form a data tag set. These data tags contain various attributes and descriptions of data assets and are the basis for subsequent statistics and analysis. Next, the number of occurrences of each data tag in the data tag set is counted, and these number of occurrences reflects the importance or relevance of the data tag. According to the preset minimum support threshold, the data tags with an occurrence frequency greater than or equal to the threshold are screened to form a first frequent item set (i.e., the first candidate item set). Frequent item sets that meet the minimum support threshold are determined from the first candidate item set, and these frequent item sets are the basis for subsequent combinations. The first frequent item set is combined to obtain a second candidate item set: the first frequent item set obtained in the previous step is combined to generate a candidate item set containing more data tags. For example, if there are two data tags "gender" and "age" in the first frequent item set, a composite tag such as "gender-age" may be obtained after the combination. The number of occurrences of the data tags is counted again to determine the second frequent item set: the data tag combinations in the second candidate item set are counted, and the combinations that meet the conditions are screened out according to the minimum support threshold to form the second frequent item set.

[0051] Repeat the process of combining and determining frequent itemsets: The above steps will be repeated until the candidate itemset is empty, that is, no more frequent itemsets that meet the minimum support threshold can be generated. This ensures that all frequent itemsets that meet the conditions are found, which are evidence of strong associations between data labels.

[0052] For each target frequent item set, all possible association rules are generated. The form of the association rule is "if A then B", where A and B are both items in the data label set, and A and B do not overlap. Confidence is a measure of the conditional probability of rule A=>B, that is, the proportion of data containing A that also contains B. Calculate the confidence of each initial association rule to evaluate the strength of the rule. According to the preset minimum confidence threshold, filter out association rules whose confidence meets the conditions. These rules reflect more stable associations between data labels. Finally, based on the above-screened association rules, determine which data labels have a strong association with the display classification of the data map. These data labels are the target data labels, which will be used to construct and display the data map to achieve multi-dimensional classification and visualization of the data.

[0053] As some optional embodiments of the present application, determining the target data tag that satisfies the association rules with the display classification in the data tag category system can be achieved by the following method: for each display classification, determining multiple first association rules that have the longest length and the highest support count that match the data tag; for each first association rule, determining the classification count based on the association rule length and the count of a single item in the association rule to obtain multiple classification counts corresponding to each display classification; for each display classification, determining the first association rule with the largest classification count as the association rule corresponding to the display classification; determining the target data tag that satisfies the association rules with the display classification in the data tag category system.

[0054] In the above embodiment, under each display classification, data mining technology is used to find multiple first association rules that match the data label. These rules should meet two conditions: one is that the length of the rule is the longest, that is, the number of data labels contained is the largest; the other is that the support count of the rule is the highest, which means that the frequency of occurrence in the data set is the highest. The purpose of this step is to find the association rule that best describes the characteristics of the classification. For each determined first association rule, the classification count is calculated based on the length of the association rule and the count of a single item (data label) in the association rule. The calculation formula for the classification count takes into account the rule length and the number of occurrences of each label to comprehensively evaluate the coverage and importance of the rule. This step ensures that the classification count not only reflects the breadth of coverage of the rule, but also considers the independent value of each label in the rule. After obtaining multiple classification counts under each display classification, the first association rule with the largest classification count is selected as the association rule of the display classification. This is intended to select the most representative and influential rules to ensure that the display classification on the data map can accurately reflect the key features and relationships of data assets. In the data tag category system, which data tags are closely related to the classification is determined according to the association rules of the display classification. These data labels are the target data labels, which will be highlighted when the data map is constructed to ensure that users can intuitively understand the data assets under this category.

[0055] In some optional embodiments of the present application, displaying the first data asset in the first display category in the data map can be achieved by the following method: obtaining user usage information of the data asset, wherein the usage information includes at least one of the following: data query and access records, data download and export records, data label modification and creation records, browsing path information on the data map; dividing users into multiple different behavior groups based on the usage information; and displaying the first data asset in the first display category in the data map in different ways for different behavior groups.

[0056] In the above embodiment, first, various operations and behavior data of users in the process of using data assets are collected and recorded, including but not limited to data query and access records, data download and export records, data tag modification and creation records, and browsing path information on the data map. This information can reflect the user's frequency of use, depth of use, and usage preferences for specific data assets. Then, based on the collected user usage information, users are divided into different behavior groups through data analysis and clustering algorithms. Each group represents a specific usage pattern or preference. For example, users who frequently perform data queries may be classified as the "data analyst" group, while users who frequently download and export data may belong to the "data user" group. This division helps to understand the needs and behavioral characteristics of different users. For different behavior groups, different ways are used to display the first data asset in the data map. This means that the display strategy and content will be adjusted according to the characteristics of the user group to provide the most relevant and valuable information. For example, for the "data analyst" group, the data map may highlight the statistical analysis and visualization of the data, while for the "data user" group, it may focus more on providing direct access and download links to the data.

[0057] The present application also provides a method for constructing a data map based on data tags, the method comprising the following steps:

[0058] Step S1, constructing a data label and data asset label category system corresponding to each data asset.

[0059] According to different business domains, application systems, importance, data distribution, update frequency, asset type, security level, confidentiality level and other dimensions, a data asset tag category system is built for data assets. Data tags include basic information such as the tag category, tag definition, tag logic, and tag value. At the same time, the data application scenario and service call status of the tag can be seen in the tag details.

[0060] It should be noted that after constructing the data tags, each data asset directory needs to be mapped with each data tag, and because one data tag may correspond to multiple data assets, and one data asset may also correspond to multiple data tags, therefore, while mapping the relationships between data tags and data assets, the data tags are also combined to assist in screening out data assets that meet the specified characteristics in the target display dimension.

[0061] Step S2, using multiple tag attributes under multiple tag categories to determine the final display classification of the data map.

[0062] Each data asset has multiple attributes and also belongs to multiple categories. There are two ways to classify the data tags. One is the data classification method, such as identifying and classifying according to the source, update frequency, and department of the data; the other is to redescribe or even reorganize the content of the data, such as the theme of organization based on characteristics, redefining a certain attribute from a business perspective, etc. After classifying the data tags, the display classification of the data map is determined according to the classification. The final display classification may include data standards, data management, data usage, data security, data business, etc.

[0063] Step S3, configuring the association relationship between each data tag and the final display classification of the data map according to the association rules and algorithms.

[0064] First, association rule mining is used to discover associations or correlations in the database where the data assets are located. That is, frequently occurring attribute value sets are found from the database, and these value sets are used to create descriptive association rules. In this process, support and confidence are measures of the association between the two rules. Then, the Apriori algorithm is used to mine from the database those association rules whose support and confidence are not less than the given minimum support threshold and minimum confidence threshold.

[0065] Association rules are to find dependencies that meet certain conditions from the database. The database is denoted by D. Generally speaking, D is a transaction record, and each data label in each record is an item. The main objects of association rule mining are In the above equation, the rule of support = s%, confidence = c% is satisfied.

[0066] Support: In database D, the support of a data item set (i.e., a data label set) X, support(X), is the ratio of the number of transactions in D containing X to the total number of transactions in D:

[0067]

[0068] The degree of association rule X=>Y is equal to the support of item set X∪Y:

[0069]

[0070] The support represents the frequency of the rule, and the minimum support is represented by minsup. If support(X) is greater than or equal to the specified minimum support minsup, then X is a frequent item set.

[0071] Confidence: If X, Y are item sets, and Implicit The support of the item set X∪Y is called the association rule. The support degree of Association Rules The confidence of is denoted as confidence(X=>Y).

[0072]

[0073] The confidence is used to measure the credibility of the association rule. The minimum confidence is represented by minconf. If the data item set X satisfies support(X)≥minsup, then X is a frequent data item set. If the rule X=>Y also satisfies confidence(X=>Y)>=minconf, then the association rule It is a strong association rule.

[0074] Association pattern: Assume that a set of items X = (x1, x2, x3, ..., xn) is given, and its all-confidence is expressed as follows:

[0075]

[0076]

[0077] Given a minimum association confidence (min_all-confidence), if all-confidence(X)≥min_all-confidence, then X is called an association pattern.

[0078] Furthermore, according to the Apriori algorithm, frequent item sets are generated based on support; strong association rules are generated based on credibility, and the core is the frequent item sets generated by support. Among them, the algorithm: Apriori uses layer-by-layer iteration based on candidate generation to find frequent item sets. Input: database D; minimum support threshold minsup. Output: frequent item set Z in D.

[0079] The basic process of the Apriori algorithm is: (1) First, scan the database D once, count the number of times each item appears, and form a candidate 1-item set; (2) Filter out frequent 1-item sets based on the minsup threshold; (3) Combine frequent 1-item sets to form candidate 2-item sets; (4) Scan the database D a second time, count each candidate 2-item set, and filter out frequent 2-item sets; (5) Repeat the above process until the candidate item set is empty; (6) Generate management rules based on the generated frequent item sets by calculating the corresponding confidence.

[0080] If an item set contains k different items, it is called a k-item set. The candidate k-item set is denoted as Ck, and the frequent k-item set is denoted as Lk.

[0081] According to the above algorithm, for each category, find all the association rules A1: (a11, a12, ..., a1j), A2 (a21, a22, ..., a2j), ... Ai: (ai1, ai2, ..., aij) that match the test data, with the longest length and the highest support count, where there is no intersection between all the association rules found, and then calculate the classification count by the formula:

[0082] C=C1×Z1×0.7+(c 11 +c 12 +…+c 1j )×0.3

[0083] +C2×Z2×0.7+(c 21 +c 22 +…+c 2j )×0.3

[0084] +C1×Z1×0.7+(c i1 +c i2 +…+c ij )×0.3

[0085] Among them, C is the classification count, Ci is the association rule technology, Zi is the association rule length, and Cij is the count of a single item in the association rule.

[0086] Finally, the classification counts of each category are compared, and the category with the larger classification count is the result category. The following takes the 8 data in Table 1 as an example:

[0087]

[0088]

[0089] Suppose the training set is data X1-X7, with support of 12% and confidence of 71%.

[0090] X8 is classified based on the association rules generated by the training set. The categories generated by the training set are: 91, 92.

[0091] Thus, for category 91: A1: (8) C1 = 3, Z1 = 1, C11 = 0; A2: (4) C2 = 1, Z2 = 1, C21 = 0; A3: (19) C3 = 1, Z3 = 1, C31 = 0; A4: (83) C4 = 1, Z4 = 1, C41 = 0. According to the formula, the classification count C(91) = 4.2. For category 92: A1: (33, 52, 75, 83), C1 = 1, Z1 = 4, c11 = 0, c12 = 0, c13 = 0, c14 = 0; A2: (83) C2 = 3, Z2 = 1, c21 = 0; A3: (8) C3 = 2, Z3 = 1, c31 = 0. According to the formula, the classification count C(92) = 6.3.

[0092] It is obtained that C(91) < C(92), so it can be determined that the category of X8 is 92.

[0093] Thus, based on the above association rules and algorithms, the data labels can be trained, and then the association rules generated by the training can be used for the final display classification. The target display dimensions generated by the data labels this time may include data standards, data management, data usage, data security, data services, etc.

[0094] Step S4, determine the positions of the data labels corresponding to each data asset according to the association relationship.

[0095] It should be noted that since the relationship mapping between the data assets and the data labels has been performed in step S1, the positions of the data labels corresponding to each data asset in the data asset catalog can be determined according to the association relationship.

[0096] Step S5, set the collection and access of the data assets corresponding to each data label on the data map according to the positions.

[0097] According to the positions of the data labels, embed the data assets corresponding to each data label on the data map in the data map in the form of a data label service interface, and perform data collection and access of the data assets for the final display classification.

[0098] Step S6, view and analyze different data types according to the data map.

[0099] According to the above steps, it can be seen that in this application, by constructing a data map based on data labels, the data asset catalog is labeled. Based on the labels, the dimensional diversity and flexibility of multi-dimensional free combination of the data map are realized. At the same time, the label is defined as a description method of an object, becoming an organizational form of classification and description that is easier to understand and identify, improving the readability of the data map for business personnel, and thus improving work efficiency.

[0100] Figure 3is a structural diagram of a data processing device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0101] The acquisition module 31 is used to acquire multiple data tags corresponding to the data assets, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories.

[0102] The first determination module 32 is used to summarize the label attributes of the data labels under the label category in the data label category system, and determine the summary results as the display categories of the data map.

[0103] The establishment module 33 is used to determine the target data tag that meets the association rule with the display classification in the data tag category system, and establish the association relationship between the target data tag and the display classification.

[0104] The second determination module 34 is used to obtain the first data asset to be displayed, determine the first target data tag to which the first data asset belongs, and determine the first display category to which the first target data tag belongs according to the association relationship.

[0105] The display module 35 is used to display the first data asset in a first display category in the data map.

[0106] Optionally, the acquisition module 31 is also used to perform the following steps: determine the preset target objects and the relationship between the preset target objects as the root directory, determine the classification dimension of the data label as the branch directory, and determine the label identifier and label value of the data label as the leaf directory; classify multiple data labels according to the root directory, branch directory and leaf directory to obtain a data label category system.

[0107] Optionally, the first determination module 32 is also used to perform the following steps: summarizing the tags whose tag attribute is the first attribute to obtain a display classification based on data standards, wherein the first attribute includes at least one of the following: data standard number, data standard name, and keywords corresponding to the data standard; summarizing the tags whose tag attribute is the second attribute to obtain a display classification based on data management, wherein the second attribute includes at least one of the following: source system, system management department, data ownership department, business demand department, data demand department; summarizing the tags whose tag attribute is the third attribute to obtain a display classification based on data security, wherein the third attribute includes at least one of the following: security classification, security level, and authorization status; summarizing the tags whose tag attribute is the fourth attribute to obtain a display classification based on data business, wherein the fourth attribute includes at least one of the following: business type, business management department, business process stage, online management information, and offline management information; summarizing the tags whose tag attribute is the fifth attribute to obtain a display classification based on data usage, wherein the fifth attribute includes at least one of the following: number of data queries and number of calls to the application where the data is located.

[0108] Optionally, module 33 is established to perform the following steps: obtain a data tag set in the data tag category system; count the number of occurrences of each data tag in the data tag set to obtain a first candidate item set; determine a first frequent item set in the first candidate item set based on the number of occurrences of each data tag in the data tag set, wherein the first frequent item set is a set of data tags whose frequency of occurrence in the data tag set is greater than or equal to a preset minimum support threshold; combine the first frequent item set to obtain a second candidate item set; count the number of occurrences of each data tag in the second candidate item set; and determine a first frequent item set based on the number of occurrences of each data tag in the second candidate item set. The number of occurrences in the set is used to determine the second frequent item set in the second candidate item set, wherein the second frequent item set is a set of data labels whose appearance frequency in the second candidate item set is greater than or equal to a preset minimum support threshold; the above statistical steps, determination steps and combination steps are repeated until the candidate item set is an empty set, and a target frequent item set is obtained; for each target frequent item set, all possible initial association rules are generated; the confidence of each initial association rule is calculated; the association rules whose confidence is greater than or equal to the minimum confidence threshold are determined in the initial association rules; the target data labels that meet the association rules are determined and displayed in the data label category system.

[0109] Optionally, module 33 is established to perform the following steps: for each display category, determine multiple first association rules that have the longest length and the highest support count that match the data label; for each first association rule, determine the classification count based on the length of the association rule and the count of a single item in the association rule to obtain multiple classification counts corresponding to each display category; for each display category, determine the first association rule with the largest classification count as the association rule corresponding to the display category; and determine the target data label that satisfies the association rule with the display category in the data label category system.

[0110] Optionally, the display module 35 is also used to perform the following steps: obtaining user usage information on data assets, wherein the usage information includes at least one of the following: data query and access records, data download and export records, data tag modification and creation records, and browsing path information on the data map; dividing users into multiple different behavior groups based on the usage information; and displaying the first data asset in the first display category in the data map in different ways for different behavior groups.

[0111] It should be noted that the above Figure 3 The modules in the embodiment may be program modules (e.g., a set of program instructions for implementing a specific function) or hardware modules. For the latter, they may be expressed in the following forms, but are not limited thereto: the expression form of the above modules is a processor, or the functions of the above modules are implemented by a processor.

[0112] It should be noted that Figure 3 The preferred implementation of the illustrated embodiment can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0113] Figure 4 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a data processing method. Figure 4 As shown, the computer terminal 40 may include one or more (402a, 402b, ..., 402n are used to illustrate) processors 402 (the processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 404 for storing data, and a transmission module 406 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0114] It should be noted that the one or more processors 402 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 40. As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0115] The memory 404 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 402 executes various functional applications and data processing by running the software programs and modules stored in the memory 404, that is, realizing the above-mentioned data processing method. The memory 404 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 404 may further include a memory remotely arranged relative to the processor 402, and these remote memories may be connected to the computer terminal 40 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0116] The transmission module 406 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 40. In one example, the transmission module 406 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0117] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 40 .

[0118] It should be noted that, in some optional embodiments, the above Figure 4 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 4 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0119] It should be noted that Figure 4 The computer terminal shown is used to execute Figure 1 The data processing method shown, therefore the relevant explanations in the execution method of the above command are also applicable to the electronic device and will not be repeated here.

[0120] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above data processing method.

[0121] A program for a non-volatile storage medium to perform the following functions: obtaining multiple data tags corresponding to a data asset, classifying the multiple data tags to obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; summarizing the tag attributes of the data tags under the tag category in the data tag category system, and determining the summary results as display categories of a data map; determining a target data tag that meets an association rule with the display category in the data tag category system, and establishing an association relationship between the target data tag and the display category; obtaining a first data asset to be displayed, determining a first target data tag to which the first data asset belongs, and determining a first display category to which the first target data tag belongs based on the association relationship; and displaying the first data asset in the first display category in the data map.

[0122] An embodiment of the present application further provides an electronic device, including: a memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the above data processing method is executed when the program is run.

[0123] The processor is used to run a program that performs the following functions: obtain multiple data tags corresponding to a data asset, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; summarize the tag attributes of the data tags under the tag category in the data tag category system, and determine the summary results as the display category of the data map; determine the target data tag that meets the association rule with the display category in the data tag category system, and establish an association relationship between the target data tag and the display category; obtain a first data asset to be displayed, determine a first target data tag to which the first data asset belongs, and determine a first display category to which the first target data tag belongs based on the association relationship; and display the first data asset in the first display category in the data map.

[0124] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0125] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0126] In the above-mentioned embodiments of the present application, the collected information is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary protection measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0131] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Acquire multiple data tags corresponding to the data asset, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; Summarizing the label attributes of the data labels under the label category in the data label category system, and determining the summary results as the display categories of the data map; Determine a target data tag that satisfies an association rule with the display classification in the data tag category system, and establish an association relationship between the target data tag and the display classification; Acquire a first data asset to be displayed, determine a first target data tag to which the first data asset belongs, and determine a first display category to which the first target data tag belongs based on the association relationship; The first data asset is displayed in the first display category in the data map.

2. The method according to claim 1, characterized in that: The plurality of data tags are classified to obtain a data tag category system, including: Determine the preset target objects and the relationship between the preset target objects as the root directory, determine the classification dimension of the data tag as the branch directory, and determine the tag identifier and tag value of the data tag as the leaf directory; According to the root directory, the branch directory and the leaf directory, the plurality of data tags are classified to obtain the data tag category system.

3. The method according to claim 1, characterized in that Summarizing the tag attributes of the data tags under the tag category in the data tag category system includes: Summarizing tags whose tag attribute is a first attribute to obtain a display classification based on a data standard, wherein the first attribute includes at least one of the following: a data standard number, a data standard name, and a keyword corresponding to the data standard; Summarizing the tags whose tag attribute is the second attribute to obtain a display classification based on data management, wherein the second attribute includes at least one of the following: source system, system management department, data ownership department, business demand department, and data demand department; Summarizing tags whose tag attributes are third attributes to obtain display classifications based on data security, wherein the third attribute includes at least one of the following: security classification, security level, and authorization status; Summarizing the tags whose tag attribute is the fourth attribute to obtain a display classification based on data services, wherein the fourth attribute includes at least one of the following: business type, business management department, business process stage, online management information, and offline management information; The tags whose tag attribute is the fifth attribute are summarized to obtain a display classification based on data usage, wherein the fifth attribute includes at least one of the following: the number of data queries and the number of calls of the application where the data is located.

4. The method according to claim 1, characterized in that: Determining a target data tag that satisfies an association rule with the display classification in the data tag category system includes: Obtaining a data tag set in the data tag category system; Counting the number of occurrences of each of the data tags in the data tag set to obtain a first candidate item set; determining a first frequent item set in the first candidate item set according to the number of occurrences of each of the data tags in the data tag set, wherein the first frequent item set is a set of data tags whose occurrence frequency in the data tag set is greater than or equal to a preset minimum support threshold; combining the first frequent item sets to obtain a second candidate item set; counting the number of occurrences of each of the data tags in the second candidate item set; determining a second frequent item set in the second candidate item set according to the number of occurrences of each of the data tags in the second candidate item set, wherein the second frequent item set is a set of data tags whose occurrence frequency in the second candidate item set is greater than or equal to a preset minimum support threshold; Repeat the above statistical steps, determination steps and combination steps until the candidate item set is an empty set and the target frequent item set is obtained; For each target frequent item set, all possible initial association rules are generated; the confidence of each of the initial association rules is calculated; and the association rules whose confidence is greater than or equal to a minimum confidence threshold are determined among the initial association rules; The target data tag that satisfies the association rule with the display classification is determined in the data tag category system.

5. The method according to claim 1, characterized in that Determining a target data tag that satisfies an association rule with the display classification in the data tag category system includes: For each of the display categories, determine a plurality of first association rules that have the longest length and the highest support count and match the data tag, and for each of the first association rules, determine a classification count according to the length of the association rule and the count of a single item in the association rule, so as to obtain a plurality of classification counts corresponding to each of the display categories; For each of the display categories, the first association rule with the largest category count is determined as the association rule corresponding to the display category; and the target data tag that satisfies the association rule with the display category is determined in the data tag category system.

6. The method according to claim 1, characterized in that Displaying the first data asset in the first display category in the data map includes: Obtaining user usage information of the data asset, wherein the usage information includes at least one of the following: data query and access records, data download and export records, data tag modification and creation records, and browsing path information on the data map; According to the usage information, the users are divided into a plurality of different behavior groups; For different behavior groups, the first data asset is displayed in the first display category in the data map in different ways.

7. A data processing device, characterized in that: include: An acquisition module, used to acquire multiple data tags corresponding to the data asset, classify the multiple data tags, and obtain a data tag category system, wherein the data tag category system includes multiple different tag categories; A first determination module is used to summarize the label attributes of the data labels under the label category in the data label category system, and determine the summary results as the display categories of the data map; An establishing module, used to determine a target data tag that satisfies an association rule with the display classification in the data tag category system, and establish an association relationship between the target data tag and the display classification; A second determination module is used to obtain a first data asset to be displayed, determine a first target data tag to which the first data asset belongs, and determine a first display category to which the first target data tag belongs based on the association relationship; A display module is used to display the first data asset in the first display category in the data map.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the data processing method according to any one of claims 1 to 6 when running.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.

Images