Data management method and device

Abstract

The invention provides a data management method and device. The method comprises the steps that fields of data to be classified and graded, field information of the fields and field sample data of the fields are obtained; configuring a model library based on the preset data features and the preset data classification and grading rule, and obtaining a preset classification and grading standard; respectively matching the field information and the field sample data according to a model library to obtain a data classification result of the field; obtaining a data classification result according to a mapping relationship between the data classification result of the field and a preset classification and classification standard; and evaluating the fields according to the data classification result and the data grading result of the fields to obtain an evaluation report, and managing the to-be-classified and graded data based on the evaluation report. Therefore, by configuring the model library, the data and the model library are efficiently and accurately matched to obtain the data classification and grading result, and unified management is performed according to the classification and grading result, so that the classification and grading efficiency and accuracy can be improved, and the data management effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of data technology, and in particular to a data management method and a data management device. Background Art

[0002] At present, data classification and grading methods cannot meet the needs of effective identification of data in the industry, such as the charging industry. At the same time, the low efficiency of manual classification and the inaccuracy of artificial intelligence lead to poor data management effects. Summary of the invention

[0003] In order to solve one of the above technical problems, the present invention proposes the following technical solution.

[0004] The first aspect of the present invention proposes a data management method, comprising the following steps: obtaining fields of data to be classified and graded and their field information and field sample data; configuring a model library based on preset data features and preset data classification and grading rules, and obtaining preset classification and grading standards; matching the field information and the field sample data respectively according to the model library to obtain the data classification result of the field; obtaining the data grading result of the data to which the field belongs according to the mapping relationship between the data classification result of the field and the preset classification and grading standard; evaluating the field according to the data classification result of the field and the data grading result to obtain an evaluation report, and managing the data to be classified and graded based on the evaluation report.

[0005] In addition, the data management method according to the above embodiment of the present invention may also have the following additional technical features.

[0006] According to an embodiment of the present invention, the fields of the data to be classified and graded and their field information and field sample data are obtained by at least one of automatic scanning, manual configuration and importing assets.

[0007] According to an embodiment of the present invention, the field information includes at least one of field name, field type, field length, field content, field comment, whether the field is a primary key, and whether the field is an index.

[0008] According to one embodiment of the present invention, the field information and the field sample data are matched respectively according to the model library, including: obtaining field attributes and table information to which the field belongs; judging whether the field attributes match the data attributes in the preset data classification and grading rules in the model library, or whether the field content and the preset data in the model library are in the same interval, or whether the field information contains the preset content in the model library or starts or ends with the preset content, or matching the field with the model library through feature matching rules to obtain a mapping relationship between the field and the preset data feature.

[0009] According to an embodiment of the present invention, the feature matching rule includes at least one of a conventional matching rule, a regular expression rule and a language parsing rule.

[0010] According to one embodiment of the present invention, the model library includes business data types, commercial data types, personal information data types, equipment data types, technical data types, and personnel data types, wherein each data type classifies data according to important data, core data, general data, personal information, sensitive data, and public data.

[0011] According to one embodiment of the present invention, the fields include charging industry field data, and different field information is annotated with its data characteristics.

[0012] According to one embodiment of the present invention, the charging industry field data includes charging order field data, and the charging order field data includes at least one of the charging order number, charging order source channel, charging start type, charging station information, charging pile information used for charging, charging gun information, charging order settlement method, order stop code, order stop reason and order status, and the data features include at least one of the charging pile numbering rules, self-operated pile numbering rules, joint operation pile numbering rules and other-operated pile numbering rules.

[0013] According to an embodiment of the present invention, the to-be-classified and graded data includes a database, a data table and a field.

[0014] The second aspect of the present invention proposes a data management device, including: an acquisition module, used to acquire the fields of the data to be classified and graded and its field information and field sample data; a configuration module, used to configure the model library based on preset data features and preset data classification and grading rules, and acquire the preset classification and grading standards; a first determination module, used to match the field information and the field sample data respectively according to the model library to obtain the data classification result of the field; a second determination module, used to obtain the data classification result of the data belonging to the field according to the mapping relationship between the data classification result of the field and the preset classification and grading standard; an evaluation module, used to evaluate the field according to the data classification result of the field and the data grading result to obtain an evaluation report, and manage the data to be classified and graded based on the evaluation report.

[0015] The technical solution of the embodiment of the present invention configures the model library to efficiently and accurately match the data with the model library to obtain data classification and grading results, and uniformly manages them according to the classification and grading results, which can improve the efficiency and accuracy of classification and grading and improve data management effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flow chart of a data management method according to an embodiment of the present invention.

[0017] Figure 2 FIG. 4 is a block diagram of a data management device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 Flow chart of a data management method according to an embodiment of the present invention.

[0020] like Figure 1 As shown, the data management method includes the following steps S1 to S5.

[0021] The data management method of the embodiment of the present invention can be applied to any scenario, for example, it can be applied to the charging industry.

[0022] S1, obtaining the fields of the data to be classified and graded, its field information and field sample data.

[0023] The data to be classified and graded includes databases, data tables and fields. The field information includes at least one of field name, field type, field length, field content, field comment, whether the field is a primary key and whether the field is an index.

[0024] Specifically, the database and database information of the data to be classified and graded, data tables and data table information, fields and their names, types, lengths, description information, and data samples of the fields can be obtained through automatic scanning, manual configuration, or importing assets.

[0025] Among them, automatic scanning refers to automatically scanning unknown data assets based on scheduled tasks, and transferring them to manual claim to configure the ownership of assets; manual configuration refers to importing unstructured asset data through an interface, and the imported content includes at least metadata information of data tables and fields and sample data; importing assets refers to automatically scanning to obtain structured asset data, real-time or periodic scanning of data assets to be tested, obtaining database information, data table information, data field information and data samples, and regularly updating data to be tested.

[0026] First, dynamically identify the database information to be classified. Various types of data information are stored in different databases, the same type of data exists in different data tables, and the various dimensional information of this type of data exists in different fields in the database table. First, obtain the database information to be detected through manual configuration or automatic identification. Unstructured data can also be imported into the system through the interface. If the data already has a classification label, access control is performed on the data with a classification label based on the data access scope in the access user's permissions and the highest level of data that the user can access. If there is no label, the data is classified and graded.

[0027] Secondly, obtain the field information to be classified and graded. The field is the smallest storage unit of data and is the target data to be scanned in this application. Before classification and grading, you need to first obtain the data table name, data table description, database name, and field information to which the field belongs, including: field name, field data type, field length, field content, field comment, whether the field is the primary key, and whether the field is an index. This will allow the field to be classified and graded based on this information in the future.

[0028] S2, configure the model library based on preset data features and preset data classification and grading rules, and obtain the preset classification and grading standards.

[0029] Among them, the model library includes business data types, commercial data types, personal information data types, equipment data types, technical data types, and personnel data types. Among them, each data type classifies data according to important data, core data, general data, personal information, sensitive data, and public data.

[0030] Specifically, based on the data features, data types, data dictionaries, data rules and data classification and grading standards formed in the industry, such as the charging industry, corresponding classification and grading rules of preset data features or preset data attributes are constructed, and the matching logic of these rules is set, and then the model library is configured. And the preset classification and grading standards of the charging industry are obtained.

[0031] S3, matching the field information and the field sample data respectively according to the model library to obtain the data classification result of the field.

[0032] Specifically, the metadata and sample data of the fields to be classified and graded are matched with the model library, and the data classification result of the field is determined by combining the features and rules through feature matching of the field name, field annotation, field type, field length or field content.

[0033] Users can configure the model library according to the actual work scenario, or directly use the configured model library, or combine the preset model library to specifically add, modify, or delete preset features or rules to improve the flexibility of the implementation scenario of this application.

[0034] Optionally, in one implementation method of the implementation scenario of the present invention, if there are separate libraries and tables, the same type of data will exist in different databases and different data tables. Before obtaining the field information, it is possible to configure which libraries of the asset represent the same library information and which tables represent the same data table information. In this way, the implementation case of the invention will only classify and grade the data in the main library and main table, thereby reducing the duplication of data and improving the efficiency of classification and grading.

[0035] Optionally, in one implementation of the present invention, before automatically detecting the classification and grading to which a field belongs, it is first checked whether the field has a predefined data classification identifier. The data classification identifier is the last scanning system or manual setting, which is used to determine the mapping relationship between the field and the preset classification and grading. If the field contains a preset classification identifier and is set to skip the confirmed data classification identifier, the field is classified and graded according to the preset classification identifier. If the setting is to ignore the confirmed data classification identifier or the data classification identifier does not exist, the metadata attributes and sample data of the field will be matched to the model library. This setting can confirm the data identifier of the field once and use it multiple times, avoiding the time-consuming process of repeated identification.

[0036] S4, obtaining the data classification result of the data belonging to the field according to the mapping relationship between the data classification result of the field and the preset classification and grading standard.

[0037] Specifically, the data classification result of the field is matched with the preset classification and grading standard to obtain a mapping relationship between the two, and the data grading result of the field data is obtained accordingly.

[0038] S5, evaluating the field according to the data classification result and data grading result of the field to obtain an evaluation report, and managing the data to be classified and graded based on the evaluation report.

[0039] Specifically, standardized assessments are conducted on all fields to obtain an assessment report of the hierarchical model. Based on the data assessment report, effective detection and early warning of the legality and compliance of the data are carried out.

[0040] Optionally, in one implementation of the implementation scenario of the present invention, the data classification and grading assessment report will include asset information, data type statistics, data classification statistics, sensitive field statistics, whether the actual field content is desensitized or encrypted, and the disposal recommendations given by the system for the field.

[0041] Compared with related technologies, the embodiments of the present invention dynamically identify the data asset information to be classified and graded; obtain the database information, data table information and field information within the detection range expected by the user; perform data matching based on the model library composed of data types, data features, data rules and data dictionaries contained in the preset charging industry data classification and grading rules to obtain matching results. The mapping relationship between data classification and data grading is used to obtain the grading results of the data, and access control is performed on the data based on the grading results of the data. At the same time, a standardized evaluation is performed on the detection data set to obtain an evaluation report of the classification and grading model. Based on the data evaluation report, effective detection and early warning of the legality and compliance of industry data are performed.

[0042] Therefore, the data management method of the embodiment of the present invention, by configuring the model library, efficiently and accurately matches the data with the model library to obtain data classification and grading results, and performs unified management based on the classification and grading results, which can improve the efficiency and accuracy of classification and grading and improve the data management effect.

[0043] In one embodiment of the present invention, step S2 is to match the field information and the field sample data respectively according to the model library, including: obtaining the field attributes and the table information to which the field belongs; judging whether the field attributes match the data attributes in the preset data classification and grading rules in the model library, or whether the field content and the preset data in the model library are in the same interval, or whether the field information contains the preset content in the model library or starts or ends with the preset content, or matching the field with the model library through feature matching rules to obtain a mapping relationship between the field and the preset data features, wherein, when matching multiple model libraries, sequential matching is performed according to the matching priority of the rules in each model library.

[0044] When matching applicable data classification and grading rules in multiple model libraries, the data will be matched sequentially according to the matching priority of the rules in the model library. The matching of priorities is to build a multi-level classification structure tree, in which each leaf node is associated with one or more data classification and grading rules. During the matching process, the system checks the data classification and grading rules associated with each node on the classification structure tree layer by layer from top to bottom, determines whether these rules match the field sample data and metadata information of the data, and determines the matching data classification results.

[0045] By setting a matching quantity threshold, it is determined whether the field to be classified and graded matches the feature, as well as the matching degree with the feature, and all matching features and feature matching degrees are obtained.

[0046] The feature matching rules include at least one of conventional matching rules, regular expression rules and language parsing rules. Specifically, based on the language model, it is determined whether the table information, field information, and field attributes of the data match the metadata attributes defined in the data classification and grading rules in the preset model library, or whether the field content is in a range, or whether the field information contains or starts or ends with the preset content, or the preset regular expression is matched with the data to obtain the mapping relationship between data and features. The data type contains the combination relationship between one or more features and rules. Through the hierarchical tree relationship between features and data types, the data type of the field is obtained, thereby obtaining the result of data classification and grading.

[0047] According to the business scenario, a hierarchical structure of data classification is constructed. For example, the charging industry model library includes: the first-level classification includes business data, commercial data, personal information data, technical data, personnel data, etc.; the second-level classification under business data includes marketing data, product sales data, operation data, etc.; the third-level classification under product sales data includes product indicator information, etc. For example, the sample data of a field contains data content such as self-operated pile numbering rules, joint-operated pile numbering rules, and other-operated pile numbering. Its data characteristics conform to the data characteristics of charging pile numbering, and the charging pile number belongs to the preset classification of equipment information under the product indicator information of the charging industry. The field can be matched to the classification of equipment information under the preset product indicator information, and the corresponding classification of the field is determined, such as S-level or A-level, etc. For example, the channel source of charging orders, as company-level master data, the field name of this field in each system is similar, and the field dictionary content is consistent. Through the language model to identify whether the field name contains specific characters, and the field content conforms to the dictionary setting of the charging order channel source, it can be determined that the classification of the field is the charging business data classification. Specific users can flexibly adjust the preset model library according to different scenario requirements.

[0048] Optionally, in one implementation method of the implementation scenario of the present invention, if a field matches multiple data types, by setting the priority of the data type, if the data type priorities are the same, a matching quantity threshold can be set to determine whether the field at the classification level matches the feature and the degree of match with the feature, so as to obtain all matched features and the degree of feature match, and obtain a better data type in the matching result through weighting.

[0049] At the same time, the classification results described in the implementation scenario of this application will be stored in the database to facilitate users to query the results. At the same time, subsequent regular scans can flexibly adjust the scanning scope to further improve the efficiency of data classification and grading when the field information in the data assets changes or the model library rules change.

[0050] In one example, the fields include charging industry field data, and different field information is annotated with its data characteristics. The charging industry field data includes charging order field data, Taking the data of charging orders as an example, the field data of charging orders include the charging order number, the source channel of the charging order, the charging start type, the charging station information, the charging pile information used for charging, the charging gun information, the settlement method of the charging order, the order stop code, the order stop reason and at least one of the order status.

[0051] Different fields of information have their own data characteristics, such as charging pile number, self-operated pile numbering rules, joint-operated pile numbering rules, and other-operated pile numbering rules. The data characteristics include at least one of the charging pile numbering rules, self-operated pile numbering rules, joint-operated pile numbering rules, and other-operated pile numbering rules.

[0052] Specifically, when the metadata attributes and field contents of the field are identified, the data type of the field is determined by detecting whether the field has the same or similar data features, and the field classification is further determined by the mapping relationship between the data type and the classification. The data in the charging industry include: personal information, which includes license plate number, frame number, and parking space number in addition to basic personal information; equipment information, charging information, activity information, etc. The equipment information includes: charging station information, charging pile information, charging gun information, charging order information, activity coupon code, etc. The data is classified according to the sensitivity, importance and potential risks of the data, and targeted data security standards and data protection measures are set. In the implementation scenario of the present invention, the specific levels that can be set are: core data S level cannot be leaked, important data A level leakage is illegal, general data B level leakage causes large losses, general data C level leakage causes general losses, public data leakage has no losses and other different security levels. The higher the security level, the stricter the control measures for the use of data, including data usage permissions, whether the data is encrypted, whether the data is desensitized, etc. Users can flexibly configure the data security level according to their needs.

[0053] The embodiment of the present invention scans all the data in the data assets that the user expects to classify and grade into the system, including databases, data tables, data fields, field attributes, and field data samples. The user can intuitively and quickly view the data information included in the current assets to achieve unified management of the data. It can also be further determined according to the master data rules whether the same type of fields meet the master data requirements during the business flow process.

[0054] In the embodiment of the present invention, the data structure of the asset to be scanned, the metadata information of the field and the sample data of the field are dynamically identified through manual configuration or automatic identification of the asset to be scanned, and it is judged whether the data has a classification and grading label. If there is no classification and grading label, the system automatically classifies and grades the data. When the data is unlabeled in the classification and grading result, the data can be manually labeled in batches. Through data access control, the data level and data range accessible to users are controlled, and the final result of classification and grading is that data without labels is not allowed to be accessed. The system presets a model rule feature library of charging industry data, including six categories of data: business data, commercial data, personal information data, equipment data, technical data, and personnel data. At the same time, the data is graded according to important data, core data, general data, personal information, sensitive data, and public data. Based on the model library of charging industry data, the field metadata and sample data to be detected are automatically and efficiently matched, and the mapping relationship between data and data categories is recommended. The mapping relationship and the standard of data classification and grading are used to perform standardized evaluation of the data set to be inspected, and an evaluation report of the classification and grading model is obtained. Based on the data evaluation report, the unified management of data and the legality and compliance of data flow are effectively monitored and warned. The data classification and grading method provided in this application improves the efficiency of data classification and grading while improving the accuracy of data classification and grading in the charging industry. In summary, the embodiments of the present invention have the following advantages: (1) A rich model library unique to the charging industry is preset to efficiently and accurately match data with the classification and grading standards and rules of the charging industry, providing the charging industry with an efficient, reliable, and explainable data classification and grading method; (2) Through automatic scanning of enterprise data assets, efficient and reliable classification and grading of data assets, access control of data is carried out according to the data classification results, and an overall asset assessment report is output, providing effective support for the unified management and security supervision of enterprise data.

[0055] Corresponding to the data management method of the above embodiment, the present invention further provides a data management device.

[0056] Figure 2 FIG. 4 is a block diagram of a data management device according to an embodiment of the present invention.

[0057] like Figure 2 As shown, the data management device includes an acquisition module 10 , a configuration module 20 , a first determination module 30 , a second determination module 40 and an evaluation module 50 .

[0058] Among them, the acquisition module 10 is used to obtain the fields of the data to be classified and graded and its field information and field sample data; the configuration module 20 is used to configure the model library based on preset data features and preset data classification and grading rules, and obtain the preset classification and grading standards; the first determination module 30 is used to match the field information and the field sample data respectively according to the model library to obtain the data classification result of the field; the second determination module 40 is used to obtain the data classification result of the data belonging to the field according to the mapping relationship between the data classification result of the field and the preset classification and grading standard; the evaluation module 50 is used to evaluate the field according to the data classification result and the data grading result of the field to obtain an evaluation report, and manage the data to be classified and graded based on the evaluation report.

[0059] It should be noted that the specific implementation of the data management device can refer to the specific implementation of the above-mentioned data management method, and to avoid redundancy, it will not be described in detail here.

[0060] The data management device of the embodiment of the present invention configures the model library to efficiently and accurately match the data with the model library to obtain data classification and grading results, and performs unified management based on the classification and grading results, which can improve the efficiency and accuracy of classification and grading and improve the data management effect.

[0061] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations on the present invention, and those of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A data management method, characterized in that: The following steps are involved: Obtain the fields of the data to be classified and graded, as well as their field information and field sample data; Configure the model library based on preset data features and preset data classification and grading rules, and obtain preset classification and grading standards; Matching the field information and the field sample data respectively according to the model library to obtain a data classification result of the field; According to the mapping relationship between the data classification result of the field and the preset classification and grading standard, the data grading result of the data belonging to the field is obtained; The field is evaluated according to the data classification result and the data grading result of the field to obtain an evaluation report, and the data to be classified and graded is managed based on the evaluation report.

2. The data management method according to claim 1, characterized in that: The fields of the data to be classified and graded and their field information and field sample data are obtained by at least one of automatic scanning, manual configuration and importing assets.

3. The data management method according to claim 1, characterized in that: The field information includes at least one of field name, field type, field length, field content, field comment, whether the field is a primary key, and whether the field is an index.

4. The data management method according to claim 3, characterized in that: Matching the field information and the field sample data respectively according to the model library includes: Get field properties and table information to which the field belongs; Determine whether the field attributes match the data attributes in the preset data classification and grading rules in the model library, or whether the field content and the preset data in the model library are in the same interval, or whether the field information contains the preset content in the model library or starts or ends with the preset content, or match the field with the model library through feature matching rules to obtain a mapping relationship between the field and the preset data features, wherein, when matching multiple model libraries, sequential matching is performed according to the matching priority of the rules in each model library.

5. The data management method according to claim 4, characterized in that: The feature matching rule includes at least one of a conventional matching rule, a regular expression rule and a language parsing rule.

6. The data management method according to claim 1, characterized in that: The model library includes business data types, commercial data types, personal information data types, equipment data types, technical data types, and personnel data types, among which each data type classifies data according to important data, core data, general data, personal information, sensitive data, and public data.

7. The data management method according to claim 1, characterized in that: The fields include charging industry field data, and different field information is marked with its data characteristics.

8. The data management method according to claim 7, characterized in that: The charging industry field data includes charging order field data, The charging order field data includes at least one of the following data: charging order number, charging order source channel, charging start type, charging station information, charging pile information, charging gun information, charging order settlement method, order stop code, order stop reason and order status. The data features include at least one of a charging pile numbering rule, a self-operated pile numbering rule, a jointly operated pile numbering rule, and a third-party operated pile numbering rule.

9. The data management method according to claim 1, characterized in that: The data to be classified and graded includes databases, data tables and fields.

10. A data management device, characterized in that: include: An acquisition module is used to acquire the fields of the data to be classified and graded and its field information and field sample data; A configuration module, used to configure the model library based on preset data features and preset data classification and grading rules, and obtain preset classification and grading standards; A first determination module, used for matching the field information and the field sample data respectively according to the model library to obtain a data classification result of the field; A second determination module is used to obtain a data classification result of the data belonging to the field according to a mapping relationship between the data classification result of the field and the preset classification and grading standard; An evaluation module is used to evaluate the field according to the data classification result and the data grading result of the field to obtain an evaluation report, and manage the data to be classified and graded based on the evaluation report.

Images