Data asset cost assessment method, device, equipment, storage medium and product

Abstract

The present invention discloses a data asset cost assessment method, device, equipment, storage medium and product, the method comprising: obtaining a data asset table to be assessed; wherein the data asset table to be assessed is obtained by processing according to at least one original data table; for each original data table, determining the original table field value cost of each field value in the original data table; determining the field value mapping relationship between each field value in the data asset table to be assessed and each field value in the original data table; determining the cost of the data asset table to be assessed according to the field value mapping relationship and the original table field value cost. The data asset cost assessment method disclosed in the present invention allocates the cost of the original data table to the field value, and tracks the processing link of the field value, and finally obtains the cost of the data asset table. By allocating and tracking the cost at the granularity of the field value, the cost propagation link can be accurately tracked, and finally the cost of the data asset table can be accurately calculated.

Description

Technical Field

[0001] The present invention relates to the technical field of data asset value assessment, and in particular to a data asset cost assessment method, device, equipment, storage medium and product. Background Art

[0002] Data assets refer to data resources owned or controlled by individuals or enterprises that can bring future economic benefits to enterprises. These data assets can be recorded physically or electronically, have data ownership (including exploration rights, use rights and ownership rights), and are valuable, measurable and readable data sets in cyberspace.

[0003] In the prior art, methods for evaluating the value of data assets usually include cost method, market method, income method and other evaluation methods, among which the cost method is the most commonly used. The cost method determines the value of data assets by evaluating the production and maintenance costs of data assets, including the costs of data collection, storage, processing and analysis. Specifically, enterprises need to incur procurement, manpower and other costs to collect raw data. The collected data needs to be imported into the database first, and then processed into the final data asset table after a series of processes such as data cleaning, quality inspection, data analysis, and data calculation. The existing data asset table cost calculation method is to make an approximate cost allocation calculation at the table granularity, which has the problem of inaccuracy when calculating the data asset table cost. Summary of the invention

[0004] The present invention provides a data asset cost assessment method, device, equipment, storage medium and product to achieve accurate assessment of the cost of data assets.

[0005] According to one aspect of the present invention, a data asset cost assessment method is provided, comprising:

[0006] Obtaining a data asset table to be evaluated; wherein the data asset table to be evaluated is obtained by processing at least one original data table;

[0007] For each of the original data tables, determining the original table field value cost of each field value in the original data table;

[0008] Determine a field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table;

[0009] The cost of the data asset table to be evaluated is determined according to the field value mapping relationship and the original table field value cost.

[0010] Further, determining the original table field value cost of each field value in the original data table includes:

[0011] Acquire the cost of the original data table, the total number of rows in the original data table, and the first cost allocation coefficient of each field in the original data table;

[0012] The quotient of the cost of the original data table and the total number of rows in the original data table is used as the row cost of the original data table;

[0013] For each field value in the original data table, the first cost allocation coefficient of the field where the field value is located is used as the target cost allocation coefficient, and the product of the row cost of the original data table and the target cost allocation coefficient is used as the original table field value cost of the field value.

[0014] Further, determining the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table includes:

[0015] Parsing the data processing structured statements of the data asset table to be evaluated, obtaining the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, and the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table;

[0016] According to the field mapping relationship and the row mapping relationship, a field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table is determined.

[0017] Further, the data processing structured statement of the data asset table to be evaluated is parsed to obtain the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, including:

[0018] Performing lineage analysis on the data processing structured statements of the data asset table to be evaluated to obtain a lineage relationship diagram between the data asset table to be evaluated and the original data table;

[0019] The blood relationship graph is traversed to convert the blood relationship graph into a field mapping relationship between each field in the to-be-evaluated data asset table and each field in the original data table.

[0020] Further, the data processing structured statement of the data asset table to be evaluated is parsed to obtain the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table, including:

[0021] Parsing the data processing structured statements of the data asset table to be evaluated, and generating row-level query statements of the original data table;

[0022] The row-level query statement is executed to obtain a row mapping relationship between each row in the to-be-evaluated data asset table and each row in the original data table.

[0023] Further, determining the cost of the data asset table to be evaluated according to the field value mapping relationship and the original table field value cost includes:

[0024] Obtaining the second cost allocation coefficients respectively corresponding to the original data tables;

[0025] Determine the asset table field value cost of each field value in the to-be-evaluated data asset table according to the field value mapping relationship, the original table field value cost, and the second cost allocation coefficient;

[0026] The sum of the asset table field value costs of each field value in the to-be-evaluated data asset table is determined as the cost of the to-be-evaluated data asset table.

[0027] According to another aspect of the present invention, there is provided a data asset cost assessment device, comprising:

[0028] The data asset table module to be evaluated is used to obtain the data asset table to be evaluated; wherein the data asset table to be evaluated is obtained by processing at least one original data table;

[0029] An original table field value cost determination module, used for determining, for each of the original data tables, an original table field value cost of each field value in the original data table;

[0030] A field value mapping relationship determination module is used to determine the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table;

[0031] The cost determination module of the data asset table to be evaluated is used to determine the cost of the data asset table to be evaluated according to the field value mapping relationship and the original table field value cost.

[0032] Optionally, the original table field value cost determination module is also used to:

[0033] Acquire the cost of the original data table, the total number of rows in the original data table, and the first cost allocation coefficient of each field in the original data table;

[0034] The quotient of the cost of the original data table and the total number of rows in the original data table is used as the row cost of the original data table;

[0035] For each field value in the original data table, the first cost allocation coefficient of the field where the field value is located is used as the target cost allocation coefficient, and the product of the row cost of the original data table and the target cost allocation coefficient is used as the original table field value cost of the field value.

[0036] Optionally, the field value mapping relationship determination module is further used to:

[0037] Parsing the data processing structured statements of the data asset table to be evaluated, obtaining the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, and the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table;

[0038] According to the field mapping relationship and the row mapping relationship, a field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table is determined.

[0039] Optionally, the field value mapping relationship determination module is further used to:

[0040] Performing lineage analysis on the data processing structured statements of the data asset table to be evaluated to obtain a lineage relationship diagram between the data asset table to be evaluated and the original data table;

[0041] The blood relationship graph is traversed to convert the blood relationship graph into a field mapping relationship between each field in the to-be-evaluated data asset table and each field in the original data table.

[0042] Optionally, the field value mapping relationship determination module is further used to:

[0043] Parsing the data processing structured statements of the data asset table to be evaluated, and generating row-level query statements of the original data table;

[0044] The row-level query statement is executed to obtain a row mapping relationship between each row in the to-be-evaluated data asset table and each row in the original data table.

[0045] Optionally, the cost determination module of the asset table of data to be evaluated is also used for:

[0046] Obtaining the second cost allocation coefficients respectively corresponding to the original data tables;

[0047] Determine the asset table field value cost of each field value in the to-be-evaluated data asset table according to the field value mapping relationship, the original table field value cost, and the second cost allocation coefficient;

[0048] The sum of the asset table field value costs of each field value in the to-be-evaluated data asset table is determined as the cost of the to-be-evaluated data asset table.

[0049] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0050] at least one processor; and

[0051] a memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data asset cost assessment method described in any embodiment of the present invention.

[0053] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data asset cost assessment method described in any embodiment of the present invention when executed.

[0054] According to another aspect of the present invention, a computer program product is provided, which includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the data asset cost assessment method described in any embodiment of the present invention are implemented.

[0055] The cost assessment method for data assets disclosed in the present invention first obtains a data asset table to be assessed; wherein the data asset table to be assessed is obtained by processing based on at least one original data table; then, for each original data table, the original table field value cost of each field value in the original data table is determined; then, the field value mapping relationship between each field value in the data asset table to be assessed and each field value in the original data table is determined; finally, the cost of the data asset table to be assessed is determined based on the field value mapping relationship and the original table field value cost. The cost assessment method for data assets disclosed in the present invention allocates the cost of the original data table to the field value, and tracks the processing link of the field value, and finally obtains the cost of the data asset table. By allocating and tracking the cost at the granularity of the field value, the cost propagation link can be accurately tracked, and finally the cost of the data asset table can be accurately calculated.

[0056] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a flow chart of a data asset cost assessment method provided according to Embodiment 1 of the present invention;

[0059] Figure 2 is a schematic diagram of a data asset processing process provided according to the first embodiment of the present invention;

[0060] Figure 3 is a flow chart of a data asset cost assessment method provided according to Embodiment 2 of the present invention;

[0061] Figure 4 is a schematic diagram of a data asset processing process provided according to the second embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the structure of a data asset cost evaluation system provided according to Embodiment 2 of the present invention;

[0063] Figure 6 is a schematic diagram of the structure of a data asset cost assessment device provided according to Embodiment 3 of the present invention;

[0064] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the data asset cost assessment method according to the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] Embodiment 1

[0068] Figure 1A flowchart of a method for cost assessment of data assets provided in Example 1 of the present invention. This embodiment is applicable to the situation where cost assessment is performed on a data asset table. The method can be executed by a cost assessment device for data assets. The cost assessment device for data assets can be implemented in the form of hardware and / or software. The cost assessment device for data assets can be configured in an electronic device.

[0069] like Figure 1 As shown, the method includes:

[0070] S110. Obtain a table of data assets to be evaluated.

[0071] The data asset table to be evaluated is a data table that needs to be cost evaluated, and the data asset table to be evaluated is obtained by processing at least one original data table.

[0072] For example, assuming that there are original data tables T1 and T2, through the data asset processing process, the data in the data asset table T3 is generated. T3 is the data asset table to be evaluated, and the cost of the T3 table needs to be calculated.

[0073] Figure 2 This is a schematic diagram of a data asset processing process provided by an embodiment of the present invention. As shown in the figure, the original data table T1 contains two fields c1 and c2, the original data table T2 contains two fields c3 and c4, x1-x6 are the field values ​​of the original data table T1, y1-y6 are the field values ​​of the original data table T2, and after a certain data processing logic, the data asset table T3 to be evaluated is generated, T3 contains a field c5, z1 and z2 are the field values ​​of the data asset table T3 to be evaluated. Among them, z1 and z2 are processed by the field values ​​in T1 and T2.

[0074] S120 . For each original data table, determine the original table field value cost of each field value in the original data table.

[0075] The original table field value cost is the result of allocating the cost of the original data table to each field value.

[0076] In this embodiment, the cost of the original data table is known. Based on the metadata of the original data table, the number of fields, number of rows and other information of each original data table can be determined. Then, based on this information, the cost of the entire original data table can be allocated to each field value to obtain the original table field value cost of each field value.

[0077] Preferably, due to the different importance of each field, the complexity of data asset processing, and the null value situation in the field, a corresponding cost allocation coefficient can be set for each field, and the sum of the cost allocation coefficients of all fields is set to 1. When calculating the cost of the original table field value, the cost allocation coefficient of the field where each field value is located can be considered, and the cost allocation coefficient can be assigned to each field value as a weight.

[0078] S130: Determine the field value mapping relationship between each field value in the data asset table to be evaluated and each field value in the original data table.

[0079] In this embodiment, since the data asset table to be evaluated is processed based on at least one original data table, there is a field value mapping relationship between the data asset table to be evaluated and each original data table at the granularity of the field value, which can indicate which field values ​​in the original data table each field value in the data asset table to be evaluated is processed from.

[0080] by Figure 2 For example, if the data processing logic is to extract the first two rows of the c2 field and the last two rows of the c4 field, and divide the c2 field by the c4 field, then the field value z1 in the c5 field is obtained by processing x2 in the c2 field and y4 in the c4 field, and z2 is obtained by processing x4 in the c2 field and y6 in the c4 field.

[0081] S140. Determine the cost of the data asset table to be evaluated according to the field value mapping relationship and the original table field value cost.

[0082] In this embodiment, after obtaining the field value mapping relationship and the original table field value cost of each field value in the original data table, the asset table field value cost of each field value in the data asset table to be evaluated can be calculated, and then the cost of each asset table field value is summed to obtain the cost of the data asset table to be evaluated.

[0083] Optionally, a method for calculating the cost of the asset table field value may be to extract the corresponding field value in the original data table for each field value in the data asset table to be evaluated according to the field value mapping relationship, and perform a weighted summation of the original table field value costs of each field value extracted from the original data table.

[0084] The cost assessment method for data assets provided by the embodiment of the present invention first obtains a data asset table to be assessed; wherein the data asset table to be assessed is obtained by processing according to at least one original data table; then, for each original data table, the original table field value cost of each field value in the original data table is determined; then, the field value mapping relationship between each field value in the data asset table to be assessed and each field value in the original data table is determined; finally, the cost of the data asset table to be assessed is determined according to the field value mapping relationship and the original table field value cost. The cost assessment method for data assets disclosed in the present invention allocates the cost of the original data table to the field value, and tracks the processing link of the field value, and finally obtains the cost of the data asset table. By allocating and tracking the cost at the granularity of the field value, the cost propagation link can be accurately tracked, and finally the cost of the data asset table can be accurately calculated.

[0085] Embodiment 2

[0086] Figure 3 This is a flow chart of a data asset cost assessment method provided in Embodiment 2 of the present invention. This embodiment is a refinement of the above embodiment. Figure 3 As shown, the method includes:

[0087] S210: Obtain a table of data assets to be evaluated.

[0088] The data asset table to be evaluated is a data table that needs to be cost evaluated, and the data asset table to be evaluated is obtained by processing at least one original data table.

[0089] S220 . For each original data table, obtain the cost of the original data table, the total number of rows in the original data table, and the first cost allocation coefficient of each field in the original data table.

[0090] The first cost allocation coefficient is the weight corresponding to different fields in the original data table.

[0091] In this embodiment, due to the different importance of each field, the complexity of data asset processing, and the null value in the field, a corresponding cost allocation coefficient can be set in advance for each field, and the sum of the cost allocation coefficients of all fields is set to 1. When calculating the cost of the original table field value, the metadata of the original data table can be collected to obtain information such as the number of fields and the number of rows of each original data table, and the cost of the original data table and the first cost allocation coefficient of each field can be obtained for use in the next calculation.

[0092] S230. The quotient of the cost of the original data table and the total number of rows in the original data table is used as the row cost of the original data table. For each field value in the original data table, the first cost allocation coefficient of the field where the field value is located is used as the target cost allocation coefficient. The product of the row cost of the original data table and the target cost allocation coefficient is used as the original table field value cost of the field value.

[0093] In this embodiment, the cost of the original data table and the total number of rows in the original data table are represented by P and n respectively, and the row cost of the original data table = p / n. Furthermore, for a field value in the original data table, the first cost allocation coefficient of the field in which it is located can be used as the target cost allocation coefficient of the field value, and the target cost allocation coefficient is represented by X, then the original table field value cost of the field value = (P / n) × X. Similarly, after determining the target cost allocation coefficient of a field value in the original data table, the corresponding original table field value cost can be calculated.

[0094] S240, parsing the data processing structured statements of the data asset table to be evaluated, obtaining the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, and the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table.

[0095] The data processing structured statement is an instruction written by the developer in the data asset processing. By executing the data processing structured statement, the data of the data asset table to be evaluated can be generated. Optionally, the data asset table to be evaluated can be written in SQL language, and the data processing structured statement is the corresponding SQL statement, for example, it can be: INSET INTO asset table SELECT <data processing logic> FROM original table<join condition> WHERE<filter condition> , where <data processing logic> is the specific data processing logic for obtaining the data asset table to be evaluated.

[0096] In this embodiment, since the data asset table to be evaluated is obtained by processing based on at least one original data table, there is a corresponding mapping relationship between the data asset table to be evaluated and each original data table at the granularity of fields and rows, that is, a field mapping relationship and a row mapping relationship. These mapping relationships can be obtained by parsing the data processing structured statements that generate the data asset table to be evaluated.

[0097] Optionally, the method for parsing the data processing structured statements of the data asset table to be evaluated and obtaining the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table may be: performing lineage analysis on the data processing structured statements of the data asset table to be evaluated and obtaining a lineage relationship diagram between the data asset table to be evaluated and the original data table; traversing the lineage relationship diagram and converting the lineage relationship diagram into a field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table.

[0098] Specifically, data lineage refers to a naturally formed relationship between data, similar to human bloodline, during the entire life cycle of data, from data generation, processing, processing, fusion, circulation to final extinction. Simply put, it is the upstream and downstream source and destination relationship between data - where the data comes from and where it goes. Therefore, after collecting data and processing structured statements, the data lineage detection module can be used to perform lineage analysis to obtain a lineage relationship diagram between tables and fields. Then, by traversing the lineage relationship diagram, processing and calculating the field-level lineage relationship, it is possible to obtain which fields of the original data table are used to process each field of the data asset table to be evaluated, and convert it into a structured field mapping relationship.

[0099] Optionally, the method for parsing the data processing structured statements of the data asset table to be evaluated and obtaining the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table may be: parsing the data processing structured statements of the data asset table to be evaluated and generating a row-level query statement of the original data table; executing the row-level query statement and obtaining the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table.

[0100] Specifically, for the convenience of representation, when creating a table, you can add a uid field to the data asset table to be evaluated and the original data table. Each field value in the uid field is a unique value corresponding to a row, which is used to indicate the row-level data flow and does not participate in data processing. When parsing the data processing structured statement of the data asset table to be evaluated, you can generate a row-level query statement for the original data table through syntax parsing and field replacement. For example, for the data processing structured statement "INSET INTO asset table SELECT <data processing logic> FROM original table<join condition> WHERE<filter condition> ", replace <data processing logic> with <original table.uid>, and get the row-level query statement "SELECT <original table.uid> FROM original table<join condition> WHERE<filter condition> By executing a row-level query statement, the uid list of the original data table corresponding to the data asset table to be evaluated can be obtained. The uid list represents which data rows of the original data table are processed to obtain each row of data in the data asset table to be evaluated, that is, the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table.

[0101] Figure 4 This is a schematic diagram of a data asset processing process provided by an embodiment of the present invention. As shown in the figure, the field values ​​of the uid field of the original data table T1 are u1, u2, and u3, and the field values ​​of the uid field of the original data table T2 are u4, u5, and u6. The brackets of the field values ​​in each original data table represent the cost of the original table field values. The field values ​​of the uid field of the data asset table to be evaluated T3 are u7 and u8. If the data processing logic is to extract the first two rows of the c2 field and the last two rows of the c4 field, and divide the c2 field by the c4 field, then through the data processing structured statement, the field mapping relationship between the data asset table to be evaluated and the original data table can be obtained: the c5 field of T3 is processed by the c2 field of T1 and the c4 field of T2; the row mapping relationship is: the u7 row of T3 is processed by the u1 row of T1 and the u5 row of T2, and the u8 row of T3 is processed by the u2 row of T1 and the u6 row of T2.

[0102] S250. Determine the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table according to the field mapping relationship and the row mapping relationship.

[0103] In this embodiment, after obtaining the field mapping relationship and the row mapping relationship, the field value mapping relationship between each field value in the data asset table to be evaluated and each field value in the original data table can be further obtained by querying the field mapping relationship and the row mapping relationship between the data asset table to be evaluated and the original data table at the granularity of the field value.

[0104] S260, obtaining the second cost allocation coefficient corresponding to each original data table, and determining the asset table field value cost of each field value in the asset table of the data to be evaluated according to the field value mapping relationship, the original table field value cost and the second cost allocation coefficient.

[0105] Among them, the second cost allocation coefficient is the weight corresponding to each original data table processed to obtain the data asset table to be evaluated.

[0106] In this embodiment, after determining the field value mapping relationship between each field value in the data asset table to be evaluated and each field value in the original data table, the original table field value cost of each original data table can be allocated to each field value in the data asset table to be evaluated to obtain the asset table field value cost. Since the importance of different original data tables is different, the second cost allocation coefficient corresponding to each original data table can be preset. For each field value in the data asset table to be evaluated, according to the field value mapping relationship, it can be determined which field values ​​of the original data tables the field value is processed from, and then the original table field value cost and the second cost allocation coefficient can be weighted summed to obtain the asset table field value cost of each field value in the data asset table to be evaluated.

[0107] by Figure 4 For example, the field value z1 is obtained by processing x2 in the c2 field and y4 in the c4 field, and the field value z2 is obtained by processing x4 in the c2 field and y6 in the c4 field. If the second cost allocation coefficients corresponding to the original data tables T1 and T2 are Y1 and Y2 respectively, and the original table field value costs of x2, x4, y4 and y6 are p2, p4, p10 and p12 respectively, then the asset table field value cost of the field value z1 in the asset table T3 of the data to be evaluated = p2×Y1+p10×Y2, and the asset table field value cost of the field value z2 = p4×Y1+p12×Y2.

[0108] S270: Determine the sum of the asset table field value costs of each field value in the data asset table to be evaluated as the cost of the data asset table to be evaluated.

[0109] In this embodiment, after the asset table field value cost of each field value in the asset table of data to be evaluated is obtained, the cost of the asset table of data to be evaluated is obtained by summing up all the asset table field value costs.

[0110] by Figure 4 For example, the cost of the data asset table to be evaluated = p2×Y1+p10×Y2+p4×Y1+p12×Y2.

[0111] Furthermore, the cost assessment method for data assets provided by any embodiment of the present invention can be executed by a cost assessment system for data assets, which can run the above-mentioned processing flow and accurately calculate the value of the data asset table to be assessed. When there are changes in the original data table, the data asset table to be assessed, and the data processing structured statements for generating the data asset table to be assessed, the system can be triggered regularly (or manually triggered by an administrator) to run and automatically calculate the cost of the data asset table to be assessed. When the system is running, relevant records such as data processing structured statements, field-level lineage relationships, and the mapping relationship between the fields, rows, and field value costs between the original data table and the data asset table to be assessed can be recorded to ensure that the asset value assessment process is explainable and traceable. Figure 5 A structural diagram of a data asset cost assessment system provided for an embodiment of the present invention, as shown in the figure, the system includes hardware such as a CPU, memory, a user-oriented operating system, and a management software data repository for storing metadata information, lineage information and other data related to the original data table and the data asset table to be assessed. The system also has built-in data asset management and assessment software, through which the cost assessment method of data assets provided by any embodiment of the present invention can be implemented. The software includes a data asset management module, a lineage analysis and management module, a data asset value assessment module, etc. In addition, a data asset storage and processing database is also built into the system to store the data asset value assessment results and the intermediate data generated during the assessment process. After the system is deployed in the IT computer room, it can be connected to the customer's own data asset storage and processing database, and a lineage plug-in can be implanted to collect lineage information for subsequent data asset cost calculations.

[0112] The cost assessment method for data assets provided in an embodiment of the present invention allocates the cost of the original data table to the field values, and tracks the processing links of the field values ​​to finally obtain the cost of the data asset table. By allocating and tracking the cost at the granularity of the field values, the cost propagation link can be accurately tracked, and finally the cost of the data asset table can be accurately calculated.

[0113] Embodiment 3

[0114] Figure 6 A schematic diagram of the structure of a data asset cost assessment device provided in Embodiment 3 of the present invention is shown in FIG. Figure 6 As shown, the device includes: a data asset table to be evaluated module 310, an original table field value cost determination module 320, a field value mapping relationship determination module 330 and a data asset table to be evaluated cost determination module 340.

[0115] The data asset table module 310 is used to obtain the data asset table to be evaluated.

[0116] The data asset table to be evaluated is obtained by processing at least one original data table.

[0117] The original table field value cost determination module 320 is used to determine the original table field value cost of each field value in the original data table for each original data table.

[0118] The field value mapping relationship determination module 330 is used to determine the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table.

[0119] The to-be-evaluated data asset table cost determination module 340 is used to determine the cost of the to-be-evaluated data asset table according to the field value mapping relationship and the original table field value cost.

[0120] Optionally, the original table field value cost determination module 320 is further configured to:

[0121] Obtain the cost of the original data table, the total number of rows in the original data table, and the first cost allocation coefficient of each field in the original data table; take the quotient of the cost of the original data table and the total number of rows in the original data table as the row cost of the original data table; for each field value in the original data table, take the first cost allocation coefficient of the field where the field value is located as the target cost allocation coefficient, and take the product of the row cost of the original data table and the target cost allocation coefficient as the original table field value cost of the field value.

[0122] Optionally, the field value mapping relationship determination module 330 is further used to:

[0123] Parse the data processing structured statements of the data asset table to be evaluated, obtain the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, and the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table; determine the field value mapping relationship between each field value in the data asset table to be evaluated and each field value in the original data table based on the field mapping relationship and the row mapping relationship.

[0124] Optionally, the field value mapping relationship determination module 330 is further used to:

[0125] Perform lineage analysis on the data processing structured statements of the data asset table to be evaluated to obtain a lineage relationship diagram between the data asset table to be evaluated and the original data table; traverse the lineage relationship diagram to convert the lineage relationship diagram into a field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table.

[0126] Optionally, the field value mapping relationship determination module 330 is further used to:

[0127] Parse the data processing structured statements of the data asset table to be evaluated, generate row-level query statements of the original data table; execute the row-level query statements to obtain the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table.

[0128] Optionally, the to-be-evaluated data asset table cost determination module 340 is further configured to:

[0129] Obtain the second cost allocation coefficient corresponding to each original data table; determine the asset table field value cost of each field value in the data asset table to be evaluated according to the field value mapping relationship, the original table field value cost and the second cost allocation coefficient; determine the sum of the asset table field value costs of each field value in the data asset table to be evaluated as the cost of the data asset table to be evaluated.

[0130] The data asset cost assessment device provided in the embodiment of the present invention can execute the data asset cost assessment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0131] Embodiment 4

[0132] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0133] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0134] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0135] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a cost assessment method for data assets.

[0136] In some embodiments, the cost assessment method for data assets may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cost assessment of the data assets described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the cost assessment method for data assets in any other appropriate manner (e.g., by means of firmware).

[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0142] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data asset cost assessment method, characterized in that: include: Obtaining a data asset table to be evaluated; wherein the data asset table to be evaluated is obtained by processing at least one original data table; For each of the original data tables, determining the original table field value cost of each field value in the original data table; Determine a field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table; The cost of the data asset table to be evaluated is determined according to the field value mapping relationship and the original table field value cost.

2. The method according to claim 1, characterized in that Determining the original table field value cost of each field value in the original data table includes: Acquire the cost of the original data table, the total number of rows in the original data table, and the first cost allocation coefficient of each field in the original data table; The quotient of the cost of the original data table and the total number of rows in the original data table is used as the row cost of the original data table; For each field value in the original data table, the first cost allocation coefficient of the field where the field value is located is used as the target cost allocation coefficient, and the product of the row cost of the original data table and the target cost allocation coefficient is used as the original table field value cost of the field value.

3. The method according to claim 1, characterized in that Determining the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table includes: Parsing the data processing structured statement of the data asset table to be evaluated, obtaining the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, and the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table; According to the field mapping relationship and the row mapping relationship, a field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table is determined.

4. The method according to claim 3, characterized in that Parsing the data processing structured statement of the data asset table to be evaluated to obtain the field mapping relationship between each field in the data asset table to be evaluated and each field in the original data table, including: Performing lineage analysis on the data processing structured statements of the data asset table to be evaluated to obtain a lineage relationship diagram between the data asset table to be evaluated and the original data table; The blood relationship graph is traversed to convert the blood relationship graph into a field mapping relationship between each field in the to-be-evaluated data asset table and each field in the original data table.

5. The method according to claim 3, characterized in that: Parsing the data processing structured statement of the data asset table to be evaluated to obtain the row mapping relationship between each row in the data asset table to be evaluated and each row in the original data table, including: Parsing the data processing structured statements of the data asset table to be evaluated, and generating row-level query statements of the original data table; The row-level query statement is executed to obtain a row mapping relationship between each row in the to-be-evaluated data asset table and each row in the original data table.

6. The method according to claim 1, characterized in that Determining the cost of the data asset table to be evaluated according to the field value mapping relationship and the original table field value cost includes: Obtaining the second cost allocation coefficients corresponding to the original data tables respectively; Determine the asset table field value cost of each field value in the to-be-evaluated data asset table according to the field value mapping relationship, the original table field value cost, and the second cost allocation coefficient; The sum of the asset table field value costs of each field value in the to-be-evaluated data asset table is determined as the cost of the to-be-evaluated data asset table.

7. A data asset cost assessment device, characterized in that: include: The data asset table module to be evaluated is used to obtain the data asset table to be evaluated; wherein the data asset table to be evaluated is obtained by processing at least one original data table; An original table field value cost determination module, used for determining, for each of the original data tables, an original table field value cost of each field value in the original data table; A field value mapping relationship determination module is used to determine the field value mapping relationship between each field value in the to-be-evaluated data asset table and each field value in the original data table; The cost determination module of the data asset table to be evaluated is used to determine the cost of the data asset table to be evaluated according to the field value mapping relationship and the original table field value cost.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data asset cost assessment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data asset cost assessment method according to any one of claims 1 to 6 when executed.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the data asset cost assessment method according to any one of claims 1 to 6 are implemented.

Images