Data object evaluation method and device, storage medium and electronic equipment

Abstract

The invention provides a data object evaluation method and device, a storage medium and electronic equipment, and the method comprises the steps: responding to a data evaluation instruction, and determining a target data object corresponding to the data evaluation instruction; acquiring data cost information, data application information and data health degree of the target data object; wherein the data cost information is determined based on the calculation cost and the storage cost of the target data object; the data application information is determined based on the use popularity, the use breadth and the data importance degree of the target data object; the data health degree is determined based on the data quality, the metadata quality, the data security degree and the data updating frequency of the target data object; and performing value evaluation on the target data object based on the data cost information, the data application information and the data health degree of the target data object to obtain a value evaluation result of the target data object. By applying the method provided by the embodiment of the invention, the value evaluation result of the data object can be accurately obtained.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data object evaluation method, device, storage medium and electronic device. Background Art

[0002] With the advent of the big data era, data has become one of the important assets of organizations and individuals. Correctly evaluating the value of data is crucial for data transactions, asset management, and data-based decision-making. However, under the existing technical framework, there is a lack of unified standards to measure the value of different types of data, which leads to subjectivity and uncertainty in the evaluation process, making it impossible to accurately evaluate the value of data. Summary of the invention

[0003] The technical problem to be solved by this application is to provide a data object evaluation method, device, storage medium and electronic device, which can accurately obtain the value evaluation result of the data object. The specific scheme is as follows:

[0004] A data object evaluation method, comprising:

[0005] In response to a data evaluation instruction, determining a target data object corresponding to the data evaluation instruction;

[0006] Acquire data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computational cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object;

[0007] The target data object is valued based on the data cost information, data application information and data health of the target data object to obtain a value evaluation result of the target data object.

[0008] Optionally, the method described above includes determining the data cost information based on the computation cost and storage cost of the target data object, including:

[0009] Obtaining the offline task computing cost, real-time task computing cost and storage cost of the target data object; the offline task computing cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task computing cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object;

[0010] Determining the computation cost of the target data object according to the offline task computation cost and the real-time task computation cost;

[0011] The calculation cost and the storage cost are summed to obtain data cost information of the target data object.

[0012] Optionally, the method described above includes determining the data application information based on the usage popularity, usage breadth, and data importance of the target data object, including:

[0013] Obtain the usage popularity, usage breadth, and data importance of the target data object; the usage popularity is determined based on the cumulative number of times the target data object is read and the popularity values ​​of subordinate data objects of the target data object; the usage breadth is determined according to the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates a report or data exchange service based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level;

[0014] The data application information is determined according to the usage popularity, usage breadth and data importance of the target data object.

[0015] Optionally, the method described above includes determining the data health based on the data quality, metadata quality, data security level, and data update frequency of the target data object, including:

[0016] The data quality, metadata quality, data security level and data update frequency of the target data object are obtained; the data quality of the target data object is calculated based on first quality assessment indicators of each preset first dimension of the target data object and the weight of each quality assessment indicator; the metadata quality of the target data object is calculated based on second quality assessment indicators of each preset second dimension of the metadata of the target data object and the weight of each second quality assessment indicator; the data security level is determined based on the security policy score, the number of anomaly detections and the anomaly repair time of the target data object;

[0017] The data health is determined according to the data quality, metadata quality, data security level and data update frequency of the target data object.

[0018] Optionally, the method described above includes performing a value assessment on the target data object based on the data cost information, data application information, and data health of the target data object to obtain a value assessment result of the target data object, including:

[0019] Dividing the data application information by the data cost information to obtain the initial data value of the target data object;

[0020] The initial data value and the data health are summed to obtain a value assessment result of the target data object.

[0021] A data object evaluation device, comprising:

[0022] a determination unit, configured to determine, in response to a data evaluation instruction, a target data object corresponding to the data evaluation instruction;

[0023] An acquisition unit, used to acquire data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object;

[0024] An evaluation unit is used to perform a value evaluation on the target data object based on data cost information, data application information and data health of the target data object to obtain a value evaluation result of the target data object.

[0025] In the above device, optionally, the acquisition unit includes:

[0026] The first acquisition subunit is used to acquire the offline task computing cost, the real-time task computing cost and the storage cost of the target data object; the offline task computing cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task computing cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object;

[0027] A first determining subunit, configured to determine a computing cost of the target data object according to the offline task computing cost and the real-time task computing cost;

[0028] The first calculation subunit is used to sum the calculation cost and the storage cost to obtain data cost information of the target data object.

[0029] In the above device, optionally, the acquisition unit includes:

[0030] The second acquisition subunit is used to acquire the usage popularity, usage breadth, and data importance of the target data object; the usage popularity is determined based on the cumulative number of times the target data object is read and the popularity value of the subordinate data objects of the target data object; the usage breadth is determined according to the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates a report or data exchange service based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level;

[0031] The second determining subunit is used to determine the data application information according to the usage popularity, usage breadth and data importance of the target data object.

[0032] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data object evaluation method as described above.

[0033] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to perform the data object evaluation method as described above.

[0034] Based on the above-mentioned implementation of this application, a data object evaluation method, device, storage medium and electronic device are provided, the method includes: in response to a data evaluation instruction, determining the target data object corresponding to the data evaluation instruction; obtaining the data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object; based on the data cost information, data application information and data health of the target data object, the target data object is value evaluated to obtain the value evaluation result of the target data object. Through the method provided by the embodiment of this application, the value evaluation result of the data object can be accurately obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0036] Figure 1 A method flow chart of a data object evaluation method provided in this application;

[0037] Figure 2 A flowchart of a process of writing data to be backed up into a first cluster provided in the present application;

[0038] Figure 3 A flowchart of a process for obtaining sample data provided by this application;

[0039] Figure 4 An example diagram of a data relationship of a target data object provided in this application;

[0040] Figure 5 An example diagram of the usage breadth of a target data object provided for this application;

[0041] Figure 6 A schematic diagram of the structure of a data object evaluation device provided by the present application;

[0042] Figure 7 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0045] The present application embodiment provides a data object evaluation method. The method can be applied to an electronic device, which can be a computer device. The method flow chart is as follows: Figure 1 As shown, specifically including:

[0046] S101: In response to a data evaluation instruction, determine a target data object corresponding to the data evaluation instruction.

[0047] In this embodiment, the target data object corresponding to the data evaluation instruction may be determined from among the data objects to be evaluated according to the instruction identifier in the data evaluation instruction.

[0048] S102: Obtain data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; and the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object.

[0049] In an embodiment provided in the present application, based on the above solution, optionally, a process of determining data cost information based on the computation cost and storage cost of the target data object is as follows: Figure 2 As shown, including:

[0050] S201: Obtain the offline task calculation cost, real-time task calculation cost and storage cost of the target data object; the offline task calculation cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task calculation cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object.

[0051] In this embodiment, the computing resources occupied by the offline task of the target data object, the computing resources occupied by the real-time task, and the storage resources occupied by the target data object can be calculated; then the offline task computing cost is determined according to the computing resources occupied by the offline task of the target data object; the real-time task computing cost is determined according to the computing resources occupied by the real-time task of the target data object; and the storage cost of the target data object is calculated according to the storage resources occupied by the target data object. Specifically, the offline task can be an offline write task, and the real-time task can be a real-time write task.

[0052] In this embodiment, the computing resources occupied by the offline task, that is, the offline task computing cost, can be calculated according to the running time of each batch of offline tasks and the number of CPUs occupied by each batch. The calculation formula is:

[0053]

[0054] Where T(i) is the running time of each historical batch of offline tasks, in hours. C(i) is the number of CPUs occupied by the batch of tasks, in C.

[0055] In this embodiment, the computing resources occupied by the real-time task can be calculated according to the online duration of the real-time task and the number of CPUs occupied. The calculation formula is:

[0056] CR=T*C

[0057] Where T is the total online time of the real-time task, in hours (H). C is the number of CPUs occupied by the task, in C.

[0058] Optionally, for every 1H*C of computing resources occupied by a real-time task or an offline task, its computing cost is cumulatively increased by 1.

[0059] In this embodiment, the storage cost (SR) of a data object refers to the disk space occupied by the data object, in GB.

[0060] Optionally, the storage cost of a data object increases by 1 for every 1 GB of storage resources it occupies.

[0061] S202: Determine the computation cost of the target data object according to the offline task computation cost and the real-time task computation cost.

[0062] In this embodiment, the offline task calculation cost and the real-time task calculation cost may be summed to obtain the calculation cost of the target data object.

[0063] S203: Sum the calculation cost and the storage cost to obtain data cost information of the target data object.

[0064] Optionally, the calculation method for obtaining the data cost information of the target data object is as follows:

[0065] DC=CR+SR

[0066] Among them, DC is the data cost information of the target data object, CR is the computing cost, and SR is the storage cost.

[0067] In an embodiment provided by the present application, based on the above solution, optionally, a process of determining data application information based on the usage popularity, usage breadth and data importance of the target data object is as follows: Figure 3 As shown, including:

[0068] S301: Obtain the usage popularity, usage breadth, and data importance of the target data object; the usage popularity is determined based on the cumulative number of times the target data object is read and the popularity values ​​of the target data object's subordinate data objects; the usage breadth is determined based on the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates reports or data exchange services based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level.

[0069] In this embodiment, the more times a data object is read, the higher the popularity of the data object.

[0070] like Figure 4 As shown, let R(a), R(b), ..., R(i) be the cumulative number of reads of data objects a, b, ..., i respectively. Using a recursive algorithm, the heat value H of each data object is equal to the cumulative number of reads of the data object plus the sum of the heat values ​​of all directly downstream data objects of the data object. The calculation formula is as follows:

[0071]

[0072] Among them, H(i) represents the heat value of each direct downstream data object of the data object.

[0073] According to the above formula, the heat value of data object a is: H(a)=R(a)+[H(b)+H(c)+H(d)]

[0074] The heat value of data object b is: H(b)=R(b)+[H(e)+H(f)].

[0075] In this embodiment, the wider the distribution of systems and personnel using the data, the higher the breadth of use.

[0076] like Figure 5 As shown in the figure, a is a data object, A1 and A2 are data API services that use the data object data, B1 and B2 are reports that use the data object data, E1 and E2 are data exchange services that use the data object data, S1 to S3 are IT systems that have subscribed to the data API service, U1 to U3 are enterprise employees who have subscribed to the report, and U4 to U6 are enterprise employees who have subscribed to the data exchange service. The calculation formula for the breadth of data object usage is:

[0077]

[0078] in, The total number of IT systems that use the API services created based on this data object, The number of employees who use reports or data exchange services created based on this data object.

[0079] According to the above usage breadth calculation formula, the usage breadth value of data object a is:

[0080] W(a)=3+6=9

[0081] In this embodiment, the importance (I) of a data object = data warehouse level score (DW) + sensitivity level score (SL) + visitor level score (VL).

[0082] Optionally, the data warehouse level can be graded in a step-by-step manner according to the most commonly used four-layer data warehouse architecture, based on the data warehouse level to which the data object belongs, as shown in Table 1:

[0083] Table 1

[0084] Data warehouse level Score Application Data Layer (ADS) 4 Data Set Summary Layer (DWS) 3 Data detail layer (DWD) 2 Original Data Layer (ODS) 1

[0085] Optionally, the sensitivity level can be graded according to the commonly used 5-level sensitivity level, and graded in a step-by-step manner according to the sensitivity level of the data object, as shown in Table 2:

[0086] Table 2

[0087]

[0088]

[0089] Optionally, the visitor level can be scored according to the role of the visitor who reads the data object. Visitors can be divided into two categories: "system" and "employee". For the system, it is divided into 3 levels according to the definition of the importance of the system. For employees, it is divided into 6 levels according to the position of the employee in the enterprise; as shown in Table 3:

[0090] Table 3

[0091]

[0092] S302: Determine data application information according to the usage popularity, usage breadth and data importance of the target data object.

[0093] In an embodiment provided by the present application, based on the above solution, optionally, the process of determining data health based on the data quality, metadata quality, data security level and data update frequency of the target data object includes:

[0094] The data quality, metadata quality, data security level and data update frequency of the target data object are obtained; the data quality of the target data object is calculated based on the first quality evaluation indicators of each preset first dimension of the target data object and the weight of each quality evaluation indicator; the metadata quality of the target data object is calculated based on the second quality evaluation indicators of each preset second dimension of the metadata of the target data object and the weight of each second quality evaluation indicator; the data security level is determined based on the security policy score of the target data object, the number of anomaly detections and the anomaly repair time;

[0095] Data health is determined based on the data quality, metadata quality, data security level, and data update frequency of the target data object.

[0096] The health degree (DH) mainly evaluates the health of data objects, which is divided into data quality (DQ), metadata quality (MQ), data update frequency (UF) and data security (DS). The calculation formula is:

[0097] DH=DQ+MQ+UF+DS

[0098] In this embodiment, the data quality assessment method can be to assess the data quality score of the data object based on the audit results of the first quality assessment indicators of each first dimension such as completeness, accuracy, compliance, uniqueness, volatility, etc. of the data quality, and the calculation formula is:

[0099] DQ = completeness score*weight + accuracy score*weight + compliance score*weight + uniqueness score*weight + volatility score*weight.

[0100] In this embodiment, the calculation method of each type of score is described below by way of example:

[0101] For the original quality score, if the data object is configured with this type of quality rules and the corresponding quality task is running, the original quality score of this type of data is 10 points. If this type of quality rules are not configured, the original quality score of this type of data is 0 points. For example, if the data object is only configured with integrity rules, its original data quality score is 10 points. If 5 types of rules are configured at the same time, its original data quality score is 50 points.

[0102] For quality anomaly deductions, 0.5 points will be deducted from the original quality score of each category of rules for each anomaly detected.

[0103] For the initiative of rectification, the time spent on quality anomaly repair can be scored by the person in charge of the data object, as shown in Table 4:

[0104] Table 4

[0105] Repair time for quality abnormalities Score <1 batch Add 0.5 points 2-5 batches No points >5 batches Deduct 0.5 points

[0106] In this embodiment, the metadata quality assessment method can be to assess the data quality score of the data object according to the audit results of the second quality assessment indicators of the second dimension of metadata quality, such as completeness, accuracy, and compliance. The calculation formula is:

[0107] DQ = Completeness score * weight + Accuracy score * weight + Compliance score * weight

[0108] In this embodiment, the calculation method of each type of score is described below by way of example:

[0109] For the raw quality score, the raw score for each type of metadata quality is 10 points.

[0110] For quality abnormalities, 0.5 points will be deducted for each abnormality detected.

[0111] For rectification enthusiasm, the person in charge of the data object can be scored based on the time it takes to repair quality anomalies. The specific method is similar to the rectification enthusiasm assessment method in data quality assessment, and can be seen in Table 4 above.

[0112] In this embodiment, the data update frequency evaluation method can be to perform step-by-step scoring according to the frequency of data update, as shown in Table 5:

[0113] Table 5

[0114]

[0115] In this embodiment, data security assessment may be performed based on the number of data risks and the duration of risk processing.

[0116] Optionally, the data security score can be determined based on the security policy and security risk assessment of the data object. The calculation formula is:

[0117] Data security score (DS) = security strategy score (SS) + security risk score (SR)

[0118] The following is an example of security strategy sub-assessment and security risk assessment:

[0119] For security policy sub-evaluation, the rules are shown in Table 6:

[0120] Table 6

[0121]

[0122]

[0123] For the safety risk score, the original score is 5 points, and 0.5 points will be deducted for each abnormality detected.

[0124] In this embodiment, the time spent on repairing quality anomalies can be scored according to the person in charge of the data object. The scoring method is similar to the evaluation method of rectification initiative in data quality assessment, and can be seen in the above Table 4.

[0125] S103: Performing a value assessment on the target data object based on the data cost information, data application information, and data health of the target data object to obtain a value assessment result of the target data object.

[0126] In this embodiment, the value assessment result of the target data object may be output and displayed on a preset display interface, or the value assessment result of the target data object may be sent to a preset terminal.

[0127] Optionally, the value assessment results of the target data object and the value assessment results of the pre-assessed data object may be sorted in descending order of value, and the sorted value assessment results of the target data object and the pre-assessed data object may be output and displayed.

[0128] In an embodiment provided by the present application, based on the above solution, optionally, the target data object is valued based on the data cost information, data application information and data health of the target data object to obtain the value evaluation result of the target data object, including:

[0129] The data application information is divided by the data cost information to obtain the initial data value of the target data object;

[0130] Sum the initial data value and the data health to obtain the value assessment result of the target data object.

[0131] In this embodiment, the specific method of obtaining the value assessment result of the target data object is: data value (DC) = data application information (DA) / data cost information (DC) + data health (DH).

[0132] Through the method provided in the embodiment of the present application, the value assessment result of the data object can be accurately obtained.

[0133] and Figure 1 Corresponding to the method, the embodiment of the present application also provides a data object evaluation device for Figure 1 The specific implementation of the method is shown in the following diagram: Figure 6 As shown, including:

[0134] A determination unit 601 is used to determine a target data object corresponding to the data evaluation instruction in response to the data evaluation instruction;

[0135] The acquisition unit 602 is used to acquire data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object;

[0136] The evaluation unit 603 is used to perform value evaluation on the target data object based on the data cost information, data application information and data health of the target data object to obtain a value evaluation result of the target data object.

[0137] In an embodiment provided in the present application, based on the above solution, optionally, the acquisition unit 602 includes:

[0138] The first acquisition subunit is used to acquire the offline task computing cost, the real-time task computing cost and the storage cost of the target data object; the offline task computing cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task computing cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object;

[0139] A first determination subunit is used to determine the calculation cost of the target data object according to the offline task calculation cost and the real-time task calculation cost;

[0140] The first calculation subunit is used to sum the calculation cost and the storage cost to obtain data cost information of the target data object.

[0141] In an embodiment provided in the present application, based on the above solution, optionally, the acquisition unit 602 includes:

[0142] The second acquisition subunit is used to acquire the usage heat, usage breadth, and data importance of the target data object; the usage heat is determined based on the cumulative number of times the target data object is read and the heat value of the subordinate data objects of the target data object; the usage breadth is determined according to the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates a report or data exchange service based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level;

[0143] The second determination subunit is used to determine data application information according to the usage popularity, usage breadth and data importance of the target data object.

[0144] The specific principles and execution processes of each unit and module in the data object evaluation device disclosed in the above embodiment of the present application are the same as the data object evaluation method disclosed in the above embodiment of the present application. Please refer to the corresponding parts of the data object evaluation method provided in the above embodiment of the present application, and will not be repeated here.

[0145] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, a device where the storage medium is located is controlled to execute the above-mentioned data object evaluation method.

[0146] The present application also provides an electronic device, whose structural diagram is as follows: Figure 7 As shown, it specifically includes a memory 701 and one or more instructions 702, wherein the one or more instructions 702 are stored in the memory 701 and are configured to be executed by one or more processors 703 to perform the following operations:

[0147] In response to the data evaluation instruction, determining a target data object corresponding to the data evaluation instruction;

[0148] Obtain data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object;

[0149] The target data object is valued based on its data cost information, data application information and data health to obtain a value assessment result of the target data object.

[0150] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0152] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0153] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0154] The above is a detailed introduction to a data object evaluation method provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data object evaluation method, characterized in that: include: In response to a data evaluation instruction, determining a target data object corresponding to the data evaluation instruction; Acquire data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computational cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object; The target data object is valued based on the data cost information, data application information and data health of the target data object to obtain a value evaluation result of the target data object.

2. The method according to claim 1, characterized in that The process of determining the data cost information based on the computation cost and the storage cost of the target data object includes: Obtaining the offline task computing cost, real-time task computing cost and storage cost of the target data object; the offline task computing cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task computing cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object; Determining the computation cost of the target data object according to the offline task computation cost and the real-time task computation cost; The calculation cost and the storage cost are summed to obtain data cost information of the target data object.

3. The method according to claim 1, characterized in that The process of determining the data application information based on the usage popularity, usage breadth and data importance of the target data object includes: Obtain the usage popularity, usage breadth, and data importance of the target data object; the usage popularity is determined based on the cumulative number of times the target data object is read and the popularity values ​​of subordinate data objects of the target data object; the usage breadth is determined according to the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates a report or data exchange service based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level; The data application information is determined according to the usage popularity, usage breadth and data importance of the target data object.

4. The method according to claim 1, characterized in that: The process of determining the data health based on the data quality, metadata quality, data security level and data update frequency of the target data object includes: The data quality, metadata quality, data security level and data update frequency of the target data object are obtained; the data quality of the target data object is calculated based on first quality assessment indicators of each preset first dimension of the target data object and the weight of each quality assessment indicator; the metadata quality of the target data object is calculated based on second quality assessment indicators of each preset second dimension of the metadata of the target data object and the weight of each second quality assessment indicator; the data security level is determined based on the security policy score, the number of anomaly detections and the anomaly repair time of the target data object; The data health is determined according to the data quality, metadata quality, data security level and data update frequency of the target data object.

5. The method according to claim 1, characterized in that The performing value assessment on the target data object based on the data cost information, data application information, and data health of the target data object to obtain the value assessment result of the target data object includes: Dividing the data application information by the data cost information to obtain the initial data value of the target data object; The initial data value and the data health are summed to obtain a value assessment result of the target data object.

6. A data object evaluation device, characterized in that: include: a determination unit, configured to determine, in response to a data evaluation instruction, a target data object corresponding to the data evaluation instruction; An acquisition unit, used to acquire data cost information, data application information and data health of the target data object; wherein the data cost information is determined based on the computing cost and storage cost of the target data object; the data application information is determined based on the usage popularity, usage breadth and data importance of the target data object; the data health is determined based on the data quality, metadata quality, data security and data update frequency of the target data object; An evaluation unit is used to perform a value evaluation on the target data object based on data cost information, data application information and data health of the target data object to obtain a value evaluation result of the target data object.

7. The device according to claim 6, characterized in that The acquisition unit comprises: The first acquisition subunit is used to acquire the offline task computing cost, the real-time task computing cost and the storage cost of the target data object; the offline task computing cost is determined based on the running time and CPU occupancy of each batch of offline tasks corresponding to the target data object; the real-time task computing cost is determined based on the CPU occupancy and online time of the online task corresponding to the target data object; the storage cost is determined based on the disk space occupied by the target data object; A first determining subunit, configured to determine a computing cost of the target data object according to the offline task computing cost and the real-time task computing cost; The first calculation subunit is used to sum the calculation cost and the storage cost to obtain data cost information of the target data object.

8. The device according to claim 6, characterized in that The acquisition unit comprises: The second acquisition subunit is used to acquire the usage popularity, usage breadth, and data importance of the target data object; the usage popularity is determined based on the cumulative number of times the target data object is read and the popularity value of the subordinate data objects of the target data object; the usage breadth is determined according to the number of target systems and the number of target users; the target system is a system that creates an API service based on the target data object; the target user is a user who creates a report or data exchange service based on the target data object; the data importance is determined based on the data warehouse level to which the target data object belongs, the sensitivity level of the target data object, and the access user level; The second determining subunit is used to determine the data application information according to the usage popularity, usage breadth and data importance of the target data object.

9. A storage medium, characterized in that: The storage medium includes storage instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the data object evaluation method according to any one of claims 1 to 5.

10. An electronic device, characterized in that: The device comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to execute the data object evaluation method according to any one of claims 1 to 5.

Images