Data product quality evaluation method and device

Abstract

The present invention provides a data product quality evaluation method and device, the method comprising: using a hierarchical analysis method to determine the weight of each secondary indicator in a pre-established data product quality evaluation indicator framework, and obtaining the total weight of the objective indicators and the total weight of the subjective indicators in the secondary indicators based on the weight of each secondary indicator; automatically collecting quality evaluation characteristic parameters of each objective indicator in the data product, and calculating the evaluation score of each objective indicator of the data product; based on the weight of each objective indicator, calculating the overall score of the objective indicators of the data product; receiving language evaluation information of each subjective indicator of the data product from multiple evaluators, converting the language evaluation information into a trapezoidal fuzzy number, and calculating the score of the data product on the current subjective indicator based on the weights of multiple evaluators; calculating the overall score of the subjective indicators of the data product based on the weights and scores of each subjective indicator; and obtaining a comprehensive score for data product quality evaluation based on the overall score of the objective indicators and the overall score of the subjective indicators.

Description

Technical Field

[0001] The present invention relates to the technical field of data product transaction, and in particular to a data product quality evaluation method and device. Background Art

[0002] After data is transformed into products, it is necessary for data to enter the data product trading market for circulation, which is a necessary means for data to maximize its utilization value. The quality level of data products is an important factor affecting whether the data product trading market can operate in an orderly and efficient manner. Low-quality data entering the market will not only reduce market transaction efficiency and increase the operating costs of the data product trading market, but also bring many data quality problems to data buyers, such as data redundancy, data distortion, data missing, data inconsistency, etc., which hinder data buyers from obtaining the maximum utilization value of data. At present, the academic community is still in the exploratory stage for the evaluation of data product quality, and there is no highly recognized effective solution.

[0003] Although some people have proposed data quality evaluation ideas, such as the existing "DAMA Data Management Knowledge System Guide", the Data Quality Assessment Framework (DQAF) constructed by Laura Sebastian-Coleman and others, and the Total Data Quality Management (TDQM) proposed by Richard Y. Wang and others, these ideas not only have the problem of inconsistent data quality evaluation indicators, but also have no reasonable and feasible quantitative methods for each evaluation indicator. Most of the existing technical solutions are qualitative analysis of data quality, and there is no reasonable indicator quantification method, which makes the data quality evaluation results mixed with too many subjective factors. In addition, the existing data quality evaluation does not take into account the product characteristics of data.

[0004] Therefore, how to measure the quality level of data products more objectively and reflect the quality level of data products more intuitively in a quantitative form is an urgent problem to be solved. Summary of the invention

[0005] In view of this, an object of the present invention is to provide a data product quality evaluation method and device to solve the problem that data product quality evaluation is difficult to reasonably quantify in the prior art.

[0006] One aspect of the present invention provides a data product quality evaluation method, the method comprising the following steps:

[0007] Determine the weight of each secondary indicator in the pre-established data product quality evaluation indicator framework by using the hierarchical analysis method, and obtain the total weight of the objective indicators and the total weight of the subjective indicators in the secondary indicators based on the weight of each secondary indicator, wherein the pre-established data product quality evaluation indicator framework includes multiple data evaluation dimensions, each data evaluation dimension includes at least one primary indicator, each primary indicator includes multiple secondary indicators, and the secondary indicators under each data evaluation dimension are divided into objective indicators and subjective indicators;

[0008] Automatically collect quality evaluation characteristic parameters of each objective indicator in the data product, and calculate the evaluation score of each objective indicator of the data product based on the collected quality evaluation characteristic parameters of each objective indicator;

[0009] Calculate the overall score of the objective indicators of the data product based on the weight and score of each objective indicator;

[0010] Receive language evaluation information on each subjective indicator of a data product from multiple evaluators, convert the language evaluation information into a trapezoidal fuzzy number, and calculate the score of the data product on the current subjective indicator based on the weights of the multiple evaluators;

[0011] Based on the weights and scores of each subjective indicator, calculate the overall score of the subjective indicators of the data product;

[0012] A comprehensive score for data product quality evaluation is obtained based on the overall score of the objective indicators and the overall score of the subjective indicators.

[0013] In one embodiment of the present invention, the step of calculating the overall score of the subjective indicators of the data product based on the weights and scores of the subjective indicators includes:

[0014] Based on the weights and scores of each subjective indicator, the comprehensive score of the subjective indicators of the data product is expressed by a trapezoidal fuzzy number;

[0015] Determine the optimal and the worst points of data product quality, use the ideal point method to calculate the distance between the subjective index comprehensive score represented by the trapezoidal fuzzy number and the optimal and the worst points, and obtain the data product quality progress coefficient;

[0016] The overall score of the subjective indicators of the data product is obtained based on the data product quality progress coefficient and the total weight of the subjective indicators.

[0017] In one embodiment of the present invention, the step of calculating the overall score of the subjective index of the data product by using the ideal point method includes: calculating the initial subjective index overall score of the data product represented by the trapezoidal fuzzy number

[0018] In one embodiment of the present invention, the step of obtaining a comprehensive score for data product quality evaluation based on the overall score of the objective indicators and the overall score of the subjective indicators includes: taking the sum of the overall score of the objective indicators and the overall score of the subjective indicators as the comprehensive score for data product quality evaluation.

[0019] In one embodiment of the present invention, the multiple data evaluation dimensions include: a data content evaluation dimension, a product packaging evaluation dimension, and a market circulation evaluation dimension.

[0020] In one embodiment of the present invention, the data content evaluation dimension includes part or all of the following first-level indicators: accuracy indicator, completeness indicator, timeliness indicator, uniqueness indicator and effectiveness indicator; the product packaging evaluation dimension includes the following first-level indicators: metadata standardization indicator; the market circulation evaluation dimension includes the following first-level indicators: service level indicator and / or market feedback indicator.

[0021] In one embodiment of the present invention, the accuracy index includes the following secondary indicators: a grammatical accuracy index and / or a semantic accuracy index; the completeness index includes some or all of the following secondary indicators: a description completeness index, a fact completeness index, a column completeness index, and a reference completeness index; the timeliness index includes the following secondary indicators: a content timeliness index and / or a collection timeliness index; the uniqueness index includes the following secondary indicators: a breadth uniqueness index and / or a depth uniqueness index; the validity index includes the following secondary indicators: a format validity index and / or a quantity validity index; the metadata standardization index includes the following secondary indicators: a metadata accuracy index and / or a metadata completeness index. Indicators; the service level indicators include the following secondary indicators: seller credit score indicators and / or buyers' favorable comments on sellers; the market feedback indicators include the following secondary indicators: product sales indicators and / or buyers' product rating indicators; the column completeness indicator, content timeliness indicator, collection timeliness indicator, depth uniqueness indicator, format validity indicator, quantity validity indicator, metadata completeness indicator, seller credit score indicator, buyers' favorable comments on sellers and buyers' product rating indicators are objective indicators; the grammatical accuracy indicator, semantic accuracy indicator, statement completeness indicator, fact completeness indicator, reference completeness indicator, breadth uniqueness indicator, metadata accuracy indicator and product sales indicator are subjective indicators.

[0022] In one embodiment of the present invention, the step of calculating the evaluation scores of the objective indicators of the data product based on the collected quality evaluation characteristic parameters of the objective indicators includes some or all of the following steps:

[0023] The following formula is used to calculate the evaluation score of the column integrity index: S1 = 1-∑[W i*(number of missing values ​​in column i / total number of values ​​in column i)]; where S1 represents the evaluation score of the column integrity index; W i Indicates the weight of the i-th column attribute in all column attributes of the data set;

[0024] The evaluation score of the content timeliness index is calculated using the following formula: S2 = e^(-Δt) = e^(t2-t1); where S2 represents the evaluation score of the content timeliness index, and t1 and t2 represent the data release time and the latest time covered by the data content, respectively;

[0025] The evaluation score S3 of the collection timeliness index is calculated using the following formula: S3 = e^(-f); where S3 represents the evaluation score of the collection timeliness index, and f represents the data collection frequency;

[0026] The evaluation score of the deep uniqueness index is calculated using the following formula: S4=1-number of repeated data rows ÷ total number of data rows, where S4 represents the evaluation score of the deep uniqueness index;

[0027] The evaluation score of the format validity index is calculated using the following formula: S5 = total number of correct format values ​​÷ total number of data record values, where S5 represents the evaluation score of the format validity index;

[0028] The evaluation score of the quantity validity index is calculated using the following formula: S6 = total number of correct quantity format values ​​÷ total number of data record values, where S6 represents the evaluation score of the collection timeliness index;

[0029] The evaluation score of the metadata integrity index is calculated using the following formula: S7 = the total number of metadata non-missing fields ÷ the total number of standard metadata fields, where S7 represents the evaluation score of the data integrity index;

[0030] Calculate the evaluation score of the seller's credit score indicator based on the seller's credit rating;

[0031] The evaluation score of the buyer's favorable evaluation index is calculated using the following formula: S9 = the total number of favorable evaluation orders from data buyers ÷ the total number of completed orders from data sellers, where S9 represents the evaluation score of the buyer's favorable evaluation index;

[0032] The evaluation score of the buyer rating index is calculated by collecting the scores given to the data product by the data buyer after the data product transaction is completed.

[0033] Another aspect of the present invention provides a data product quality evaluation device, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of any one of the methods described in claims 1-8.

[0034] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned method when executed by a processor.

[0035] The data product quality evaluation method and evaluation device provided by the present invention start from the characteristics of the data product content itself, consider the data quality from five dimensions: data accuracy, data integrity, data timeliness, data uniqueness, and data validity, and design the corresponding refined secondary indicators under each dimension to fully and comprehensively evaluate the quality level of data resources. The present invention can scientifically and intuitively display the quality level of data products in digital form.

[0036] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0038] Figure 1 Schematic diagram of a data product quality evaluation framework in one embodiment of the present invention.

[0039] Figure 2 Schematic diagram of a flow chart of a data product quality evaluation method in one embodiment of the present invention.

[0040] Figure 3 The figure is a flow chart of a subjective indicator quantification method in one embodiment of the present invention.

[0041] Figure 4 Schematic diagram of the function image of the gradient fuzzy number. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0043] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0044] It should be emphasized that the terms “include / comprises / has” when used herein refer to the presence of features, elements, steps or components, but do not exclude the presence or addition of one or more other features, elements, steps or components.

[0045] The present invention aims at data products in the data trading market, starting from the three aspects of data content, product packaging and market circulation, and on the basis of the established data quality evaluation index system, constructs a data product quality evaluation index system (or data product quality evaluation index framework) containing eight primary indicators and eighteen secondary indicators. At the same time, an indicator measurement and quantification method based on the data product quality evaluation index system is designed to realize the intuitive display of the quality level of data products in the form of numbers.

[0046] Among them, the data quality evaluation index system on which the present invention is based is a complete and comprehensive data quality evaluation index system. It starts from the characteristics of the data content itself, considers the data quality from five dimensions: data accuracy, data integrity, data timeliness, data uniqueness, and data validity, and designs corresponding detailed secondary indicators under each dimension, so as to be able to fully and comprehensively evaluate the quality level of data resources.

[0047] The data product quality evaluation index system constructed based on the data quality evaluation index system in the embodiment of the present invention is shown in Table 1 below.

[0048] Table 1. Data product quality evaluation index system:

[0049]

[0050]

[0051] As can be seen from Table 1, based on the data quality evaluation index system, the data product quality evaluation index system of the present invention further explains the characteristics of data productization from the perspective of data sellers and the data product trading market, mainly including metadata standardization, data seller service level and market feedback. These characteristics are data productization characteristics that can be automatically extracted by the trading system.

[0052] As shown in Table 1, each secondary indicator can be divided into two categories: objective indicators and subjective indicators, among which:

[0053] Objective indicators may include: column completeness index, content timeliness index, collection timeliness index, depth uniqueness index, format validity index, quantity validity index, metadata completeness index, seller credit score index, buyer's seller praise index and buyer's product rating index.

[0054] Subjective indicators may include: grammatical accuracy indicators, semantic accuracy indicators, statement completeness indicators, factual completeness indicators, reference completeness indicators, breadth uniqueness indicators, metadata accuracy indicators and product sales indicators.

[0055] The present invention designs different quantification methods for subjective indicators and objective indicators. Figure 1 As shown, the overall indicator quantification idea of ​​the present invention is as follows: In order to minimize the influence of human subjective factors on the data product quality evaluation results and improve the reliability of the data product quality evaluation method, the present invention divides the eighteen secondary indicators in the data product quality evaluation indicator system into ten objective indicators and eight subjective indicators, and adopts different methods to measure and quantify the objective indicators and subjective indicators respectively. For example, for objective indicators, the score of the indicator is calculated based on the characteristics of the data product itself, and the overall score of the objective indicator can be obtained based on the weight of the objective indicator; for the main indicator, experts can evaluate each indicator and convert it into a trapezoidal fuzzy number, and then use the ideal point method (i.e., TOPSIS method) based on the expert weight and the trapezoidal fuzzy number of each indicator to obtain the overall score of the subjective indicator. When using the data product quality evaluation indicator system to evaluate data quality, firstly, the weight of each secondary indicator is determined by experts using the hierarchical analysis method, and the sum of the weights of the subjective indicators is calculated to obtain the total weight of the subjective indicators, and then the performance scores S and C of the data product on the objective indicators and subjective indicators are respectively calculated, so as to obtain the comprehensive evaluation score Q=S+C of the data product quality.

[0056] Figure 2 FIG. 2 is a flow chart of a data product quality evaluation method according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:

[0057] Step S110, using the hierarchical analysis method to determine the weight of each secondary indicator in the pre-established data product quality evaluation indicator framework, and based on the weight of each secondary indicator, obtain the total weight of the objective indicators and the total weight of the subjective indicators in the secondary indicators.

[0058] The pre-established data product quality evaluation indicator framework includes multiple data evaluation dimensions, each data evaluation dimension includes at least one primary indicator, each primary indicator includes multiple secondary indicators, and the secondary indicators under each data evaluation dimension are divided into objective indicators and subjective indicators. Preferably, the pre-established data product quality evaluation indicator framework has the structure shown in Table 1, but the present invention is not limited thereto, and the secondary indicators can also be divided into more or fewer indicators.

[0059] The analytic hierarchy process involved in this step is an existing system analysis method. By establishing the hierarchical relationship of the problem, constructing the judgment matrix, determining the weight vector and performing consistency test on the weight vector, performing the total hierarchical sorting and performing consistency test on it, the weights of the indicators at each level are systematically determined, and then the optimal solution among the alternative solutions is determined. The analytic hierarchy process has a complete theoretical basis, a rigorous structure, and is concise in solving problems, especially in solving unstructured problems.

[0060] Since the specific implementation of determining the weights of different indicators based on the hierarchical analysis method belongs to the existing technology, it will not be described here.

[0061] Step S120: automatically collect quality evaluation characteristic parameters of each objective indicator related to the data product, and calculate the evaluation score of each objective indicator of the data product based on the collected quality evaluation characteristic parameters of each objective indicator.

[0062] This step is used to quantify the quality evaluation of objective indicators. In this step, by analyzing the content of data products or data product transaction-related information, the quality evaluation characteristic parameters of each objective indicator are collected from the data product or the trading platform, and the evaluation scores of each objective indicator of the data product are calculated. Objective indicators are indicators that can be directly quantified by automated means. The quantification of each objective indicator will be described in detail later.

[0063] In this step, it is preferred to calculate the evaluation score of each objective indicator of the data product based on the quality evaluation characteristic parameters of all 10 objective indicators shown in the previous Table 1, but the present invention is not limited to this, and more or fewer objective indicators can also be selected to quantify the quality evaluation.

[0064] Step S130, based on the weights and scores of the various objective indicators, calculate the overall score S of the objective indicators of the data product.

[0065] Step S140, receiving language evaluation information of multiple evaluators on each subjective indicator of the data product, converting the language evaluation information into a trapezoidal fuzzy number, and calculating the score of the data product on the current subjective indicator based on the weights of the multiple evaluators.

[0066] This step belongs to the quantification step of quality evaluation of subjective indicators.

[0067] Step S150, calculating the overall subjective index score C of the data product based on the weights and scores of each subjective index.

[0068] In step S160, a comprehensive score for data product quality evaluation is obtained based on the obtained overall score of the objective indicators and the overall score of the subjective indicators.

[0069] For example, by adding the overall score S of the objective indicators and the overall score C of the subjective indicators, we can get the comprehensive score Q of the data product quality evaluation: Q=S+C.

[0070] The quantification method of the objective indicator in step S120 is described below.

[0071] The calculation of objective indicators is mainly based on the performance of the content of the data product itself in various indicators to evaluate the quality of the data product. The following will explain the specific calculation method of each secondary indicator.

[0072] 1) Column integrity

[0073] Column integrity is reflected by the proportion of missing values ​​in the column. The specific calculation method for the score S1 of the data product in column integrity is:

[0074] S1=1-∑[W i *(number of missing values ​​in column i / total number of values ​​in column i)];

[0075] Among them, W i It refers to the weight of the i-th column attribute in all column attributes of the data set, 0 <W i <1. The calculated S1 is usually a value between 0 and 1.

[0076] 2) Timeliness of content

[0077] The timeliness of the content is reflected by the difference Δt between the data release time t1 and the latest time t2 covered by the data content, where the latest time covered by the data content refers to the time point recorded in the data that is closest to the current time. -x To standardize Δt, we can get the score S2 of the data product in terms of content timeliness. The specific calculation method is:

[0078] S2 = e^(-Δt) = e^(t2-t1).

[0079] 3) Timeliness of collection

[0080] The timeliness of data content collection can be measured by the number of data collection times per unit time, that is, the collection frequency f. Similar to the timeliness of content, the exponential function y=e is used in the present invention. -x To standardize the collection frequency f, we can get the score S3 of the data product on the collection timeliness index. The specific calculation method is:

[0081] S3 = e^(-f).

[0082] 4) Deep uniqueness

[0083] Deep uniqueness can be reflected by the ratio of repeated data rows. The specific calculation method for the deep uniqueness score S4 of a data product is as follows:

[0084] S4 = 1-number of repeated data rows ÷ total number of data rows.

[0085] 5) Format validity

[0086] Format validity is to consider and evaluate the data value from the aspects of data type, predefined enumeration value, storage format, etc., to determine whether the data value meets the special requirements of the attribute data type in a certain attribute. For example, the ID card number format can only be numeric and the length can only be 18 digits. Format validity is reflected by the proportion of values ​​in the correct format in the data. The specific calculation method for the data product's score S5 in format validity is:

[0087] S5 = total number of values ​​with correct format ÷ total number of data record values

[0088] 6) Quantity effectiveness

[0089] Quantity validity refers to whether the data value meets the requirements in terms of accuracy and range. For example, age is a positive integer and is generally less than 130. Quantity validity is reflected by the proportion of valid records in the quantity format in the data value. The specific calculation method for the data product's score S6 in quantity validity is as follows:

[0090] S6 = total number of values ​​with correct quantity format ÷ total number of data record values.

[0091] 7) Metadata integrity

[0092] Metadata integrity is reflected by the ratio of complete fields in the metadata to the total number of fields, that is, the ratio of non-missing fields in the metadata to the total number of fields. The specific calculation method for the metadata integrity score S7 of a data product is as follows:

[0093] S7 = total number of non-missing metadata fields ÷ total number of standard metadata fields.

[0094] 8) Seller credit score

[0095] The seller's credit score is reflected by the data seller's social influence and the historical data left by the data seller in the data trading system. The data trading system automatically evaluates the seller's credit rating based on the data seller's historical behavior and gives the data seller a credit score S8 (0≤S8≤1). For example, the data seller's credit rating can be evaluated based on the data seller's historical behavior and performance ability in the data trading system.

[0096] 9) Buyer's positive reviews

[0097] Buyer favorable comments are reflected by the proportion of the total number of orders in which data buyers give favorable comments to data sellers in the total number of completed orders for data product transactions. The specific calculation method for the score S9 of the data product in buyer favorable comments is as follows:

[0098] S9 = total number of orders with favorable reviews from data buyers ÷ total number of orders completed by data sellers.

[0099] In other embodiments of the present invention, an average star rating score may also be calculated based on the scores corresponding to the star ratings of the data sellers by the data buyers.

[0100] 10) Buyer Ratings

[0101] Buyer ratings are reflected in the scores given by data buyers to data products after the data product transaction is completed. The transaction system automatically collects and calculates the comprehensive score S of the data product based on the buyer's ratings. 10 (0≤S 10 ≤1).

[0102] To sum up, the scores of data products on various objective indicators are determined by the attribute characteristics of the data itself and are not affected by other human factors, thus ensuring the objectivity and effectiveness of the data product quality assessment results.

[0103] In step S130, after obtaining the score of the data product on a single objective indicator, the weights of each objective indicator are combined to calculate the score S of the data product quality on the objective indicator. The larger S is, the better the data product performs on the objective indicator and the higher the data quality level.

[0104] The subjective index quantification method in step S140 is described below.

[0105] Subjective indicators are indicators that are difficult to quantify directly through automated means, and mainly include: grammatical accuracy, semantic accuracy, description completeness, factual completeness, reference completeness, breadth uniqueness, metadata accuracy, and product sales.

[0106] In the present invention, Figure 3 As shown, experts first evaluate the performance of the data product on each subjective indicator, and then use fuzzy mathematics theory to convert the textual evaluation into trapezoidal fuzzy numbers to quantify the subjective indicator evaluation. The following will introduce the specific implementation steps of the subjective indicator quantification method.

[0107] 1. Expert language evaluation

[0108] Invite m experts who are authoritative in data quality assessment to conduct language evaluation on the performance of data products on various subjective indicators, and obtain expert language evaluation information L ij(i.e., the linguistic evaluation of the data product by the i-th expert on the j-th subjective quality indicator, 1≤i≤m, 1≤j≤8, and all are integers), and determine the weight P of each expert i In the expert language evaluation process, different experts are allowed to have different language evaluation scales to reduce the deviation of the data product quality evaluation results caused by inconsistent evaluation standards of different experts. Among them, the language scale refers to the standard scale used by people to evaluate a thing with language, and the language granularity refers to the coarseness of the division of the language phrase set. For example, {very bad, bad, relatively poor, average, good, good, very good} can be used as a language scale for evaluating things, which is a language phrase set with a language granularity of 7.

[0109] (II) Using fuzzy theory to transform language evaluation information into trapezoidal fuzzy numbers

[0110] Trapezoidal fuzzy numbers are an important concept in fuzzy mathematics and an effective tool for expressing uncertainty problems in life in mathematical language. In fuzzy mathematics, it is believed that if a fuzzy set on a certain domain U is given, for any x∈U, there is a unique u(x)∈[0,1] corresponding to it, which represents the membership of x to U. u(x) is called the membership function of x.

[0111] If A is a trapezoidal fuzzy number, then A=(a,b,c,d), and its membership function is:

[0112]

[0113] u(x)=1,x∈[b,c]

[0114]

[0115] u(x)=0, others.

[0116] The function graph corresponding to the trapezoidal fuzzy number is as follows Figure 4 shown.

[0117] The specific conversion formula for converting language evaluation information into trapezoidal fuzzy numbers is designed as follows:

[0118]

[0119] Among them, τ is a natural number greater than 0, 2τ+1 represents the granularity of the language phrase set, k is an integer between [0, 2τ+1], and Ak represents the kth element in the language phrase set.

[0120] The conversion relationship between language evaluation and trapezoidal fuzzy number is used to obtain the evaluation information A of each expert on the quality of data products expressed in trapezoidal fuzzy number. ij (a ij , b ij , cij , d ij ), where A ij (a ij , b ij , c ij , d ij ) represents the trapezoidal fuzzy number corresponding to the language evaluation information of the i-th expert on the j-th subjective indicator, 1≤i≤m, 1≤j≤8, i and j are both integers.

[0121] (III) Using trapezoidal fuzzy numbers to represent the comprehensive scores of each subjective indicator of data products

[0122] By combining the language evaluation information of each expert on the quality of data products A ij The weight of each expert can determine the comprehensive score of the data product on a certain subjective indicator. j , the specific calculation method is:

[0123]

[0124] (IV) Using trapezoidal fuzzy numbers to represent the overall score of the data product on the subjective indicators

[0125] By A j and the weight of each subjective indicator I j The comprehensive score A of the data product expressed as a trapezoidal fuzzy number on the subjective index can be calculated. The specific calculation formula is:

[0126]

[0127] (V) Furthermore, the ideal point method can be used to calculate the quality of data products and obtain a comprehensive score for data product quality assessment.

[0128] The ideal point method, also known as the TOPSIS method, is a commonly used calculation method in multi-objective decision-making when doing decision analysis. The ideal point method calculates the distance between the evaluated object and the optimal point and the worst point to obtain the progress coefficient of the evaluated object, and then determines the optimal solution in the decision problem by sorting the progress coefficient. Only when the evaluated object is closest to the optimal point and farthest from the worst point can it be selected as the optimal solution.

[0129] In the present invention, the worst point of data quality is d0 (0, 0, 0, 0), and the optimal point is d1 (1, 1, 1, 1). By calculating the distance between A (a, b, c, d) and the optimal and worst points, the data product quality progress coefficient can be obtained, and combined with the total weight of the subjective indicators, the final score C of the data product quality subjective indicator evaluation can be obtained.

[0130] The specific calculation process is:

[0131] ① Calculate the distance D1 and D0 between A(a, b, c, d) and the best and worst points respectively:

[0132]

[0133]

[0134] ② Calculate the data product quality progress coefficient:

[0135]

[0136] The larger the progress coefficient c is, the closer A(a, b, c, d) is to the worst point and the closer it is to the best point, and the higher the quality level of the data product in terms of subjective indicators.

[0137] ③ Calculate the comprehensive score C of data product quality in subjective indicators:

[0138] C=c*I t

[0139] Among them, I t It is the weight of subjective index in all indexes.

[0140] After calculating the overall score S of the objective indicators and the overall score C of the subjective indicators, the comprehensive score Q of the data product quality evaluation can be obtained in step S160: Q=S+C.

[0141] Based on the above method of the present invention, the quality level of data products can be effectively evaluated based on the characteristics of the data itself and expert language evaluation, and the quality level of data products can be intuitively displayed in digital form.

[0142] The present invention has the following advantages:

[0143] A complete and comprehensive data quality evaluation index system has been established. On the basis of the data quality evaluation index system, a universal data product quality evaluation index system has been constructed in combination with the product characteristics of the data that can be automatically extracted by the trading system. The quality level of data products is comprehensively measured from aspects such as data content, data product packaging, and data product market circulation, providing a reference for solving data product quality evaluation issues.

[0144] This invention proposes a method for measuring and quantifying each secondary indicator in the data product quality evaluation index system, which can effectively evaluate the quality level of data products based on the characteristics of the data itself and expert language evaluation, and intuitively display the quality level of data products in digital form.

[0145] The present invention also provides a data product quality evaluation device, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0146] The present invention also relates to a storage medium on which computer program code can be stored, and when the program code is executed, various embodiments of the method of the present invention can be implemented. The storage medium can be a tangible storage medium, such as a CD, a USB flash drive, a floppy disk, a hard disk, etc.

[0147] It should be understood by those of ordinary skill in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information.

[0148] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0149] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data product quality evaluation method, characterized in that: The method comprises the following steps: The weight of each secondary indicator in the pre-established data product quality evaluation indicator framework is determined by using the hierarchical analysis method, and the total weight of the objective indicators and the total weight of the subjective indicators in the secondary indicators are obtained based on the weight of each secondary indicator, wherein the pre-established data product quality evaluation indicator framework includes multiple data evaluation dimensions, each data evaluation dimension includes at least one primary indicator, each primary indicator includes multiple secondary indicators, and the secondary indicators under each data evaluation dimension are divided into objective indicators and subjective indicators; the multiple data evaluation dimensions include: data content evaluation dimension, product packaging evaluation dimension and market circulation evaluation dimension; Automatically collect quality evaluation characteristic parameters of each objective indicator in the data product, and calculate the evaluation score of each objective indicator of the data product based on the collected quality evaluation characteristic parameters of each objective indicator; Calculate the overall score of the objective indicators of the data product based on the weight and score of each objective indicator; Receive language evaluation information on each subjective indicator of a data product from multiple evaluators, convert the language evaluation information into a trapezoidal fuzzy number, and calculate the score of the data product on the current subjective indicator based on the weights of the multiple evaluators; Based on the weights and scores of each subjective indicator, calculate the overall score of the subjective indicators of the data product; A comprehensive score for data product quality evaluation is obtained based on the overall score of the objective indicators and the overall score of the subjective indicators.

2. The method according to claim 1, characterized in that: The step of calculating the overall score of the subjective indicators of the data product based on the weights and scores of the subjective indicators includes: Based on the weights and scores of each subjective indicator, the comprehensive score of the subjective indicators of the data product is expressed by a trapezoidal fuzzy number; Determine the optimal and the worst points of data product quality, use the ideal point method to calculate the distance between the subjective index comprehensive score represented by the trapezoidal fuzzy number and the optimal and the worst points, and obtain the data product quality progress coefficient; The overall score of the subjective indicators of the data product is obtained based on the data product quality progress coefficient and the total weight of the subjective indicators.

3. The method according to claim 1, characterized in that The step of calculating the overall score of the subjective index of the data product by using the ideal point method includes: Calculate the overall score of the initial subjective indicators of the data product represented by trapezoidal fuzzy numbers.

4. The method according to claim 1, characterized in that The step of obtaining a comprehensive score for data product quality evaluation based on the overall score of the objective indicators and the overall score of the subjective indicators comprises: The sum of the overall scores of objective indicators and subjective indicators is used as the comprehensive score for data product quality evaluation.

5. The method according to claim 1, characterized in that: The data content evaluation dimensions include some or all of the following primary indicators: accuracy indicator, completeness indicator, timeliness indicator, uniqueness indicator and effectiveness indicator; The product packaging evaluation dimensions include the following primary indicators: metadata normative indicators; The market circulation evaluation dimension includes the following primary indicators: service level indicators and / or market feedback indicators.

6. The method according to claim 5, characterized in that: The accuracy index includes the following secondary indexes: grammatical accuracy index and / or semantic accuracy index; The integrity index includes some or all of the following secondary indicators: description integrity index, fact integrity index, column integrity index and reference integrity index; The timeliness index includes the following secondary indexes: content timeliness index and / or collection timeliness index; The uniqueness index includes the following secondary indexes: breadth uniqueness index and / or depth uniqueness index; The effectiveness index includes the following secondary indicators: format effectiveness index and / or quantity effectiveness index; The metadata standardization indicators include the following secondary indicators: metadata accuracy indicators and / or metadata completeness indicators; The service level indicators include the following secondary indicators: seller credit score indicators and / or buyers’ favorable comments on sellers; The market feedback indicators include the following secondary indicators: product sales indicators and / or buyer rating indicators for products; The column integrity index, content timeliness index, collection timeliness index, depth uniqueness index, format validity index, quantity validity index, metadata integrity index, seller credit score index, buyer's seller praise index and buyer's product rating index are objective indicators; The grammatical accuracy index, semantic accuracy index, statement completeness index, fact completeness index, reference completeness index, breadth uniqueness index, metadata accuracy index and product sales index are subjective indicators.

7. The method according to claim 6, characterized in that The step of calculating the evaluation scores of the objective indicators of the data product based on the collected quality evaluation characteristic parameters of the objective indicators includes some or all of the following steps: The following formula is used to calculate the evaluation score of the column integrity index: S1 = 1-∑[W i *(number of missing values ​​in column i / total number of values ​​in column i)]; where S1 represents the evaluation score of the column integrity index; W i Indicates the weight of the i-th column attribute in all column attributes of the data set; The evaluation score of the content timeliness index is calculated using the following formula: S2 = e^(-Δt) = e^(t2-t1); where S2 represents the evaluation score of the content timeliness index, and t1 and t2 represent the data release time and the latest time covered by the data content, respectively; The evaluation score S3 of the collection timeliness index is calculated using the following formula: S3 = e^(-f); where S3 represents the evaluation score of the collection timeliness index, and f represents the data collection frequency; The evaluation score of the deep uniqueness index is calculated using the following formula: S4=1-number of repeated data rows ÷ total number of data rows, where S4 represents the evaluation score of the deep uniqueness index; The evaluation score of the format validity index is calculated using the following formula: S5 = total number of correct format values ​​÷ total number of data record values, where S5 represents the evaluation score of the format validity index; The evaluation score of the quantity validity index is calculated using the following formula: S6 = total number of correct quantity format values ​​÷ total number of data record values, where S6 represents the evaluation score of the collection timeliness index; The evaluation score of the metadata integrity index is calculated using the following formula: S7 = the total number of metadata non-missing fields ÷ the total number of standard metadata fields, where S7 represents the evaluation score of the data integrity index; Calculate the evaluation score of the seller's credit score indicator based on the seller's credit rating; The evaluation score of the buyer's favorable evaluation index is calculated using the following formula: S9 = the total number of favorable evaluation orders from data buyers ÷ the total number of completed orders from data sellers, where S9 represents the evaluation score of the buyer's favorable evaluation index; The evaluation score of the buyer rating index is calculated by collecting the scores given to the data product by the data buyer after the data product transaction is completed.

8. A data product quality evaluation device, comprising a processor and a memory, characterized in that: The memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of any one of the methods described in claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images