Data quality evaluation method and device, equipment, storage medium and program product
Through comprehensive evaluation and fuzzy hierarchical analysis of multiple key data quality indicators, the problem of insufficient evaluation of single indicators in the existing technology is solved, the credibility and comprehensiveness of data quality assessment is improved, and powerful data is provided for artificial intelligence. support.
Patent Information
- Application Number
- CN202411077512.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
Existing data quality assessment methods only focus on a single indicator and cannot comprehensively consider the mutual influence between multiple indicators, resulting in insufficient credibility of data assessment.
Through data extraction, scoring calculation and fuzzy hierarchy analysis, the interaction between multiple key indicators (completeness, accuracy, consistency, diversity, balance, logic, volatility and timeliness) is comprehensively evaluated to determine the data. Quality assessment results.
It improves the credibility of data quality assessment, realizes a comprehensive dynamic assessment of the overall quality of data, and provides more accurate and reliable artificial intelligence data support.
Smart Images

Figure CN119089110A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data quality assessment method, device, equipment, storage medium and program product. Background Art
[0002] With the rapid development of computer software and hardware technology, artificial intelligence (AI) has made significant breakthroughs and has been increasingly applied in many fields of human production and life, such as finance, medical care, and autonomous driving. However, with the widespread application of artificial intelligence, the continuous emergence of data quality problems such as insufficient data volume and lack of representativeness has become an obstacle for artificial intelligence to penetrate into practical application scenarios. In order to solve this problem, in the past, research was mainly focused on optimizing model structure or enhancing learning algorithms to improve the credibility of artificial intelligence. However, due to problems related to data quality or labeling and lack of confidence in data quality encountered during the research process, the research focus later shifted from model optimization to data improvement to improve data quality. Compared with model training, selection, and deployment, the time spent on data loading, cleaning, and visualization is almost twice that of the former, but thanks to the development of high-quality data, breakthroughs in artificial intelligence have also been achieved.
[0003] At present, existing data quality assessments often focus on a single assessment indicator, such as sample quantity, sample integrity or consistency, and then determine whether the data quality is qualified by comparing the threshold. However, the various indicators of data quality are interrelated, and changes in one indicator may affect other indicators. For example, excessive pursuit of data integrity may sacrifice data accuracy, thereby affecting the credibility of data assessment. Summary of the invention
[0004] In view of the problems existing in the prior art, the embodiments of the present invention provide a data quality assessment method, device, equipment, storage medium and program product, which can comprehensively consider and balance various indicators, achieve a comprehensive assessment of data quality, and improve the credibility of data quality assessment.
[0005] In a first aspect, an embodiment of the present invention provides a data quality assessment method, comprising:
[0006] Extract data from the target data to be evaluated and obtain data parameters of multiple key indicators related to data quality evaluation;
[0007] Calculate the score of each of the key indicators according to the data parameters;
[0008] Performing fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator;
[0009] The data quality assessment result of the target data is determined according to the scores and weights of the key indicators.
[0010] As an improvement of the above solution, the method further includes:
[0011] Establish rules based on predefined indicators, perform full life cycle analysis on the target data, and determine multiple key indicators of the target data.
[0012] As an improvement of the above scheme, the key indicators include at least two indicators among an integrity indicator, an accuracy indicator, a consistency indicator, a diversity indicator, a balance indicator, a logic indicator, a volatility indicator, a uniqueness indicator and a timeliness indicator.
[0013] As an improvement of the above solution, the data parameters include: the actual sample number of samples in the target data, the first sample number with null values and the reference sample number, the actual feature number and the reference feature number of each sample in the target sample;
[0014] Calculating the scores of the key indicators according to the data parameters includes:
[0015] The score of the integrity indicator is calculated based on the actual sample number, the first sample number, the benchmark sample number, the actual number of features and the benchmark number of features.
[0016] As an improvement of the above solution, the step of calculating the score of the integrity indicator according to the actual number of samples, the first number of samples, the benchmark number of samples, the actual number of features, and the benchmark number of features includes:
[0017] Calculating a feature integrity score according to the actual number of features and the benchmark number of features;
[0018] Calculate a sample integrity score based on the actual sample size, the first sample size, and the benchmark sample size;
[0019] Calculate a data size adequacy score based on the actual sample size and the benchmark sample size;
[0020] The score of the integrity indicator is calculated based on the feature integrity score, the sample integrity score and the data scale adequacy score.
[0021] As an improvement of the above solution, the data parameters include: the actual feature number and the reference feature number of each sample in the target sample;
[0022] Calculating the scores of the key indicators according to the data parameters includes:
[0023] A score of the accuracy index is calculated based on the actual number of features and the benchmark number of features.
[0024] As an improvement of the above solution, the data parameters include: the actual sample number of samples in the target data, the second sample number of format conflicts, and the third sample number of content conflicts;
[0025] Calculating the scores of the key indicators according to the data parameters includes:
[0026] The score of the consistency index is calculated according to the actual sample size, the second sample size, and the third sample size.
[0027] As an improvement of the above solution, the step of calculating the score of the consistency index according to the actual sample size, the second sample size, and the third sample size includes:
[0028] Calculating a first sample score according to the actual sample size and the second sample size;
[0029] Calculating a second sample score according to the actual sample size and the third sample size;
[0030] A score of the consistency index is calculated based on the first sample score and the second sample score.
[0031] As an improvement of the above solution, the data parameters include: the number of actual data channels and the number of reference data channels of the samples in the target data, the number of fourth samples belonging to different categories and the number of reference samples in the target data;
[0032] Calculating the scores of the key indicators according to the data parameters includes:
[0033] The score of the diversity indicator is calculated according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number.
[0034] As an improvement of the above solution, the step of calculating the score of the diversity indicator according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number includes:
[0035] Calculate a data source breadth score according to the actual number of data channels and the benchmark number of data channels;
[0036] Calculating a data category richness score based on the fourth sample size and the benchmark sample size;
[0037] The score of the diversity indicator is calculated according to the data source breadth score and the data category richness score.
[0038] As an improvement of the above solution, the data parameter includes: the number of fourth samples belonging to different categories in the target data;
[0039] Calculating the scores of the key indicators according to the data parameters includes:
[0040] The score of the balance index is calculated according to the number of fourth samples belonging to different categories in the target data.
[0041] As an improvement of the above solution, the step of calculating the score of the balance index according to the number of fourth samples belonging to different categories in the target data includes:
[0042] Obtaining a maximum value and a minimum value of the number of samples in the fourth number of samples belonging to different categories in the target data;
[0043] Calculating a sample quantity mean of a fourth sample quantity belonging to different categories in the target data;
[0044] The score of the balance index is calculated based on the maximum sample size, the minimum sample size and the mean sample size.
[0045] As an improvement of the above solution, the data parameters include: the actual sample number of samples in the target data, and the fifth sample number of logical errors;
[0046] Calculating the scores of the key indicators according to the data parameters includes:
[0047] The score of the logic index is calculated based on the actual sample size and the fifth sample size.
[0048] As an improvement of the above solution, the data parameters include: the number of sixth samples of the target data in different life cycles;
[0049] Calculating the scores of the key indicators according to the data parameters includes:
[0050] The score of the volatility index is calculated according to the sixth sample quantity of the target data in different life cycles.
[0051] As an improvement of the above solution, the step of calculating the score of the volatility index according to the sixth sample quantity of the target data in different life cycles includes:
[0052] Obtaining the sixth sample quantity corresponding to the latest life cycle in the target data and the sixth sample quantity corresponding to the target life cycle; wherein the target life cycle is any other life cycle in the target data except the latest life cycle before the latest life cycle;
[0053] The score of the volatility indicator is calculated according to the sixth sample quantity corresponding to the latest life cycle and the sixth sample quantity corresponding to the target life cycle.
[0054] As an improvement of the above solution, the data parameters include: the actual sample number of samples in the target data, the seventh sample number of repeated samples;
[0055] Calculating the scores of the key indicators according to the data parameters includes:
[0056] The score of the uniqueness indicator is calculated based on the actual sample size and the seventh sample size.
[0057] As an improvement of the above solution, the data parameters include: the non-updated time length and the benchmark update time length of the target data;
[0058] Calculating the scores of the key indicators according to the data parameters includes:
[0059] The score of the timeliness index is calculated according to the non-updated time length of the target data and the benchmark update time length.
[0060] As an improvement of the above solution, the fuzzy hierarchical analysis is performed on the multiple key indicators to obtain the weight of each key indicator, including:
[0061] According to the importance of each of the key indicators, a proportional scale table is established; wherein the proportional scale table includes quantitative values for comparing two of the key indicators;
[0062] According to the quantitative values of the comparison of the two key indicators in the ratio scale table, a fuzzy factor of the comparison of the corresponding two key indicators is calculated by a fuzzy membership function;
[0063] The weight of each of the key indicators is determined according to the fuzzy factor.
[0064] As an improvement of the above solution, determining the weight of each key indicator according to the fuzzy factor includes:
[0065] According to the fuzzy factors, a judgment matrix is established;
[0066] According to the judgment matrix, the weight of each key indicator is determined.
[0067] As an improvement of the above solution, determining the weight of each key indicator according to the judgment matrix includes:
[0068] Performing a consistency check on the judgment matrix;
[0069] In the case of passing the consistency check, extracting the eigenvector where the maximum eigenvalue is located from the judgment matrix; wherein each eigenvalue in the eigenvector corresponds to one of the key indicators;
[0070] Normalizing the feature vector to obtain a weight vector; wherein each weight value in the weight vector corresponds to one of the key indicators;
[0071] Each weight value in the weight vector is used as the weight of its corresponding key indicator.
[0072] In a second aspect, an embodiment of the present invention provides a data quality assessment device, comprising:
[0073] A data extraction module is used to extract data from the target data to be evaluated and obtain data parameters of multiple key indicators related to data quality evaluation;
[0074] A scoring calculation module, used to calculate the scores of each of the key indicators according to the data parameters;
[0075] A fuzzy hierarchical analysis module is used to perform fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator;
[0076] The data quality assessment module is used to determine the data quality assessment result of the target data according to the scores and weights of each of the key indicators.
[0077] In a third aspect, an embodiment of the present invention provides a data quality assessment device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the data quality assessment method as described in any one of the first aspects is implemented.
[0078] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the data quality assessment method as described in any one of the first aspects.
[0079] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the data quality assessment method as described in any one of the first aspects.
[0080] Compared with the prior art, a data quality assessment method, device, equipment, storage medium and program product in an embodiment of the present invention extracts data from the target data to be assessed to obtain data parameters of multiple key indicators related to data quality assessment; then, based on the data parameters, the scores of each of the key indicators are calculated, and the data situation is measured from different dimensions, so that the advantages and disadvantages of these dimensions can be more clearly understood; then, fuzzy hierarchical analysis is performed on the multiple key indicators to obtain the weights of each of the key indicators; based on the scores and weights of each of the key indicators, the data quality assessment results of the target data are determined, and by performing fuzzy hierarchical analysis on the key indicators, the interaction between the key indicators can be focused on, and the various indicators can be comprehensively considered and balanced to achieve a comprehensive and dynamic assessment of the overall data quality, improve the credibility of the data quality assessment, and provide strong data support for the development of trustworthy artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] In order to more clearly illustrate the technical solution of the present invention, the drawings used in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0082] Figure 1 is a flow chart of a data quality assessment method provided by an embodiment of the present invention;
[0083] Figure 2 is a structural block diagram of a data quality assessment device provided by an embodiment of the present invention;
[0084] Figure 3 It is a structural block diagram of a data quality assessment device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0085] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0086] It is understood that the various numbers involved in the embodiments of the present invention are only for the convenience of description and are not intended to limit the scope of the present application. The size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.
[0087] In embodiments of the present invention, relational terms such as first, second, third, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "includes..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. The term "multiple or several" refers to two or more. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0088] See also Figure 1 , Figure 1 : is a flow chart of a data quality assessment method provided by an embodiment of the present invention. The data quality assessment method specifically includes:
[0089] S11: extracting the target data to be evaluated to obtain data parameters of multiple key indicators related to data quality evaluation;
[0090] S12: Calculating the score of each of the key indicators according to the data parameters;
[0091] S13: Performing fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator;
[0092] S14: Determine the data quality assessment result of the target data according to the scores and weights of the key indicators.
[0093] It should be noted that the data quality assessment method described in the embodiment of the present invention can be implemented by computing devices such as servers and computers. First, the data parameters of multiple key indicators related to data quality assessment are mined from the target data to be evaluated, and the scores of each key indicator are calculated based on the mined data parameters. By measuring the data situation from different dimensions, the advantages and disadvantages of these dimensions can be more clearly understood, providing theoretical guidance and feedback for researchers in implementing improvements; then, fuzzy hierarchical analysis is performed on multiple key indicators, the weights of each key indicator are calculated, and the scores and weights of each key indicator are determined to determine the final data quality assessment results of the target data. By performing fuzzy hierarchical analysis on the key indicators, the interaction between each key indicator can be focused on, and each indicator can be comprehensively considered and balanced. By harmoniously unifying subjective and objective standards, the data subject to multi-dimensional constraints can be truly evaluated, the influence of human subjectivity can be alleviated, and a comprehensive dynamic evaluation of the overall quality of the data can be achieved, the credibility of the data quality evaluation can be improved, and strong data support can be provided for the development of trustworthy artificial intelligence.
[0094] Furthermore, the method further comprises:
[0095] Establish rules based on predefined indicators, perform full life cycle analysis on the target data, and determine multiple key indicators of the target data.
[0096] Among them, the target data can be a training data set for artificial intelligence model training, or it can be other data sets, which is not specifically limited in the embodiments of the present invention. In the embodiment of the present invention, the training data set for artificial intelligence model training is taken as the target data as an example to describe the data quality assessment process in detail, and the target data includes several samples. In different life cycles, the samples in the target data may be updated, so that the samples and sample quantities of the target data in different life cycles may not be different. The key indicators related to data quality assessment can be determined by performing a full life cycle analysis on the target data, and the data status can be evaluated from different levels / dimensions by calculating the separate scores of each key indicator.
[0097] The key indicators include at least two indicators among a completeness indicator, an accuracy indicator, a consistency indicator, a diversity indicator, a balance indicator, a logic indicator, a volatility indicator, a uniqueness indicator and a timeliness indicator.
[0098] The predefined indicator establishment rules (also described as indicator system) include: indicator definitions of each key indicator and data parameters required for the evaluation of the corresponding key indicators, as follows:
[0099] The indicator definition of the data integrity indicator is: there are no gaps or missing values (also described as null values) in the target data. Since data containing missing values can only provide partial information, it will cause the artificial intelligence model to make inaccurate assumptions and illusions, so it is necessary to evaluate the data integrity of the target data. Data integrity includes the integrity of the features in the sample, the completeness of the sample, and the adequacy of the data scale.
[0100] Data parameters required for data integrity indicator evaluation: the actual sample number of samples in the target data, the first sample number with null values and the benchmark sample number, the actual feature number and the benchmark feature number of each sample in the target sample;
[0101] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0102] The score of the integrity indicator is calculated based on the actual sample number, the first sample number, the benchmark sample number, the actual number of features and the benchmark number of features.
[0103] Further, the calculating the score of the integrity indicator according to the actual sample number, the first sample number, the benchmark sample number, the actual number of features and the benchmark number of features includes:
[0104] Calculating a feature integrity score according to the actual number of features and the benchmark number of features;
[0105] Calculate a sample integrity score based on the actual sample size, the first sample size, and the benchmark sample size;
[0106] Calculate a data size adequacy score based on the actual sample size and the benchmark sample size;
[0107] The score of the integrity indicator is calculated based on the feature integrity score, the sample integrity score and the data scale adequacy score.
[0108] Exemplarily, the feature integrity score of the target data is equal to the average of the quotients of the actual number of features of all samples and the benchmark number of features, the sample integrity score is equal to the quotient of the actual number of samples and the difference between the actual number of samples and the first number of samples with null values, and the data scale adequacy score is equal to the quotient of the actual number of samples and the benchmark number of samples; the score of the integrity indicator is obtained by calculating the average of the feature integrity score, the sample integrity score and the data scale adequacy score.
[0109] In the embodiment of the present invention, the score of the integrity index of the target data is calculated from three aspects: the integrity of the features in the sample, the integrity of the sample, and the adequacy of the data scale. The specific calculation process is as follows:
[0110] The feature completeness score S of a sample n 11n = actual number of features / number of benchmark features; then the feature integrity score of the target data is Among them, n∈N, N represents the actual number of samples in the target data.
[0111] Sample integrity score S 12 =(actual sample size - first sample size with null value) / actual sample size.
[0112] Data size adequacy score S 13 =Actual sample size / benchmark sample size.
[0113] Scoring of completeness indicators
[0114] Definition of the accuracy index: the degree of closeness between the target data and the normal true value. Since abnormal data can mislead the decision-making results of the artificial intelligence model and seriously affect the reliability of the artificial intelligence model, it is necessary to evaluate the accuracy of the target data. For example, abnormal data in the target data can be found through anomaly detection technology.
[0115] Data parameters required for accuracy indicator evaluation: the actual number of features and the benchmark number of features of each sample in the target sample.
[0116] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0117] A score of the accuracy index is calculated based on the actual number of features and the benchmark number of features.
[0118] For example, the accuracy score of the target data is equal to the average of the quotients between the actual number of features and the number of benchmark features of all samples. For example, the accuracy score S of a sample n is 2n = actual number of features / number of benchmark features; then the score of the accuracy index of the target data is
[0119] Definition of consistency index: Samples will not contradict each other. Contradictory samples in the target data may confuse AI models and prevent them from understanding the correct relationship between data sets; this confusion will reduce the prediction stability of AI models in various scenarios and even over time, so consistency assessment of target data is required. The consistency of target data includes format consistency and content consistency; format consistency means that the feature values of the same feature will not appear in two or more modes; content consistency means that the feature values of the data are the same but the label values are different.
[0120] The data parameters required for consistency index evaluation include: the actual sample number of samples in the target data, the second sample number of format conflicts, and the third sample number of content conflicts.
[0121] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0122] The score of the consistency index is calculated according to the actual sample size, the second sample size, and the third sample size.
[0123] Further, calculating the score of the consistency index according to the actual sample size, the second sample size, and the third sample size includes:
[0124] Calculating a first sample score according to the actual sample size and the second sample size;
[0125] Calculating a second sample score according to the actual sample size and the third sample size;
[0126] A score of the consistency index is calculated based on the first sample score and the second sample score.
[0127] Exemplarily, the first sample score of the format conflict is equal to the quotient of the difference between the actual sample number and the second sample number indicating the format conflict and the actual sample number; the second sample score of the content conflict is equal to the quotient of the difference between the actual sample number and the third sample number indicating the content conflict and the actual sample number; the score of the consistency index is obtained by calculating the average of the first sample score of the format conflict and the second sample score of the content conflict.
[0128] In the embodiment of the present invention, the score of the consistency index of the target data is calculated from the two aspects of the format consistency and content consistency of the sample target data. The specific calculation process is as follows:
[0129] First sample score S for format conflict 31 =(actual number of samples-number of second samples with format conflict) / actual number of samples.
[0130] The second sample score S of content conflict 32 =(actual sample number-third sample number of content conflicts) / actual sample number.
[0131] Scoring of consistency indicators
[0132] Definition of diversity index: rich sample types and wide sources. When the target data samples lack diversity, the AI model may overfit. Therefore, in order to ensure the versatility of AI models in different scenarios, it is crucial to evaluate the breadth of data sources and the richness of categories of samples in the target data.
[0133] The data parameters required for diversity index evaluation include: the number of actual data channels and the number of benchmark data channels of samples in the target data, the number of fourth samples belonging to different categories and the number of benchmark samples in the target data.
[0134] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0135] The score of the diversity indicator is calculated according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number.
[0136] Further, the calculating the score of the diversity indicator according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number includes:
[0137] Calculate a data source breadth score according to the actual number of data channels and the benchmark number of data channels;
[0138] Calculating a data category richness score based on the fourth sample size and the benchmark sample size;
[0139] The score of the diversity indicator is calculated according to the data source breadth score and the data category richness score.
[0140] Exemplarily, the data source breadth score is equal to the quotient of the actual number of data channels and the benchmark number of data channels; the data category richness score is equal to the average of the quotients of the fourth sample number of all categories and the benchmark sample number; the score of the diversity indicator is obtained by calculating the average of the data source breadth score and the data category richness score.
[0141] In the embodiment of the present invention, the score of the diversity index of the target data is calculated from two aspects: the breadth of the data source and the richness of the data categories of the samples of the target data. The specific calculation process is as follows:
[0142] Data source breadth score S 41 =Actual data channel number / benchmark data channel number.
[0143] The data category richness score S of a category m 42m = the fourth sample number under category m / the benchmark sample number; then the data category richness score of the target data Among them, m∈M, M represents the number of sample categories in the target data.
[0144] Scoring of diversity indicators
[0145] Definition of the balance index: There is no significant deviation in the number of samples in each category. Significant differences in the number of samples in different categories may lead to discriminatory decision-making results of the artificial intelligence model, so it is necessary to evaluate the balance of the target data.
[0146] The data parameters required for the balance indicator evaluation include: the number of fourth samples belonging to different categories in the target data.
[0147] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0148] The score of the balance index is calculated according to the number of fourth samples belonging to different categories in the target data.
[0149] Further, calculating the score of the balance index according to the number of fourth samples belonging to different categories in the target data includes:
[0150] Obtaining a maximum value and a minimum value of the number of samples in the fourth number of samples belonging to different categories in the target data;
[0151] Calculating a sample quantity mean of a fourth sample quantity belonging to different categories in the target data;
[0152] The score of the balance index is calculated based on the maximum sample size, the minimum sample size and the mean sample size.
[0153] For example, the calculation of the score S5 of the balance index needs to first find the maximum value (i.e. the above-mentioned maximum value of the sample number) and the minimum value (i.e. the above-mentioned minimum value of the sample number) of the fourth sample numbers belonging to different categories in the target data, and calculate the sample number mean of the fourth sample numbers of all categories; then calculate the first difference between the maximum value of the sample number and the sample number mean, and the second difference between the maximum value of the sample number and the minimum value of the sample number; then calculate the quotient between the third difference between 1 and the first difference and the second difference to obtain the score of the balance index; the specific calculation formula is as follows:
[0154] The score of the balance index S5=1-(maximum value of the sample size-mean value of the first sample data) / (maximum value of the sample size-minimum value of the sample size).
[0155] Definition of logical indicators: whether the relationship between the characteristics of samples in the target data conforms to facts or common sense. Data logical evaluation can find data errors left after data accuracy evaluation.
[0156] The data parameters required for the evaluation of the logic index include: the actual sample number of samples in the target data, and the fifth sample number of logical errors.
[0157] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0158] The score of the logic index is calculated based on the actual sample size and the fifth sample size.
[0159] For example, the score of the balance index is obtained by calculating the quotient between the difference between the actual sample number and the fifth sample number indicating a logical error and the actual sample number; the specific calculation formula is as follows:
[0160] The score S6 of the balance index = (actual sample number - fifth sample number of logical errors) / actual sample number.
[0161] The indicator definition of volatility index is the difference between historical samples and the latest samples. The volatility evaluation of target data can be used to study the periodic changes or distribution patterns of target data.
[0162] The data parameters required for volatility indicator evaluation include: the number of sixth samples of the target data in different life cycles.
[0163] Specifically: the step of calculating the scores of the key indicators according to the data parameters includes:
[0164] The score of the volatility index is calculated according to the sixth sample quantity of the target data in different life cycles.
[0165] Further, calculating the score of the volatility index according to the sixth sample quantity of the target data in different life cycles includes:
[0166] Obtaining the sixth sample quantity corresponding to the latest life cycle in the target data and the sixth sample quantity corresponding to the target life cycle; wherein the target life cycle is any other life cycle in the target data except the latest life cycle before the latest life cycle;
[0167] The score of the volatility indicator is calculated according to the sixth sample quantity corresponding to the latest life cycle and the sixth sample quantity corresponding to the target life cycle.
[0168] In the embodiment of the present invention, the target life cycle can also be understood as a certain life cycle in history. The length of the life cycle can be preset and defined, and is not specifically limited in the embodiment of the present invention, such as 1 week, 1 month, etc. For example, the quotient between the difference between the sixth sample number corresponding to a certain target life in history and the sixth sample number corresponding to the latest life cycle and the sixth sample number corresponding to the target life cycle is calculated, and the score of the balance index is obtained; the specific calculation formula is as follows:
[0169] The score S7 of the volatility index = (the sixth sample number corresponding to the target life cycle - the fifth and sixth sample numbers corresponding to the latest life cycle) / the sixth sample number corresponding to the target life cycle.
[0170] Definition of the uniqueness indicator: samples that appear repeatedly in the target data. Excessive repeated samples often lead to overfitting of the AI model, so it is necessary to evaluate the uniqueness of the target data.
[0171] The data parameters required for uniqueness index evaluation include: the actual sample number of samples in the target data and the seventh sample number of repeated samples.
[0172] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0173] The score of the uniqueness indicator is calculated based on the actual sample size and the seventh sample size.
[0174] For example, the score of the uniqueness index is obtained by calculating the quotient between the difference between the actual sample number and the seventh sample number indicating the repeated sample and the actual sample number; the specific calculation formula is as follows:
[0175] The score S8 of the volatility index = (actual sample number - seventh sample number of repeated samples) / actual sample number.
[0176] Definition of the indicator of timeliness: Whether the target data is updated in a timely manner. Artificial intelligence models need to be trained with the latest data to ensure optimal performance. Failure to update the data in a timely manner may prevent the model from obtaining the latest information, ultimately reducing the accuracy of its predictions or decisions.
[0177] The data parameters required for timeliness index evaluation include: the non-updated time of the target data and the benchmark update time.
[0178] Specifically, calculating the score of each of the key indicators according to the data parameters includes:
[0179] The score of the timeliness index is calculated according to the non-updated time length of the target data and the benchmark update time length.
[0180] For example, the timeliness index score is obtained by calculating the quotient between the difference between the benchmark update time and the target data's unupdated time to date and the benchmark update time; in the embodiment of the present invention, the time unit of the target data update time can be predefined, and is not specifically limited in the embodiment of the present invention, such as day, week, month, etc. In the embodiment of the present invention, the time unit is day, and the timeliness index score calculation formula of the target data is as follows:
[0181] The timeliness index score S9=(baseline update days-no update days) / baseline update days.
[0182] It should be noted that the benchmark parameters used for key indicator scoring in the embodiments of the present invention (such as the number of benchmark samples, the number of benchmark features, the number of benchmark data channels, and the benchmark update time) can be pre-set or set according to the needs of the artificial intelligence model, and are not specifically limited in the embodiments of the present invention.
[0183] Specifically, the fuzzy hierarchical analysis is performed on the multiple key indicators to obtain the weight of each key indicator, including:
[0184] According to the importance of each of the key indicators, a proportional scale table is established; wherein the proportional scale table includes quantitative values for comparing two of the key indicators;
[0185] According to the quantitative values of the comparison of the two key indicators in the ratio scale table, a fuzzy factor of the comparison of the corresponding two key indicators is calculated by a fuzzy membership function;
[0186] The weight of each of the key indicators is determined according to the fuzzy factor.
[0187] Further, determining the weight of each of the key indicators according to the fuzzy factor includes:
[0188] According to the fuzzy factors, a judgment matrix is established;
[0189] According to the judgment matrix, the weight of each key indicator is determined.
[0190] Further, determining the weight of each of the key indicators according to the judgment matrix includes:
[0191] Performing a consistency check on the judgment matrix;
[0192] In the case of passing the consistency check, extracting the eigenvector where the maximum eigenvalue is located from the judgment matrix; wherein each eigenvalue in the eigenvector corresponds to one of the key indicators;
[0193] Normalizing the feature vector to obtain a weight vector; wherein each weight value in the weight vector corresponds to one of the key indicators;
[0194] Each weight value in the weight vector is used as the weight of its corresponding key indicator.
[0195] In the actual production environment, not all key indicators are applicable. Therefore, after calculating the scores of multiple key indicators (such as integrity indicators, accuracy indicators, consistency indicators, diversity indicators, balance indicators, logic indicators, volatility indicators, uniqueness indicators and timeliness indicators), it is necessary to conduct a comprehensive evaluation of multiple key indicators. Assuming that the target data used to train a certain artificial intelligence scenario is evaluated using k key indicators to evaluate the overall quality of the data, the data quality evaluation result of the target data can be calculated by weighted summation. The specific calculation formula is as follows:
[0196] Ω=ω1S1+ω2S2+...+ω k S k (1);
[0197] Wherein, k∈K, K represents the number of key indicators, and in the embodiment of the present invention, K=9.
[0198] According to the above formula, it can be known that the weight determination of key indicators is the key to objectively and accurately evaluate the overall quality of target data. In the embodiment of the present invention, the weight of each key indicator is determined by using the Analytic Hierarchy Process (AHP), which can make data quality assessment simple and easy to implement. First, the 9 importance levels and their values given by Saaty in the existing AHP are shown in the following table.
[0199]
[0200]
[0201] By comparing the key indicators pairwise, we can get the initial judgment matrix A of the key indicators. However, for the above factors, the element a in the judgment matrix A ijWhen assigning values, since the values of each key indicator in the judgment matrix A are obtained by the decision maker based on his own experience and knowledge, they are subjective and uncertain. Different decision makers may come up with different weights, and even conflicts may occur. Especially when there are many key indicators (more than 9), the workload of key indicator scaling is too large, the process cycle is long and complicated, and it is easier to cause confusion in judgment, reduce the quality of the results of the hierarchical analysis method, and even lead to the failure of the hierarchical analysis method decision.
[0202] In order to improve the result quality of the hierarchical analysis method and the success rate of the hierarchical analysis method, the embodiment of the present invention introduces fuzzy factors into the hierarchical analysis method to calculate the weights of key indicators. The fuzzy factors are based on fuzzy membership functions, which convert qualitative evaluation into quantitative evaluation, and solve the problems of difficulty in quantification and uncertainty by harmoniously unifying subjective and objective standards, so as to truly evaluate things or objects subject to multi-dimensional constraints.
[0203] Among them, a ij =∑x ij F(x ij );F(·) represents a fuzzy membership function, and a suitable membership function can be selected according to a specific scenario, such as expert voting, Gaussian function, gradient function, piecewise function, etc., which is not specifically limited in the embodiment of the present invention. ij is the quantitative value of the comparison between factor i and factor j, a ij The fuzzy factor representing the comparison between factor i and factor j indicates the final quantitative value of the comparison between factor i and factor j after the fuzzy membership function is calculated.
[0204] The feasibility of the hierarchical analysis method after the introduction of fuzzy factors is verified below. When using the Saaty1-9 scaling method to judge the importance of two key indicators, it is a fuzzy concept in itself, and experts are easily confused when making judgments. In addition, different experts may have different final results. Therefore, the introduction of fuzzy membership function can express the "real degree" of an element belonging to a fuzzy set. For example, in the scenario where the two elements of balance and uniqueness are compared, the specific quantitative values of balance and accuracy cannot be determined. If balance and accuracy are equally important, after the calculation of the fuzzy membership function, the membership degree of balance and accuracy is equally important is F(1)=0.6, and the final quantitative score is 1×0.6=0.6; if balance and accuracy are slightly important, after the calculation of the fuzzy membership function, the membership degree of balance and accuracy is slightly important is F(3)=0.4, and the final quantitative score is 3×0.4=1.2; therefore, the final quantitative value between the comparison of balance and accuracy is 0.6+1.2=1.8.
[0205] Allowing misalignment of judgment between two key indicators is in line with the reality of limited rationality in the decision-making process. In addition, the quantitative values 2, 4, 6, and 8 can also be regarded as the values of the membership function of the two key indicators of equal importance. If balance and accuracy are equally important, after the calculation of the membership function, the membership degree of balance and accuracy is 0.5, the quantitative score is 1×0.5=0.5; if balance and accuracy are slightly more important, after the calculation of the membership function, the membership degree of balance and accuracy is 0.5, the quantitative score is 3×0.5=1.5, and the final quantitative score is 0.5+1.5=2; so the fuzzy factor is introduced, and the Saaty1-9 scale table can be simplified to the scale table shown in the following table.
[0206] Key indicator i is higher than key indicator j Quantization value Equally important 1 Slightly important 3 Strong and important 5 Strongly important 7 Extremely important 9
[0207] Based on the above established proportional scale table, the fuzzy factors of the comparison of the corresponding two key indicators are calculated by the fuzzy membership function, and the final judgment matrix indicating the weights of the key indicators is constructed, which is as follows:
[0208]
[0209] Although the introduction of fuzzy factors can minimize and alleviate the errors caused by subjective judgment on the evaluation results, it cannot be completely avoided. Therefore, the embodiment of the present invention performs consistency check on the judgment matrix constructed based on fuzzy factors to further ensure the accuracy of the decision. The specific consistency ratio calculation formula is as follows:
[0210]
[0211] Among them, λ max represents the maximum eigenvalue in the judgment matrix; p represents the number of non-zero eigenvectors; RI represents the average random consistency index, which can be seen from the table. If CR<1, the judgment matrix passes the consistency test, otherwise it does not have satisfactory consistency and the judgment matrix A needs to be adjusted.
[0212] After the judgment matrix A passes the consistency check, the largest eigenvector λ max The feature vector (for example, the horizontal feature vector) is normalized, and the normalized feature vector is the required weight vector. Each element in the weight vector is the weight of the corresponding key indicator. Then, the weighted sum is performed by combining the scores of the key indicators through formula (1) to obtain the final data quality assessment result of the target data. When the score in the data quality assessment result is greater than the preset threshold, it can be determined that the data quality of the target data meets the training data requirements of the artificial intelligence model.
[0213] The following uses the KDD Cup 99 dataset as the target data to describe the data quality assessment method described in the embodiment of the present invention in detail. The KDD Cup 99 dataset is an open source dataset for network intrusion detection, which contains a large amount of network traffic data and corresponding labels for training and evaluating intrusion detection systems. Based on the 10% training subset in the KDDcup99 dataset as an example, the key indicators are determined as follows:
[0214] For the training data in the intrusion detection scenario, the seven indicators of integrity, accuracy, consistency, diversity, balance, logic and uniqueness are used for evaluation. In addition, this experiment does not evaluate volatility and timeliness. The reasons are as follows: Volatility: The intrusion detection data does not conform to periodic changes, so volatility evaluation is not performed. Timeliness: These training data come from 1999, and the timeliness index is not very meaningful for evaluating its current quality, so timeliness evaluation is not performed.
[0215] The scores of each key indicator are calculated as shown in the following table:
[0216]
[0217] In order to be more intuitive and clear, the calculation of membership degree is based on the comprehensive opinions of experts. For example, 20 experts were collected, and 8 experts believed that the accuracy of the KDDcup99 dataset is equally important than the completeness, 6 experts believed that the accuracy of the KDDcup99 dataset is more important than the completeness, and the remaining experts believed that the accuracy of the KDDcup99 dataset is more important than the completeness. The membership degrees are 0.4, 0.3, and 0.3 respectively. Therefore, the quantitative value of the comparison between the completeness index and the accuracy index is a 12 =0.4×1+0.3×5+0.3×7=4, calculate the remaining quantization values in sequence to form a judgment matrix B.
[0218]
[0219] After solving, the maximum eigenvalue is 7.7, and the number of non-zero eigenvalues is 7. By querying the table, it is found that when the matrix order is 7, the value of RI is 1.32. Next, perform a consistency check:
[0220]
[0221] The consistency requirement is met. Therefore, after normalizing the eigenvector of the maximum eigenvalue, the weight vector obtained is [0.0716, 0.2104, 0.3354, 0.0806, 0.0194, 0.2593, 0.0232]. Therefore, the comprehensive data quality of the 10% training subset in the KDDcup99 dataset is:
[0222] Ω=ω1S1+ω2S2+...+ω k S k
[0223] =0.0716×92.24+0.2104×96+0.3354×100+0.0806×54.5+0.0194×26+0.2593×100+0.0232×30;
[0224] =91.87
[0225] Therefore, the 10% training subset of this open source KDDcup99 dataset meets the requirements for training data samples for artificial intelligence model training. Although the balance and uniqueness scores of this dataset are low, combined with other indicators, the dataset can still be used as a research basis for network intrusion detection after comprehensive evaluation.
[0226] Compared with the prior art, the embodiments of the present invention determine the scores of multiple key indicators in the target data and measure the data situation from different dimensions, so as to more clearly understand the advantages and disadvantages of these dimensions and provide theoretical guidance and feedback for researchers in implementing improvements; then, fuzzy hierarchical analysis is performed on the multiple key indicators, the weights of each key indicator are calculated, and the scores and weights of each key indicator are determined to determine the final data quality assessment results of the target data. By performing fuzzy hierarchical analysis on the key indicators, it is possible to focus on the interaction between the key indicators, comprehensively consider and balance the various indicators, and through the harmonious unification of subjective and objective standards, a true evaluation of data subject to multi-dimensional constraints is performed, thereby weakening the influence of human subjectivity, realizing a comprehensive and dynamic evaluation of the overall data quality, and improving the credibility of data quality evaluation, providing strong data support for the development of trustworthy artificial intelligence.
[0227] See also Figure 2 , Figure 2 1 is a structural block diagram of a data quality assessment device provided in an embodiment of the present invention, wherein the data quality assessment device comprises:
[0228] A data extraction module 11 is used to extract data from target data to be evaluated, and obtain data parameters of multiple key indicators related to data quality evaluation;
[0229] A score calculation module 12, used to calculate the score of each of the key indicators according to the data parameters;
[0230] A fuzzy hierarchical analysis module 13 is used to perform fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator;
[0231] The data quality assessment module 14 is used to determine the data quality assessment result of the target data according to the score and weight of each of the key indicators.
[0232] In an optional embodiment, the device further includes:
[0233] The key indicator determination module is used to establish rules according to predefined indicators, perform full life cycle analysis on the target data, and determine multiple key indicators of the target data.
[0234] In an optional embodiment, the key indicators include at least two indicators among a completeness indicator, an accuracy indicator, a consistency indicator, a diversity indicator, a balance indicator, a logic indicator, a volatility indicator, a uniqueness indicator and a timeliness indicator.
[0235] In an optional embodiment, the data parameters include: the actual sample number of samples in the target data, the first sample number with null values and the reference sample number, the actual feature number and the reference feature number of each sample in the target sample;
[0236] The score calculation module 12 includes:
[0237] An integrity index calculation unit is used to calculate the score of the integrity index based on the actual sample number, the first sample number, the benchmark sample number, the actual feature number and the benchmark feature number.
[0238] In an optional embodiment, the integrity indicator calculation unit includes:
[0239] A feature integrity scoring subunit, configured to calculate a feature integrity score according to the actual number of features and the reference number of features;
[0240] A sample integrity scoring subunit, configured to calculate a sample integrity score based on the actual sample size, the first sample size, and the benchmark sample size;
[0241] A data size adequacy scoring subunit, used to calculate a data size adequacy score according to the actual sample size and the benchmark sample size;
[0242] The integrity indicator scoring subunit is used to calculate the score of the integrity indicator based on the feature integrity score, the sample integrity score and the data scale adequacy score.
[0243] In an optional embodiment, the data parameters include: the actual feature number and the reference feature number of each sample in the target sample;
[0244] The score calculation module 12 includes:
[0245] The accuracy index calculation unit is used to calculate the score of the accuracy index according to the actual number of features and the reference number of features.
[0246] In an optional embodiment, the data parameters include: the actual sample number of samples in the target data, the second sample number of format conflicts, and the third sample number of content conflicts;
[0247] The score calculation module 12 includes:
[0248] A consistency index calculation unit is used to calculate the score of the consistency index according to the actual sample size, the second sample size, and the third sample size.
[0249] In an optional embodiment, the consistency index calculation unit includes:
[0250] A first sample scoring subunit, configured to calculate a first sample score according to the actual sample size and the second sample size;
[0251] A second sample scoring subunit, configured to calculate a second sample score according to the actual sample size and the third sample size;
[0252] The consistency index scoring subunit is used to calculate the score of the consistency index according to the first sample score and the second sample score.
[0253] In an optional embodiment, the data parameters include: the number of actual data channels and the number of reference data channels of samples in the target data, the number of fourth samples belonging to different categories and the number of reference samples in the target data;
[0254] The score calculation module 12 includes:
[0255] A diversity index calculation unit is used to calculate the score of the diversity index according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number.
[0256] In an optional embodiment, the diversity index calculation unit includes:
[0257] A data source breadth scoring subunit, configured to calculate a data source breadth score according to the actual number of data channels and the benchmark number of data channels;
[0258] A data category richness scoring subunit, used to calculate a data category richness score according to the fourth sample number and the benchmark sample number;
[0259] The diversity index scoring subunit is used to calculate the score of the diversity index according to the data source breadth score and the data category richness score.
[0260] In an optional embodiment, the data parameter includes: the number of fourth samples belonging to different categories in the target data;
[0261] The score calculation module 12 includes:
[0262] The balance index calculation unit is used to calculate the score of the balance index according to the number of fourth samples belonging to different categories in the target data.
[0263] In an optional embodiment, the balance index calculation unit includes:
[0264] A first sample quantity acquisition subunit is used to acquire a maximum sample quantity and a minimum sample quantity among fourth sample quantities belonging to different categories in the target data;
[0265] A sample quantity mean calculation subunit, used for calculating the sample quantity mean of a fourth sample quantity belonging to different categories in the target data;
[0266] The balance index scoring subunit is used to calculate the score of the balance index based on the maximum sample size, the minimum sample size and the mean sample size.
[0267] In an optional embodiment, the data parameters include: the actual sample number of samples in the target data, the fifth sample number of logical errors;
[0268] The score calculation module 12 includes:
[0269] A logic index calculation unit is used to calculate the score of the logic index according to the actual sample size and the fifth sample size.
[0270] In an optional embodiment, the data parameter includes: the number of sixth samples of the target data in different life cycles;
[0271] The score calculation module 12 includes:
[0272] The volatility index calculation unit is used to calculate the score of the volatility index according to the sixth sample quantity of the target data in different life cycles.
[0273] In an optional embodiment, the volatility index calculation unit includes:
[0274] A second sample quantity acquisition subunit is used to acquire the sixth sample quantity corresponding to the latest life cycle in the target data and the sixth sample quantity corresponding to the target life cycle; wherein the target life cycle is any other life cycle in the target data except the latest life cycle before the latest life cycle;
[0275] A second sample quantity mean value calculation subunit, used to calculate a second sample quantity mean value of a sixth sample quantity of the target data in different life cycles;
[0276] The volatility index scoring subunit is used to calculate the score of the volatility index according to the sixth sample number corresponding to the latest life cycle and the sixth sample number corresponding to the target life cycle.
[0277] In an optional embodiment, the data parameters include: the actual sample number of samples in the target data, the seventh sample number of repeated samples;
[0278] The score calculation module 12 includes:
[0279] A uniqueness index calculation unit is used to calculate the score of the uniqueness index according to the actual sample number and the seventh sample number.
[0280] In an optional embodiment, the data parameters include: a non-updated time length and a benchmark update time length of the target data;
[0281] The score calculation module 12 includes:
[0282] The timeliness index calculation unit is used to calculate the score of the timeliness index according to the non-updated time length and the benchmark update time length of the target data.
[0283] In an optional embodiment, the fuzzy hierarchical analysis module 13 includes:
[0284] A ratio scale table establishment unit, used to establish a ratio scale table according to the importance of each of the key indicators; wherein the ratio scale table includes quantitative values for comparing two of the key indicators;
[0285] A fuzzy factor calculation unit, used to calculate the fuzzy factor of the comparison between the two key indicators according to the quantitative value of the comparison between the two key indicators in the ratio scale table through a fuzzy membership function;
[0286] The weight determination unit is used to determine the weight of each of the key indicators according to the fuzzy factors.
[0287] In an optional embodiment, the weight determination unit includes:
[0288] A judgment matrix establishment subunit, used to establish a judgment matrix according to the fuzzy factor;
[0289] The indicator weight determination subunit is used to determine the weight of each of the key indicators according to the judgment matrix.
[0290] In an optional embodiment, the indicator weight determination subunit includes:
[0291] A consistency check subunit, used to perform a consistency check on the judgment matrix;
[0292] A maximum eigenvector extraction subunit is used to extract the eigenvector where the maximum eigenvalue is located from the judgment matrix when the consistency check is passed; wherein each eigenvalue in the eigenvector corresponds to one of the key indicators;
[0293] The normalization processing subunit is used to normalize the feature vector to obtain a weight vector; wherein each weight value in the weight vector corresponds to one of the key indicators, and each weight value in the weight vector is used as the weight of the corresponding key indicator.
[0294] It should be noted that the working process of each module in the data quality assessment device described in the embodiment of the present invention can refer to the working process of the data quality assessment method described in the above embodiment, and the technical effect achieved is the same as the data quality assessment method described in the above embodiment, which will not be repeated here.
[0295] See also Figure 3 , Figure 3 2 is a block diagram of a data quality assessment device provided in an embodiment of the present invention. The data quality assessment device includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps in the above-mentioned data quality assessment method embodiments are implemented, such as steps S11 to S14.
[0296] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the data quality assessment device.
[0297] The data quality assessment device may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of a data quality assessment device and does not constitute a limitation on the data quality assessment device. The device may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the data quality assessment device may also include input and output devices, network access devices, buses, etc.
[0298] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 21 is the control center of the data quality assessment device, and uses various interfaces and lines to connect various parts of the entire data quality assessment device.
[0299] The memory 22 can be used to store the computer program and / or module, and the processor 21 realizes various functions of the data quality assessment device by running or executing the computer program and / or module stored in the memory 22, and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0300] Wherein, if the module / unit integrated in the data quality assessment device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor 21. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-On lyMemory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0301] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.
[0302] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, many improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A data quality assessment method, characterized in that: include: Extract data from the target data to be evaluated and obtain data parameters of multiple key indicators related to data quality evaluation; Calculate the score of each key indicator according to the data parameters; Performing fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator; The data quality assessment result of the target data is determined according to the scores and weights of the key indicators.
2. The data quality assessment method according to claim 1, characterized in that: The method further comprises: Establish rules based on predefined indicators, perform full life cycle analysis on the target data, and determine multiple key indicators of the target data.
3. The data quality assessment method according to claim 1, characterized in that: The key indicators include at least two indicators among a completeness indicator, an accuracy indicator, a consistency indicator, a diversity indicator, a balance indicator, a logic indicator, a volatility indicator, a uniqueness indicator and a timeliness indicator.
4. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the actual sample number of samples in the target data, the first sample number with null values and the reference sample number, the actual feature number and the reference feature number of each sample in the target sample; Calculating the scores of the key indicators according to the data parameters includes: The score of the integrity indicator is calculated based on the actual sample number, the first sample number, the benchmark sample number, the actual number of features and the benchmark number of features.
5. The data quality assessment method according to claim 4, characterized in that: The calculating the score of the integrity indicator according to the actual sample number, the first sample number, the benchmark sample number, the actual number of features and the benchmark number of features includes: Calculating a feature integrity score according to the actual number of features and the benchmark number of features; Calculate a sample integrity score based on the actual sample size, the first sample size, and the benchmark sample size; Calculate a data size adequacy score based on the actual sample size and the benchmark sample size; The score of the integrity indicator is calculated based on the feature integrity score, the sample integrity score and the data scale adequacy score.
6. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the actual feature number and the reference feature number of each sample in the target sample; Calculating the scores of the key indicators according to the data parameters includes: A score of the accuracy index is calculated based on the actual number of features and the benchmark number of features.
7. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the actual sample number of samples in the target data, the second sample number of format conflicts, and the third sample number of content conflicts; Calculating the scores of the key indicators according to the data parameters includes: The score of the consistency index is calculated according to the actual sample size, the second sample size, and the third sample size.
8. The data quality assessment method according to claim 7, characterized in that: The step of calculating the score of the consistency index according to the actual sample size, the second sample size, and the third sample size includes: Calculating a first sample score according to the actual sample size and the second sample size; Calculating a second sample score according to the actual sample size and the third sample size; A score of the consistency index is calculated based on the first sample score and the second sample score.
9. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the number of actual data channels and the number of reference data channels of the samples in the target data, the number of fourth samples belonging to different categories and the number of reference samples in the target data; Calculating the scores of the key indicators according to the data parameters includes: The score of the diversity indicator is calculated according to the number of actual data channels, the number of benchmark data channels, the fourth sample number and the benchmark sample number.
10. The data quality assessment method according to claim 9, characterized in that: Calculating the score of the diversity indicator according to the number of actual data channels, the number of benchmark data channels, the fourth sample number, and the benchmark sample number includes: Calculate a data source breadth score according to the actual number of data channels and the benchmark number of data channels; Calculating a data category richness score based on the fourth sample size and the benchmark sample size; The score of the diversity indicator is calculated according to the data source breadth score and the data category richness score.
11. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the number of fourth samples belonging to different categories in the target data; Calculating the scores of the key indicators according to the data parameters includes: The score of the balance index is calculated according to the number of fourth samples belonging to different categories in the target data.
12. The data quality assessment method according to claim 11, characterized in that: The step of calculating the score of the balance index according to the number of fourth samples belonging to different categories in the target data includes: Obtaining a maximum value and a minimum value of the number of samples in the fourth number of samples belonging to different categories in the target data; Calculating a sample quantity mean of a fourth sample quantity belonging to different categories in the target data; The score of the balance index is calculated based on the maximum sample size, the minimum sample size and the mean sample size.
13. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the actual sample number of samples in the target data, and the fifth sample number of logical errors; Calculating the scores of the key indicators according to the data parameters includes: The score of the logic index is calculated based on the actual sample size and the fifth sample size.
14. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the number of sixth samples of the target data in different life cycles; Calculating the scores of the key indicators according to the data parameters includes: The score of the volatility index is calculated according to the sixth sample quantity of the target data in different life cycles.
15. The data quality assessment method according to claim 14, characterized in that: The step of calculating the score of the volatility index according to the sixth sample quantity of the target data in different life cycles includes: Obtaining the sixth sample quantity corresponding to the latest life cycle in the target data and the sixth sample quantity corresponding to the target life cycle; wherein the target life cycle is any other life cycle in the target data except the latest life cycle before the latest life cycle; The score of the volatility indicator is calculated according to the sixth sample quantity corresponding to the latest life cycle and the sixth sample quantity corresponding to the target life cycle.
16. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the actual sample number of samples in the target data, the seventh sample number of repeated samples; Calculating the scores of the key indicators according to the data parameters includes: The score of the uniqueness indicator is calculated based on the actual sample size and the seventh sample size.
17. The data quality assessment method according to claim 3, characterized in that: The data parameters include: the non-updated time length and the benchmark update time length of the target data; Calculating the scores of the key indicators according to the data parameters includes: The score of the timeliness index is calculated according to the non-updated time length of the target data and the benchmark update time length.
18. The data quality assessment method according to claim 1, characterized in that: The fuzzy hierarchical analysis is performed on the multiple key indicators to obtain the weight of each key indicator, including: According to the importance of each of the key indicators, a proportional scale table is established; wherein the proportional scale table includes quantitative values for comparing two of the key indicators; According to the quantitative values of the comparison of the two key indicators in the ratio scale table, a fuzzy factor of the comparison of the corresponding two key indicators is calculated by a fuzzy membership function; The weight of each of the key indicators is determined according to the fuzzy factor.
19. The data quality assessment method according to claim 18, characterized in that: Determining the weight of each of the key indicators according to the fuzzy factor includes: According to the fuzzy factors, a judgment matrix is established; According to the judgment matrix, the weight of each key indicator is determined.
20. The data quality assessment method according to claim 19, characterized in that: Determining the weight of each of the key indicators according to the judgment matrix includes: Performing a consistency check on the judgment matrix; In the case of passing the consistency check, extracting the eigenvector where the maximum eigenvalue is located from the judgment matrix; wherein each eigenvalue in the eigenvector corresponds to one of the key indicators; Normalizing the feature vector to obtain a weight vector; wherein each weight value in the weight vector corresponds to one of the key indicators; Each weight value in the weight vector is used as the weight of its corresponding key indicator.
21. A data quality assessment device, characterized in that: include: A data extraction module is used to extract data from the target data to be evaluated and obtain data parameters of multiple key indicators related to data quality evaluation; A scoring calculation module, used to calculate the scores of each of the key indicators according to the data parameters; A fuzzy hierarchical analysis module is used to perform fuzzy hierarchical analysis on the multiple key indicators to obtain the weight of each key indicator; The data quality assessment module is used to determine the data quality assessment result of the target data according to the scores and weights of each of the key indicators.
22. A data quality assessment device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the data quality assessment method according to any one of claims 1 to 20 when executing the computer program.
23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the data quality assessment method according to any one of claims 1 to 20.
24. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the data quality assessment method described in any one of claims 1 to 20 is implemented.
Citation Information
Cited By
Product quality evaluation method and device based on artificial intelligence attention mechanism
CN120409890A
