A data transaction quality assessment system based on big data
Patent Information
- Application Number
- CN202411231109.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-04
AI Technical Summary
Traditional data transaction quality assessment systems are difficult to track fast-changing market environments and faults or efficiency problems intertwined by multiple factors in real time, resulting in increased transaction risks and reduced market trust.
A data transaction quality evaluation system based on big data was designed, including data transaction collection module, preprocessing module, rule management database, quality evaluation information acquisition module, analysis module and comprehensive evaluation report module. Through data cleaning, unified format and missing Value processing and outlier detection, establish a comprehensive quality evaluation model, and generate timeliness and accuracy evaluation reports.
It improves the accuracy and operational efficiency of quality evaluation of data transactions, ensures the timeliness and accuracy of evaluation results, reduces transaction risks, and enhances market trust.
Smart Images

Figure CN118760670B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data science and technology, and more specifically, to a data transaction quality assessment system based on big data. Background Art
[0002] With the rapid development of the digital economy and the increasing popularity of data-driven decision-making, data transactions, as a key link in data circulation, have attracted widespread attention for their quality and efficiency. Traditional data quality assessment methods mainly focus on the accuracy and completeness of data, but with the increasing demand for data applications, the requirements for data quality are also increasing. Data consumers have a broader understanding of data quality than information system professionals. They not only pay attention to the accuracy of data, but also to the contextual adaptability, representation, and accessibility of data.
[0003] In recent years, the rapid progress of information technology has provided innovative solutions for the intelligent evaluation of data transaction quality. Data transaction quality evaluation is evolving towards a higher level of intelligence and automation. The introduction of advanced algorithms such as AGA-BP neural network and the use of quantitative calculation methods such as Shapley value have greatly improved the accuracy and operational efficiency of the evaluation work. The integration of information technology and cutting-edge data management systems has made real-time monitoring of data transaction processes possible, further optimizing data quality management and transaction risk control mechanisms, thereby enhancing the overall efficiency and security of data transactions.
[0004] However, there are still some shortcomings in its actual use. For example, the system often needs to cope with the rapidly changing market environment, in which the frequent fluctuations of key information such as data quality, circulation speed, and security make it difficult for traditional data management strategies to achieve real-time and accurate tracking of these dynamic changes. In addition, failures or efficiency problems in data transaction quality assessment systems are often the result of the interweaving of multiple factors, including data inaccuracy, opacity of the transaction process, and inconsistency of assessment standards. The comprehensive analysis and evaluation of these factors exceed the processing capabilities of traditional methods. Ultimately, due to the limitations of the level of automation, the shortcomings of traditional systems in processing speed and operational accuracy may lead to increased transaction risks and reduced market trust. Summary of the invention
[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a data transaction quality assessment system based on big data, which solves the problems raised in the above-mentioned background technology through the following scheme.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] A data transaction quality assessment system based on big data, comprising:
[0008] Data transaction collection module: used to obtain data transaction information from the target data source and preliminarily screen to form the original data set;
[0009] Data transaction preprocessing module: used to preprocess the original data set formed by preliminary screening from the target data source, build a higher quality desensitized data set, number the desensitized data set in chronological order, assign a unique serial number to each data set, mark it as 1, 2, ..., n in sequence, and transmit it to the quality assessment information acquisition module;
[0010] Rule management database: used to store original data sets, desensitized data sets, and modeled data, store data protection rules, and store quality standard thresholds;
[0011] Quality assessment information acquisition module: used to obtain the pre-processed desensitized data set from the target data source, screen and extract key information from it, and transmit this information to the quality assessment information analysis module; including accurate and timely data acquisition unit, coherent and complete data acquisition unit, compliant and reachable data acquisition unit and diverse and stable data acquisition unit;
[0012] Quality assessment information analysis module: used to quantify the information transmitted from the quality assessment information acquisition module, generate modeled data after analysis and processing, and transmit it to the comprehensive quality assessment module; including precise timeliness quantification function, coherence and integrity quantification function, compliance accessibility quantification function and diverse stability quantification function;
[0013] Comprehensive quality assessment module: used to analyze the characteristic data of the modeled data in the quality assessment information analysis module, establish a comprehensive quality assessment model, and transmit it to the comprehensive assessment report module;
[0014] Comprehensive evaluation report module: used to evaluate the quality status of data transactions, put forward targeted improvement suggestions, and form a comprehensive evaluation report.
[0015] Preferably, the quality assessment information acquisition module includes an accurate and timely data acquisition unit, a coherent and complete data acquisition unit, a compliant and reachable data acquisition unit, and a diverse and stable data acquisition unit;
[0016] It should be noted that the precision timeliness data acquisition unit is used to obtain the pre-processed desensitized data set in the target data source, accurately screen and extract information related to the precision timeliness of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of precision timeliness in data transactions;
[0017] It should be noted that the information related to the precise timeliness of data includes the benchmark data set of the authoritative data source in the rule database, the record value in the desensitized data set, the data arrival time, the processing delay time, and the frequency of data update, which are recorded as , , , as well as ;
[0018] It should be noted that the coherent and complete data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the coherence and integrity of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of the coherence and integrity in data transactions;
[0019] It should be noted that the information related to the consistency and integrity of the data includes the number of missing values in the desensitized data set, the total number of records in the desensitized data set, the number of evaluation rules, and the number of rule-inconsistent values, which are marked as , , as well as ;
[0020] It should be noted that the compliance accessibility data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the compliance accessibility of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of compliance accessibility in data transactions;
[0021] It should be noted that the information related to the compliance and accessibility of data includes the desensitized dataset compliance score and the desensitized dataset accessibility score, which are marked as , ; It should be noted that the compliance score is 0 or 1, 1 if it complies with the specification, and 0 if it does not comply with the specification; It should be noted that the accessibility score is 0 or 1, 0 if it is accessible, and 1 if it is inaccessible;
[0022] It should be noted that the diverse and stable data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the diverse stability of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of the diverse stability in data transactions;
[0023] It should be noted that the information related to the diversity and stability of data includes the proportion of data types in the desensitized data set, the stability observation value, and the observation mean, which are marked as , , .
[0024] Preferably, the precision timeliness quantification function in the quality assessment information analysis module is specifically expressed as: , Expressed as A quantitative function of the accuracy and timeliness of a desensitized dataset, Represents the influencing factors for obtaining data set records, is the number of records in the dataset, Represented as the record value in each desensitized data set, It is represented as the weight of the record value in each desensitized data set. Expressed as Benchmark datasets, Represents the influencing factors for obtaining data at a time point, It is expressed as the influencing factor of arrival time, Expressed as The arrival time of data in the anonymized data set, Expressed as The processing delay time of the desensitized data set is Expressed as the influencing factor of update frequency, Expressed as The frequency of data update in a desensitized dataset.
[0025] Preferably, the coherence integrity quantification function in the quality assessment information analysis module is specifically expressed as: , Expressed as A quantitative function of the coherence and completeness of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The number of missing values in the masked dataset, Expressed as The total number of records in the desensitized dataset, is the number of records in the dataset, is the number of rules evaluated, Expressed as The record is in The weight under the rule, Expressed as The record is in The number of inconsistent values under a rule.
[0026] Preferably, the compliance reachability quantification function in the quality assessment information analysis module is specifically expressed as: , Expressed as A quantitative function of the compliance accessibility of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The compliance scores of the desensitized datasets, Expressed as Accessibility scores of desensitized datasets.
[0027] Preferably, the diverse stability quantification function in the quality assessment information analysis module is specifically expressed as: , Expressed as A desensitized dataset to evaluate the quantitative function of diverse stability, Expressed as The proportion of data types in the desensitized datasets, Expressed as Stability observations of desensitized datasets, Expressed as the observed mean.
[0028] Preferably, the comprehensive quality assessment model in the comprehensive assessment report module is specifically expressed as: , Represented as each evaluation function in the modeled data The evaluation value of Indicated as except A subset of other evaluation functions besides Represented as a subset The sum of the evaluation results of all functions in , It is expressed as the number of elements in the subset S, and the value of j can be any one of 1, 2, 3, and 4.
[0029] Technical effects and advantages of the present invention:
[0030] 1. The data transaction preprocessing module will perform data cleaning, format unification, missing value processing and outlier detection on each data through the preprocessing process to eliminate redundancy and inconsistency in the data, ensure that the data meets the preliminary quality standards, and lay a solid foundation for further quality assessment, quantitative analysis and evaluation;
[0031] 2. Comprehensive quality assessment model By establishing a comprehensive quality assessment model, the evaluation values of different evaluation functions can be effectively integrated to form a unified quality evaluation standard;
[0032] 3. The comprehensive assessment report module sets the assessment results that meet the quality standards as the quality standard threshold and transmits it to the rule management database. The quality standard threshold is continuously adjusted as the data transaction environment changes dynamically to ensure the timeliness and accuracy of the assessment results. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] As attached Figure 1 A data transaction quality assessment system based on big data is shown, including a data transaction collection module, a data transaction preprocessing module, a rule management database, a quality assessment information acquisition module, a quality assessment information analysis module, a comprehensive quality assessment module and a comprehensive assessment report module.
[0036] The data transaction collection module is used to obtain data transaction information from the target data source and perform preliminary screening to form the original data set.
[0037] It should be noted that the original data set is represented by a set of unprocessed data obtained from the target data source; it should be noted that the original data set includes basic content such as creation time, source details, version history, and collection time point, and is transmitted to the rule management database for easy comparison with the desensitized data set.
[0038] In this example, the data transaction collection module mainly performs the functions of data identification, capture and preliminary integration to ensure that the collected data can truly reflect the status of the data source. The original data set will be passed to the data transaction preprocessing module to provide input for the next step of data cleaning and standardization, maintain the integrity and availability of the data, and lay the foundation for the subsequent stages of data transaction quality assessment.
[0039] The data transaction preprocessing module is used to perform preprocessing on the original data set formed by preliminary screening from the target data source, construct a higher quality desensitized data set, number the desensitized data set in chronological order, assign a unique serial number to each data set, mark them as 1, 2, ..., n in sequence, and transmit them to the quality assessment information acquisition module.
[0040] It should be noted that the preprocessing process of the data transaction preprocessing module will perform steps such as data cleaning, format unification, missing value processing and outlier detection on each data to eliminate redundancy and inconsistency in the data, ensure that the data meets the preliminary quality standards, and lay a solid foundation for further quality assessment, quantitative analysis and evaluation.
[0041] In this example, the desensitized data set is used to balance the availability and privacy protection of the data set, ensure that data protection rules are followed during the analysis of data transactions, and ensure that the entire data transaction quality assessment process complies with regulations.
[0042] The rule management database is used to store original data sets, desensitized data sets, and modeled data, store data protection rules, and store quality standard thresholds.
[0043] The quality assessment information acquisition module is used to obtain the pre-processed desensitized data set in the target data source, screen and extract key information from it, and transmit this information to the quality assessment information analysis module; it includes an accurate and timely data acquisition unit, a coherent and complete data acquisition unit, a compliant and reachable data acquisition unit, and a diverse and stable data acquisition unit.
[0044] It should be noted that the precision timeliness data acquisition unit is used to obtain the pre-processed desensitized data set in the target data source, accurately screen and extract information related to the precision timeliness of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of precision timeliness in data transactions;
[0045] It should be noted that the information related to the precise timeliness of data includes the benchmark data set of the authoritative data source in the rule database, the record value in the desensitized data set, the data arrival time, the processing delay time, and the frequency of data update, which are recorded as , , , as well as .
[0046] It should be noted that the coherent and complete data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the coherence and integrity of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of the coherence and integrity in data transactions;
[0047] It should be noted that the information related to the consistency and integrity of the data includes the number of missing values in the desensitized data set, the total number of records in the desensitized data set, the number of evaluation rules, and the number of rule-inconsistent values, which are marked as , , as well as .
[0048] It should be noted that the compliance accessibility data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the compliance accessibility of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of compliance accessibility in data transactions;
[0049] It should be noted that the information related to the compliance and accessibility of data includes the desensitized dataset compliance score and the desensitized dataset accessibility score, which are marked as , .
[0050] In this example, the compliance score takes a value of 0 or 1, where 1 means compliance with the specification and 0 means non-compliance with the specification. It should be noted that the accessibility score takes a value of 0 or 1, where 0 means accessible and 1 means inaccessible.
[0051] It should be noted that the diverse and stable data acquisition unit is used to obtain the pre-processed desensitized data set in the target data source, accurately screen and extract information related to the diversity and stability of the data, and further pass this key information to the quality assessment information analysis module to facilitate the quality analysis of diversity and stability in data transactions.
[0052] It should be noted that the information related to the diversity and stability of data includes the proportion of data types in the desensitized data set, the stability observation value, and the observation mean, which are marked as , , .
[0053] The quality assessment information analysis module is used to quantify the information transmitted from the quality assessment information acquisition module, generate modeled data after analysis and processing, and transmit it to the comprehensive quality assessment module, including the precision timeliness quantification unit, the coherence and integrity quantification unit, the compliance accessibility quantification unit and the diversity stability quantification unit.
[0054] It should be noted that the modeled data is represented by data processed by the quantitative model to improve the evaluation efficiency and accuracy; it includes precise timeliness quantification function, coherence and integrity quantification function, compliance and accessibility quantification function and diverse stability quantification function.
[0055] It should be noted that the precision timeliness quantification unit is used to process the information of the precision timeliness data acquisition unit in the quality assessment information acquisition module, construct a quantitative function model for evaluating precision timeliness, and transmit it to the comprehensive quality assessment module; it should be noted that the precision timeliness quantification function is specifically expressed as: , Expressed as A quantitative function of the accuracy and timeliness of a desensitized dataset, Represents the influencing factors for obtaining data set records, is the number of records in the dataset, Represented as the record value in each desensitized data set, It is represented as the weight of the record value in each desensitized data set. Expressed as Benchmark datasets, Represents the influencing factors for obtaining data at a time point, It is expressed as the influencing factor of arrival time, Expressed as The arrival time of data in the anonymized data set, Expressed as The processing delay time of the desensitized data set is Expressed as the influencing factor of update frequency, Expressed as The frequency of data update in a desensitized dataset.
[0056] In this example, the precise timeliness measurement unit uses regression analysis methods to establish a quantitative function model of precise timeliness to quantify the consistency between the desensitized data set obtained in the data transaction preprocessing module and the benchmark data set of the rule database management module; at the same time, time series analysis is used to determine the update frequency and timestamp of the desensitized data set obtained in the data transaction preprocessing module. It should be noted that the benchmark data set is the authoritative data set selected in the rule database management module as a reference standard for evaluating the accuracy of the desensitized data set.
[0057] It should be noted that the coherence integrity quantification unit is used to process the information of the coherence and integrity data acquisition unit in the quality assessment information acquisition module, construct a quantitative function model for evaluating the coherence integrity, and transmit it to the comprehensive quality assessment module; it should be noted that the coherence integrity quantification function is specifically expressed as: , Expressed as A quantitative function of the coherence and completeness of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The number of missing values in the masked dataset, Expressed as The total number of records in the desensitized dataset, is the number of records in the dataset, is the number of rules evaluated, Expressed as The record is in The weight under the rule, Expressed as The record is in The number of inconsistent values under a rule.
[0058] In this example, the consistency and integrity measurement unit ensures high-standard consistency of data by clarifying multiple key dimensions of data consistency and setting detailed evaluation rules for each dimension; accurately identifies data items that violate predefined rules in the data set, and then reasonably assigns weights to each rule to ensure that the evaluation results can accurately reflect the substantive impact of data quality on the business; calculates the consistency score by applying these rules to each data record and recording the number of inconsistency violations for each data item; and integrates them through weighted averaging to form the overall consistency and integrity score of the data set to quantify the consistency level of the data.
[0059] It should be noted that the compliance reachability quantification unit is used to process the information of the compliance reachability data acquisition unit in the quality assessment information acquisition module, construct a quantitative function model for evaluating compliance reachability, and transmit it to the comprehensive quality assessment module; it should be noted that the compliance reachability quantification function is specifically expressed as: , Expressed as A quantitative function of the compliance accessibility of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The compliance scores of the desensitized datasets, Expressed as Accessibility scores of desensitized datasets.
[0060] In this example, the compliance and accessibility unit module quantifies the compliance score with a clear value of 0 or 1 to accurately reflect whether the data set complies with the established specifications; by setting the accessibility score to 0 or 1, this module clearly defines the accessible status of the data set, making it easier to identify access restrictions; through the constructed quantitative function model, it can comprehensively consider compliance and accessibility to provide a comprehensive compliance and accessibility assessment.
[0061] It should be noted that the diversity stability quantization unit is used to process the information of the diversity stability data acquisition unit in the quality assessment information acquisition module, construct a quantitative function model for evaluating diversity stability, and transmit it to the comprehensive quality assessment module; it should be noted that the diversity stability quantization function is specifically expressed as: , Expressed as A desensitized dataset to evaluate the quantitative function of diverse stability, Expressed as The proportion of data types in the desensitized datasets, Expressed as Stability observations of desensitized datasets, Expressed as the observed mean.
[0062] In this example, the diversity stability unit can perform customized analysis on the characteristics of different desensitized data sets by constructing a quantitative function model for evaluating diverse stability. The stability level of each data set can be accurately measured by the difference between the stability observation value and the observation mean. The key input parameters are provided for the overall data quality assessment by transmitting the quantitative results to the comprehensive quality assessment module. Through statistical analysis methods, a quantitative function model is established to quantify the diversity of the desensitized data sets obtained in the data trading preprocessing module.
[0063] The comprehensive quality assessment module is used to analyze the characteristic data of the modeled data in the quality assessment information analysis module, establish a comprehensive quality assessment model, and transmit it to the comprehensive assessment report module.
[0064] It should be noted that the comprehensive quality assessment model is specifically expressed as: , Represented as each evaluation function in the modeled data The evaluation value of Indicated as except A subset of other evaluation functions besides Represented as a subset The sum of the evaluation results of all functions in , It is expressed as the number of elements in the subset S, and the value of j can be any one of 1, 2, 3, and 4.
[0065] In this example, the comprehensive quality assessment model can effectively integrate the evaluation values of different evaluation functions to form a unified quality evaluation standard by establishing a comprehensive quality assessment model; by summing the evaluation results of all functions in the subset, it provides a method to quantify the overall quality of the data; by expressing it as a subset of other evaluation functions except for a specific evaluation function, it can flexibly evaluate the performance of the data set in different quality dimensions; through the comprehensive evaluation model, the quality of data transactions is improved.
[0066] The comprehensive evaluation report module is used to evaluate the quality status of data transactions, put forward targeted improvement suggestions, and form a comprehensive evaluation report.
[0067] It should be noted that the comprehensive evaluation report compares and analyzes the evaluation indicators of each key quality dimension in the data transaction with the preset data quality standard threshold to determine the current quality status of the data transaction; if the evaluation indicator in any key quality dimension of the data transaction is less than the preset quality standard threshold, it is recorded as , data sets with quality risks will be identified and marked, and corresponding quality control measures will be taken; by iteratively analyzing the key quality dimensions of data quality, potential causes will be explored, and targeted improvement suggestions will be put forward; the evaluation results, analysis conclusions and improvement suggestions will be integrated into a comprehensive evaluation report.
[0068] In this example, the comprehensive assessment report module sets the assessment results that meet the quality standards as the quality standard threshold and transmits it to the rule management database. The quality standard threshold is constantly adjusted as the data transaction environment changes dynamically to ensure the timeliness and accuracy of the assessment results. By monitoring the key quality dimensions of data transactions, the data sets that do not meet the preset standards are marked, and notifications are automatically generated and quality control measures are triggered to ensure that problems can be handled in a timely manner. By integrating assessment results, analysis conclusions and improvement suggestions, the timeliness and accuracy of the assessment results are guaranteed.
[0069] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;
[0070] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A data transaction quality assessment system based on big data, characterized in that: include: Data transaction collection module: used to obtain data transaction information from the target data source and preliminarily screen to form the original data set; Data transaction preprocessing module: used to preprocess the original data set formed by preliminary screening from the target data source, build a higher quality desensitized data set, number the desensitized data set in chronological order, assign a unique serial number to each data set, mark it as 1, 2, ..., n in sequence, and transmit it to the quality assessment information acquisition module; Rule management database: used to store original data sets, desensitized data sets, and modeled data, store data protection rules, and store quality standard thresholds; Quality assessment information acquisition module: used to obtain the pre-processed desensitized data set from the target data source, filter and extract key information from it, and transmit this information to the quality assessment information analysis module; Including accurate and timely data acquisition unit, coherent and complete data acquisition unit, compliant and reachable data acquisition unit and diverse and stable data acquisition unit; The precise and timely data acquisition unit is used to acquire the pre-processed desensitized data set from the target data source, and accurately screen and extract information related to the precise and timely nature of the data; It includes the benchmark data set of the authoritative data source in the rule database, the record value in the desensitized data set, the data arrival time, the processing delay time, and the frequency of data update, which are recorded as , , , as well as ; The coherent and complete data acquisition unit is used to obtain the pre-processed desensitized data set in the target data source, accurately screen and extract information related to the coherence and integrity of the data, including the number of missing values in the desensitized data set, the total number of records in the desensitized data set, the number of evaluation rules, and the number of inconsistent values in the rules, which are marked as , , as well as ; The compliant and accessible data acquisition unit is used to obtain the pre-processed desensitized data set in the target data source, accurately screen and extract information related to the compliance and accessibility of the data, including the compliance score of the desensitized data set and the accessibility score of the desensitized data set, which are marked as , sa; the compliance score is 0 or 1, 1 if it complies with the specification, and 0 if it does not comply with the specification; the accessibility score is 0 or 1, 0 if it is accessible, and 1 if it is inaccessible; The diverse and stable data acquisition unit is used to obtain the pre-processed desensitized data set from the target data source, accurately screen and extract information related to the diverse stability of the data, including the proportion of the data type of the desensitized data set, the stability observation value and the observation mean, which are marked as , , ; Quality assessment information analysis module: used to quantify the information transmitted from the quality assessment information acquisition module, generate modeled data after analysis and processing, and transmit it to the comprehensive quality assessment module; It includes precision timeliness quantification unit, coherence and integrity quantification unit, compliance accessibility quantification unit and diversity stability quantification unit; The precision timeliness quantification unit is used to process the information of the precision timeliness data acquisition unit in the quality assessment information acquisition module and construct a quantitative function model for evaluating precision timeliness, which is specifically expressed as: , Expressed as A quantitative function of the accuracy and timeliness of a desensitized dataset, It represents the influencing factors for obtaining data set records, and n represents the number of desensitized data sets. Represented as the record value in each desensitized data set, It is represented as the weight of the record value in each desensitized data set. Expressed as Benchmark datasets, Represents the influencing factors for obtaining data at a time point, It is expressed as the influencing factor of arrival time, Expressed as The arrival time of data in the desensitized data set, Expressed as The processing delay time of the desensitized data set is Expressed as the influencing factor of update frequency, Expressed as The frequency of data updates in the desensitized dataset; The coherence and integrity quantification unit is used to process the information of the coherence and integrity data acquisition unit in the quality assessment information acquisition module and construct a quantitative function model for evaluating coherence and integrity, which is specifically expressed as: , Expressed as A quantitative function of the coherence and completeness of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The number of missing values in the masked dataset, Expressed as The total number of records in the desensitized data set, n represents the number of desensitized data sets, is the number of rules evaluated, It is expressed as the weight of the k-th record under the j-th rule, It is expressed as the number of inconsistent values of the k-th record under the j-th rule; The compliance reachability quantification unit is used to process the information of the compliance reachability data acquisition unit in the quality assessment information acquisition module and construct a quantitative function model for evaluating compliance reachability, which is specifically expressed as: , Expressed as A quantitative function of the compliance accessibility of a desensitized dataset, Expressed as The weight of the desensitized dataset, Expressed as The compliance score of the anonymized dataset, sa i Expressed as Accessibility scores of desensitized datasets; The diversity stability quantification unit is used to process the information of the diversity stability data acquisition unit in the quality assessment information acquisition module and construct a quantitative function model for evaluating diversity stability, which is specifically expressed as: , Expressed as A desensitized dataset to evaluate the quantitative function of diverse stability, Expressed as The proportion of data types in the desensitized datasets, Expressed as Stability observations of desensitized datasets, is expressed as the observed mean; Comprehensive quality assessment module: used to analyze the characteristic data of the modeled data in the quality assessment information analysis module, establish a comprehensive quality assessment model, and transmit it to the comprehensive assessment report module; Comprehensive evaluation report module: used to evaluate the quality status of data transactions, put forward targeted improvement suggestions, and form a comprehensive evaluation report.
2. The data transaction quality assessment system based on big data according to claim 1, characterized in that: The comprehensive quality assessment model in the comprehensive assessment report module is specifically expressed as follows: , Expressed as , , , The evaluation value of Indicated as except A subset of other evaluation functions besides Represented as a subset The sum of the evaluation results of all functions in , It is expressed as the number of elements in the subset S, and the value of j can be any one of 1, 2, 3, and 4.
Citation Information
Patent Citations
College student party member continuous education tracking evaluation system and method based on multi-modal data analysis
CN116777690A
