Data asset multi-dimensional evaluation method and system based on block chain and machine learning
By introducing blockchain and machine learning technology into data asset evaluation, a multi-dimensional evaluation index system and weight optimization model was built, the existing evaluation methods have solved the problems of single dimensions, opaqueness and strong subjectivity, and the science of data asset value has been realized Evaluation and market-oriented pricing.
Patent Information
- Application Number
- CN202411439160.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-21
AI Technical Summary
The existing data asset appraisal methods have problems such as single dimensions, opaqueness and strong subjectivity, and it is difficult to fully reflect the intrinsic value of data assets and be recognized by all parties.
Using a multi-dimensional evaluation method based on blockchain and machine learning, the multi-dimensional data asset evaluation index system is built, and the metadata, ownership information, and transaction record information of data assets are stored on the chain, and the evaluation index is formed using machine learning algorithm training. Weight optimization model and market-oriented pricing is formed through dynamic bidding and joint pricing.
It has realized scientific evaluation, market-oriented pricing and credible publicity of the value of data assets, improved the objectivity and credibility of the evaluation results, and overcome the problems of single dimensions, strong subjectivity and opaqueness of the traditional methods.
Smart Images

Figure CN119494575A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data asset evaluation, and in particular to a multi-dimensional data asset evaluation method and system based on blockchain and machine learning. Background Art
[0002] With the rapid development of emerging technologies such as big data and artificial intelligence, data has become a key production factor driving the digital economy. However, existing data asset evaluation methods mainly rely on traditional statistical analysis and expert experience judgment, which has the following problems:
[0003] 1. The evaluation dimension is single, and it is difficult to fully reflect the intrinsic value of data assets. Traditional evaluation methods usually only focus on the quantity and scale of data assets, ignoring multiple dimensions such as data quality, data security, and data application value, and are unable to accurately evaluate the true value of data assets. For example, the cost method mainly considers the acquisition cost of data assets, the income method focuses on the discount of future cash flows, and the market method needs to refer to the transaction price of similar data assets. These methods are difficult to fully measure the multi-dimensional value of data assets.
[0004] 2. The evaluation process is not transparent, lacks the necessary credible mechanism, and the evaluation results are difficult to be recognized by all parties. The traditional evaluation process is hidden within the evaluation agency, lacks external supervision, and the evaluation basis is unclear, resulting in insufficient credibility of the evaluation results. Key information such as the ownership, quality, and security status of data assets is difficult to be effectively verified by the parties involved in the evaluation, and problems such as data tampering and information omissions during the evaluation process cannot be discovered in a timely manner.
[0005] 3. The evaluation results are highly subjective and easily influenced by the personal experience and judgment of the evaluators, lacking objectivity and consistency. The evaluation results of the same data asset by different evaluators often vary greatly, which is mainly due to the different subjective judgments on the weights of evaluation indicators. In traditional methods, the weights of evaluation indicators are usually fixed based on expert experience, ignoring the differences in the characteristics of data assets in different industries and scenarios, and it is difficult to reflect the dynamic changes in the value of data assets.
[0006] These problems make it difficult for existing evaluation methods to meet the growing demand for market-based allocation of data elements, and a new data asset evaluation solution is urgently needed.
[0007] Patent document CN117575529A discloses a method and system for automatic scrapping of equipment based on smart contracts and scoring cards. The data preprocessing module is responsible for preprocessing and feature extraction of the collected data as needed; the scoring card generation module uses machine learning algorithms and historical data to generate a scoring card model for evaluating the health status of the equipment based on its characteristics; the automatic scrapping module uses smart contract technology to embed the scoring card model into the scrapping process. Through the asset automatic scrapping contract, the system automatically scores the IoT equipment according to the scoring card model and automatically determines whether it has been approved according to the set threshold. However, this patent cannot completely solve the above-mentioned technical problems, nor can it meet the needs of the present invention. Summary of the invention
[0008] In view of the defects in the prior art, the purpose of the present invention is to provide a multi-dimensional evaluation method and system for data assets based on blockchain and machine learning.
[0009] The multi-dimensional data asset evaluation method based on blockchain and machine learning provided by the present invention includes:
[0010] Step 1: Build a multi-dimensional data asset evaluation indicator system, with evaluation indicators including data quality, data value, data risk and data security dimensions;
[0011] Step 2: The metadata, ownership information, and transaction record information of data assets are stored on the blockchain through blockchain technology, and the distributed ledger, consensus mechanism, and cryptographic security features of blockchain are used to achieve trusted digital management of data assets throughout their life cycle; smart contracts are used to regulate and supervise the collection, cleaning, and storage of data assets, and the evaluation results and evaluation certificates of data assets are publicized through blockchain;
[0012] Step 3: Based on historical data asset evaluation cases, the evaluation indicator weight optimization model is formed through machine learning algorithm training. The machine learning algorithm includes logistic regression, decision tree and support vector machine. The evaluation indicator weight optimization model adaptively adjusts the importance weight of each evaluation indicator according to the characteristics of data assets in different industries and scenarios. It learns from the newly added evaluation data to achieve iterative update of evaluation knowledge and self-optimization of the algorithm.
[0013] Step 4: Consider the quality level of data assets, market supply and demand, and expected returns, and form market-based pricing for data assets through dynamic bidding and joint pricing.
[0014] Preferably, the step 1 comprises:
[0015] Conduct data quality dimension assessment, including:
[0016] Step 1.1: Accuracy assessment: Check whether there are anomalies or errors in the data records by comparing with authoritative data sources, including field value errors, missing values, and null values; use data quality verification tools to verify the accuracy of the data and identify outliers; perform manual sampling inspections, and have business experts determine whether the data records are accurate;
[0017] Step 1.2: Completeness assessment; evaluate the fill rate and missing rate of the field through the record value and unique value in the data statistics; compare the data between different data sources to check whether there is data missing; use data exploration tools to analyze the completeness level of data tables and fields;
[0018] Step 1.3: Consistency assessment; check whether the data conforms to the unified format specifications, including field data type, format, and value range; check whether the logical relationship between the data is consistent, including the size relationship between the calculation indicators; compare the newly collected data with the historical data to identify whether there is a large preset deviation; use consistency verification rules to detect conflicts between data;
[0019] Step 1.4: Timeliness assessment; monitor the delays in data collection, processing, and application, and assess data timeliness; set data timeliness thresholds based on data usage scenarios, and check whether the data meets the requirements; for data with high timeliness requirements, establish an automated data delay monitoring and early warning mechanism;
[0020] Conduct data value dimension assessment, including:
[0021] Step 2.1: Scarcity assessment: Analyze the market supply of similar data to determine the scarcity of the data; assign a higher scarcity weight to data that is highly unique and irreplaceable;
[0022] Step 2.2: Relevance assessment: Analyze the relevance of data with other data. Data with high relevance has higher value. The greater the breadth and depth of data relevance, the greater the value space that can be mined.
[0023] Step 2.3: Uniqueness assessment: assess whether the data is unique and exclusive. Data with high uniqueness has higher value. Analyze whether the data is difficult to copy. Data with high copying cost has higher value.
[0024] Step 2.4: Timeliness assessment: Based on the frequency of data updates, assess the timeliness value of data to the business; highly assess the timeliness of data in combination with specific scenarios;
[0025] Conduct data risk dimension assessment, including:
[0026] Step 3.1: Privacy risk assessment; Identify whether the data contains personal privacy information; Evaluate whether the data processing activities comply with the requirements of relevant laws and regulations on privacy protection; The larger the amount of private data and the higher the sensitivity, the greater the privacy risk;
[0027] Step 3.2: Compliance risk assessment; assess whether data processing activities comply with industry regulatory requirements; identify sensitive information in data assets and assess whether desensitization is in place; assess whether security measures for cross-border data transmission comply with legal requirements;
[0028] Conduct data security dimension assessment, including:
[0029] Step 4.1: Confidentiality assessment: Determine the confidentiality level of the data based on the sensitivity of the data; assess whether the access control and permission management of each link in the data life cycle meet the confidentiality requirements;
[0030] Step 4.2: Integrity assessment: Use consistency check and hash check technology to verify whether the data has been tampered with during storage and transmission; evaluate whether the data backup mechanism is complete and whether it can ensure data recoverability;
[0031] Step 4.3: Availability assessment: conduct availability assessment on data storage, computing, and transmission infrastructure, paying attention to whether there is a single point of failure; assess the impact of data quality issues on data availability; test the feasibility of off-site disaster recovery and emergency plans to ensure the continuous availability of data;
[0032] Conduct multi-dimensional comprehensive scoring, including: quantifying scores for each evaluation indicator to form a unified scoring system; using the analytic hierarchy process to determine the weight of each indicator, and using the weighted average method to calculate the comprehensive score; setting score thresholds, grading data assets, and forming a five-star rating model; using the rating results as a reference for data quality improvement and data asset pricing.
[0033] Preferably, the trusted digital management of data assets throughout their life cycle in step 2 includes:
[0034] Use distributed ledgers to achieve full life cycle management of data assets, specifically: Data asset collection stage: record the metadata information of data assets in the distributed ledger of the blockchain in the form of transactions; Data asset storage stage: store the data assets themselves in the distributed storage system under the chain, and record the storage location hash value of the data assets as a transaction in the blockchain ledger to establish a mapping relationship between data assets and storage locations; Data asset use stage: write the access control policy of the data assets into the smart contract and deploy it in the blockchain network; data users obtain access authorization to data assets by calling smart contracts when the access control conditions are met; record the use of data assets in the form of transactions in the blockchain ledger to achieve full traceability of data use; Data asset transaction stage: record the transaction information of data assets in the blockchain ledger; use the openness and transparency of the blockchain to achieve fairness and justice in the data transaction process; use digital signature technology to ensure the authenticity of the identities of both parties to the transaction; Data asset destruction stage: record the data destruction record in the form of transactions in the blockchain ledger, including the destruction time, execution subject, and destruction proof;
[0035] Use consensus mechanisms to ensure the consistency and credibility of data asset management. Specifically, blockchain nodes reach consensus on transactions related to data assets through consensus mechanisms to ensure the consistency of ledger data at each node. The consensus mechanism encourages nodes to keep honest accounts and punishes malicious nodes. Different consensus mechanisms are suitable for different application scenarios. Select the corresponding consensus mechanism based on the needs of data asset management.
[0036] Cryptography technology is applied to ensure the security of data asset management, specifically: using hash algorithms to generate unique digital fingerprints for data assets and their metadata; using asymmetric encryption algorithms to encrypt data assets; using digital signature technology to sign transactions and verify the identity of the transaction initiator; using zero-knowledge proof and homomorphic encryption technology to achieve calculation and analysis of data assets without leaking the original data.
[0037] Preferably, the step 3 comprises:
[0038] The goal of the evaluation index weight optimization model is to dynamically adjust the importance weight of each evaluation index through a machine learning algorithm. Suppose there are n evaluation indexes, the weight vector is \mathbf{w}=[w_1,w_2,\ldots,w_n], and the score vector of the evaluation index is \mathbf{x}=[x_1,x_2,\ldots,x_n], then the comprehensive score S can be expressed as: S=\sum_{i=1}^{n}w_i x_i; where: w_i represents the weight of the i-th evaluation index, and x_i represents the score of the i-th evaluation index;
[0039] Optimize weights \mathbf{w} using logistic regression, decision trees, and support vector machines;
[0040] Logistic regression maps the output of the linear regression model to [0, 1] through a logistic function, indicating the probability of belonging to a certain category;
[0041] Model representation: h_\theta(x)=\frac{1}{1+e^{-\theta^T x}}, where: h_\theta(x) is the predicted value; \theta is the model parameter vector; x is the input feature vector;
[0042] Loss function: J(\theta) = -\frac{1}{m}\sum_{i=1}^{m}\left[y^{(i)}\log(h_\theta(x^{(i)}))+(1y^{(i)})\log(1h_\theta(x^{(i)}))\right], where: m is the number of samples; y^{(i)} is the true label of the i-th sample; x^{(i)} is the feature vector of the i-th sample;
[0043] Gradient descent: \theta: = \theta\alpha\frac{\partial J(\theta)}{\partial\theta}, where: \alpha is the learning rate;
[0044] Decision trees achieve classification or regression by splitting features;
[0045] Information gain: IG(D, A) = H(D)\sum_{v\in\text{Values}(A)}\frac{|D_v|}{|D|}H(D_v), where: IG(D, A) is the information gain of feature A on data set D; H(D) is the entropy of data set D; D_v is the subset of feature A whose value is v;
[0046] Entropy: H(D) = -\sum_{i=1}^{k}p_i\log_2(p_i), where: k is the number of categories; p_i is the probability of category i;
[0047] Pruning: Pre-pruning: In the process of generating a decision tree, the splitting is stopped by setting a threshold; Post-pruning: After the complete decision tree is generated, the redundant branches are pruned by calculating the error rate;
[0048] Support vector machines achieve classification by finding a hyperplane that maximizes the margin;
[0049] Optimization goal: \min_{w,b}\frac{1}{2}\|w\|^2, where: w is the normal vector; b is the bias;
[0050] Constraints: y_i(w\cdot x_i+b)\geq 1, where: y_i is the label of the i-th sample; x_i is the feature vector of the i-th sample;
[0051] Dual problem: \max_{\alpha}\sum_{i=1}^{m}\alpha_i\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_j y_i y_j(x_i\cdot x_j), where: \alpha_i is the Lagrange multiplier;
[0052] Kernel function: K(x_i, x_j) = \phi(x_i)^T\phi(x_j);
[0053] The specific process of adjusting the importance weights of each evaluation indicator is as follows: define the objective function, including mean square error and cross entropy loss; initialize the weight vector \mathbf{w} to a random value or a uniformly distributed value; calculate the gradient of the objective function with respect to the weight \mathbf{w} \nabla J(\mathbf{w}); use the gradient descent method to update the weight: \mathbf{w}: = \mathbf{w}-\alpha\nabla J(\mathbf{w}), where \alpha is the learning rate; iterate the optimization until the objective function converges or the preset number of iterations is reached.
[0054] Preferably, step 4 comprises:
[0055] Dynamic bidding pricing: Data demanders publish data demand information on the data trading platform based on their own business needs, including data types, quantities, quality requirements, and expected uses; data suppliers review and evaluate demand information based on their own data assets to determine whether there are matching data assets; if there are matching data assets, the data supplier sets a reserve price, i.e., the lowest selling price, based on the data quality level and expected returns; multiple data suppliers quote for the same demand information, forming a dynamic bidding process, and the one with the highest bid has a greater chance of reaching a deal; a time limit for bidding is set on the data trading platform; during the bidding process, data suppliers dynamically adjust their bids based on the bids of other competitors; after the bidding ends, the data demander selects the best bid based on data quality and price factors and reaches a deal with the data supplier; the data trading platform records and supervises the transaction process;
[0056] Joint pricing: Multiple data demanders are interested in the same data asset and form a joint purchasing group; the joint purchasing group elects representatives to negotiate with the data supplier, taking data quality, expected returns, and purchase quantity into consideration during the negotiation; the data trading platform provides technical support for joint purchasing, including the release of demand information, group formation, and recording of the bargaining process; a unified selling price is reached through bargaining; the data trading platform records and supervises the bargaining process; members of the joint purchasing group pay the purchase price and share the right to use the data asset in accordance with the pre-agreed sharing method; the data trading platform manages the payment and allocation of funds.
[0057] The multi-dimensional data asset evaluation system based on blockchain and machine learning provided by the present invention includes:
[0058] Module M1: Construct a multi-dimensional data asset evaluation indicator system, with evaluation indicators including data quality, data value, data risk and data security dimensions;
[0059] Module M2: The metadata, ownership information, and transaction record information of data assets are stored on the blockchain through blockchain technology, and the distributed ledger, consensus mechanism, and cryptographic security features of blockchain are used to achieve trusted digital management of data assets throughout their life cycle; smart contracts are used to regulate and supervise the collection, cleaning, and storage of data assets, and the evaluation results and evaluation certificates of data assets are publicized through blockchain;
[0060] Module M3: Based on historical data asset evaluation cases, the evaluation indicator weight optimization model is formed through machine learning algorithm training. The machine learning algorithm includes logistic regression, decision tree and support vector machine. The evaluation indicator weight optimization model adaptively adjusts the importance weight of each evaluation indicator according to the characteristics of data assets in different industries and scenarios. It learns from the newly added evaluation data to achieve iterative update of evaluation knowledge and self-optimization of the algorithm.
[0061] Module M4: Comprehensively consider the quality level of data assets, market supply and demand, and expected returns, and form market-based pricing for data assets through dynamic bidding and joint pricing.
[0062] Preferably, the module M1 comprises:
[0063] Conduct data quality dimension assessment, including:
[0064] Module M1.1: Accuracy assessment: Check whether there are anomalies or errors in the data records by comparing with authoritative data sources, including field value errors, missing values, and null values; use data quality verification tools to verify the accuracy of the data and identify outliers; perform manual sampling inspections, and have business experts determine whether the data records are accurate;
[0065] Module M1.2: Completeness assessment; evaluate the fill rate and missing rate of fields through record values and unique values in data statistics; compare data between different data sources to check whether there is data missing; use data exploration tools to analyze the integrity level of data tables and fields;
[0066] Module M1.3: Consistency assessment; check whether the data conforms to the unified format specifications, including field data type, format, and value range; check whether the logical relationship between the data is consistent, including the size relationship between the calculation indicators; compare the newly collected data with the historical data to identify whether there is a large preset deviation; use consistency verification rules to detect conflicts between data;
[0067] Module M1.4: Timeliness assessment; monitor the delays in data collection, processing, and application, and assess data timeliness; set data timeliness thresholds based on data usage scenarios, and check whether the data meets the requirements; for data with high timeliness requirements, establish an automated data delay monitoring and early warning mechanism;
[0068] Conduct data value dimension assessment, including:
[0069] Module M2.1: Scarcity assessment; Analyze the market supply of similar data and determine the scarcity of data; assign a higher scarcity weight to data that is highly unique and irreplaceable;
[0070] Module M2.2: Relevance assessment; analyze the relevance of data with other data. Data with high relevance has higher value. The greater the breadth and depth of data relevance, the greater the value space that can be mined.
[0071] Module M2.3: Uniqueness assessment: assess whether the data is unique and exclusive. Data with high uniqueness has higher value. Analyze whether the data is difficult to copy. Data with high copying cost has higher value.
[0072] Module M2.4: Timeliness assessment: Evaluate the timeliness value of data to the business based on the frequency of data updates; highly evaluate the timeliness of data in combination with specific scenarios;
[0073] Conduct data risk dimension assessment, including:
[0074] Module M3.1: Privacy risk assessment; Identify whether the data contains personal privacy information; Evaluate whether the data processing activities comply with the requirements of relevant laws and regulations on privacy protection; The larger the amount of privacy data and the higher the sensitivity, the greater the privacy risk;
[0075] Module M3.2: Compliance risk assessment; assess whether data processing activities comply with industry regulatory requirements; identify sensitive information in data assets and assess whether desensitization is in place; assess whether security measures for cross-border data transmission comply with legal requirements;
[0076] Conduct data security dimension assessment, including:
[0077] Module M4.1: Confidentiality assessment: Determine the confidentiality level of data based on the sensitivity of the data; assess whether access control and permission management at all stages of the data life cycle meet confidentiality requirements;
[0078] Module M4.2: Integrity assessment: Verify whether the data has been tampered with during storage and transmission through consistency check and hash check technology; evaluate whether the data backup mechanism is complete and whether it can ensure the recoverability of the data;
[0079] Module M4.3: Availability assessment; conduct availability assessment on data storage, computing, and transmission infrastructure, paying attention to whether there is a single point of failure; assess the impact of data quality issues on data availability; test the feasibility of off-site disaster recovery and emergency plans to ensure the continuous availability of data;
[0080] Conduct multi-dimensional comprehensive scoring, including: quantifying scores for each evaluation indicator to form a unified scoring system; using the analytic hierarchy process to determine the weight of each indicator, and using the weighted average method to calculate the comprehensive score; setting score thresholds, grading data assets, and forming a five-star rating model; using the rating results as a reference for data quality improvement and data asset pricing.
[0081] Preferably, the trusted digital management of data assets throughout their life cycle in the module M2 includes:
[0082] Use distributed ledgers to achieve full life cycle management of data assets, specifically: Data asset collection stage: record the metadata information of data assets in the distributed ledger of the blockchain in the form of transactions; Data asset storage stage: store the data assets themselves in the distributed storage system under the chain, and record the storage location hash value of the data assets as a transaction in the blockchain ledger to establish a mapping relationship between data assets and storage locations; Data asset use stage: write the access control policy of the data assets into the smart contract and deploy it in the blockchain network; data users obtain access authorization to data assets by calling smart contracts when the access control conditions are met; record the use of data assets in the form of transactions in the blockchain ledger to achieve full traceability of data use; Data asset transaction stage: record the transaction information of data assets in the blockchain ledger; use the openness and transparency of the blockchain to achieve fairness and justice in the data transaction process; use digital signature technology to ensure the authenticity of the identities of both parties to the transaction; Data asset destruction stage: record the data destruction record in the form of transactions in the blockchain ledger, including the destruction time, execution subject, and destruction proof;
[0083] Use consensus mechanisms to ensure the consistency and credibility of data asset management. Specifically, blockchain nodes reach consensus on transactions related to data assets through consensus mechanisms to ensure the consistency of ledger data at each node. The consensus mechanism encourages nodes to keep honest accounts and punishes malicious nodes. Different consensus mechanisms are suitable for different application scenarios. Select the corresponding consensus mechanism based on the needs of data asset management.
[0084] Cryptography technology is applied to ensure the security of data asset management, specifically: using hash algorithms to generate unique digital fingerprints for data assets and their metadata; using asymmetric encryption algorithms to encrypt data assets; using digital signature technology to sign transactions and verify the identity of the transaction initiator; using zero-knowledge proof and homomorphic encryption technology to achieve calculation and analysis of data assets without leaking the original data.
[0085] Preferably, the module M3 comprises:
[0086] The goal of the evaluation index weight optimization model is to dynamically adjust the importance weight of each evaluation index through a machine learning algorithm. Suppose there are n evaluation indexes, the weight vector is \mathbf{w}=[w_1,w_2,\ldots,w_n], and the score vector of the evaluation index is \mathbf{x}=[x_1,x_2,\ldots,x_n], then the comprehensive score S can be expressed as: S=\sum_{i=1}^{n}w_i x_i; where: w_i represents the weight of the i-th evaluation index, and x_i represents the score of the i-th evaluation index;
[0087] Optimize weights \mathbf{w} using logistic regression, decision trees, and support vector machines;
[0088] Logistic regression maps the output of the linear regression model to [0, 1] through a logistic function, indicating the probability of belonging to a certain category;
[0089] Model representation: h_\theta(x)=\frac{1}{1+e^{-\theta^T x}}, where: h_\theta(x) is the predicted value; \theta is the model parameter vector; x is the input feature vector;
[0090] Loss function: J(\theta) = -\frac{1}{m}\sum_{i=1}^{m}\left[y^{(i)}\log(h_\theta(x^{(i)}))+(1y^{(i)})\log(1h_\theta(x^{(i)}))\right], where: m is the number of samples; y^{(i)} is the true label of the i-th sample; x^{(i)} is the feature vector of the i-th sample;
[0091] Gradient descent: \theta: = \theta\alpha\frac{\partial J(\theta)}{\partial\theta}, where: \alpha is the learning rate;
[0092] Decision trees achieve classification or regression by splitting features;
[0093] Information gain: IG(D, A) = H(D)\sum_{v\in\text{Values}(A)}\frac{|D_v|}{|D|}H(D_v), where: IG(D, A) is the information gain of feature A on data set D; H(D) is the entropy of data set D; D_v is the subset of feature A whose value is v;
[0094] Entropy: H(D) = -\sum_{i=1}^{k}p_i\log_2(p_i), where: k is the number of categories; p_i is the probability of category i;
[0095] Pruning: Pre-pruning: In the process of generating a decision tree, the splitting is stopped by setting a threshold; Post-pruning: After the complete decision tree is generated, the redundant branches are pruned by calculating the error rate;
[0096] Support vector machines achieve classification by finding a hyperplane that maximizes the margin;
[0097] Optimization goal: \min_{w,b}\frac{1}{2}\|w\|^2, where: w is the normal vector; b is the bias;
[0098] Constraints: y_i(w\cdot x_i+b)\geq 1, where: y_i is the label of the i-th sample; x_i is the feature vector of the i-th sample;
[0099] Dual problem: \max_{\alpha}\sum_{i=1}^{m}\alpha_i\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_j y_i y_j(x_i\cdot x_j), where: \alpha_i is the Lagrange multiplier;
[0100] Kernel function: K(x_i, x_j) = \phi(x_i)^T\phi(x_j);
[0101] The specific process of adjusting the importance weights of each evaluation indicator is as follows: define the objective function, including mean square error and cross entropy loss; initialize the weight vector \mathbf{w} to a random value or a uniformly distributed value; calculate the gradient of the objective function with respect to the weight \mathbf{w} \nabla J(\mathbf{w}); use the gradient descent method to update the weight: \mathbf{w}: = \mathbf{w}-\alpha\nabla J(\mathbf{w}), where \alpha is the learning rate; iterate the optimization until the objective function converges or the preset number of iterations is reached.
[0102] Preferably, the module M4 comprises:
[0103] Dynamic bidding pricing: Data demanders publish data demand information on the data trading platform based on their own business needs, including data types, quantities, quality requirements, and expected uses; data suppliers review and evaluate demand information based on their own data assets to determine whether there are matching data assets; if there are matching data assets, the data supplier sets a reserve price, i.e., the lowest selling price, based on the data quality level and expected returns; multiple data suppliers quote for the same demand information, forming a dynamic bidding process, and the one with the highest bid has a greater chance of reaching a deal; a time limit for bidding is set on the data trading platform; during the bidding process, data suppliers dynamically adjust their bids based on the bids of other competitors; after the bidding ends, the data demander selects the best bid based on data quality and price factors and reaches a deal with the data supplier; the data trading platform records and supervises the transaction process;
[0104] Joint pricing: Multiple data demanders are interested in the same data asset and form a joint purchasing group; the joint purchasing group elects representatives to negotiate with the data supplier, taking data quality, expected returns, and purchase quantity into consideration during the negotiation; the data trading platform provides technical support for joint purchasing, including the release of demand information, group formation, and recording of the bargaining process; a unified selling price is reached through bargaining; the data trading platform records and supervises the bargaining process; members of the joint purchasing group pay the purchase price and share the right to use the data asset in accordance with the pre-agreed sharing method; the data trading platform manages the payment and allocation of funds.
[0105] Compared with the prior art, the present invention has the following beneficial effects:
[0106] (1) The present invention uses blockchain technology to record data asset metadata and transaction information, ensuring the credibility of the evaluation process and results; introduces a machine learning model to dynamically optimize the evaluation index weights, thereby improving the objectivity and applicability of the evaluation results;
[0107] (2) Through a series of processes such as data collection, metadata on-chain, evaluation engine calculation, pricing mechanism formation, blockchain disclosure, and visual presentation, the present invention ultimately achieves scientific evaluation, market-based pricing, and credible disclosure of data asset value, providing an important reference for the circulation of data elements;
[0108] (3) The present invention adopts modular design and containerized deployment, has good scalability, and each module of the system communicates based on industrial standard protocols, ensuring interface compatibility and interoperability;
[0109] (4) The present invention overcomes the problems of existing data asset evaluation methods, such as single dimension, strong subjectivity and lack of credibility, and realizes scientific evaluation, market-based pricing and credible disclosure of data asset value, providing an important reference for the circulation of data elements, which is of great significance for promoting the prosperity and development of the data element market. BRIEF DESCRIPTION OF THE DRAWINGS
[0110] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0111] Figure 1 This is the architecture diagram of the data asset assessment system;
[0112] Figure 2 It is the workflow diagram of the data asset assessment system;
[0113] Figure 3 This is a schematic diagram of the multi-dimensional evaluation indicator system for data assets;
[0114] Figure 4 A schematic diagram of blockchain-based data asset evaluation information storage;
[0115] Figure 5 Flowchart for optimizing evaluation metric weights for machine learning models;
[0116] Figure 6 This is a schematic diagram of the market-based pricing mechanism for data assets;
[0117] Figure 7 Shows an example diagram for the visual interactive interface. DETAILED DESCRIPTION
[0118] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0119] Example 1
[0120] The present invention provides a multi-dimensional data asset evaluation method based on blockchain and machine learning, comprising the following steps:
[0121] Step 1: Build a multi-dimensional data asset evaluation indicator system and design evaluation indicators from the dimensions of data quality, data value, data risk, and data security, such as Figure 3 .
[0122] In the dimension of data quality, focus on evaluating the accuracy, completeness, consistency, timeliness, etc. of the data;
[0123] In the dimension of data value, focus on evaluating the scarcity, relevance, uniqueness, timeliness, etc. of data;
[0124] In the data risk dimension, focus on assessing data privacy risks, legal compliance risks, etc.
[0125] In the dimension of data security, the focus is on evaluating the confidentiality, integrity, availability, etc. of the data.
[0126] Through the comprehensive scoring of multi-dimensional indicators, a "five-star" rating model for data assets is formed.
[0127] Specific process and technical means for multi-dimensional evaluation of data assets:
[0128] 1. Data quality dimension assessment
[0129] 1.1 Accuracy Assessment
[0130] By comparing with authoritative data sources, check whether there are anomalies or errors in the data records, such as incorrect, missing, or null field values.
[0131] Use data quality checking tools to verify data accuracy and identify outliers.
[0132] Manual sampling inspection, in which business experts determine whether the data records are accurate.
[0133] 1.2 Integrity Assessment
[0134] Evaluate the fill rate and missing rate of the field through the record values and unique values in the data statistics.
[0135] Compare data between different data sources to check for missing data.
[0136] Use data exploration tools to analyze the completeness level of data tables and fields.
[0137] 1.3 Consistency Assessment
[0138] Check whether the data conforms to the unified format specifications, such as field data type, format, value range, etc. Check whether the logical relationship between the data is consistent, such as the size relationship between the calculation indicators.
[0139] Compare the newly collected data with the historical data to identify whether there are major deviations.
[0140] Use consistency check rules to automatically detect conflicts between data.
[0141] 1.4 Timely Assessment
[0142] Monitor the delays in each link of data collection, processing, and application, and evaluate the timeliness of data.
[0143] According to the data usage scenario, set the threshold of data timeliness requirements and check whether the data meets the requirements. For data with high timeliness requirements, establish an automated data delay monitoring and early warning mechanism.
[0144] 2. Data value dimension assessment
[0145] 2.1 Scarcity Assessment
[0146] Analyze the market supply of similar data and determine the scarcity of the data.
[0147] Assign a higher scarcity weight to data that is highly unique and irreplaceable.
[0148] 2.2 Relevance Assessment
[0149] Analyze the correlation between data and other data. Data with high correlation has higher value.
[0150] The greater the breadth and depth of data associations, the greater the value space that can be mined.
[0151] 2.3 Uniqueness Assessment
[0152] Evaluate whether the data is unique and exclusive; data with high uniqueness has higher value.
[0153] Analyze whether the data is difficult to copy. Data with high copy costs is more valuable.
[0154] 2.4 Timeliness Assessment
[0155] Evaluate the timeliness of data to the business based on the frequency of data updates.
[0156] Data timeliness is highly correlated to the application scenario and should be evaluated in conjunction with specific applications.
[0157] 3. Data risk dimension assessment
[0158] 3.1 Privacy Risk Assessment
[0159] Identify whether the data contains personal privacy information, such as identity, property and other sensitive information.
[0160] Assess whether data processing activities comply with relevant laws and regulations on privacy protection.
[0161] The larger the amount of private data and the higher its sensitivity, the greater the privacy risk.
[0162] 3.2 Compliance Risk Assessment
[0163] Assess whether data processing activities comply with industry regulatory requirements.
[0164] Identify sensitive information in data assets and assess whether desensitization is in place.
[0165] Assess whether security measures for cross-border data transfers comply with legal requirements.
[0166] 4. Data security dimension assessment
[0167] 4.1 Confidentiality Assessment
[0168] Determine the confidentiality level of the data based on the sensitivity of the data.
[0169] Evaluate whether access control and permission management at all stages of the data life cycle meet confidentiality requirements.
[0170] 4.2 Integrity Assessment
[0171] Through consistency verification, hash verification and other technical means, verify whether the data has been tampered with during storage and transmission.
[0172] Evaluate whether the data backup mechanism is complete and whether it can guarantee data recoverability.
[0173] 4.3 Usability Evaluation
[0174] Conduct availability assessments on infrastructure such as data storage, computing, and transmission, paying attention to whether there are single points of failure.
[0175] Assess the impact of data quality issues on data usability.
[0176] Test the feasibility of off-site disaster recovery and emergency plans to ensure the continuous availability of data.
[0177] 5. Multi-dimensional comprehensive scoring
[0178] Quantify and score each evaluation indicator to form a unified scoring system.
[0179] The analytic hierarchy process was used to determine the weight of each indicator, and the weighted average method was used to calculate the comprehensive score.
[0180] Set score thresholds, classify data assets, and form a five-star rating model.
[0181] The rating results can serve as an important reference for improving data quality and pricing of data assets.
[0182] This multi-dimensional evaluation index system makes up for the deficiency of traditional methods that only focus on data scale, and can more comprehensively reflect the intrinsic value of data assets. Through systematic consideration and quantitative evaluation of multiple dimensions such as data quality, value, risk, and security, a scientific and reasonable data asset evaluation framework has been constructed, providing an important basis for value discovery of data assets.
[0183] Step 2: Use blockchain technology to record data asset metadata and transaction information to ensure the credibility of the evaluation process and results.
[0184] The metadata, ownership information, transaction records and other key information of data assets are stored on the blockchain through blockchain technology. The distributed ledger, consensus mechanism, cryptographic security and other characteristics of blockchain are used to achieve trusted digital management of data assets throughout their life cycle, providing a verifiable and tamper-proof data foundation for the evaluation process.
[0185] Use smart contracts to regulate and supervise all aspects of data asset collection, cleaning, storage, etc. to ensure that the data source is authentic and reliable, and the flow is safe and controllable;
[0186] The evaluation results and evaluation certificates of data assets will be made public through blockchain, subject to supervision by stakeholders, and the credibility of the evaluation process and results will be improved.
[0187] The innovative application of blockchain technology in the field of data asset evaluation solves the problem of opacity and difficulty in verification in the traditional evaluation process. By storing key information and evaluation certificates on the chain, and utilizing the traceability and tamper-proofness of blockchain, a credible audit of the entire evaluation process is achieved, greatly enhancing the credibility of the evaluation results.
[0188] The specific process of how to apply blockchain technology in the trusted digital management of data assets throughout the life cycle:
[0189] 1. Use distributed ledgers to achieve full life cycle management of data assets
[0190] 1.1 Data asset collection stage
[0191] The metadata information of data assets (such as data name, format, source, collection time, etc.) is recorded in the distributed ledger of the blockchain in the form of transactions.
[0192] The tamper-proof characteristics of blockchain are utilized to ensure the authenticity and traceability of metadata information.
[0193] 1.2 Data asset storage stage
[0194] Store the data assets themselves in off-chain distributed storage systems, such as IPFS, FileCoin, etc.
[0195] The storage location hash value of the data asset is recorded as a transaction in the blockchain ledger to establish a mapping relationship between the data asset and the storage location.
[0196] Leverage the immutability of blockchain to ensure the authenticity and integrity of data asset storage locations.
[0197] 1.3 Data asset use stage
[0198] The access control policy of data assets is written into smart contracts and deployed in the blockchain network.
[0199] Data users obtain access authorization to data assets by calling smart contracts if the access control conditions are met.
[0200] The usage records of data assets (such as who accessed which data and when) are recorded in the blockchain ledger in the form of transactions, making the entire data usage process traceable.
[0201] 1.4 Data asset transaction stage
[0202] The transaction information of data assets (such as transaction subject, transaction time, transaction price, etc.) is recorded in the blockchain ledger.
[0203] Utilize the openness and transparency of blockchain to achieve fairness and justice in the data transaction process.
[0204] Through digital signature technology, the authenticity of the identities of both parties to the transaction is guaranteed to prevent denial.
[0205] 1.5 Data asset destruction phase
[0206] The data destruction records are recorded in the blockchain ledger in the form of transactions, including the destruction time, execution entity, destruction proof, etc.
[0207] The immutability of blockchain is utilized to ensure the authenticity and irreversibility of the data destruction process.
[0208] 2. Use consensus mechanisms to ensure consistency and credibility of data asset management
[0209] Blockchain nodes reach agreement on transactions related to data assets through a consensus mechanism, ensuring the consistency of ledger data on each node.
[0210] The consensus mechanism encourages nodes to keep honest accounts, punishes malicious nodes, and improves the credibility of ledger data.
[0211] Different consensus mechanisms (such as PoW, PoS, PBFT, etc.) are suitable for different application scenarios, and the appropriate consensus mechanism can be selected according to the needs of data asset management.
[0212] 3. Apply cryptography technology to ensure the security of data asset management
[0213] Use hash algorithms to generate unique digital fingerprints for data assets and their metadata to prevent data tampering.
[0214] Use asymmetric encryption algorithms to encrypt data assets to protect data privacy. Only users with the corresponding private keys can decrypt and access them.
[0215] Use digital signature technology to sign transactions, verify the identity of the transaction initiator, and prevent repudiation.
[0216] Use cryptographic techniques such as zero-knowledge proof and homomorphic encryption to achieve calculation and analysis of data assets without leaking the original data.
[0217] In summary, blockchain technology can play an important role in the management of the entire life cycle of data assets. By using distributed ledgers to record the metadata, storage location, access control policies, usage records, transaction information, etc. of data assets, using consensus mechanisms to ensure the consistency and credibility of ledger data among multiple parties, and applying cryptographic technology to ensure the integrity, privacy and security of data, we can ultimately achieve trusted digitization of the entire process of data asset management.
[0218] Step 3: Introduce a machine learning model to dynamically optimize the weights of evaluation indicators to improve the objectivity and applicability of the evaluation results.
[0219] Based on massive historical data asset evaluation cases, we train machine learning algorithms such as logistic regression, decision trees, and support vector machines to form an evaluation index weight optimization model;
[0220] The model can adaptively adjust the importance weight of each evaluation indicator according to the characteristics of data assets in different industries and scenarios, overcoming the defects of traditional methods, which are highly subjective and poorly applicable.
[0221] The machine learning model can continuously learn from new evaluation data to achieve iterative updating of evaluation knowledge and self-optimization of the algorithm, making the evaluation results more intelligent and accurate.
[0222] The application of machine learning technology in the optimization of indicator weights is an important improvement on the traditional expert experience judgment. By automatically learning the weight rules from the data and dynamically adjusting the indicator weights, the differences in the characteristics of different data assets are fully considered, which improves the objectivity and applicability of the evaluation results. At the same time, the continuous learning ability of the model also provides important support for the updating and improvement of evaluation knowledge.
[0223] The goal of the evaluation indicator weight optimization model is to dynamically adjust the importance weights of each evaluation indicator through a machine learning algorithm. Assuming there are n evaluation indicators, the weight vector is \mathbf{w}=[w_1,w_2,\ldots,w_n], and the score vector of the evaluation indicator is \mathbf{x}=[x_1,x_2,\ldots,x_n], then the comprehensive score S can be expressed as: S=\sum_{i=1}^{n}w_i x_i; where: w_i represents the weight of the i-th evaluation indicator, and x_i represents the score of the i-th evaluation indicator;
[0224] To optimize the weights \mathbf{w}, machine learning algorithms such as logistic regression, decision trees, and support vector machines can be used. The following are the training processes and calculation formulas of these algorithms.
[0225] Logistic regression algorithm
[0226] Logistic regression is a statistical learning method for binary classification problems. Its core is to map the output of the linear regression model to [0, 1] through a logistic function (Sigmoid function), indicating the probability of belonging to a certain category.
[0227] 1. Model representation:
[0228] h_\theta(x)=\frac{1}{1+e^{-\theta^T x}}
[0229] Where: h_\theta(x) is the predicted value; \theta is the model parameter vector; x is the input feature vector;
[0230] 2. Loss function:
[0231] J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}\left[y^{(i)}\log(h_\theta(x^{(i)}))+(1y^{(i)})\log(1h_\theta(x^{(i)}))\right]
[0232] Where: m is the number of samples; y^{(i)} is the true label of the i-th sample; x^{(i)} is the feature vector of the i-th sample;
[0233] 3. Gradient Descent:
[0234] \theta:=\theta\alpha\frac{\partial J(\theta)}{\partial\theta}
[0235] Where: \alpha is the learning rate;
[0236] Decision Tree Algorithm
[0237] A decision tree is a tree-structured model used for classification and regression, which achieves classification or regression by splitting features.
[0238] 1. Information Gain:
[0239] IG(D,A)=H(D)\sum_{v\in\text{Values}(A)}\frac{|D_v|}{|D|}H(D_v)
[0240] Where: IG(D, A) is the information gain of feature A on data set D; H(D) is the entropy of data set D; D_v is the subset of feature A with value v;
[0241] 2. Entropy:
[0242] H(D)=-\sum_{i=1}^{k}p_i\log_2(p_i)
[0243] Where: k is the number of categories; p_i is the probability of category i;
[0244] 3. Pruning:
[0245] Pre-pruning: In the process of generating a decision tree, a threshold is set to stop splitting.
[0246] Post-pruning: After generating a complete decision tree, redundant branches are pruned by calculating the error rate.
[0247] Support Vector Machine (SVM) Algorithm
[0248] Support vector machine is a machine learning algorithm for binary classification problems, which achieves classification by finding a hyperplane that maximizes the margin.
[0249] 1. Optimization objectives:
[0250] \min_{w,b}\frac{1}{2}\|w\|^2
[0251] Where: w is the normal vector; b is the bias;
[0252] 2. Constraints:
[0253] y_i(w\cdot x_i+b)\geq 1
[0254] Among them: y_i is the label of the i-th sample; x_i is the feature vector of the i-th sample;
[0255] 3. Dual problem:
[0256] \max_{\alpha}\sum_{i=1}^{m}\alpha_i\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_j y_i y_j(x_i\cdot x_j)
[0257] Where: \alpha_i is the Lagrange multiplier;
[0258] 4. Kernel function:
[0259] K(x_i,x_j)=\phi(x_i)^T\phi(x_j)
[0260] Commonly used kernel functions include linear kernel, polynomial kernel and Gaussian kernel.
[0261] Through the training process of the above algorithm, the weights of the evaluation indicators can be dynamically optimized, making the evaluation results more objective and applicable.
[0262] The specific process of adjusting the importance weights of each evaluation indicator is as follows:
[0263] 1. Define the objective function:
[0264] The objective function can be an indicator designed according to the requirements to measure the performance of the model, such as mean square error (MSE), cross entropy loss, etc.
[0265] 2. Initialize weights:
[0266] Initialize the weight vector \mathbf{w} to a random value or a uniformly distributed value.
[0267] 3. Calculate the gradient:
[0268] Calculate the gradient of the objective function with respect to the weight \mathbf{w} \nabla J(\mathbf{w}).
[0269] 4. Update weights:
[0270] Update the weights using gradient descent or other optimization algorithms:
[0271] \mathbf{w}:=\mathbf{w}-\alpha\nabla J(\mathbf{w})
[0272] Among them, \alpha is the learning rate.
[0273] 5. Iterative optimization:
[0274] Repeat steps 3 and 4 until the objective function converges or the preset number of iterations is reached.
[0275] Through the training process of the above algorithm, the weights of the evaluation indicators can be dynamically optimized, making the evaluation results more objective and applicable.
[0276] The specific process of iterative update of the model and self-optimization of the algorithm:
[0277] Model iterative updating refers to gradually optimizing and improving the performance of the model by continuously introducing new data and feedback. The following are several common model iterative updating methods:
[0278] 1. Retraining with all data:
[0279] Process: Merge historical training data with new data to retrain a new model.
[0280] Advantages and disadvantages: This method usually works best, but it is time-consuming, resource-intensive, and has poor real-time performance, especially in big data scenarios.
[0281] 2. Model Fusion:
[0282] Process: Use the prediction results of the old model as a new feature and train a new model on the new data.
[0283] Advantages and disadvantages: Training takes a short time, increases the complexity of decision-making, and the amount of new data must be sufficient to ensure the fusion effect.
[0284] 3. Incremental (online) learning:
[0285] Process: Use the new data to further update the original model. For example, use the partial_fit method in sklearn for incremental learning.
[0286] Advantages and disadvantages: Memory-friendly, fast model iteration and high efficiency, especially suitable for learning neural networks.
[0287] Step 4: Explore the market-based pricing mechanism for data assets and provide pricing references for the circulation of data elements.
[0288] Comprehensively consider factors such as the quality level of data assets, market supply and demand, and expected returns to form market-based pricing for data assets and provide a price benchmark for the trading and circulation of data assets;
[0289] Introduce innovative pricing methods such as dynamic bidding and joint pricing to give full play to the decisive role of the market in the allocation of data factors;
[0290] The following is a supplement and improvement to the specific implementation process of "dynamic bidding" and "joint pricing" in the market-based pricing mechanism of data assets, and a detailed explanation based on factors such as the quality level of data assets, market supply and demand, and expected returns.
[0291] 1. Dynamic bidding pricing
[0292] 1.1 Pricing Process
[0293] Data demanders publish data demand information on the data trading platform based on their own business needs, including data type, quantity, quality requirements, intended use, etc.
[0294] The data supplier reviews and evaluates the demand information based on its own data assets to determine whether there are matching data assets.
[0295] If there are matching data assets, the data supplier sets a reserve price, i.e. the minimum selling price, based on factors such as data quality level and expected returns.
[0296] Multiple data suppliers bid for the same demand information, forming a dynamic bidding process. The one with the highest bid has a greater chance of reaching a deal.
[0297] Data trading platforms can set time limits for bidding, such as 24 hours, 48 hours, etc., to improve transaction efficiency.
[0298] During the bidding process, data suppliers can dynamically adjust their bids based on the bids of other competitors to increase the chances of a deal.
[0299] After the bidding ends, the data demander will comprehensively consider factors such as data quality and price, select the best bid, and reach a deal with the data supplier.
[0300] The data trading platform records and supervises the transaction process to ensure fairness, justice and transparency of transactions.
[0301] 1.2 Factors affecting price
[0302] The quality level of data assets is an important factor affecting the price. The higher the quality level, the higher the price.
[0303] Market supply and demand will affect prices. If the supply of similar data assets is sufficient, the price will be relatively low; if the supply is scarce, the price will rise.
[0304] The expected profit of the data supplier will also affect the price. The higher the expected profit, the higher the price will usually be.
[0305] The budget and willingness to pay of the data demander will also affect the transaction price. The higher the budget and the stronger the willingness to pay, the higher the price will usually be.
[0306] The timeliness of data assets also affects the price. The fresher and more time-sensitive the data is, the higher the price will generally be.
[0307] The uniqueness and scarcity of data assets also affect the price. The higher the uniqueness and scarcity, the higher the price will usually be.
[0308] 2. Joint pricing
[0309] 2.1 Pricing Process
[0310] Multiple data demanders are interested in the same data asset and form a joint purchasing group.
[0311] The joint purchasing group elects a representative to negotiate with the data supplier, taking into account factors such as data quality, expected returns, and purchase quantity.
[0312] The data trading platform provides technical support for joint purchasing, such as demand information release, group formation, and bargaining process recording.
[0313] A unified price is reached through negotiation. This price is usually lower than the price of individual purchases, but it can bring a larger transaction scale to the data supplier.
[0314] The data trading platform records and supervises the bargaining process to ensure fairness, impartiality and transparency of the bargaining.
[0315] Members of the joint purchasing group pay the purchase price and share the right to use the data assets according to the pre-agreed sharing method.
[0316] The data trading platform manages the payment and allocation of funds to ensure the security and timely arrival of funds.
[0317] 2.2 Factors affecting price
[0318] The more data demanders that purchase data together, the stronger their bargaining power will be, and they can often get more favorable prices.
[0319] Different data demanders may have different data quality requirements, which need to be agreed upon during the bargaining process and reflected in the price.
[0320] The expected revenue of the data supplier will also affect the selling price. If joint purchase can bring considerable total revenue, the data supplier may offer a more favorable price.
[0321] The larger the scale of data assets involved in the joint purchase, the stronger the bargaining power of the data supplier may be.
[0322] The timeliness requirements of data assets will also affect the price. If the joint purchasing group has higher requirements for data timeliness, they may need to pay a higher price.
[0323] The stability and purchase frequency of the joint purchasing group will also affect the price. If it is a long-term and stable cooperative relationship, the data supplier may offer a more favorable price.
[0324] The quality level of data assets, market supply and demand, expected returns and other factors play an important role in the pricing process. Through innovative methods such as dynamic bidding and joint pricing, the market can better play a decisive role in the allocation of data elements and promote the circulation and value realization of data elements.
[0325] At the same time, data trading platforms play an important role in this process. They need to provide comprehensive technical support and trading rules, and record and supervise the trading process as necessary to ensure that the transaction is fair, orderly and secure. In addition, they need to be closely integrated with the data asset evaluation system to provide a more scientific and accurate basis for data pricing.
[0326] Use the blockchain platform to make the pricing process and results open and transparent, accept supervision from all parties, and prevent price manipulation.
[0327] Data asset pricing is a key link in the market-based allocation of data elements. Based on data asset evaluation, this paper comprehensively considers factors such as market supply and demand and proposes a scientific and reasonable pricing mechanism. By providing reference prices for data asset transactions and guiding the rational flow and optimal allocation of data resources, it is of great significance to promote the prosperity of the data element market.
[0328] Example 2
[0329] like Figure 1 The present invention also provides a data asset evaluation system, comprising:
[0330] Data asset management module, used for data asset collection, cleaning, storage, classification and metadata extraction;
[0331] By connecting with various business systems within the enterprise, such as ERP and CRM, the structured, semi-structured and unstructured data generated by the enterprise in R&D, production, sales, service and other links are collected, and external data is obtained through crawlers, data procurement and other methods to form enterprise data assets;
[0332] Perform pre-processing such as deduplication, denoising, and format conversion on the collected raw data to improve data quality;
[0333] According to the type, scale, and application scenario of data assets, select appropriate storage media such as relational databases, distributed storage systems, and storage methods, taking into account storage security and desensitizing sensitive data;
[0334] Manage data assets by categories according to business attributes, data features, etc., form a data asset catalog, and implement automatic classification based on machine learning algorithms such as clustering and classification;
[0335] Extract key metadata information of data assets, such as name, format, size, source, timestamp, quality score, etc., and store them on the blockchain. Figure 4 .
[0336] Data asset management is the basic work for data asset evaluation. The data asset management module of the present invention brings together internal and external data of the enterprise to form a large-scale and diversified data asset pool. At the same time, practical technical solutions are proposed for the collection, cleaning, storage, classification and other aspects of data assets. With data asset catalog and metadata management as the starting point, standardized and refined management of data assets is achieved, laying a solid foundation for subsequent evaluation work.
[0337] Blockchain network, used for metadata upload, evaluation information storage and evaluation certification publicity;
[0338] Establish consensus with multiple nodes in the blockchain through consensus algorithms such as PBFT to ensure data consistency and credibility;
[0339] The specific process of establishing consensus with multiple nodes in the blockchain through the consensus algorithm:
[0340] The consensus algorithm is a core component of blockchain technology, which enables nodes in a distributed network to reach a consensus on the state of the blockchain. The following are the specific processes of several common consensus algorithms, including Proof of Work (PoW), Proof of Stake (PoS), Practical Byzantine Fault Tolerance (PBFT), etc.
[0341] 1. Proof of Work (PoW)
[0342] Proof of Work is the earliest consensus algorithm used in blockchain systems. Its core idea is to compete for the right to record blockchain accounts by calculating complex mathematical problems.
[0343] 1.1. Node competition: All participating nodes solve a mathematical puzzle by calculating the hash value. This puzzle is usually to find a hash value that meets certain conditions.
[0344] 1.2. Verification and broadcast: The first node that finds a hash value that satisfies the conditions broadcasts the new block to other nodes in the network.
[0345] 1.3. Verification: Other nodes verify the validity of the block. If the verification is successful, these nodes add the block to their own blockchain.
[0346] 1.4. Rewards: The node that successfully finds the hash value receives block rewards and transaction fees.
[0347] plaintext
[0348] PoW process:
[0349] (1) Nodes A, B, and C compete to calculate the hash value.
[0350] (2) Node A first finds a hash value that meets the conditions and generates a new block.
[0351] (3) Node A broadcasts the new block to nodes B and C.
[0352] (4) Nodes B and C verify the validity of the new block.
[0353] (5) After verification, nodes B and C add the new block to their respective blockchains.
[0354] (6) Node A obtains block rewards and transaction fees.
[0355] 2. Proof of Stake (PoS)
[0356] Proof of Stake requires nodes to stake a certain amount of cryptocurrency to obtain the right to keep accounts. The more coins staked, the greater the probability of obtaining the right to keep accounts.
[0357] 2.1. Staking: A node locks a certain amount of cryptocurrency into a smart contract.
[0358] 2.2. Election: A node is selected as a block producing node based on the amount of stake and a random algorithm.
[0359] 2.3. Verification and broadcasting: The selected node generates a new block and broadcasts it to other nodes in the network.
[0360] 2.4. Verification: Other nodes verify the validity of the block. If the verification is successful, these nodes add the block to their own blockchain.
[0361] 2.5. Rewards: Nodes that successfully generate blocks receive block rewards.
[0362] plaintext
[0363] PoS process:
[0364] (1) Nodes A, B, and C pledge cryptocurrency.
[0365] (2) Based on the pledge amount and the random algorithm, node B is selected as the block producing node.
[0366] (3) Node B generates a new block and broadcasts it to nodes A and C.
[0367] (4) Nodes A and C verify the validity of the new block.
[0368] (5) After verification, nodes A and C add the new block to their respective blockchains.
[0369] (6) Node B receives the block reward.
[0370] 3. Practical Byzantine Fault Tolerance (PBFT)
[0371] PBFT is a consensus algorithm suitable for permissioned chains that can reach consensus even when a small number of nodes act maliciously.
[0372] 3.1. Pre-prepare:
[0373] The primary node generates a new block and broadcasts it to all replica nodes.
[0374] 3.2. Preparation phase (Prepare):
[0375] After receiving the new block, the replica node verifies its validity and broadcasts the prepare message to other nodes.
[0376] 3.3. Commit phase:
[0377] When a node receives at least $$2f+1$$ prepare messages (where $$f$$ is the tolerable number of malicious nodes), it broadcasts a commit message.
[0378] When a node receives at least $$2f+1$$ commit messages, it adds the new block to the blockchain.
[0379] plaintext
[0380] PBFT process:
[0381] (1) The master node P generates a new block and broadcasts it to replica nodes R1, R2, and R3.
[0382] (2) Replica nodes R1, R2, and R3 verify the validity of the new block and broadcast a prepare message.
[0383] (3) After receiving at least 2f+1 prepare messages, nodes R1, R2, and R3 broadcast commit messages.
[0384] (4) After nodes R1, R2, and R3 receive at least 2f+1 commit messages, they add the new block to the blockchain.
[0385] 4. Raft consensus algorithm
[0386] Raft is a consensus algorithm for distributed systems, especially suitable for consortium chains.
[0387] 4.1. Leader election:
[0388] When a node does not receive a heartbeat message from the leader within the specified time, it becomes a candidate node and initiates an election.
[0389] A candidate node becomes a leader after receiving votes from the majority of nodes.
[0390] 4.2. Log replication:
[0391] The leader receives client requests and adds them to the log as log entries.
[0392] The leader replicates the log entries to all followers.
[0393] 4.3. Log submission:
[0394] When the log entry has been replicated to a majority of nodes, the leader commits it and applies it to the state machine.
[0395] After receiving the commit message, the slave node also applies the log entry to the state machine.
[0396] plaintext
[0397] Raft process:
[0398] (1) Nodes A, B, and C elect a leader.
[0399] (2) Node A becomes the leader.
[0400] (3) The client sends a request to node A.
[0401] (4) Node A adds the request as a log entry to the log and replicates it to nodes B and C.
[0402] (5) Nodes B and C receive and store the log entry.
[0403] (6) Node A confirms that the log entry has been stored by the majority of nodes, commits the log entry and applies it to the state machine.
[0404] (7) Nodes B and C receive the commit message and apply the log entry to the state machine.
[0405] Through the above consensus algorithm, the blockchain system can achieve data consistency and security in a distributed environment and ensure that all nodes reach a consensus on the blockchain status.
[0406] Use smart contracts to verify the collected data to ensure that the data is authentic;
[0407] Data verification is a key step to ensure data accuracy, completeness and consistency. The following is the specific process of data verification based on blockchain technology:
[0408] 1. Data entry and initial verification:
[0409] Data Entry: Participants enter data through the client application.
[0410] Initial validation: The client application performs basic format and range checks (e.g., date format, numeric range, etc.).
[0411] 2. Data transmission and relay server verification:
[0412] Data transmission: The client sends data to two randomly selected relay servers.
[0413] Relay server verification: The relay server checks the integrity and consistency of the data and forwards the data to the blockchain network.
[0414] 3. Blockchain network verification:
[0415] Receiving data: The blockchain network receives data from the relay server.
[0416] Hash chain verification: The blockchain network calculates a new hash value based on the secret hash key and the previous hash value saved by the client device, and compares it with the hash value registered to the blockchain network to verify the integrity of the data.
[0417] Double verification: The blockchain network confirms that the data records sent by two relay servers are consistent to detect potential fraud.
[0418] 4. Consensus mechanism verification:
[0419] Consensus mechanism: The blockchain network uses a consensus mechanism (such as PoW, PoS, PBFT, etc.) to verify the legitimacy of data. Each node independently verifies the data and reaches a consensus through the consensus algorithm.
[0420] Block creation: Once consensus is reached, the data is added to a new block and linked to the blockchain.
[0421] 5. Data storage and disclosure:
[0422] Data storage: Verified data is permanently stored in the blockchain to ensure that the data cannot be tampered with.
[0423] Data disclosure: Evaluation results and verification proofs are disclosed through the blockchain event mechanism and are subject to supervision by stakeholders.
[0424] When data validation fails, the following actions need to be taken to process the data:
[0425] 1. Records and Logs:
[0426] Error log: Record detailed information about verification failures in the error log, including failure reasons, timestamps, related data, etc.
[0427] Notification: Send notifications to relevant personnel (such as data administrators and participants) to inform them of the verification failure and reasons.
[0428] 2. Data retransmission and retry:
[0429] Data retransmission: Allows the client to resend data. A retry mechanism can be set up to automatically try to resend data, especially in the event of network failure or temporary errors.
[0430] Exponential backoff: Use an exponential backoff algorithm in the retry mechanism to avoid system overload caused by frequent retries.
[0431] 3. Data correction and re-verification:
[0432] Data correction: Participants or data managers correct the data and resubmit it based on the information in the error log.
[0433] Re-verification: The corrected data re-enters the verification process for re-verification.
[0434] 4. Isolation and marking:
[0435] Data isolation: Data that fails verification is stored in isolation to prevent it from affecting the processing of other data.
[0436] Data marking: Mark the data that failed verification to facilitate subsequent analysis and processing.
[0437] 5. Audit and Analysis:
[0438] Audit: Regularly audit data that has failed verification, analyze the reasons for failure, and identify potential system vulnerabilities or data quality issues.
[0439] Improvement measures: Based on the audit results, optimize the data verification process and system configuration to reduce the occurrence of verification failures.
[0440] Through the above data verification and processing mechanism, the accuracy and completeness of the data can be ensured and the reliability and security of the system can be improved.
[0441] The key information such as data asset metadata, ownership information, transaction records, etc. is stored on the chain through data structures such as Merkle trees to achieve trusted management throughout the life cycle;
[0442] Utilize the tamper-proof and traceable characteristics of blockchain to provide a verifiable data basis for the evaluation process;
[0443] The evaluation results and certificates will be made public through the blockchain event mechanism, subject to supervision and improving credibility.
[0444] The blockchain network is a key infrastructure to ensure the credibility of the data asset evaluation process and the authority of the evaluation results. By putting the metadata, ownership, transaction and other information of data assets on the chain, and taking advantage of the advantages of blockchain technology, the trusted management and traceability of the entire life cycle of data assets is achieved. At the same time, the blockchain evidence of the evaluation process and results is subject to public supervision by stakeholders, which enhances the authority and credibility of the evaluation work.
[0445] Evaluation indicator system library, used to store multi-dimensional evaluation indicators and scoring rules;
[0446] From the perspectives of the intrinsic value and external application of data assets, we build a multi-dimensional evaluation indicator system covering data quality, data value, data risk, data security, etc.
[0447] A hierarchical structure is adopted, with the first layer being the comprehensive dimensions, the second layer being the specific indicators under each dimension, and the third layer being the supporting indicators, forming a complete qualitative evaluation framework;
[0448] Construct an indicator system based on the principles of comprehensiveness, systematicness, operability, and timeliness to ensure the scientificity and rationality of the evaluation;
[0449] The scoring rules for each indicator are stored to guide the specific scoring process of the evaluation and ensure the reliability of the evaluation results.
[0450] The evaluation index system is the core basis for conducting data asset value assessment. The multi-dimensional evaluation index system constructed by the present invention comprehensively considers the intrinsic value and external application of data assets, covers key factors such as data quality, value, risk, and security, and forms a scientific and systematic evaluation framework. By quantifying qualitative indicators and making implicit values explicit, it provides operational norms and standards for the comprehensive evaluation of data assets.
[0451] Machine learning models are used to dynamically optimize the weights of evaluation indicators, such as Figure 5 ;
[0452] Based on massive historical data asset evaluation cases, we use machine learning algorithms such as logistic regression, decision tree, and support vector machine to train and form an evaluation index weight optimization model;
[0453] By dynamically adjusting the indicator weights, the subjective defects of traditional methods can be overcome, and the objectivity and applicability of the evaluation can be improved;
[0454] Adaptively adjust indicator weights according to the characteristics of data assets in different industries and scenarios to form a dynamically optimized weight model;
[0455] Continuously learn from new evaluation data to achieve iterative updates of evaluation knowledge and self-optimization of algorithms.
[0456] Machine learning models are a key technical means to achieve intelligent data asset evaluation. By studying massive evaluation cases, machine learning models can automatically discover evaluation rules from data and learn to form the most optimized indicator weight combination. Compared with the traditional static indicator weight setting, machine learning models can dynamically adjust indicator weights according to the characteristics of different data assets, improving the objectivity and applicability of evaluation results. At the same time, the model also has the ability to continuously learn, and can continuously iterate and optimize from practice to improve the evaluation level.
[0457] Evaluation engine, which is used to integrate the evaluation indicator system and machine learning model to generate data asset evaluation reports;
[0458] Obtain target data asset metadata from the blockchain and retrieve the evaluation indicator system and scoring rules;
[0459] Call machine learning models to dynamically optimize indicator weights;
[0460] Score data assets item by item from multiple dimensions such as data quality, value, risk, and security;
[0461] Summarize the scores of each item to form a comprehensive evaluation result and generate a complete evaluation report, including the total score, grade, scores of each dimension, key advantages, improvement suggestions, etc.
[0462] The assessment report will be made public through blockchain and subject to supervision to ensure the credibility of the assessment process and results.
[0463] The evaluation engine is the executive body of data asset evaluation. It uses the evaluation index library and machine learning models to conduct scientific and rigorous value evaluation of target data assets. By scoring data assets in multiple dimensions, an objective and fair comprehensive evaluation result is formed. The evaluation report not only presents the value level of data assets, but also puts forward improvement suggestions on its advantages and disadvantages, which has a strong guiding significance. At the same time, the publicity of the evaluation report on the blockchain has also been subject to supervision from all walks of life, further enhancing the authority of the evaluation work.
[0464] Pricing engine, used for market-based pricing of data assets, such as Figure 6 ;
[0465] Based on the evaluation results, market-based pricing is formed by comprehensively considering factors such as data asset quality level, market supply and demand, and expected returns;
[0466] A dynamic bidding mechanism is adopted, whereby the buyer submits an application and quotation based on demand and judgment, and the transaction price is determined based on the principle of the highest bidder wins;
[0467] A joint pricing mechanism is adopted. When multiple buyers are interested in the same data asset, they jointly negotiate to form a unified price to avoid vicious competition.
[0468] The pricing process is implemented through blockchain smart contracts, and the bidding records are stored on the chain, using the characteristics of non-tamperability and traceability to ensure fairness and transparency;
[0469] The market-based pricing formed provides a price benchmark for the circulation and trading of data elements, and promotes the release of the value of data assets.
[0470] The pricing engine is a market-based application of the results of data asset evaluation. Based on the evaluation results, it fully considers factors such as market supply and demand to form a market-based pricing for data assets. Through innovative mechanisms such as dynamic bidding and joint pricing, and using blockchain technology to ensure a fair and transparent pricing process, a reasonable data asset transaction price is finally formed. This provides a price benchmark for the market-based circulation of data elements, is conducive to the full release of the value of data assets, and is of great significance for promoting the prosperity of the data element market.
[0471] Visual interactive interface, used to present evaluation and pricing results and provide interactive queries, such as Figure 7 .
[0472] Display the evaluation results such as comprehensive score, grade, and sub-score of each dimension of data assets in an intuitive and easy-to-understand way;
[0473] Clearly display market-based pricing information, including reference price, transaction price, pricing basis, etc., to improve pricing transparency and credibility;
[0474] List the key information hash values and links stored in the blockchain during the assessment process for users to verify the assessment proof;
[0475] Provide interactive query functions such as keyword search, filtering, and sorting to help users quickly find the required assessment reports and pricing information;
[0476] It uses rich visual charts to present the evaluation results, provides common functions such as result downloading and sharing, and embeds interactive tools such as online customer service.
[0477] The visual interactive interface is an important part of the data asset evaluation system and a key window for the system to exchange information, present results, and interact with users. By presenting the evaluation and pricing results in a graphical and visual way and providing a rich interactive query function, the user's usage threshold is greatly reduced and the system's usability is improved. At the same time, the verification of blockchain proof information is integrated into the interface, further enhancing the credibility of the evaluation results. A good user experience allows more users to easily understand and apply the value of data assets, promoting the development of the data factor market.
[0478] like Figure 2 , the workflow of the present invention is as follows:
[0479] 1. Collect and process target assessment data assets through the data asset management module, extract key metadata and upload them to the blockchain;
[0480] 2. The evaluation engine obtains data asset metadata from the blockchain and retrieves the corresponding evaluation rules from the evaluation indicator system library;
[0481] 3. The evaluation engine uses the machine learning model to dynamically optimize the weights of evaluation indicators;
[0482] 4. The evaluation engine scores each data asset item, summarizes the comprehensive evaluation results, and generates an evaluation report;
[0483] 5. Based on the evaluation results, the pricing engine introduces a market-based pricing mechanism to form the data asset price;
[0484] 6. Publicize data asset evaluation reports, pricing information and evaluation certificates through blockchain to complete credible evaluation;
[0485] 7. The visual interactive interface presents evaluation results and pricing information to users and provides interactive query services.
[0486] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.
[0487] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. A multi-dimensional evaluation method for data assets based on blockchain and machine learning, characterized in that: include: Step 1: Build a multi-dimensional data asset evaluation indicator system, with evaluation indicators including data quality, data value, data risk and data security dimensions; Step 2: The metadata, ownership information, and transaction record information of data assets are stored on the blockchain through blockchain technology, and the distributed ledger, consensus mechanism, and cryptographic security features of blockchain are used to achieve trusted digital management of data assets throughout their life cycle; smart contracts are used to regulate and supervise the collection, cleaning, and storage of data assets, and the evaluation results and evaluation certificates of data assets are publicized through blockchain; Step 3: Based on historical data asset evaluation cases, an evaluation indicator weight optimization model is formed through machine learning algorithm training. The machine learning algorithm includes logistic regression, decision tree and support vector machine. The evaluation indicator weight optimization model adaptively adjusts the importance weight of each evaluation indicator according to the characteristics of data assets in different industries and scenarios. Learn from newly added evaluation data to achieve iterative updates of evaluation knowledge and self-optimization of algorithms; Step 4: Comprehensively consider the quality level of data assets, market supply and demand, and expected returns, and form market-based pricing for data assets through dynamic bidding and joint pricing. The step 1 comprises: Conduct data quality dimension assessment, including: Step 1.1: Accuracy assessment: Check whether there are anomalies or errors in the data records by comparing with authoritative data sources, including field value errors, missing values, and null values; use data quality verification tools to verify the accuracy of the data and identify outliers; perform manual sampling inspections, and have business experts determine whether the data records are accurate; Step 1.2: Completeness assessment; evaluate the fill rate and missing rate of the field through the record value and unique value in the data statistics; compare the data between different data sources to check whether there is data missing; use data exploration tools to analyze the completeness level of data tables and fields; Step 1.3: Consistency assessment; check whether the data conforms to the unified format specifications, including field data type, format, and value range; check whether the logical relationship between the data is consistent, including the size relationship between the calculation indicators; compare the newly collected data with the historical data to identify whether there is a large preset deviation; use consistency verification rules to detect conflicts between data; Step 1.4: Timeliness assessment; monitor the delays in data collection, processing, and application, and assess data timeliness; set data timeliness thresholds based on data usage scenarios, and check whether the data meets the requirements; for data with high timeliness requirements, establish an automated data delay monitoring and early warning mechanism; Conduct data value dimension assessment, including: Step 2.1: Scarcity assessment: Analyze the market supply of similar data to determine the scarcity of data; assign a higher scarcity weight to data that is highly unique and irreplaceable; Step 2.2: Relevance assessment: Analyze the relevance of data with other data. Data with high relevance has higher value. The greater the breadth and depth of data relevance, the greater the value space that can be mined. Step 2.3: Uniqueness assessment: assess whether the data is unique and exclusive. Data with high uniqueness has higher value. Analyze whether the data is difficult to copy. Data with high copying cost has higher value. Step 2.4: Timeliness assessment: Based on the frequency of data updates, assess the timeliness value of data to the business; highly assess the timeliness of data in combination with specific scenarios; Conduct data risk dimension assessment, including: Step 3.1: Privacy risk assessment; Identify whether the data contains personal privacy information; Evaluate whether the data processing activities comply with the requirements of relevant laws and regulations on privacy protection; The larger the amount of private data and the higher the sensitivity, the greater the privacy risk; Step 3.2: Compliance risk assessment; assess whether data processing activities comply with industry regulatory requirements; identify sensitive information in data assets and assess whether desensitization is in place; assess whether security measures for cross-border data transmission comply with legal requirements; Conduct data security dimension assessment, including: Step 4.1: Confidentiality assessment: Determine the confidentiality level of the data based on the sensitivity of the data; assess whether the access control and permission management of each link in the data life cycle meet the confidentiality requirements; Step 4.2: Integrity assessment: Use consistency check and hash check technology to verify whether the data has been tampered with during storage and transmission; evaluate whether the data backup mechanism is complete and whether it can ensure data recoverability; Step 4.3: Availability assessment: conduct availability assessment on data storage, computing, and transmission infrastructure, paying attention to whether there is a single point of failure; assess the impact of data quality issues on data availability; test the feasibility of off-site disaster recovery and emergency plans to ensure the continuous availability of data; Conduct multi-dimensional comprehensive scoring, including: quantitatively scoring each evaluation indicator to form a unified scoring system; using the analytic hierarchy process to determine the weight of each indicator and using the weighted average method to calculate the comprehensive score; setting score thresholds, grading data assets, and forming a five-star rating model; using the rating results as a reference for data quality improvement and data asset pricing; The step 3 comprises: The goal of the evaluation index weight optimization model is to dynamically adjust the importance weight of each evaluation index through a machine learning algorithm. Suppose there are n evaluation indexes, the weight vector is \mathbf{w}=[w_1,w_2,\ldots,w_n], and the score vector of the evaluation index is \mathbf{x}=[x_1,x_2,\ldots,x_n], then the comprehensive score S can be expressed as: S=\sum_{i=1}^{n}w_i x_i; where: w_i represents the weight of the i-th evaluation index, and x_i represents the score of the i-th evaluation index; Optimize weights \mathbf{w} using logistic regression, decision trees, and support vector machines; Logistic regression maps the output of the linear regression model to [0, 1] through a logistic function, indicating the probability of belonging to a certain category; Model representation: h_\theta(x)=\frac{1}{1+e^{-\theta^T x}}, where: h_\theta(x) is the predicted value; \theta is the model parameter vector; x is the input feature vector; Loss function: J(\theta) = -\frac{1}{m}\sum_{i=1}^{m}\left[y^{(i)}\log(h_\theta(x^{(i)}))+(1y^{(i)})\log(1h_\theta(x^{(i)}))\right], where: m is the number of samples; y^{(i)} is the true label of the i-th sample; x^{(i)} is the feature vector of the i-th sample; Gradient descent: \theta: = \theta\alpha\frac{\partial J(\theta)}{\partial\theta}, where: \alpha is the learning rate; Decision trees achieve classification or regression by splitting features; Information gain: IG(D, A) = H(D)\sum_{v\in\text{Values}(A)}\frac{|D_v|}{|D|}H(D_v), where: IG(D, A) is the information gain of feature A on data set D; H(D) is the entropy of data set D; D_v is the subset of feature A whose value is v; Entropy: H(D) = -\sum_{i=1}^{k}p_i\log_2(p_i), where: k is the number of categories; p_i is the probability of category i; Pruning: Pre-pruning: In the process of generating a decision tree, the splitting is stopped by setting a threshold; Post-pruning: After the complete decision tree is generated, the redundant branches are pruned by calculating the error rate; Support vector machines achieve classification by finding a hyperplane that maximizes the margin; Optimization goal: \min_{w,b}\frac{1}{2}\|w\|^2, where: w is the normal vector; b is the bias; Constraints: y_i(w\cdot x_i+b)\geq 1, where: y_i is the label of the i-th sample; x_i is the feature vector of the i-th sample; Dual problem: \max_{\alpha}\sum_{i=1}^{m}\alpha_i\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_j y_i y_j(x_i\cdot x_j), where: \alpha_i is the Lagrange multiplier; Kernel function: K(x_i, x_j) = \phi(x_i)^T\phi(x_j); The specific process of adjusting the importance weights of each evaluation indicator is as follows: define the objective function, including mean square error and cross entropy loss; initialize the weight vector \mathbf{w} to a random value or a uniformly distributed value; calculate the gradient of the objective function with respect to the weight \mathbf{w} \nabla J(\mathbf{w}); update the weight using the gradient descent method: \mathbf{w}: = \mathbf{w}-\alpha\nabla J(\mathbf{w}), where \alpha is the learning rate; iterate the optimization until the objective function converges or the preset number of iterations is reached; The step 4 comprises: Dynamic bidding pricing: Data demanders publish data demand information on the data trading platform based on their own business needs, including data types, quantities, quality requirements, and expected uses; data suppliers review and evaluate demand information based on their own data assets to determine whether there are matching data assets; if there are matching data assets, the data supplier sets a reserve price, i.e., the lowest selling price, based on the data quality level and expected returns; multiple data suppliers quote for the same demand information, forming a dynamic bidding process, and the one with the highest bid has a greater chance of reaching a deal; a time limit for bidding is set on the data trading platform; during the bidding process, data suppliers dynamically adjust their bids based on the bids of other competitors; after the bidding ends, the data demander selects the best bid based on data quality and price factors and reaches a deal with the data supplier; the data trading platform records and supervises the transaction process; Joint pricing: Multiple data demanders are interested in the same data asset and form a joint purchasing group; the joint purchasing group elects representatives to negotiate with the data supplier, taking data quality, expected returns, and purchase quantity into consideration during the negotiation; the data trading platform provides technical support for joint purchasing, including the release of demand information, group formation, and recording of the bargaining process; a unified selling price is reached through bargaining; the data trading platform records and supervises the bargaining process; members of the joint purchasing group pay the purchase price and share the right to use the data asset in accordance with the pre-agreed sharing method; the data trading platform manages the payment and allocation of funds.
2. The multi-dimensional data asset evaluation method based on blockchain and machine learning according to claim 1 is characterized in that: The trusted digital management of data assets throughout their life cycle in step 2 includes: Use distributed ledgers to achieve full life cycle management of data assets, specifically: Data asset collection stage: record the metadata information of data assets in the distributed ledger of the blockchain in the form of transactions; Data asset storage stage: store the data assets themselves in the distributed storage system under the chain, and record the storage location hash value of the data assets as a transaction in the blockchain ledger to establish a mapping relationship between data assets and storage locations; Data asset use stage: write the access control policy of the data assets into the smart contract and deploy it in the blockchain network; data users obtain access authorization to data assets by calling smart contracts when the access control conditions are met; record the use of data assets in the form of transactions in the blockchain ledger to achieve full traceability of data use; Data asset transaction stage: record the transaction information of data assets in the blockchain ledger; use the openness and transparency of the blockchain to achieve fairness and justice in the data transaction process; use digital signature technology to ensure the authenticity of the identities of both parties to the transaction; Data asset destruction stage: record the data destruction record in the form of transactions in the blockchain ledger, including the destruction time, execution subject, and destruction proof; Use consensus mechanisms to ensure the consistency and credibility of data asset management. Specifically, blockchain nodes reach consensus on transactions related to data assets through consensus mechanisms to ensure the consistency of ledger data at each node. The consensus mechanism encourages nodes to keep honest accounts and punishes malicious nodes. Different consensus mechanisms are suitable for different application scenarios. Select the corresponding consensus mechanism based on the needs of data asset management. Cryptography technology is applied to ensure the security of data asset management, specifically: using hash algorithms to generate unique digital fingerprints for data assets and their metadata; using asymmetric encryption algorithms to encrypt data assets; using digital signature technology to sign transactions and verify the identity of the transaction initiator; using zero-knowledge proof and homomorphic encryption technology to achieve calculation and analysis of data assets without leaking the original data.
3. A multi-dimensional data asset evaluation system based on blockchain and machine learning, characterized in that: include: Module M1: Construct a multi-dimensional data asset evaluation indicator system, with evaluation indicators including data quality, data value, data risk and data security dimensions; Module M2: The metadata, ownership information, and transaction record information of data assets are stored on the blockchain through blockchain technology, and the distributed ledger, consensus mechanism, and cryptographic security features of blockchain are used to achieve trusted digital management of data assets throughout their life cycle; smart contracts are used to regulate and supervise the collection, cleaning, and storage of data assets, and the evaluation results and evaluation certificates of data assets are publicized through blockchain; Module M3: Based on historical data asset evaluation cases, an evaluation indicator weight optimization model is formed through machine learning algorithm training. The machine learning algorithm includes logistic regression, decision tree and support vector machine. The evaluation indicator weight optimization model adaptively adjusts the importance weight of each evaluation indicator according to the characteristics of data assets in different industries and scenarios. Learn from newly added evaluation data to achieve iterative updates of evaluation knowledge and self-optimization of algorithms; Module M4: Based on the quality level, market supply and demand relationship and expected returns of data assets, market-based pricing of data assets is formed through dynamic bidding and joint pricing. The module M1 comprises: Conduct data quality dimension assessment, including: Module M1.1: Accuracy assessment: Check whether there are anomalies or errors in the data records by comparing with authoritative data sources, including field value errors, missing values, and null values; use data quality verification tools to verify the accuracy of the data and identify outliers; perform manual sampling inspections, and have business experts determine whether the data records are accurate; Module M1.2: Completeness assessment; evaluate the fill rate and missing rate of fields through record values and unique values in data statistics; compare data between different data sources to check whether there is data missing; use data exploration tools to analyze the integrity level of data tables and fields; Module M1.3: Consistency assessment; check whether the data conforms to the unified format specifications, including field data type, format, and value range; check whether the logical relationship between the data is consistent, including the size relationship between the calculation indicators; compare the newly collected data with the historical data to identify whether there is a large preset deviation; use consistency verification rules to detect conflicts between data; Module M1.4: Timeliness assessment; monitor the delays in data collection, processing, and application, and assess data timeliness; set data timeliness thresholds based on data usage scenarios, and check whether the data meets the requirements; for data with high timeliness requirements, establish an automated data delay monitoring and early warning mechanism; Conduct data value dimension assessment, including: Module M2.1: Scarcity assessment; analyze the market supply of similar data and determine the scarcity of data; assign a higher scarcity weight to data that is highly unique and irreplaceable; Module M2.2: Relevance assessment; analyze the relevance of data with other data. Data with high relevance has higher value. The greater the breadth and depth of data relevance, the greater the value space that can be mined. Module M2.3: Uniqueness assessment: assess whether the data is unique and exclusive. Data with high uniqueness has higher value. Analyze whether the data is difficult to copy. Data with high copying cost has higher value. Module M2.4: Timeliness assessment: Evaluate the timeliness value of data to the business based on the frequency of data updates; highly evaluate the timeliness of data in combination with specific scenarios; Conduct data risk dimension assessment, including: Module M3.1: Privacy risk assessment; Identify whether the data contains personal privacy information; Evaluate whether the data processing activities comply with the requirements of relevant laws and regulations on privacy protection; The larger the amount of privacy data and the higher the sensitivity, the greater the privacy risk; Module M3.2: Compliance risk assessment; assess whether data processing activities comply with industry regulatory requirements; identify sensitive information in data assets and assess whether desensitization is in place; assess whether security measures for cross-border data transmission comply with legal requirements; Conduct data security dimension assessment, including: Module M4.1: Confidentiality assessment: Determine the confidentiality level of data based on the sensitivity of the data; assess whether access control and permission management at all stages of the data life cycle meet confidentiality requirements; Module M4.2: Integrity assessment: Verify whether the data has been tampered with during storage and transmission through consistency check and hash check technology; evaluate whether the data backup mechanism is complete and whether it can ensure the recoverability of the data; Module M4.3: Availability assessment; conduct availability assessment on data storage, computing, and transmission infrastructure, paying attention to whether there is a single point of failure; assess the impact of data quality issues on data availability; test the feasibility of off-site disaster recovery and emergency plans to ensure the continuous availability of data; Conduct multi-dimensional comprehensive scoring, including: quantitatively scoring each evaluation indicator to form a unified scoring system; using the analytic hierarchy process to determine the weight of each indicator and using the weighted average method to calculate the comprehensive score; setting score thresholds, grading data assets, and forming a five-star rating model; using the rating results as a reference for data quality improvement and data asset pricing; The module M3 comprises: The goal of the evaluation index weight optimization model is to dynamically adjust the importance weight of each evaluation index through a machine learning algorithm. Suppose there are n evaluation indexes, the weight vector is \mathbf{w}=[w_1,w_2,\ldots,w_n], and the score vector of the evaluation index is \mathbf{x}=[x_1,x_2,\ldots,x_n], then the comprehensive score S can be expressed as: S=\sum_{i=1}^{n}w_i x_i; where: w_i represents the weight of the i-th evaluation index, and x_i represents the score of the i-th evaluation index; Optimize weights \mathbf{w} using logistic regression, decision trees, and support vector machines; Logistic regression maps the output of the linear regression model to [0, 1] through a logistic function, indicating the probability of belonging to a certain category; Model representation: h_\theta(x)=\frac{1}{1+e^{-\theta^T x}}, where: h_\theta(x) is the predicted value; \theta is the model parameter vector; x is the input feature vector; Loss function: J(\theta) = -\frac{1}{m}\sum_{i=1}^{m}\left[y^{(i)}\log(h_\theta(x^{(i)}))+(1y^{(i)})\log(1h_\theta(x^{(i)}))\right], where: m is the number of samples; y^{(i)} is the true label of the i-th sample; x^{(i)} is the feature vector of the i-th sample; Gradient descent: \theta: = \theta\alpha\frac{\partial J(\theta)}{\partial\theta}, where: \alpha is the learning rate; Decision trees achieve classification or regression by splitting features; Information gain: IG(D, A) = H(D)\sum_{v\in\text{Values}(A)}\frac{|D_v|}{|D|}H(D_v), where: IG(D, A) is the information gain of feature A on data set D; H(D) is the entropy of data set D; D_v is the subset of feature A whose value is v; Entropy: H(D) = -\sum_{i=1}^{k}p_i\log_2(p_i), where: k is the number of categories; p_i is the probability of category i; Pruning: Pre-pruning: In the process of generating a decision tree, the splitting is stopped by setting a threshold; Post-pruning: After the complete decision tree is generated, the redundant branches are pruned by calculating the error rate; Support vector machines achieve classification by finding a hyperplane that maximizes the margin; Optimization goal: \min_{w,b}\frac{1}{2}\|w\|^2, where: w is the normal vector; b is the bias; Constraints: y_i(w\cdot x_i+b)\geq 1, where: y_i is the label of the i-th sample; x_i is the feature vector of the i-th sample; Dual problem: \max_{\alpha}\sum_{i=1}^{m}\alpha_i\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_j y_i y_j(x_i\cdot x_j), where: \alpha_i is the Lagrange multiplier; Kernel function: K(x_i, x_j) = \phi(x_i)^T\phi(x_j); The specific process of adjusting the importance weights of each evaluation indicator is as follows: define the objective function, including mean square error and cross entropy loss; initialize the weight vector \mathbf{w} to a random value or a uniformly distributed value; calculate the gradient of the objective function with respect to the weight \mathbf{w} \nabla J(\mathbf{w}); update the weight using the gradient descent method: \mathbf{w}: = \mathbf{w}-\alpha\nabla J(\mathbf{w}), where \alpha is the learning rate; iterate the optimization until the objective function converges or the preset number of iterations is reached; The module M4 comprises: Dynamic bidding pricing: Data demanders publish data demand information on the data trading platform based on their own business needs, including data types, quantities, quality requirements, and expected uses; data suppliers review and evaluate demand information based on their own data assets to determine whether there are matching data assets; if there are matching data assets, the data supplier sets a reserve price, i.e., the lowest selling price, based on the data quality level and expected returns; multiple data suppliers quote for the same demand information, forming a dynamic bidding process, and the one with the highest bid has a greater chance of reaching a deal; a time limit for bidding is set on the data trading platform; during the bidding process, data suppliers dynamically adjust their bids based on the bids of other competitors; after the bidding ends, the data demander selects the best bid based on data quality and price factors and reaches a deal with the data supplier; the data trading platform records and supervises the transaction process; Joint pricing: Multiple data demanders are interested in the same data asset and form a joint purchasing group; the joint purchasing group elects representatives to negotiate with the data supplier, taking data quality, expected returns, and purchase quantity into consideration during the negotiation; the data trading platform provides technical support for joint purchasing, including the release of demand information, group formation, and recording of the bargaining process; a unified selling price is reached through bargaining; the data trading platform records and supervises the bargaining process; members of the joint purchasing group pay the purchase price and share the right to use the data asset in accordance with the pre-agreed sharing method; the data trading platform manages the payment and allocation of funds.
4. The multi-dimensional data asset evaluation system based on blockchain and machine learning according to claim 3 is characterized in that: The trusted digital management of data assets throughout their life cycle in module M2 includes: Use distributed ledgers to achieve full life cycle management of data assets, specifically: Data asset collection stage: record the metadata information of data assets in the distributed ledger of the blockchain in the form of transactions; Data asset storage stage: store the data assets themselves in the distributed storage system under the chain, and record the storage location hash value of the data assets as a transaction in the blockchain ledger to establish a mapping relationship between data assets and storage locations; Data asset use stage: write the access control policy of the data assets into the smart contract and deploy it in the blockchain network; data users obtain access authorization to data assets by calling smart contracts when the access control conditions are met; record the use of data assets in the form of transactions in the blockchain ledger to achieve full traceability of data use; Data asset transaction stage: record the transaction information of data assets in the blockchain ledger; use the openness and transparency of the blockchain to achieve fairness and justice in the data transaction process; use digital signature technology to ensure the authenticity of the identities of both parties to the transaction; Data asset destruction stage: record the data destruction record in the form of transactions in the blockchain ledger, including the destruction time, execution subject, and destruction proof; Use consensus mechanisms to ensure the consistency and credibility of data asset management. Specifically, blockchain nodes reach consensus on transactions related to data assets through consensus mechanisms to ensure the consistency of ledger data at each node. The consensus mechanism encourages nodes to keep honest accounts and punishes malicious nodes. Different consensus mechanisms are suitable for different application scenarios. Select the corresponding consensus mechanism based on the needs of data asset management. Cryptography technology is applied to ensure the security of data asset management, specifically: using hash algorithms to generate unique digital fingerprints for data assets and their metadata; using asymmetric encryption algorithms to encrypt data assets; using digital signature technology to sign transactions and verify the identity of the transaction initiator; using zero-knowledge proof and homomorphic encryption technology to achieve calculation and analysis of data assets without leaking the original data.
Citation Information
Patent Citations
Method and system for automatically scrapping equipment based on smart contract and score card
CN117575529A
Cited By
Industrial report automatic generation system based on large language model
CN119961330A
Meteorological data multi-model fusion value evaluation system and method based on block chain
CN120047203A
Multi-dimensional data analysis system and method for data element market
CN120216568A
Large language model high-quality text data set construction method and system
CN120336527A
Container cluster-oriented big data job intelligent submission method and system
CN120371487A