Data asset assessment method and system based on big data

By building a graph network and using blockchain to confirm rights, combined with causal inference and regression analysis, the problems of dynamic changes and insufficient relationship identification in data asset evaluation are solved, and a more accurate and comprehensive data asset value evaluation is achieved.

CN119006165AActive Publication Date: 2024-11-22CHINA NAT INST OF STANDARDIZATION

Patent Information

Application Number
CN202410936211.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-22
Estimated Expiration
2044-07-12

AI Technical Summary

Technical Problem

Existing data asset appraisal methods are difficult to accurately capture the dynamic changes and important relationships between data assets, resulting in insufficient comprehensiveness and accuracy of value appraisal.

Method used

By collecting multi-source data and building a graph network, using blockchain rights confirmation and Neo4j graph database to capture the time changes in data relationships, extract node characteristics, and construct an asset evaluation model based on causal inference and regression analysis to monitor and update the value of data assets in real time. and risk attributes.

Benefits of technology

It significantly improves the depth and breadth of data asset appraisal, provides a more dynamic and comprehensive evaluation perspective, and improves the credibility and practicality of evaluation results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006165A_ABST
    Figure CN119006165A_ABST
Patent Text Reader

Abstract

The invention discloses a data asset assessment method and system based on big data, and relates to the technical field of data processing, and the method comprises the steps: collecting multi-source data, storing the multi-source data to a central database, and uploading the multi-source data to a block chain for data right confirmation and recording; constructing a graph network of data assets based on the central database, capturing time change of a data relationship in the graph network, and extracting node features from the graph network; and identifying factors influencing the data asset value based on the extracted node features, and constructing an asset evaluation model to evaluate the data asset value. According to the method, the graph network of the data assets is constructed by collecting the multi-source data, the node features of the graph network are extracted to identify the factors influencing the value of the data assets, the asset evaluation model is constructed for asset evaluation, the dynamic relationship between the data assets can be accurately analyzed, key nodes and important relationships can be identified, and the data assets are evaluated. The depth and the breadth of data analysis are improved, a more dynamic and comprehensive evaluation view angle is provided, and the credibility and the practicability of an evaluation result are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data asset evaluation method and system based on big data. Background Art

[0002] Against the backdrop of the rapid development of modern information technology, big data has become a core asset supporting corporate strategic decision-making and operational optimization. Data asset evaluation methods are an important research direction in the fields of information science and data management. The key lies in how to accurately and efficiently evaluate and utilize the value of these data assets. With the improvement of computing power and the advancement of data acquisition technology, the integration and analysis of multi-source data has become possible, but it also brings new challenges. Existing methods often lack effective capture of the dynamic changes in the relationship between data assets, are insufficient in identifying the key nodes and important relationships of data assets, and need to be further improved in the comprehensiveness and accuracy of data asset value assessment. Therefore, a more dynamic and in-depth data asset evaluation method is needed. Summary of the invention

[0003] The purpose of the present invention is to provide a data asset evaluation method and system based on big data to solve the above problems.

[0004] The present invention is achieved through the following technical solutions:

[0005] A data asset evaluation method and system based on big data, comprising:

[0006] Collect multi-source data, store it in a central database, and upload it to the blockchain for data ownership confirmation and recording;

[0007] Build a graph network of data assets based on a central database, capture the temporal changes of data relationships in the graph network, and extract node features from the graph network;

[0008] Identify factors that affect the value of data assets based on extracted node features, and build an asset evaluation model to evaluate the value of data assets;

[0009] Store data generated during the data asset assessment process and implement access control.

[0010] As a preferred solution of the data asset evaluation method based on big data described in the present invention, wherein: the collection of multi-source data to a central database and uploading it to the blockchain for data confirmation and recording refers to collecting data from various data sources and storing it in a central database after pre-processing, using the Solidity language to write smart contracts, deploying smart contracts on the Ethereum platform, and storing the data on the blockchain.

[0011] As a preferred solution of the data asset evaluation method based on big data of the present invention, wherein: the graph network of data assets constructed based on the central database refers to installing Neo4j graph database and Elasticsearch, constructing a graph data model, and using the transaction relationship between two nodes as an edge to connect related data asset nodes and user nodes;

[0012] Add multi-dimensional attributes of data assets to the basic model and set edge weights based on the transaction frequency and amount of data assets;

[0013] The centrality of data assets is defined using the data asset influence scoring formula based on the intensity and frequency of interactions between data assets:

[0014]

[0015] In the formula, S(v) represents the influence score of the asset node, w uv is the relationship weight between nodes u and v, t uv is the most recent interaction time, and f is the time decay function;

[0016] Calculate the influence score of each data asset node and use the influence score as the node attribute;

[0017] Establish a real-time data monitoring system to monitor the dynamic changes of data assets in real time and push updates to Neo4j, and use exponential decay functions to adjust the initial value attributes of data asset nodes in real time;

[0018] Use the batch import tool to import data assets and transaction records into Neo4j, and use the risk assessment formula to update the risk attributes of data asset nodes based on market fluctuations and historical data assets;

[0019] When changes are detected in the initial value and risk of data assets, the value attributes and risk attributes of the data asset nodes in the graph network are automatically updated.

[0020] As a preferred solution of the data asset evaluation method based on big data described in the present invention, wherein: the capturing of the time changes of data relationships and extracting node features from the graph network refers to adding time attributes to each edge and defining a time window, performing time series analysis on the data asset relationship, calculating the statistical indicators of each data asset node in different time windows, using Cypher queries in Neo4j to generate statistical indicators, and adding the statistical indicators as attributes to the corresponding data asset nodes;

[0021] Get all nodes and corresponding edges in the graph network, configure GraphSAGE model parameters, learn node embedding by aggregating neighbor information of each data asset node and perform iterative updates;

[0022] Update the model weights through back-propagation and define the loss function. When the loss of the GraphSAGE model no longer decreases significantly during the continuous iteration, stop iterating and output the model parameters to update the GraphSAGE model.

[0023] After training is completed, the features of nodes and edges are extracted from the GraphSAGE model.

[0024] As a preferred solution of the data asset evaluation method based on big data described in the present invention, wherein: the identification of factors affecting the value of data assets based on extracted node features refers to defining a list of hypothetical factors affecting the initial value of data assets based on business knowledge and node features extracted from a graph network, using the DoWhy library to draw a causal graph, determining the transaction frequency of data assets as the cause variable, and the initial value of data assets as the result variable;

[0025] Use regression analysis to estimate the impact of factors on the initial value of data assets for causal inference. The results of causal inference include the coefficient of each factor. Check the p-value of each factor and set the judgment threshold Q. Compare the p-value of each factor with the judgment threshold Q to determine the impact of the factor on the result variable. If the p-value of the factor is less than Q, the factor is statistically significant, indicating that the factor affects the result variable, otherwise it has no effect.

[0026] Build an XGBoost model to quantify the contribution of each factor to the initial value of the data asset, use the features extracted from the graph network to train the XGBoost model, use the cross-validation method to verify the performance of the XGBoost model, and stop training if the model performance does not improve, and obtain the trained XGBoost model;

[0027] The parameters of the XGBoost model are adjusted based on the causal inference results, and the factors screened from the causal inference are used as input variables to identify the factors that have the greatest impact on the initial value of the data asset.

[0028] As a preferred solution of the data asset evaluation method based on big data described in the present invention, wherein: the construction of an asset evaluation model to evaluate the value of data assets refers to constructing a linear regression model as an asset evaluation model, taking the initial value of the data asset as the dependent variable, the maximum influencing factor as the independent variable, using the least squares method to estimate the model parameters, and using historical influencing factors and historical data asset value data as training data to input into the asset evaluation model for iterative training;

[0029] Define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the asset valuation model no longer decreases significantly during the continuous iteration process, stop iterating and output the model parameters to update the asset valuation model.

[0030] Input the data to be evaluated into the asset evaluation model to obtain the value evaluation results of the data assets.

[0031] As a preferred solution of the data asset evaluation method based on big data described in the present invention, wherein: the storage of data generated in the data asset evaluation process and the implementation of access control refers to the synchronization of storing the results in the central database after the data asset value evaluation results are verified through smart contracts and recorded in the blockchain, collecting the data generated in the data asset evaluation process and storing them in the central database and implementing access control, allowing authorized users to query and access the data.

[0032] Another object of the present invention is to provide a data asset evaluation system based on big data, which comprises:

[0033] The data collection module is used to determine and configure the types of data sources that need to be integrated, collect data for pre-processing, and perform blockchain authentication and recording, and store the data in a central database;

[0034] The graph network construction module is used to build a graph database based on the central database and update node attributes in real time according to changes in node data and extract node features in the graph network;

[0035] The causal inference module is used to draw causal diagrams using the DoWhy library and perform causal inference using regression analysis methods, and to quantify the contribution of each factor using the XGBoost model;

[0036] The asset evaluation module is used to build an asset evaluation model based on the most influencing factors to evaluate the value of data assets and verify the evaluation results of data asset value through smart contracts, and store the data generated during the evaluation process in the central database.

[0037] A computer device includes: a memory and a processor; the memory stores a computer program, and the processor implements steps of data asset evaluation based on big data when executing the computer program.

[0038] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements steps for data asset evaluation based on big data.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] The present invention constructs a graph network of data assets by collecting multi-source data and extracts node features of the graph network to identify factors affecting the value of data assets, and constructs an asset valuation model for asset valuation. It can accurately analyze the dynamic relationships between data assets, identify key nodes and important relationships, improve the depth and breadth of data analysis, provide a more dynamic and comprehensive evaluation perspective, and significantly improve the credibility and practicality of the evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0042] Figure 1 It is a flowchart of a data asset evaluation method based on big data;

[0043] Figure 2 Schematic diagram of the implementation of graph network construction and update;

[0044] Figure 3 This is a structural diagram of the data asset evaluation method based on big data. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and drawings. The schematic implementation modes and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention. It should be noted that the present invention is already in the actual development and use stage.

[0046] Embodiment 1, as Figure 1 and Figure 2 As shown, it is the first embodiment of the present invention, which provides a data asset evaluation method based on big data, and the data asset evaluation method based on big data includes:

[0047] S1. Collect multi-source data, store it in a central database and upload it to the blockchain for data ownership confirmation and recording;

[0048] Specifically, collecting multi-source data to store in the central database and uploading it to the blockchain for data confirmation and recording means determining the type of data source that needs to be integrated, configuring the NiFi data flow group for each data source, setting the data flow error handling logic and setting the processor in NiFi, using SQL-like queries to remove invalid, blank and duplicate data records from the data collected from each data source, and standardizing the data before storing the data in the central database;

[0049] Calculate the hash value of the data, write a smart contract using the Solidity language, deploy the smart contract on the Ethereum platform, call the smart contract function by writing a client application, and store the hash value, current block timestamp, and data source address on the blockchain.

[0050] By collecting and integrating multi-source data, the comprehensiveness and diversity of data are ensured. This integration of multi-source data not only provides a richer data foundation, but also improves the depth and breadth of data analysis. Storing data in a central database can realize centralized management and efficient access to data, and support large-scale data processing and analysis. Uploading to the blockchain for data confirmation and recording ensures the immutability and high credibility of the data, and solves the problems of data tampering and loss in traditional data storage methods. By configuring the NiFi data flow group, automated data flow management from each data source to the central database can be achieved. This automated management not only improves the efficiency of data processing, but also reduces manual intervention and reduces the error rate in the data processing process. Setting the data flow error handling logic and processor ensures the stability and reliability of the data processing process, solves various abnormal situations that may occur in the data processing process, and by calculating the hash value of the data, a unique identifier can be generated for each data item to ensure the integrity and consistency of the data. Use the Solidity language to write smart contracts and deploy smart contracts on the Ethereum platform. Through smart contracts, the hash value, block timestamp, and data source address are stored on the blockchain to realize data confirmation and recording. This approach not only improves the security and reliability of data storage, but also realizes the automation and transparency of data processing.

[0051] S2. Build a graph network of data assets based on the central database, capture the temporal changes of data relationships in the graph network, and extract node features from the graph network;

[0052] Specifically, building a graph network of data assets based on a central database means installing the Neo4j graph database, starting a Neo4j instance and performing preliminary configuration, including setting authentication information, configuring network interfaces and ports, installing Elasticsearch, and setting cluster configuration and indexing strategies to integrate with Neo4j, setting the node name, cluster name, and data path of Elasticsearch, and synchronizing the data between the two through the Neo4j-to-Elasticsearch plug-in;

[0053] Build a graph data model, use Cypher language to create data asset nodes, user nodes, transaction relationships and their corresponding attributes in Neo4j, and use the transaction relationship between two nodes as an edge to connect related data assets and user nodes;

[0054] Add multi-dimensional attributes of data assets to the basic model, including static attributes (ID, type and initial value) and dynamic attributes (market value, risk level) of data assets, and set edge weights according to the transaction frequency and amount of data assets;

[0055] The centrality of data assets is defined using the data asset influence scoring formula based on the intensity and frequency of interactions between data assets:

[0056]

[0057] In the formula, S(v) represents the influence score of the asset node, w uv is the relationship weight between nodes u and v, t uv is the most recent interaction time, f is the time decay function, d uv is the interaction frequency between nodes u and v. The higher the frequency, the greater the weight. uv is the time difference of the most recent interaction, and λ is the decay coefficient, which indicates the effect of time on the interaction weight;

[0058] Calculate the influence score of each data asset node and use the influence score as the node attribute;

[0059] Establish a real-time data monitoring system to monitor the dynamic changes of data assets in real time and push updates to Neo4j, and use the exponential decay function to adjust the initial value attributes of data asset nodes in real time:

[0060] V t =V0·e -δt ,

[0061] Where V t is the value of the data asset at time t, V0 is the initial data asset value, and δ is the decay constant, which is obtained through historical data regression analysis and represents the decay rate of the data asset value over time;

[0062] When the initial value change of the data asset is detected, the value attribute of the data asset node in the graph network is automatically updated;

[0063] Use the batch import tool to import data assets and transaction records into Neo4j, and use the risk assessment formula to update the risk attributes of data asset nodes based on market fluctuations and historical data assets:

[0064] R(v)=α·σ(v)+β·μ(v),

[0065] Where R(v) is the risk score of data asset v, σ(v) is the standard deviation of the return of data asset v, indicating the volatility of the return, μ(v) is the expected return of the data asset, and α and β are parameters that adjust risk perception, which are determined through expert opinions or historical data regression analysis;

[0066] In the formula, r i is the return of the data asset in period i, ε(v) is the average return of the data asset, and N is the total number of periods;

[0067] When risk changes in data assets are monitored, the risk attributes of data asset nodes in the graph network are automatically updated.

[0068] By installing the Neo4j graph database and performing preliminary configuration, including setting authentication information, configuring network interfaces and ports, efficient storage and secure access to data assets are ensured. Elasticsearch is integrated and cluster configuration and indexing strategies are set to make data search and query more efficient. The Neo4j-to-Elasticsearch plug-in is used to synchronize the data of the two, which enhances the system's data management capabilities and query performance. By using the Cypher language to create data asset nodes, user nodes, and transaction relationships in Neo4j, and defining their respective properties, the complex relationships between data assets can be intuitively represented. The transaction relationship is used as an edge to connect related data assets and user nodes, and a complete graph data model is constructed, which can more accurately reflect the data interaction in actual business. Multi-dimensional attributes of data assets, including static attributes and dynamic attributes, are added to the basic model, making the description of data assets more comprehensive and fine-grained. The weight of the edge is set according to the transaction frequency and amount of the data asset, which enhances the expressiveness of the graph model, enabling it to reflect more complex interaction patterns and value assessments. The interaction frequency and time factors are comprehensively considered by using the data asset influence scoring formula. The influence of each data asset in the graph network can be quantified, which can more accurately reflect the dynamic influence of data assets. By establishing a real-time data monitoring system, the changes of data assets can be dynamically monitored, and the initial value of data asset nodes can be adjusted in real time using an exponential decay function to promptly reflect market changes and improve the real-time and accuracy of data asset value assessment. By using risk assessment formulas, the risk attributes of data assets can be dynamically evaluated and updated, which can accurately assess the risk level of data assets and help enterprises better manage and control risks in the decision-making process.

[0069] Furthermore, we capture the temporal changes of data relationships and extract node feature values ​​from the graph network. We add time attributes to each edge and define time windows. We perform time series analysis on data asset relationships and calculate statistical indicators for each data asset node in different time windows, including total transaction amount, average transaction amount, and maximum and minimum transaction amount. We use Cypher queries in Neo4j to generate statistical indicators and add them as attributes to the corresponding data asset nodes.

[0070] Get all nodes and corresponding edges in the graph network, configure GraphSAGE model parameters, learn node embedding by aggregating neighbor information of each data asset node and perform iterative updates;

[0071] Update the model weights through back-propagation and define the loss function. When the loss of the GraphSAGE model no longer decreases significantly during the continuous iteration, stop iterating and output the model parameters to update the GraphSAGE model.

[0072] After training is completed, the features of nodes and edges are extracted from the GraphSAGE model.

[0073] By adding a time attribute to each edge, the time point of each data interaction can be accurately recorded, thus laying the foundation for subsequent time series analysis. Defining a time window can divide continuous time data into several fixed-length intervals, which is convenient for data statistics and analysis within these intervals. Doing so can capture the changes in data asset relationships over time and improve the understanding of the dynamic nature of data relationships. By performing time series analysis on data asset relationships, the changing patterns of data relationships can be identified, and statistical indicators of each data asset node in different time windows can be calculated, such as total transaction amount, average transaction amount, and maximum and minimum transaction amount. These statistical indicators can reflect the activity and importance of data assets and provide valuable information for subsequent data analysis and decision-making. By using Cypher queries, data can be efficiently extracted from the graph database and statistical indicators can be calculated, and these indicators can be added as attributes to the corresponding data asset nodes. This method can enhance the representation ability of the graph network, so that it contains not only structural information but also rich attribute information, which is convenient for subsequent graph analysis tasks. By configuring the GraphSAGE model parameters, the neighbor information of each data asset node is aggregated to learn node embedding, which can capture the structural information and characteristics of the node and its neighbors. The GraphSAGE model iteratively updates node embeddings, enabling efficient representation learning in large-scale graph networks, improving the representation ability and analysis effect of graph networks. Through the back-propagation algorithm, the weights of the GraphSAGE model can be updated according to the gradient of the loss function to minimize the prediction error. Define a reasonable loss function to ensure that the loss of the model gradually decreases during continuous iterations until it reaches a convergence state. This can ensure the effectiveness and stability of model training and improve the quality of node embedding. By extracting node and edge features from the trained GraphSAGE model, high-quality graph representation can be obtained to capture the structural information and features of nodes and their neighbors.

[0074] S3. Identify factors that affect the value of data assets based on the extracted node features, and build an asset evaluation model to evaluate the value of data assets;

[0075] Specifically, identifying factors affecting the value of data assets based on extracted node features means defining a list of hypothetical factors affecting the initial value of data assets based on business knowledge and node features extracted from the graph network, including but not limited to transaction frequency, market volatility, risk score and other factors, using the DoWhy library to draw a causal graph, determining the transaction frequency of data assets as the cause variable and the initial value of data assets as the result variable;

[0076] Use regression analysis to estimate the impact of factors on the initial value of data assets for causal inference. The results of causal inference include the coefficient of each factor. Check the p-value of each factor and set the judgment threshold Q. Compare the p-value of each factor with the judgment threshold Q to determine the impact of the factor on the result variable. If the p-value of the factor is less than Q, the factor is statistically significant, indicating that the factor affects the result variable, otherwise it has no effect.

[0077] In statistics, the p-value is a probability value that indicates the probability of observing a statistical result (or a more extreme result) under the condition that the null hypothesis is true. The null hypothesis usually assumes that the factor has no effect on the result (that is, the coefficient is zero). If the p-value is lower than the pre-set threshold, the null hypothesis is rejected and the factor is considered to have a significant effect on the result.

[0078] Build an XGBoost model to quantify the contribution of each factor to the initial value of the data asset, use the features extracted from the graph network to train the XGBoost model, use the cross-validation method to verify the performance of the XGBoost model, and stop training if the model performance does not improve, and obtain the trained XGBoost model;

[0079] The parameters of the XGBoost model are adjusted based on the causal inference results, and the factors screened from the causal inference are used as input variables to identify the factors that have the greatest impact on the initial value of the data asset.

[0080] The judgment threshold Q is set based on actual application scenarios and the experience of professionals. Based on business knowledge and node features extracted from the graph network, a list of hypothetical factors affecting the initial value of data assets is defined, including transaction frequency, market volatility, risk score and other factors. The key factors affecting the value of data assets can be systematically identified and analyzed. Using the DoWhy library to draw a causal graph, the causal relationship between these factors can be clearly displayed and understood, and the transaction frequency of data assets is determined as the cause variable and the initial value of data assets is determined as the result variable, thus providing a scientific and intuitive analysis method. The impact of each factor on the initial value of data assets is estimated by regression analysis, and causal inference is performed to quantify the specific impact of each factor. The results of causal inference include the coefficient of each factor, and the statistical significance of the factor is determined by checking the p value. If the p value of a factor is less than the set judgment threshold Q, the factor is statistically significant, indicating that the factor has a significant impact on the result variable. This method improves the credibility and scientificity of the analysis results. By constructing an XGBoost model to quantify the contribution of each factor to the initial value of data assets, the model is trained using features extracted from the graph network, and the model performance is verified using the cross-validation method to ensure the accuracy and stability of the model. If the model performance does not improve, stop training and obtain the optimized XGBoost model. Adjust the parameters of the XGBoost model based on the causal inference results so that the model can more accurately identify and quantify the factors that have the greatest impact on the initial value of data assets.

[0081] Further, constructing an asset evaluation model to evaluate the value of data assets means constructing a linear regression model as an asset evaluation model, taking the initial value of the data asset as the dependent variable, the maximum influencing factor as the independent variable, estimating the model parameters using the least squares method, and using historical influencing factors and historical data asset value data as training data to input into the asset evaluation model for iterative training;

[0082] Define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the asset valuation model no longer decreases significantly during the continuous iteration process, stop iterating and output the model parameters to update the asset valuation model.

[0083] Input the data to be evaluated into the asset evaluation model to obtain the value evaluation results of the data assets.

[0084] By constructing a linear regression model, the linear relationship between the value of data assets and their influencing factors can be effectively captured. Taking the initial value of data assets as the dependent variable and the largest influencing factor as the independent variable can simplify the complexity of the model while retaining the main explanatory power of the asset value. The least squares method is used to estimate the model parameters to ensure that the model can minimize the prediction error and improve the accuracy and stability of the prediction. The historical influencing factors and historical data asset value data are used as training data to make full use of past actual data and improve the prediction ability of the model. The iterative training process can continuously optimize the model parameters to make it more suitable for the current data characteristics and market environment. The definition of a reasonable loss function can effectively measure the error between the model prediction value and the actual value, providing a clear goal for model optimization. The Adam optimizer is used for parameter optimization because of its high computational efficiency and ability to adaptively adjust the learning rate, making the model training process more stable and faster. When the loss of the asset valuation model no longer decreases significantly during the continuous iteration process, the iteration is stopped to ensure that the model reaches the optimal state. The data to be evaluated is input into the trained asset valuation model to quickly obtain the value evaluation results of the data asset. The model can make accurate value predictions based on the features in the input data and combined with the trained parameters, thereby improving the efficiency and accuracy of the evaluation work.

[0085] S4. Store the data generated during the data asset assessment process and implement access control;

[0086] Specifically, storing the data generated during the data asset evaluation process and implementing access control means that after the data asset value evaluation results are verified through smart contracts and recorded in the blockchain, the results are synchronously stored in the central database, and the data generated during the data asset evaluation process are collected and stored in the central database and access control is implemented, allowing authorized users to query and access the data.

[0087] The results of data asset value assessment are verified by smart contracts and recorded on the blockchain, ensuring the transparency and immutability of the assessment results. The automatic execution characteristics of smart contracts reduce manual intervention, operational risks and error rates. At the same time, the decentralized and distributed storage characteristics of blockchain further enhance the security and credibility of data. The assessment results verified by smart contracts are synchronously stored in the central database to achieve centralized management of data. The central database provides efficient data query and access capabilities to ensure that the assessment results can be quickly retrieved and used. This centralized storage method also facilitates the subsequent processing and analysis of data and improves the overall efficiency of data management. In the process of data asset assessment, a large amount of intermediate data and result data will be generated. By collecting this data and storing it in the central database, the assessment process can be fully recorded, providing detailed data tracking and auditing capabilities. This method not only improves the integrity of the data, but also facilitates the backtracking and analysis of the assessment process, further enhancing the credibility and transparency of the assessment.

[0088] Embodiment 2, as Figure 3 As shown, it is a second embodiment of the present invention. This embodiment is different from the previous embodiment and provides a data asset evaluation system based on big data, which includes:

[0089] The data collection module is used to determine and configure the types of data sources that need to be integrated, collect data for pre-processing, and perform blockchain authentication and recording, and store the data in a central database;

[0090] The graph network construction module is used to build a graph database based on the central database and update node attributes in real time according to changes in node data and extract node features in the graph network;

[0091] The causal inference module is used to draw causal diagrams using the DoWhy library and perform causal inference using regression analysis methods, and to quantify the contribution of each factor using the XGBoost model;

[0092] The asset evaluation module is used to build an asset evaluation model based on the most influencing factors to evaluate the value of data assets and verify the evaluation results of data asset value through smart contracts, and store the data generated during the evaluation process in the central database.

[0093] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0094] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0095] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0096] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0097] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data asset evaluation method based on big data , The invention comprises, wherein: Collect multi-source data, store it in a central database, and upload it to the blockchain for data ownership confirmation and recording; Build a graph network of data assets based on a central database, capture the temporal changes of data relationships in the graph network, and extract node features from the graph network; Based on the extracted node features, we identify the factors that affect the value of data assets and build an asset evaluation model to evaluate the value of data assets. Store data generated during the data asset assessment process and implement access control; The said collecting multi-source data to store in a central database and uploading to the blockchain for data confirmation and recording means collecting data from various data sources and storing it in a central database after pre-processing, writing smart contracts in Solidity language, deploying smart contracts on the Ethereum platform, and storing the data on the blockchain; The said construction of a graph network of data assets based on a central database refers to installing a Neo4j graph database and Elasticsearch, building a graph data model, and using the transaction relationship between two nodes as an edge to connect related data asset nodes and user nodes; Add multi-dimensional attributes of data assets to the basic model and set edge weights based on the transaction frequency and amount of data assets; The centrality of data assets is defined using the data asset influence scoring formula based on the intensity and frequency of interactions between data assets: In the formula, S(v) represents the influence score of the asset node, w uv is the relationship weight between nodes u and v, t uv is the most recent interaction time, and f is the time decay function; Calculate the influence score of each data asset node and use the influence score as the node attribute; Establish a real-time data monitoring system to monitor the dynamic changes of data assets in real time and push updates to Neo4j, and use exponential decay functions to adjust the initial value attributes of data asset nodes in real time; Use the batch import tool to import data assets and transaction records into Neo4j, and use the risk assessment formula to update the risk attributes of data asset nodes based on market fluctuations and historical data assets; When changes are detected in the initial value and risk of data assets, the value attributes and risk attributes of the data asset nodes in the graph network are automatically updated.

2. According to the data asset evaluation method based on big data in claim 1, Features: Capturing the temporal changes of data relationships and extracting node features from the graph network refers to adding a time attribute to each edge and defining a time window, performing a time series analysis on the data asset relationship, calculating the statistical indicators of each data asset node in different time windows, using Cypher queries in Neo4j to generate the statistical indicators, and adding the statistical indicators as attributes to the corresponding data asset nodes; Get all nodes and corresponding edges in the graph network, configure GraphSAGE model parameters, learn node embedding by aggregating neighbor information of each data asset node and perform iterative updates; Update the model weights through back-propagation and define the loss function. When the loss of the GraphSAGE model no longer decreases significantly during the continuous iteration, stop iterating and output the model parameters to update the GraphSAGE model. After training is completed, the features of nodes and edges are extracted from the GraphSAGE model.

3. According to the data asset evaluation method based on big data as described in claim 2, Features: The identifying factors affecting the value of data assets based on extracted node features refers to defining a list of hypothetical factors affecting the initial value of data assets based on business knowledge and node features extracted from a graph network, drawing a causal graph using the DoWhy library, determining the transaction frequency of data assets as the cause variable, and the initial value of data assets as the result variable; Use regression analysis to estimate the impact of factors on the initial value of data assets for causal inference. The results of causal inference include the coefficient of each factor. Check the p-value of each factor and set the judgment threshold Q. Compare the p-value of each factor with the judgment threshold Q to determine the impact of the factor on the result variable. If the p-value of the factor is less than Q, the factor is statistically significant, indicating that the factor affects the result variable, otherwise it has no effect. Build an XGBoost model to quantify the contribution of each factor to the initial value of the data asset, use the features extracted from the graph network to train the XGBoost model, use the cross-validation method to verify the performance of the XGBoost model, and stop training if the model performance does not improve, and obtain the trained XGBoost model; The parameters of the XGBoost model are adjusted based on the causal inference results, and the factors screened from the causal inference are used as input variables to identify the factors that have the greatest impact on the initial value of the data asset.

4. According to the data asset evaluation method based on big data as described in claim 3, Features: The said construction of an asset evaluation model to evaluate the value of data assets refers to constructing a linear regression model as an asset evaluation model, taking the initial value of the data asset as the dependent variable, the maximum influencing factor as the independent variable, estimating the model parameters using the least squares method, and using historical influencing factors and historical data asset value data as training data to input into the asset evaluation model for iterative training; Define the loss function and Adam optimizer to iteratively optimize the model parameters. When the loss of the asset valuation model no longer decreases significantly during the continuous iteration process, stop iterating and output the model parameters to update the asset valuation model. Input the data to be evaluated into the asset evaluation model to obtain the value evaluation results of the data assets.

5. According to the data asset evaluation method based on big data in claim 4, Features: The storing of data generated during the data asset evaluation process and the implementation of access control means that after the data asset value evaluation results are verified through smart contracts and recorded in the blockchain, the results are synchronously stored in the central database, the data generated during the data asset evaluation process are collected and stored in the central database, and access control is implemented to allow authorized users to query and access the data.

6. A data asset evaluation system based on big data based on the data asset evaluation method based on big data according to any one of claims 1 to 5, Features: include, The data collection module is used to determine and configure the types of data sources that need to be integrated, collect data for pre-processing, and perform blockchain authentication and recording, and store the data in a central database; The graph network construction module is used to build a graph database based on the central database and update node attributes in real time according to changes in node data and extract node features in the graph network; The causal inference module is used to draw causal diagrams using the DoWhy library and perform causal inference using regression analysis methods, and to quantify the contribution of each factor using the XGBoost model; The asset evaluation module is used to build an asset evaluation model based on the most influencing factors to evaluate the value of data assets and verify the evaluation results of data asset value through smart contracts, and store the data generated during the evaluation process in the central database.

7. A computer device, include: A memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the data asset evaluation method based on big data described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the steps of the data asset evaluation method based on big data described in any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Scholarship influence calculation method and system based on scholarship paper relationship network

    CN110851659A

  • Data operating method based on block chain, related equipment and storage medium

    CN111260468A

  • Asset value evaluation method and device, model training method and device and readable storage medium

    CN116194911A

  • Method and device for realizing data asset value evaluation processing based on machine learning in credential environment, processor and storage medium thereof

    CN116228268A

  • Virtual resource prediction release amount determination and value evaluation model training method

    CN116308734A

Cited By

  • Geological disaster intelligent early warning method and system based on multi-source remote sensing data

    CN119380496A

  • Data asset value evaluation method and system based on dynamic network analysis

    CN120145403A

  • New energy charging station asset credibility evaluation optimization method based on machine learning

    CN120764833A

  • Data asset comprehensive evaluation system based on artificial intelligence and block chain

    CN121562976A