Data asset cost measurement method based on data blood relationship and compliance coefficient

Abstract

The application discloses a data asset cost measurement method based on data blood relationship and compliance coefficient, and belongs to the technical field of data asset cost measurement. The method responds to a blood relationship topological change event of a data asset, acquires resource residence time length and resource specification snapshots of each processing node in a data link, and allocates a node original cost. A compliance coefficient is obtained through a security grading label and a compliance cost mapping table, and a weighted compliance cost is calculated. Then, a business gain coefficient is calculated according to a calling frequency and a calling result state of a downstream business system, and the weighted compliance cost is allocated according to the business gain coefficient to generate a total cost measurement value. The scheme can include compliance management loss and actual business contribution into cost accounting, so that the data asset cost can truly reflect security compliance cost and business value.

Description

Technical Field

[0001] This invention relates to the field of data asset cost measurement technology, specifically a data asset cost measurement method based on data lineage and compliance coefficient. Background Technology

[0002] Existing data asset cost measurement schemes mostly use direct resource consumption as the accounting benchmark. They only record the duration and specifications of computing, storage, and network resources occupied by each processing node in the data chain, and obtain the total cost by multiplying the unit price of resources by the usage. This type of method does not consider the additional costs incurred by data assets during the circulation process due to security classification and compliance requirements, nor does it reflect the differences in the actual use value of data assets in downstream businesses.

[0003] When data assets flow across systems and regions, compliance actions such as encryption, de-identification, and access auditing must be performed based on the classification and grading identifiers they carry. These actions consume additional computing and storage resources, which are not identified and included in asset costs in traditional measurement schemes, leading to severely distorted cost data. Simultaneously, the same data asset is accessed by multiple downstream business systems through different processing nodes. Each node handles different query frequencies and returns varying amounts of data. Traditional solutions either amortize all costs equally or allocate costs solely based on resource usage, failing to differentiate the cost contributions of high-value, high-access nodes from low-value nodes, causing accounting results to deviate from actual business efficiency.

[0004] Regarding real-time capture of resource overhead, existing technologies typically rely on application layer or middleware logs, making it difficult to accurately obtain the precise time and instantaneous resource specifications of data packets entering and leaving nodes, resulting in coarse-grained calculations of raw node costs. In terms of quantifying security and compliance costs, there is a lack of a mechanism to dynamically map data security classification to compliance overhead weights, preventing the automatic breakdown and allocation of compliance costs to data flow nodes. Regarding business value measurement, there is a lack of methods to quantify business gains based on call output and call frequency, making it difficult to reasonably allocate compliance-weighted costs towards business benefits. How to accurately quantify the compliance load in the lineage chain and combine it with resource costs, and how to use downstream call behavior to reflect cost allocation logic, constitute the core issues that need to be addressed. Summary of the Invention

[0005] The purpose of this invention is to provide a data asset cost measurement method based on data lineage and compliance coefficients, so as to solve the problems that existing data asset cost measurement cannot include additional costs for security and compliance and cannot reflect the differences in the value of downstream business calls.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The purpose of the present invention is to provide a data asset cost measurement method based on data lineage and compliance coefficient, which aims to solve the problems that existing data asset cost measurement methods cannot dynamically perceive changes in data links, cannot reflect security and compliance costs, and ignore the actual value of business calls.

[0007] To achieve the above objectives, the technical solution adopted by this invention is as follows: In response to a data asset lineage topology change event, the resource residence time and resource specification snapshot of the data asset on each processing node in the data link are obtained; the original cost of the node is allocated to each processing node according to the resource residence time and resource specification snapshot; the security classification label triggered when the data asset flows in the data link is identified, and the compliance coefficient of each processing node is obtained by querying a preset compliance cost mapping table; the weighted compliance cost of each processing node is calculated using the original cost of the node and the compliance coefficient; the call frequency and call result status of the data asset called by downstream business systems within a preset accounting period are obtained, and the business gain coefficient of each processing node is calculated accordingly; the weighted compliance cost of each processing node is allocated according to the business gain coefficient to generate a total cost metric of the data asset.

[0008] As a preferred technical solution of the present invention, when acquiring the resource residency duration and resource specification snapshot of data assets on each processing node, an eBPF probe is attached to the kernel state of each processing node in the data link to capture the timestamps when the data assets enter and leave each processing node in the form of data packets. The difference between the departure timestamp and the entry timestamp is used as the resource residency duration. Simultaneously, the eBPF probe collects the number of virtual CPU cores allocated to the processing node, the maximum memory bandwidth, and the storage IOPS quota during the data asset residency period. These parameters are then combined to form a resource specification snapshot. This solution can accurately capture the actual resource consumption of data assets at each processing stage without intrusion, providing a refined data foundation for subsequent cost accounting.

[0009] In a further technical implementation, the process of allocating the original cost of each processing node based on resource residency duration and resource specification snapshots is implemented as follows: The CPU cost component is obtained by multiplying the number of virtual CPU cores by a preset CPU unit price; the memory cost component is obtained by multiplying the memory bandwidth limit by a preset memory bandwidth unit price; and the storage cost component is obtained by multiplying the storage IOPS quota by a preset storage IOPS unit price. The CPU cost component, memory cost component, and storage cost component are summed to obtain the unit time cost, and then the unit time cost is multiplied by the resource residency duration to obtain the original node cost. Thus, the original node cost accurately reflects the rigid resource overhead of data assets at each processing node, calculated comprehensively based on specifications and time.

[0010] As a preferred embodiment of the present invention, the specific steps for identifying security classification labels and obtaining compliance coefficients are as follows: Before the data asset enters each processing node, the data asset's payload header is parsed to extract the data classification identifier and data level identifier carried by the data asset, and these two are concatenated to form a security classification label; using the security classification label as an index key, the corresponding encryption overhead weight and access audit weight are looked up in the compliance cost mapping table, and added together to obtain the compliance coefficient. This process automatically maps the security attributes of the data asset into quantifiable compliance cost factors without manual intervention.

[0011] Preferably, the encryption overhead weight and access audit weight in the compliance cost mapping table are also dynamically adjusted according to the geographical location of the processing nodes through which the data assets are processed. The geographical location is read from the deployment configuration file of the processing node, so that the calculation of the compliance coefficient can be adapted to the differences in regulatory requirements in different regions.

[0012] In the weighted compliance cost calculation stage, the security protection level of the physical host where the processing node resides is obtained. Based on the security protection level, a pre-defined compliance cost mapping table is consulted to obtain the environment hardening coefficient of the physical host. The compliance coefficient is multiplied by the environment hardening coefficient to obtain the comprehensive compliance multiplier. Then, the original cost of the node is multiplied by the comprehensive compliance multiplier to obtain the weighted compliance cost. The weighted compliance cost not only reflects the security compliance requirements of the data itself, but also takes into account the security hardening investment of the operating environment, making the security cost measurement more comprehensive.

[0013] This invention optimizes the calculation of the business gain coefficient: It captures the amount of data from data query requests received by each processing node within a preset accounting period using a log collector, as the call frequency; it analyzes the number of rows and columns in the returned dataset corresponding to each data query request, multiplying the number of rows and columns as the output value of a single call, and sums all single call output values ​​within the period to obtain the total call output value; the ratio of the total call output value to the call frequency is used as the average single output, and after normalization, the business gain coefficient is obtained. This coefficient truly reflects the effective data supply capacity of the processing node to downstream businesses, avoiding the bias caused by evaluating solely based on the number of calls.

[0014] As an improvement, the call result status further incorporates the response time of the data query request and the null value rate of the returned data fields; when the response time exceeds a preset threshold or the field null value rate exceeds a preset null value ratio, the output value of that single call is set to zero. This eliminates the contribution of inefficient or invalid calls to business gains, making the business gain coefficient closer to actual business value.

[0015] During the total cost metric generation phase, the business gain coefficients of all processing nodes on the data link are summed to obtain the total gain. The business gain coefficient of each processing node is divided by the total gain to obtain the allocation ratio of that processing node. The weighted compliance cost of each processing node is multiplied by its allocation ratio to obtain the allocated cost. The total cost metric of the data asset is formed by summing the allocated costs of all processing nodes. The total cost metric organically unifies rigid resource consumption, compliance-related costs, and business contribution, truly reflecting the comprehensive consumption and output of the data asset throughout its entire lifecycle.

[0016] Preferably, when the data asset leaves the processing node, the eBPF probe encapsulates the resource dwell time and resource specification snapshot into a metadata message and sends it to the downstream processing node along with the data asset, so that each node in the lineage can obtain upstream cost information, supporting cost tracking and backtracking throughout the entire chain.

[0017] This invention achieves multi-dimensional, end-to-end measurement of data asset costs by dynamically sensing changes in data lineage and introducing compliance and business gain coefficients. The measurement results can dynamically fluctuate with the security level of data assets, the state of environmental hardening, and the value of business calls, providing an accurate basis for the refined pricing, internal accounting, and optimization of data assets.

[0018] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0019] As data assets flow along the lineage link, for each processing node, an eBPF probe is mounted to the kernel mode to capture the timestamps of data packets entering and leaving the node and calculate the resource dwell time. Simultaneously, the number of virtual CPU cores allocated to the node, the maximum memory bandwidth, and the storage IOPS quota are collected to form a resource specification snapshot. The original node cost is obtained by multiplying the number of virtual CPU cores by the CPU unit price, the maximum memory bandwidth by the memory bandwidth unit price, and the storage IOPS quota by the storage IOPS unit price, summing these three cost components, and multiplying by the resource dwell time. When the data asset leaves the processing node, the eBPF probe encapsulates the resource dwell time and resource specification snapshot into a metadata message and sends it along with the data to downstream nodes, ensuring that cost factors are passed hop-by-hop throughout the entire link. Before the data enters the processing node, the data asset payload header is parsed to extract the data classification identifier and data level identifier, which are concatenated into a security rating label. This label is then used as an index key to look up the matching encryption overhead weight and access audit weight in the compliance cost mapping table, and the sum of these two is the compliance coefficient. The encryption overhead weight and access audit weight in the compliance cost mapping table can be dynamically adjusted based on the geographical location of the processing node, which is read from the deployment configuration file of the processing node. The weighted compliance cost is calculated by multiplying the original cost of the node by a compliance coefficient, thus directly internalizing the compliance costs such as encryption and auditing arising from data security classification into the cost structure of each node. By establishing a quantitative correlation between the resource consumption of compliance actions and the security level, the missing compliance costs in asset metrics are avoided, ensuring that cost data truly reflects the actual cost of regulatory compliance and security protection requirements.

[0020] The value of downstream business call behavior is quantified. The call frequency is calculated by accumulating the amount of data query requests received by each processing node within a preset accounting period using a log collector. The number of rows and columns in the returned dataset for each query request is analyzed, and the product of the row and column counts is used as the output value of a single call. The total call output value is obtained by summing all single call output values ​​within the period. The ratio of the total call output value to the call frequency is used as the average single call output, which is then normalized and used as the business gain coefficient. When calculating the call output value, if the response time of a call exceeds a preset threshold or the null value rate of the returned data exceeds a preset null value ratio, the single call output value of that call is set to zero to eliminate the interference of low-quality calls on the gain coefficient. The total gain is obtained by summing the business gain coefficients of all processing nodes in the data link. The business gain coefficient of each node is divided by the total gain to obtain the allocation ratio. The weighted compliance cost of each processing node is multiplied by the allocation ratio to obtain the allocated cost, and these are accumulated to generate the total cost metric. By allocating weighted compliance costs proportionally according to business gain coefficients, the cost measurement results are linked to the actual contribution of data assets to downstream businesses. Nodes with high output and high call frequency are allocated more costs, while low-value nodes are allocated less costs. This achieves a balance between cost and business efficiency, providing a quantitative basis that conforms to business logic for data asset pricing and internal settlement. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 This is a flowchart of a data asset cost measurement method based on data lineage and compliance coefficients; Figure 2 This is a flowchart of data asset link cost measurement and hop-by-hop transmission based on eBPF probes; Figure 3 This is a flowchart of the dynamic calculation process for compliance coefficients based on security rating labels and geographical regions. Figure 4 This is a diagram illustrating the relationship between the original cost of a node and the duration of resource residence. Figure 5 This is a trend chart showing the changes in compliance coefficients corresponding to security rating labels in different geographical regions; Figure 6 This is a diagram illustrating the relationship between weighted compliance costs and original node costs for different security protection levels. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See Figure 1 This invention provides a data asset cost measurement method based on data lineage and compliance coefficient, comprising: responding to a data asset lineage topology change event, acquiring the resource residence time and resource specification snapshot of the data asset on each processing node in the data link; allocating the original node cost to each processing node according to the resource residence time and the resource specification snapshot; identifying the security classification label triggered when the data asset flows in the data link, querying a preset compliance cost mapping table according to the security classification label to obtain the compliance coefficient of each processing node; calculating the weighted compliance cost of each processing node according to the original node cost and the compliance coefficient; acquiring the call frequency and call result status of the data asset called by downstream business systems within a preset accounting period, calculating the business gain coefficient of each processing node according to the call frequency and the call result status; and allocating the weighted compliance cost of each processing node according to the business gain coefficient to generate a total cost metric for the data asset.

[0025] Example 1 In specific implementation, please refer to Figure 2 By attaching eBPF probes to the kernel mode of each processing node in the data link, fine-grained observation of data assets at the network layer is achieved. The eBPF probes are loaded into the kernel of the processing node's operating system as eBPF programs, with the attachment point selected on the data transmission and reception path of the virtual network interface card (NIC) driver. When a data asset arrives at the processing node as a data packet, the eBPF probe inserts hook code in the interrupt handler function where the data packet enters the virtual NIC, reading the time value from the kernel timestamp counter and recording it as the timestamp of the data asset entering the processing node. When the data asset completes all transformation logic within the processing node and is ready to be sent from the same virtual NIC, the eBPF probe reads the kernel timestamp in the hook code on the sending path where the data packet leaves the virtual NIC, recording it as the timestamp of the data asset leaving the processing node. The difference between the departure timestamp and the arrival timestamp is used as the resource residence time of the data asset on the processing node, measured in microseconds.

[0026] While the data asset resides on the processing node, the eBPF probe continuously collects the computing and storage resource usage quotas allocated by the processing node to the data asset. The collection method is as follows: The eBPF probe reads the cpu.cfs_quota_us file of the process group to which the data asset belongs through the cgroup interface related to process scheduling in the kernel, obtaining the number of virtual CPU cores. The number of virtual CPU cores represents the number of logical CPU cores allocated by the processing node to the data asset, and is a positive integer. It also obtains the memory bandwidth limit by reading the memory.max_bandwidth file or an equivalent memory bandwidth control file. The memory bandwidth limit represents the maximum number of bytes that can be consumed per second for memory read / write operations, in MB / s. Finally, it reads the blkio.throttle.read_iops_device and blkio.throttle.write_iops_device files, and uses the sum of the read IOPS limit and write IOPS limit as the storage IOPS quota. The storage IOPS quota represents the maximum number of input / output operations that can be performed per second, in operations per second. The collected virtual CPU core count, memory bandwidth limit, and storage IOPS quota are combined into a triplet to form a resource specification snapshot.

[0027] After obtaining the resource residency duration and resource specification snapshots, a node raw cost is allocated to each processing node. The calculation of the node raw cost is expressed using a comprehensive formula, which is: ; in, This represents the original cost of the processing node. The value is a floating-point number and the unit is a preset currency unit. This represents the number of virtual CPU cores, extracted from the resource specification snapshot, and has a value greater than 0. The preset CPU unit price is the pre-configured cost per virtual CPU core per unit of time, expressed in preset currency units per (virtual CPU core·microsecond). The preset CPU unit price is calculated based on the purchase cost, depreciation period, and per unit virtual CPU core amortization share of the physical host where the processing node is located and is stored in the cost parameter configuration file. This represents the maximum memory bandwidth value, extracted from a resource specification snapshot, in MB / s. The preset memory bandwidth unit price is the cost of a unit of memory bandwidth per unit of time, which is pre-configured. The unit is the preset currency unit / (MB / s·microsecond). The preset memory bandwidth unit price is calculated and stored based on the ratio of memory module cost to bandwidth capacity. This represents the storage IOPS quota, which is extracted from the resource specification snapshot, and is expressed in operations per second. The preset storage IOPS unit price is the cost per unit of IOPS per unit of time that is pre-configured. The unit is the preset currency unit / (number of operations / second·microsecond). The preset storage IOPS unit price is calculated and stored based on the ratio of storage device cost to total IOPS supply capacity. The resource residency duration is obtained through timestamp differences and is measured in microseconds. The part within parentheses in the formula first calculates the unit time cost, which represents the weighted sum of computational, memory bandwidth, and storage I / O resources consumed per microsecond on the processing node. Then, the unit time cost is multiplied by the resource residency duration to obtain the node's original cost.

[0028] In one implementation, when a data asset leaves a processing node, the eBPF probe does not simply report the resource residency duration and resource specification snapshot to the centralized collector. Instead, it encapsulates these parameters into a metadata message and sends it along with the data asset's data packet to downstream processing nodes. During encapsulation, the eBPF probe appends a fixed-format metadata block to the end of the data packet's application-layer payload. This metadata block contains a resource residency duration field, a virtual CPU core number field, a memory bandwidth limit field, and a storage IOPS quota field, all encoded in network byte order. Upon receiving the data packet, downstream processing nodes can extract and parse this information from the metadata block, thus obtaining the amount of resources used by upstream nodes without needing to trace back to central storage, enabling hop-by-hop transmission of cost information in the data link.

[0029] See Figure 4 This figure illustrates the relationship between resource dwell time and node raw cost collected by the eBPF probe in Example 1. Scattered distributions and corresponding fitted curves were plotted for three different virtual CPU core configurations (2 cores, 4 cores, and 8 cores). The horizontal axis represents resource dwell time in microseconds, and the vertical axis represents node raw cost in a preset currency unit. In the figure, blue dashed lines and dots represent nodes with 2 virtual CPU cores, orange dashed lines and squares represent nodes with 4 virtual CPU cores, and green dashed lines and triangles represent nodes with 8 virtual CPU cores.

[0030] As shown in the figure, the original cost of a node increases overall with the increase in resource residency time, and the cost value and resource residency time have an approximately linear relationship, which is consistent with the product relationship of resource residency time in the original cost calculation formula of the node in Example 1. The slopes of the three fitted curves increase sequentially, and the more virtual CPU cores there are, the higher the cost per unit time, reflecting the significant impact of the number of virtual CPU cores in the resource specification snapshot on the original cost of the node.

[0031] Specifically, in the range of low resource dwell time (approximately 0 to 10,000 microseconds), the original node costs for all three virtual CPU core counts are low and relatively concentrated, with no significant differences. As the resource dwell time gradually increases to around 30,000 microseconds, the differences in original costs gradually become apparent. The cost of an 8-core node is significantly higher than that of a 4-core or 2-core node, and the distribution of nodes is more dispersed, indicating that under higher resource consumption, the original node cost becomes more sensitive to the number of virtual CPU cores.

[0032] Example 2 In specific implementation, please refer to Figure 3 When data assets flow along the data link and are about to enter a processing node, the payload header of the data assets is intercepted and its fields are parsed. Data assets are transmitted over the network using a structured message format. The message header contains fields for carrying data classification and data level identifiers. The offset and length of the data classification identifier field are uniformly defined by the data governance platform and written into the message pattern definition file. In one implementation, the data classification identifier field is located at offset 0x10 bytes in the message header, with a length of 2 bytes, using unsigned integer encoding, representing the business data category code to which the data asset belongs, such as financial data, personal identification information data, or publicly available data. The data level identifier field is located at offset 0x12 bytes in the message header, with a length of 1 byte, using unsigned integer encoding, representing the security sensitivity level of the data asset, divided into multiple levels from low to high, with integer values ​​from 1 to 5 corresponding to levels 1 to 5 respectively. The field parsing process is as follows: In the pre-processing network hook at the entry point of the data asset, the specified offset byte sequence in the message header is read. The values ​​of the data classification identifier and the data level identifier are parsed out respectively. Then, the two values ​​are converted into readable string label codes according to a predefined mapping table. The data classification identifier value is mapped to the classification code string, for example, the value 0x0001 is mapped to "PII", the value 0x0002 is mapped to "FIN", etc.; the data level identifier value is directly converted to numeric characters, for example, the value 3 is converted to "3". The data classification identifier string and the data level identifier string are concatenated with a fixed connector, which uses the underscore character "_", to form a security level label. The final form of the security level label is the classification code string followed by an underscore followed by the level numeric string, for example, "PII_3".

[0033] After obtaining the security rating label, a query operation is performed in the pre-defined compliance cost mapping table using the security rating label as the index key. The compliance cost mapping table is an associative storage structure with the security rating label as the key and the compliance cost structure as the value. The compliance cost structure includes an encryption overhead weight field and an access audit weight field. The compliance cost mapping table is loaded into memory by the security policy management module during system initialization and is organized as a hash table. The key is the security rating label string, and the value is a structure containing two floating-point numbers. The query operation specifically involves calculating the hash value using the security rating label string, locating the corresponding storage slot, and if the key in the slot completely matches the query key, then retrieving the encryption overhead weight and access audit weight from that slot. The encryption overhead weight represents the cost weighting factor of the additional computing resource consumption and key management overhead caused by implementing transmission encryption and static encryption of data assets. The value of the encryption overhead weight is set by security auditors during initial configuration based on the encryption protocol type, key length, and rotation frequency, and is a floating-point number greater than or equal to zero. Access audit weight represents the cost weighting factor required for logging, storing audit logs, and performing audit analysis for each access behavior to data assets. The value of access audit weight is also set by security auditors based on the storage period, sampling rate, and analysis depth of the audit logs, and is a floating-point number greater than or equal to zero.

[0034] In one implementation, the encryption overhead weights and access audit weights in the compliance cost mapping table are not fixed to stored baseline values, but are dynamically adjusted according to the geographical location of the processing nodes through which the data assets are processed. The geographical location is read from the deployment configuration file of the processing node. During the deployment phase of each processing node, operations personnel write a region identifier field into the node's configuration file. The value of the region identifier field adopts the international region code standard, such as "cn-north-1" or "eu-west-2". When compliance coefficients need to be calculated, the system reads the geographical location identifier of the processing node from its local configuration file or from the central configuration center. The compliance cost mapping table additionally stores encryption adjustment coefficients and audit adjustment coefficients for each geographical location. These coefficients are stored separately in a regional adjustment mapping area, using the geographical location identifier as the key. During the query process, a set of baseline encryption overhead weights and baseline access audit weights are first obtained using the security rating label. Then, the regional adjustment mapping area is queried using the geographical location identifier to obtain the encryption adjustment coefficients and audit adjustment coefficients corresponding to that geographical location. The encryption adjustment factor is a floating-point number representing a multiplicative adjustment factor applied to the baseline encryption overhead weight due to varying compliance policy stringency in the region. Its value typically falls between 0.8 and 1.5, with a default value of 1.0. The audit adjustment factor is also a floating-point number, representing a multiplicative adjustment factor applied to the baseline access audit weight due to varying audit regulatory requirements in the region. Its value typically falls between 0.7 and 1.8, with a default value of 1.0. Multiplying the baseline encryption overhead weight by the encryption adjustment factor yields the actual encryption overhead weight used in this calculation, and multiplying the baseline access audit weight by the audit adjustment factor yields the actual access audit weight used in this calculation.

[0035] The compliance coefficient is obtained by adding the dynamically adjusted encryption overhead weight to the access audit weight, expressed by the formula: ; in, This represents the compliance coefficient, which is a dimensionless weighted factor used for subsequent compliance adjustments to the original cost of the node. This represents the encryption overhead weight, which is the value of the encryption overhead weight after adjustment based on the geographical location. This represents the access audit weight, which is a value adjusted for geographical location. Through this method, the compliance coefficient of each processing node combines the basic compliance requirements corresponding to the security classification label of the data asset itself, as well as the regional differences in compliance strength caused by the geographical location of the processing node. This allows the compliance coefficient to dynamically reflect the differences in compliance costs when processing data assets of the same security level in different regions.

[0036] See Figure 5 The vertical axis of the graph represents the compliance coefficient, a dimensionless value reflecting the weighted average of compliance costs for data assets with different security rating labels at different processing nodes. The horizontal axis represents the security rating label, which is formed by concatenating data classification identifiers and data grading identifiers, specifically including six types: "PII_1", "PII_2", "PII_3", "FIN_1", "FIN_2", and "PUB_1". The graph shows the compliance coefficient trends for the geographical locations of three representative processing nodes: "cn-north-1" (blue dots), "eu-west-2" (orange squares), and "us-east-1" (green diamonds). The compliance coefficient for each geographical region is dynamically calculated by adjusting the encryption overhead weight and access audit weight corresponding to the security rating label using a geographical location adjustment coefficient, reflecting the differences in the stringency of compliance policies across different regions.

[0037] As can be seen from the curve trend in the graph, the compliance coefficient generally exhibits a fluctuating pattern of first increasing and then decreasing. Specifically, at the "PII_1" security rating label, the compliance coefficient is at a relatively low level, averaging around 0.3. Subsequently, at the "PII_2" label, the compliance coefficient increases significantly, reaching around 0.5. It peaks at the "PII_3" label, with the highest compliance coefficient concentrated between 0.7 and 0.8, reflecting the greatest compliance intensity required for data with higher security sensitivity levels. Next, at the "FIN_1" label, the compliance coefficient drops sharply to around 0.4, indicating that the compliance burden of low-level financial data is relatively low. Subsequently, at the "FIN_2" label, it rises again to the range of 0.6 to 0.7, reflecting the increased compliance cost of medium-level financial data. Finally, at the "PUB_1" label, the compliance coefficient drops sharply to around 0.15, indicating that the compliance cost of publicly available data is the lowest.

[0038] The compliance coefficient curves for the three geographical regions generally exhibit similar patterns, but significant numerical differences exist. Specifically, the compliance coefficient for the "eu-west-2" region is generally higher than that of the other two regions, reflecting stricter compliance policies and a larger compliance cost adjustment factor in this region. "us-east-1" follows closely, while the "cn-north-1" region has the lowest compliance coefficient, indicating a relatively smaller compliance cost adjustment factor in this region. The data points are relatively concentrated, suggesting that the calculation of the compliance coefficient is stable but significantly influenced by security rating labels and geographical region factors.

[0039] Example 3 In practical implementation, the security protection level of the physical host where the processing node resides is obtained and used as the basis for the hardening coefficient of the computing environment. Each processing node runs on a physical host. Before deployment, the physical host undergoes a security configuration hardening process. The operation and maintenance security system assigns a security protection level to the physical host based on the completeness of the security control measures deployed on the physical host. The security protection level is stored in the asset configuration management database of the physical host and maintained in key-value pairs. The key is the unique identifier of the physical host, and the value is the security protection level string or numerical code. During system operation, a query request is sent to the asset configuration management database, passing in the unique identifier of the physical host corresponding to the current processing node, to obtain the security protection level of the physical host. In one implementation, the security protection level is described by four levels: Level 1 represents basic protection, which only deploys the operating system's built-in firewall and basic access control; Level 2 represents standard protection, which additionally deploys a host intrusion detection system on top of basic protection; Level 3 represents enhanced protection, which adds kernel-level mandatory access control and file integrity monitoring on top of standard protection; and Level 4 represents the highest level of protection, which enables a hardware trusted execution environment and memory encryption on top of enhanced protection. The security protection level is assigned by security operations and maintenance personnel based on the security configuration checklist during host online acceptance testing, and the score is written into the asset configuration management database.

[0040] After obtaining the security protection level of the physical host where the processing node resides, the system uses the security protection level as the query key to retrieve the corresponding environment hardening coefficient from the pre-defined compliance cost mapping table. This pre-defined compliance cost mapping table is located in the same hash table space storing the compliance coefficient query structure, and includes an environment hardening coefficient mapping area. The environment hardening coefficient mapping area uses the security protection level code as the key and the environment hardening coefficient as the value. The environment hardening coefficient is a floating-point number representing the incremental adjustment factor for compliance costs caused by different physical host security protection levels. The environment hardening coefficient is set by the security cost accounting module during the initial configuration phase. Level 1 has a hardening coefficient of 0.05, based on the fact that basic protection at Level 1 involves only a few default security components, and the additional compliance resource consumption is negligible. Level 2 has a hardening coefficient of 0.15, based on the fact that standard protection at Level 2 requires additional CPU and memory resources to be allocated to the host intrusion detection system for signature matching and alarm generation. Level 3 has a hardening coefficient of 0.30, based on the fact that enhanced protection at Level 3 introduces mandatory access control policy checks and file integrity verification, requiring each data access to be verified by the policy engine, resulting in significant CPU overhead. Level 4 has a hardening coefficient of 0.50, based on the fact that the highest level of protection at Level 4 enables a trusted hardware execution environment and memory encryption, involving encryption and decryption operations during data transmission within the host memory and processor, resulting in the most significant resource consumption. The environment hardening coefficient acts as a multiplicative factor on the compliance coefficient during calculation, reflecting a positive correlation between higher physical host security protection levels and higher additional compliance costs when handling data assets of the same security level.

[0041] After obtaining the compliance coefficient and environmental hardening coefficient, the compliance coefficient and environmental hardening coefficient are multiplied together to obtain the comprehensive compliance multiplier. The comprehensive compliance multiplier is a dimensionless multiplier factor used to uniformly weight the original cost of a node based on both compliance and environmental dimensions. The comprehensive compliance multiplier is calculated by directly multiplying the value of the compliance coefficient by the value of the environmental hardening coefficient; the product is the comprehensive compliance multiplier.

[0042] In practice, the weighted compliance cost is obtained by multiplying the original cost of a node by the comprehensive compliance multiplier. The calculation process is expressed by the following formula: ; in, This represents the weighted compliance cost of the processing node. The weighted compliance cost is a floating-point number in a preset currency unit, and represents the cost value after the original resource usage cost has been adjusted by a combination of compliance strength and environmental protection strength. This represents the original cost of the processing node. The original cost of the node is the unadjusted cost calculated based on the resource residency time and resource specification snapshot, and the unit is the preset currency unit. The compliance coefficient is the sum of the encryption overhead weight and the access audit weight obtained by querying the preset compliance cost mapping table based on the security rating label. The compliance coefficient is a dimensionless floating-point number with a value not less than 0. This represents the environment hardening factor, which is a physical host environment adjustment factor obtained by querying a preset compliance cost mapping table based on the security protection level of the physical host where the processing node resides. The environment hardening factor is a dimensionless floating-point number, and its value is determined by the security protection level. (The text within the parentheses in the formula is irrelevant and can be omitted.) and First, multiply the costs to obtain the overall compliance multiplier. Then, multiply the original cost of the node by the overall compliance multiplier. The product is the weighted compliance cost of that processing node. The weighted compliance cost reflects all compliance-related computational and resource costs that a processing node should be allocated after considering the security sensitivity of the data assets it processes and the protection strength of its operating environment.

[0043] See Figure 6 The diagram illustrates the relationship between the weighted compliance cost and the original cost of processing nodes corresponding to different security protection levels. The horizontal axis represents the original cost of the node in a preset currency unit, and the vertical axis represents the weighted compliance cost in the same preset currency unit. The diagram shows node data points for security protection levels 1, 2, 3, and 4, connected by dashed lines. Different colors and markers distinguish the four security protection levels in the legend.

[0044] As shown in the graph, the weighted compliance cost for all security protection levels increases positively with the increase of the original node cost, and there is a clear stratification of the weighted compliance cost curves between levels. Specifically, Level 1 (blue dotted line) has the lowest overall weighted compliance cost, with the weighted compliance cost not exceeding 30 pre-set currency units within the range of original node costs from 0 to approximately 600 pre-set currency units. The curve for Level 2 (orange square dotted line) is above Level 1, and within the same range of original node costs, the weighted compliance cost is roughly distributed between 0 and 60 pre-set currency units. Level 3 (green diamond dotted line) has a wider distribution of points, but the overall curve is higher than Level 2, with the weighted compliance cost reaching approximately 180 pre-set currency units within the range of original node costs from 0 to approximately 600 pre-set currency units. The weighted compliance cost for Level 4 (red triangle dotted line) is significantly higher than other levels, and it shows a steeper upward trend with the increase of original node costs; when the original node cost exceeds 1000, the weighted compliance cost can reach more than 500 pre-set currency units.

[0045] Example 4 In practical implementation, a log collector captures the amount of data from data query requests received by processing nodes within a preset accounting period. The log collector is deployed as a daemon process in the operating system user space of each processing node, listening to the access log files output by the data service process on the processing node. The format of the access log file is determined by the log collector's parsing configuration. In one implementation, each line of the access log corresponds to one data query request and includes the following fields: request arrival timestamp, request source business system identifier, request operation type, requested data volume in bytes, number of rows in the returned dataset, number of columns in the returned dataset, response time in milliseconds, and null value rate of the returned data fields. The log collector uses a preset accounting period as the statistical window. The preset accounting period is a time period configured by the cost accounting system, typically set to a granularity of one calendar month. The time period between the start and end timestamps constitutes a complete accounting interval. At the end of the preset accounting period, the log collector filters out all records in the log file that fall within that time period and extracts the requested data volume field for each record. The requested data volume field represents the size of the request body when the business system sends a query request to the processing node, in bytes. The values ​​of the request data volume field in all the filtered records are summed up. The summation result is the call frequency, which represents the total amount of request data received by the processing node within the preset accounting period, in bytes.

[0046] In practice, for each data query request, the log collector parses the returned dataset row number field and returned dataset column number field of the same record in the access log. The returned dataset row number field represents the number of records in the result set returned to the business system after the processing node executes the query, and is an integer greater than or equal to zero. The returned dataset column number field represents the number of fields contained in the result set, and is an integer greater than or equal to zero. Multiplying the number of returned dataset rows by the number of returned dataset columns yields the single call output value, which measures the number of data units produced by a single query operation, in units of data units. The single call output values ​​of all data query requests within a preset accounting period are summed, and the sum is the total call output value, which represents the total number of data units output by the processing node to the downstream business system within the preset accounting period.

[0047] In practical implementation, the call result status also includes the response time of the data query request and the null value rate of the returned data fields. The response time field is read directly from the access log, representing the time taken by the processing node from receiving the query request to returning the complete result set, in milliseconds. The null value rate field is also read from the access log, representing the proportion of fields with null values ​​in the returned result set out of all fields, recorded as a floating-point number, ranging from 0 to 1. Before calculating the total call output value by accumulating the output values ​​of a single call, a pass / fail check is performed on the call result status of each data query request. The response time is compared with a preset threshold, which is a pre-configured time limit. In this implementation, the preset threshold is set to 3000 milliseconds, based on the fact that the interactive query response time for online analysis processing is typically required to be no more than 3 seconds; queries exceeding 3 seconds are considered unable to meet the real-time requirements of the business. When the response time exceeds the preset threshold, the single call output value corresponding to that data query request is set to zero. Simultaneously, the null value rate of the returned data fields is compared with a preset null value ratio. The preset null value ratio is a pre-configured maximum acceptable null value rate. In this embodiment, the preset null value ratio is set to 0.5. This setting is based on the data quality governance specification, which stipulates that when more than half of the field values ​​in the returned data are null, the dataset is considered invalid output and cannot provide effective information for business decisions. When the null value rate of the returned data fields exceeds the preset null value ratio, the output value of a single call corresponding to this data query request is also set to zero. Only data query requests with a response time not exceeding a preset threshold and a null value rate of the returned data fields not exceeding the preset null value ratio are accumulated with the original calculated value.

[0048] After obtaining the call frequency and total call output, the total call output is divided by the call frequency, and the quotient is used as the average output per call. The physical meaning of the average output per call is the number of data units produced by the processing node for processing a unit of requested data byte, and the unit is data unit / byte.

[0049] To eliminate the impact of output dimensions caused by differences in business characteristics between different processing nodes, the average single-output is normalized to obtain a business gain coefficient. The normalization process is as follows: The average single-output values ​​of all processing nodes on the data link within the same preset calculation period are aggregated to form a numerical set. The maximum average single-output value from this numerical set is taken as the normalization benchmark value. If the normalization benchmark value is greater than zero, the average single-output of the current processing node is divided by the normalization benchmark value to obtain the business gain coefficient; if the normalization benchmark value is zero, the business gain coefficient of the current processing node is directly set to zero. The formula for calculating the business gain coefficient is: ; in, The business gain coefficient of the processing node is a dimensionless floating-point number with a value between 0 and 1, representing the relative level of the processing node's output per unit of requested data. This represents the average output per call of the current processing node. The average output per call is the quotient obtained by dividing the total output value of the current processing node by the call frequency, and the unit is data unit / byte. This represents the maximum average single-output value selected from the average single-output values ​​of all processing nodes in the data link. The value is obtained by collecting the average single output of all processing nodes within the same preset calculation cycle and taking the maximum value, with the unit being data units / byte. When When equal to zero, The value is zero. The larger the business gain coefficient, the more critical the processing node is in terms of business output efficiency.

[0050] Example 5 In practical implementation, the business gain coefficients of all processing nodes on the data link are summed to obtain the total gain. The data link is an ordered set of processing nodes through which data assets flow from the source generation node to the final consumption node. This set of processing nodes is maintained by a lineage topology data structure. At the end of each preset accounting cycle, the cost aggregation module traverses all processing nodes in the lineage topology, retrieving the business gain coefficient of each processing node within the preset accounting cycle from its metric storage record. The business gain coefficient is a dimensionless floating-point number representing the relative level of business output of the processing node within the preset accounting cycle. All retrieved business gain coefficient values ​​are summed, and the sum is the total gain, also a floating-point number, representing the total weight of business output of all processing nodes on the data link.

[0051] In practice, for each processing node on the data link, the business gain coefficient of that node is divided by the total gain, and the quotient is used as the allocation ratio for that node. The allocation ratio is calculated as follows: the business gain coefficient of the current processing node is read from the metric storage record, and then divided by the calculated total gain. If the total gain is zero, the allocation ratio of the current processing node is directly set to zero. The allocation ratio is a dimensionless floating-point number, ranging from 0 to 1, representing the weight of the current processing node's business output in the entire data link. The sum of the allocation ratios of all processing nodes equals 1.

[0052] In practice, the weighted compliance cost of each processing node is multiplied by its allocation ratio to obtain the allocated cost for that node. The weighted compliance cost is the resource cost value adjusted for compliance, calculated in the previous steps based on the node's original cost, compliance coefficient, and environmental hardening coefficient. The unit is a preset currency unit. The allocated cost is the result of weighting the weighted compliance cost according to the proportion of business output, also measured in a preset currency unit. The calculation method for the allocated cost is as follows: the weighted compliance cost of the current processing node is read from the cost storage record, the allocation ratio of the current processing node is read from the measurement storage record, and the weighted compliance cost and allocation ratio are multiplied together; the product is the allocated cost.

[0053] In practice, the allocated costs of all processing nodes are summed up, and the sum is the total cost metric for the data asset. The calculation process for the total cost metric is expressed by the following formula: ; in, This represents the total cost metric of the data asset. The total cost metric is a floating-point number in a preset currency unit. It represents the final cost of the data asset after compliance adjustments and business output allocation within a preset accounting period. This represents the traversal index, used to iterate through the number variables of all processing nodes in the data link. Take from 1 ; This indicates the total number of processing nodes in the data link. It is a positive integer, determined by the number of nodes in the kinship topology graph; Indicates the first The weighted compliance cost of each processing node, where the weighted compliance cost is a floating-point number in a preset currency unit; Indicates the first The business gain coefficient of each processing node is a dimensionless floating-point number with a value range between 0 and 1. It represents the relative level of the amount of data produced per unit of request after normalization. This represents the sum of gains, which is the sum of the service gain coefficients of all processing nodes on the data link. The sum of gains is a floating-point number. Indicates the first The allocation ratio for each processing node, where the allocation ratio is a dimensionless floating-point number, when... When the summation ratio is zero, the sharing ratio is zero. The summation symbol in the formula represents the summation of data links from the first to the second... The allocated costs of each processing node are summed sequentially, and the sum is the total cost metric for the data asset. The total cost metric comprehensively reflects the balance between the computing resources consumed by the data asset as it passes through each processing node in the data chain, compliance-related costs, and the value generated for the business system.

[0054] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for data asset cost measurement based on data lineage and compliance coefficient, characterized in that, In response to a data asset lineage topology change event, obtain a snapshot of the resource residence time and resource specifications of the data asset on each processing node in the data link; Based on the resource residency duration and the resource specification snapshot, allocate the original node cost to each processing node; Identify the security classification label triggered when the data asset flows through the data link, and query the preset compliance cost mapping table based on the security classification label to obtain the compliance coefficient of each processing node; Calculate the weighted compliance cost of each processing node based on the original cost of the node and the compliance coefficient; The frequency of calls to the data assets by downstream business systems and the status of the call results are obtained within a preset accounting period. The business gain coefficient of each processing node is calculated based on the call frequency and the status of the call results. The weighted compliance cost of each processing node is allocated according to the business gain coefficient to generate a total cost metric for the data asset.

2. The method for data asset cost measurement based on data consanguinity and compliance coefficient according to claim 1, characterized in that, The specific steps for obtaining snapshots of the resource residency duration and resource specifications of the data asset at each processing node in the data link are as follows: By attaching an eBPF probe to the kernel state of each processing node in the data link, the timestamps of the data assets entering and leaving each processing node in the form of data packets are captured, and the difference between the departure timestamp and the entry timestamp is used as the resource residence time. The eBPF probe collects the number of virtual CPU cores, memory bandwidth limit, and storage IOPS quota allocated to the processing node during the data asset residency period, and combines the number of virtual CPU cores, the memory bandwidth limit, and the storage IOPS quota into a resource specification snapshot.

3. The method for data asset cost measurement based on data consanguinity and compliance coefficient according to claim 2, characterized in that, The process of allocating the original cost of a node to each processing node based on the resource residency duration and the resource specification snapshot is as follows: The CPU cost component is obtained by multiplying the number of virtual CPU cores by the preset CPU unit price, the memory cost component is obtained by multiplying the memory bandwidth limit by the preset memory bandwidth unit price, and the storage cost component is obtained by multiplying the storage IOPS quota by the preset storage IOPS unit price. The unit time cost is obtained by adding the CPU cost component, the memory cost component, and the storage cost component, and the original cost of the node is obtained by multiplying the unit time cost by the resource residence time.

4. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 1, characterized in that, The process involves identifying security classification labels triggered when the data assets flow along the data link, querying a preset compliance cost mapping table based on the security classification labels, and obtaining the compliance coefficient for each processing node. Specifically: Before the data asset enters each of the processing nodes, the payload header of the data asset is parsed to extract the data classification identifier and data classification identifier carried by the data asset, and the data classification identifier and the data classification identifier are concatenated to form the security classification label; Using the security rating label as the index key, the encryption overhead weight and access audit weight that match the index key are found in the compliance cost mapping table. The compliance coefficient is obtained by adding the encryption overhead weight and the access audit weight.

5. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 1, characterized in that, The calculation of the weighted compliance cost of each processing node based on the original cost of the node and the compliance coefficient is specifically as follows: Obtain the security protection level of the physical host where the processing node is located, and query the preset compliance cost mapping table based on the security protection level to obtain the environmental hardening coefficient of the physical host; The compliance coefficient is multiplied by the environmental hardening coefficient to obtain the comprehensive compliance multiplier, and the original cost of the node is multiplied by the comprehensive compliance multiplier to obtain the weighted compliance cost.

6. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 1, characterized in that, The process of obtaining the call frequency and call result status of the data asset by downstream business systems within a preset accounting period, and calculating the business gain coefficient of each processing node based on the call frequency and call result status, specifically involves: The log collector captures the amount of data query requests received by each processing node within the preset accounting period, and the data amounts of the data query requests are accumulated to obtain the call frequency. The number of rows and columns of the returned dataset corresponding to each data query request is analyzed, and the product of the number of rows and the number of columns is used as the output value of a single call. The total call output value is obtained by summing all the output values ​​of a single call within the preset calculation period. The ratio of the total call output value to the call frequency is used as the average single output, and the normalized average single output is used as the business gain coefficient.

7. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 1, characterized in that, The step of allocating the weighted compliance cost of each processing node according to the business gain coefficient to generate a total cost metric for the data asset is as follows: The service gain coefficients of all processing nodes on the data link are summed to obtain the total gain, and the service gain coefficient of each processing node is divided by the total gain to obtain the sharing ratio of that processing node. The weighted compliance cost of each processing node is multiplied by the sharing ratio of that processing node to obtain the shared cost of that processing node. The shared costs of all processing nodes are then summed to obtain the total cost metric.

8. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 3, characterized in that, When the data asset leaves the processing node, the eBPF probe encapsulates the resource dwell time and the resource specification snapshot into a metadata message and sends it to the downstream processing node along with the data asset.

9. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 4, characterized in that, The encryption overhead weight and access audit weight in the compliance cost mapping table are dynamically adjusted according to the geographical location of the processing node through which the data asset passes, and the geographical location is read from the deployment configuration file of the processing node.

10. The data asset cost measurement method based on data lineage and compliance coefficient as described in claim 6, characterized in that, The call result status also includes the response time of the data query request and the field null value rate of the returned data. When the response time exceeds a preset threshold or the field null value rate exceeds a preset null value ratio, the single call output value of this call is set to zero.

Images