Machine learning-based multi-source heterogeneous data security governance method

Abstract

The present application belongs to the technical field of data security, and discloses a multi-source heterogeneous data security management method based on machine learning, which comprises the following steps: performing alignment on each data source record based on a cross-source identification mapping relationship to obtain a candidate association pair set; calculating an association confidence weight according to a cross-source co-occurrence rule, filtering the candidate association pair set, generating a cross-source association edge with a confidence weight from the retained candidate association pairs, and jointly constructing a heterogeneous association graph with each type of node and a homogenous relationship edge; respectively encoding each type of node to obtain a node code; performing message passing in a manner of participating in control by the association confidence weight along each edge in the graph with the node code as the starting point to obtain a fusion feature vector; and outputting a disposal action instruction from a disposal action prediction model with the fusion feature vector as the input. The original time sequence relationship of the cross-system records is retained, so that data fields with content features that are not sensitive but have high-risk cross-system behavior patterns can be correctly processed.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and more specifically, to a method for managing the security of multi-source heterogeneous data based on machine learning. Background Technology

[0002] Enterprises typically operate multiple independent data systems simultaneously, including business databases, access log systems, and file servers. Data security governance is a crucial means of ensuring the security of these data assets. Existing solutions usually use pre-configured data integration pipelines to periodically aggregate data from various systems to a unified platform, then analyze the content characteristics of data fields to identify sensitive fields and trigger corresponding action measures.

[0003] However, many data fields within enterprises, while not inherently sensitive in their content characteristics, are actually highly sensitive in practice. For example, customer ratings and account points fields stored as integer codes are difficult to match at the lexical level, yet they are often the primary targets in data breaches. The risks of these fields lie more in cross-system operations—the correlation between high-frequency queries in the database and subsequent batch exports from the file server is often more telling than the field content itself. Due to batch processing delays in data integration pipelines, the original temporal relationships of records from different systems are difficult to reconstruct after aggregation, making cross-system behavioral correlations difficult to utilize effectively with existing solutions. Moreover, even if cross-system behavioral correlations can be captured, they are not always reliable. Two operations by the same account close in time may have a substantial causal relationship, or they may simply be coincidental. Treating both cases together allows occasional noisy correlations to interfere with the identification of genuine risk signals, negatively impacting the final security assessment results. Summary of the Invention

[0004] To overcome the aforementioned problems in existing technologies, this invention proposes a multi-source heterogeneous data security governance method based on machine learning to address these issues.

[0005] This invention provides the following technical solution: Machine learning-based methods for multi-source heterogeneous data security governance include: Based on the pre-established cross-source identifier mapping relationship, cross-source alignment is performed on records in each data source to obtain a set of candidate association pairs; The association credibility weight is calculated based on the cross-source co-occurrence pattern of each candidate association pair. The candidate association pair set is filtered, and the retained candidate association pairs are used to generate cross-source association edges with credibility weights. The cross-source association edges are then combined with nodes of various types and homogeneous relationship edges to construct a heterogeneous association graph. Each type of node in the heterogeneous association graph is encoded to obtain the node code for each type of node; Starting with the node encoding, message passing is performed along the edges of the heterogeneous association graph in a manner that uses association confidence weights for control, thereby obtaining the fused feature vector of each data field node; Using the fused feature vector as input, a pre-trained action prediction model is used to output action instructions.

[0006] Preferably, the method for establishing the cross-source identifier mapping relationship includes: Scan the metadata of each data source and extract candidate identifier fields whose field names or field value formats conform to the entity identifier characteristics; For each candidate identifier field from each data source, calculate the percentage of the intersection of their value ranges. Two candidate identifier fields whose percentage of the intersection of their value ranges is not less than a preset threshold are identified as cross-source identifiers of the same entity. Record the conversion rules for converting the two identifier values ​​into a unified form that can be directly compared. By summarizing all the judgment results and corresponding conversion rules, the cross-source identifier mapping relationship is obtained.

[0007] Preferably, the cross-source alignment of records in each data source to obtain a candidate association pair set includes: The time difference distribution of confirmed cross-source related event pairs is statistically analyzed from historical data, and the first preset percentile of the time difference distribution is taken as the time window; Within the time window, after converting the entity identifier values ​​in each data source into a unified form according to the conversion rules in the cross-source identifier mapping relationship, records from different data sources with the same converted identifier value are identified as candidate association pairs. Summarize all candidate association pairs to obtain the candidate association pair set.

[0008] Preferably, the step of calculating the association confidence weight based on the cross-source co-occurrence pattern of each candidate association pair and filtering the candidate association pair set includes: For each candidate association pair, the association credibility weight is calculated based on the frequency of co-occurrence of the two data source records within the historical statistical period and the degree to which subsequent events stably follow the occurrence of the preceding event. The filtering threshold is adaptively determined based on the association confidence weight distribution of all candidate association pairs in the current processing batch, and candidate association pairs with association confidence weights not lower than the filtering threshold are retained.

[0009] Preferably, the frequency of co-occurrence is the ratio of the number of windows in which the candidate association pair co-occurs within the time window during the historical statistical period to the total number of observation windows; the degree of stable following occurrence is the ratio of the number of subsequent events occurring within a preset following time after the preceding event to the total number of preceding events; and the filtering threshold is the median of the association confidence weights of all candidate association pairs in the current processing batch.

[0010] Preferably, the step of constructing a heterogeneous association graph by combining the cross-source association edges with nodes of various types and edges with similar origins includes: Extract data field nodes, user account nodes, access event nodes, and file object nodes from each data source respectively; Based on the original data relationships between records in each data source, the same-source relationship edges are determined, including the initiation relationship edge between the user account node and the access event node, the involvement relationship edge between the access event node and the data field node, and the cross-source association between the access event node and the file object node represented by the cross-source association edge; Construct a heterogeneous association graph by combining all nodes, edges with the same origin, and cross-origin association edges with credibility weights.

[0011] Preferably, the step of encoding each type of node in the heterogeneous association graph to obtain the node encoding of each type of node includes: For data field nodes, the data field node is encoded into a fixed-dimensional vector based on the field name, field data type, and field value range statistical characteristics. This vector is then linearly projected onto the target dimension to obtain the node encoding of the data field node. For access event nodes, extract the key slots corresponding to the operation type, operation time, and operation result from the access log text and encode them into a fixed-dimensional vector. Then, project the vector linearly onto the target dimension to obtain the node encoding of the access event node. For user account nodes and file object nodes, their original attributes are encoded into fixed-dimensional vectors, which are then linearly projected onto the target dimension to obtain the corresponding node codes. The linear projection matrices of each type of node are independent of each other, and the target dimension is the same, thus obtaining the node encoding of each type of node.

[0012] Preferably, the step of executing message passing along the edges of the heterogeneous association graph, starting with the node encoding and using association confidence weights for control, to obtain the fused feature vector of each data field node includes: For each edge, the attention weights are calculated from the corresponding learnable projection matrix based on the triplet combination of source node type, edge type, and target node type. For cross-source association edges, a gating value is calculated based on the association credibility weight of the edge, and the product of the gating value and the attention weight is used as the comprehensive transmission weight; for same-source association edges, the attention weight is directly used as the comprehensive transmission weight. The source node's current node representation is weighted by the comprehensive transmission weight and used as the transmission message. The target node updates its own node representation based on the transmission messages from all neighboring nodes. This process is repeated several times, with the number of rounds determined by the prediction accuracy of the action on the validation set. For each data field node after the update is completed, calculate the mean of the confidence weights of all its direct cross-source incoming edges, and concatenate the mean with the updated node representation of the data field node to obtain the fused feature vector.

[0013] Preferably, the calculation of the gate value based on the association credibility weight of the edge includes: The source node's current node representation, the target node's current node representation, and the association credibility weight of the cross-source association edge are concatenated along the feature dimension, compressed into a scalar by a learnable linear transformation, and then mapped to the (0,1) interval to obtain the gating value.

[0014] Preferably, the training method for the action prediction model includes: Using the fused feature vectors of each data field node in historical data as training input, and the corresponding confirmed action categories in historical security governance records as initial training labels, the initial training labels of training samples whose scalars reflecting the overall credibility of cross-system related evidence in the fused feature vectors are lower than the preset credibility threshold are adjusted to action categories with higher security strength, and a training sample set is constructed. The training weight of each training sample is determined based on the scalar that reflects the overall credibility of cross-system correlation evidence in the fused feature vector of each training sample. The weighted sum of the classification loss of the action is calculated using the training weight of each training sample to form the confidence-weighted training loss. The action prediction model is trained based on the confidence-weighted training loss to obtain the action prediction model.

[0015] This invention provides a multi-source heterogeneous data security governance method based on machine learning, which has the following beneficial effects: This invention identifies cross-source identifier fields by the proportion of value domain intersection and records format conversion rules. Based on this, a time window is determined according to the time difference distribution of historically confirmed cross-system related event pairs. Within the time window, the identifier values ​​are converted into a unified form and cross-source alignment is performed to construct a candidate set of related pairs from the original records of each independent data system. This process directly affects the original records of each system, and the temporal sequence relationship between records is naturally preserved during the alignment process. The time interval of several minutes between database queries and file export operations can be directly captured and incorporated into subsequent graph structure modeling. This makes up for the shortcomings of existing solutions, which are affected by batch processing delays in data integration pipelines and have difficulty in effectively utilizing cross-system temporal behavioral associations. It enables data fields with insensitive content features but high-risk cross-system behavioral patterns to be correctly processed.

[0016] This invention calculates the co-occurrence frequency and time-series following rate of each candidate association pair within a historical statistical period and integrates them into a credibility weight. This weight is applied throughout the entire process of graph construction, message passing, and model training through dynamic threshold filtering, gated propagation control, feature appending, and training weighting. This allows for the differentiation between occasional co-occurrence and stable causal behaviors in cross-system operations of the same account that are close in time, thereby effectively suppressing the interference of occasional noise associations on risk assessment and handling decisions of data fields. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the multi-source heterogeneous data security governance method based on machine learning of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example:

[0020] Please see Figure 1 In this embodiment, the multi-source heterogeneous data security governance method based on machine learning includes: S1. Based on the pre-established cross-source identifier mapping relationship, perform cross-source alignment on the records in each data source to obtain a set of candidate association pairs; Methods for establishing cross-source identifier mapping relationships include: Scan the metadata of each data source and extract candidate identifier fields whose field names or field value formats conform to the entity identifier characteristics; For each candidate identifier field from each data source, calculate the percentage of the intersection of their value ranges. Two candidate identifier fields whose percentage of the intersection of their value ranges is not less than a preset threshold are identified as cross-source identifiers of the same entity. Record the conversion rules for converting the two identifier values ​​into a unified form that can be directly compared. By summarizing all the judgment results and corresponding conversion rules, the cross-source identifier mapping relationship is obtained.

[0021] Cross-source alignment is performed on records from each data source, resulting in a set of candidate association pairs including: The time difference distribution of confirmed cross-source related event pairs is statistically analyzed from historical data, and the first preset percentile of the time difference distribution is taken as the time window; Within the time window, after converting the entity identifier values ​​in each data source into a unified form according to the conversion rules in the cross-source identifier mapping relationship, records from different data sources with the same converted identifier value are identified as candidate association pairs. Summarize all candidate association pairs to obtain the candidate association pair set.

[0022] In this embodiment, it should be noted that the enterprise's internal business database, access log system, file server, and other data systems are independent of each other. The same operation account is often recorded with different field names and formats in different systems. When scanning the metadata of each data source, fields whose field names contain keywords such as "id," "user," "account," or "operator," or whose field value format conforms to common entity identifier formats such as account name, email, or IP address, are all considered as candidate identifier fields. The intersection ratio of the value ranges of each pair of candidate identifier fields is calculated, that is, the size of the intersection of the value sets of two fields divided by the smaller value of the size of the two value sets. In this example, the intersection ratio of the value sets of the operator_id field in the access log and the file_operator field in the file server log is 0.85. The preset ratio threshold is set to 0.3, and it is determined that the two identify the same type of entity. The selection of the preset ratio threshold needs to take into account both coverage and accuracy. 0.3 is a commonly used reference starting point in practice.

[0023] After determining the cross-source identifier, the conversion rules also need to be recorded. The value formats of the two candidate identifier fields may be the same or different. For example, one system may use the pure integer `user_id` to record accounts, while another system may use the string format `uid_ + integer` to record the same account. The conversion rules describe how to convert one format to another unified form that can be directly compared. When performing cross-source alignment, the identifier value of each record is converted according to the conversion rules before comparison. If the formats of the two fields are completely identical, the conversion rule is direct comparison, and no additional processing is required.

[0024] The time window is determined by statistical analysis of historical data. Historical data includes cross-system operation pairs that have been clearly identified as causally related through business audits or security incident reviews, such as a database query by an account and its subsequent file export. The time difference between the two operations in these confirmed event pairs is statistically analyzed, and a certain percentile of the time difference distribution is used as the time window. The selection of the percentile reflects the trade-off between covering truly related events and suppressing occasional co-occurrence noise: a higher percentile covers a larger proportion of truly related events, but the time window is also longer, increasing the probability of introducing occasional co-occurrence; a lower percentile has the opposite effect. For example, the 90th percentile means that 90% of historically truly related events can be captured within the corresponding time window. In this example, the 90th percentile of the time difference between historically confirmed event pairs is 3.2 minutes, so the time window is set to 5 minutes. If there are no historically confirmed event pairs when the system is initially deployed, the initial time window can be 10 times the typical operation response latency of the business system, and gradually updated as confirmed event pairs accumulate during operation.

[0025] It should be noted that when performing cross-source alignment, the entity identifier value of each record in each data source is first converted into a unified form according to the conversion rules of the records in the cross-source identifier mapping relationship. Then, the converted identifier values ​​of records from different data sources are compared within a time window. Records from different data sources with the same identifier value are identified as candidate association pairs and added to the candidate association pair set. For example, if the identifier values ​​of a query operation by analyze_01 on the credit_score field of the customer table and a subsequent file export operation are the same after conversion, and the time difference is within the time window, they are identified as a candidate association pair.

[0026] S2. Calculate the association credibility weight based on the cross-source co-occurrence rules of each candidate association pair, filter the candidate association pair set, generate cross-source association edges with credibility weights from the retained candidate association pairs, and construct a heterogeneous association graph together with each type of node and homo-source relationship edge. The association confidence weight is calculated based on the cross-source co-occurrence pattern of each candidate association pair, and the filtered candidate association pair set includes: For each candidate association pair, the association credibility weight is calculated based on the frequency of co-occurrence of the two data source records within the historical statistical period and the degree to which subsequent events stably follow the occurrence of the preceding event. The filtering threshold is adaptively determined based on the association confidence weight distribution of all candidate association pairs in the current processing batch, and candidate association pairs with association confidence weights not lower than the filtering threshold are retained.

[0027] The frequency of co-occurrence is the ratio of the number of windows in which the candidate association pair co-occurs within a time window in the historical statistical period to the total number of observation windows. The degree of stable following occurrence is the ratio of the number of times subsequent events occur within a preset following time after the preceding event to the total number of preceding events. The filtering threshold is the median of the association confidence weights of all candidate association pairs in the current processing batch.

[0028] The heterogeneous association graph is constructed by combining cross-source association edges with nodes of various types and edges with the same origin, including: Extract data field nodes, user account nodes, access event nodes, and file object nodes from each data source respectively; Based on the original data relationships between records in each data source, determine the same-source relationship edges, including the initiation relationship edge between the user account node and the access event node, the involvement relationship edge between the access event node and the data field node, and the cross-source association between the access event node and the file object node represented by the cross-source association edge; Construct a heterogeneous association graph by combining all nodes, edges with the same origin, and cross-origin association edges with credibility weights.

[0029] In this embodiment, it should be noted that not all record pairs in the candidate association pair set have a substantial causal relationship; some may only be occasional co-occurrences in time. The association confidence weight is calculated by combining historical statistics from two dimensions, with the historical statistical period being a rolling window of 30 days prior to the current moment. The first dimension is the frequency of co-occurrence: within 30 days, the timeline is divided into several observation windows according to the time window length. The number of windows in which accounts with similar identifier relationships generate both database access records and file operation records within the same observation window is counted, and this number is divided by the total number of observation windows to obtain the co-occurrence frequency. The second dimension is the degree of stable following occurrence: within 30 days, the number of times a file operation event follows a database access event within half the time window is counted, and this number is divided by the total number of database access events to obtain the time-series following rate. Taking half the time window as the following judgment interval is to verify whether subsequent events closely follow preceding events rather than being randomly distributed throughout the entire time window. The association confidence weight is calculated by combining the two dimensions. The calculation method can be the product of the two dimensions, a weighted average, or the smaller value, as long as the requirement is met that a high weight indicates a high value for both dimensions, and a low weight indicates a lower weight. In this example, there are 200 observation windows over 30 days. `analyst_01` showed both database access and file operation records in 144 windows, with a co-occurrence frequency of 0.72. `analyst_01` initiated 180 database accesses, of which 146 occurred within 2.5 minutes, resulting in a time-series follow-up rate of 0.81. The association confidence weight calculated using the product method is 0.583.

[0030] When filtering the candidate association pair set, the median of the association confidence weights of all current candidate association pairs is used as the filtering threshold, retaining candidate association pairs with weights not lower than this threshold. The median is used as the threshold because the activity level of the data source varies at different times. A fixed threshold might filter out most candidate pairs during low-activity periods and retain too many low-confidence pairs during high-activity periods, while the median always retains about half of the candidate pairs with relatively high confidence, thus exhibiting adaptability. In this example, the median association confidence weight of all current candidate association pairs is 0.41. The weight of the candidate association pair corresponding to analyze_01, 0.583, is higher than 0.41 and is retained. A cross-source association edge with a confidence weight of 0.583 is generated, connecting the node corresponding to this access event record and the file operation record.

[0031] The heterogeneous relationship graph consists of four types of nodes and two types of edges. The four types of nodes are extracted from various data systems: data field nodes are extracted from database metadata, with one node for each field, such as the `credit_score` field in the `customer` table; user account nodes are extracted from account records in various data systems, such as `analyst_01`; access event nodes are extracted from access logs, with one node for each access log record; and file object nodes are extracted from file server logs, with one node for each file operation record. Among the two types of edges, the origin-related edges are determined based on the original data relationships between records within each data system and are guaranteed to exist: an initiation relationship edge is established between a user account node and an access event node, indicating that the account initiated the corresponding access operation; an involvement relationship edge is established between an access event node and a data field node, indicating that the access operation involves the corresponding data field. Specifically, this is determined by parsing the SQL statements in the access logs to extract the table names and field names involved. For example, "SELECT customer.credit_score" can be parsed to reveal that the access event involves the `credit_score` field in the `customer` table, and an involvement relationship edge is established accordingly. Same-source edges reflect established record relationships within a single data system and do not carry confidence weights. Cross-source edges connect access event nodes and file object nodes, carrying confidence weights and reflecting the degree of confidence in a causal relationship between cross-system operations. All nodes, same-source edges, and cross-source edges with confidence weights are aggregated to construct a heterogeneous relationship graph.

[0032] S3. Encode each type of node in the heterogeneous association graph to obtain the node code for each type of node; Encode each type of node in the heterogeneous association graph separately, resulting in the following node codes for each type: For data field nodes, the data field node is encoded into a fixed-dimensional vector based on the field name, field data type, and field value range statistical characteristics. This vector is then linearly projected onto the target dimension to obtain the node encoding of the data field node. For access event nodes, extract the key slots corresponding to the operation type, operation time, and operation result from the access log text and encode them into a fixed-dimensional vector. Then, project the vector linearly onto the target dimension to obtain the node encoding of the access event node. For user account nodes and file object nodes, their original attributes are encoded into fixed-dimensional vectors, which are then linearly projected onto the target dimension to obtain the corresponding node codes. The linear projection matrices of each type of node are independent of each other, and the target dimension is the same, thus obtaining the node encoding of each type of node.

[0033] In this embodiment, it should be noted that the original attributes of the four types of nodes differ in type and quantity. Directly using them for calculation would result in inconsistent processing due to format discrepancies. Therefore, it is necessary to convert the original attributes of each type of node into numerical vectors of fixed dimensions, and then project them onto the same target dimension using their respective independent linear projection matrices to obtain the node encoding. Each type of node uses an independent linear projection matrix, preserving the semantic differences in the attributes of different types of nodes; the unified target dimension ensures that the representations of each type of node can directly participate in the computation during subsequent message passing calculations. In this embodiment, the target dimension is 128 dimensions.

[0034] The original attributes of a data field node include the field name, field data type, and field value range statistical characteristics. The field name is converted to a vector using character-level encoding, the field data type is represented using one-hot encoding, and the field value range statistical characteristics include scalars such as maximum, minimum, and the proportion of unique values. These three attribute codes are concatenated and linearly projected to 128 dimensions to obtain the node encoding of the data field node. Taking the `credit_score` field as an example, its field name encoding reflects the semantic characteristics of a rating field, the data type is an integer corresponding to the corresponding bit in the one-hot encoding, and the value range statistical characteristics reflect the rating distribution where its values ​​are concentrated within a finite range. These three attributes are concatenated and projected to 128 dimensions.

[0035] The original attributes of access event nodes come from the access log text, which is variable-length semi-structured content. Log text lengths and formats vary at different times for the same type of access operation, making direct processing with a fixed dimension impossible. To address this, a log template parsing method is employed. This method identifies the fixed template portion and the parameter portion that changes with the operation within the log text, extracting key slots. Taking the Drain algorithm as an example, this algorithm automatically identifies log templates through online clustering, eliminating the need for manual pre-definition and making it suitable for real-world scenarios with diverse log formats. The extracted key slots include operation type (SELECT, INSERT, UPDATE, DELETE, etc., represented by one-hot encoding), operation time (hours and minutes extracted, retaining the periodicity of time through sine and cosine encoding), and operation result (such as the number of rows returned, processed through logarithmic normalization). The encoded slots are concatenated and linearly projected to 128 dimensions. Taking the access log record "14:23:01, analyze_01, SELECT, customer.credit_score, ROWS:150" as an example, the operation type SELECT, time 14:23, and number of returned rows 150 are parsed. After encoding them respectively, they are concatenated and projected to obtain the 128-dimensional node encoding of the access event node.

[0036] The original attributes of user account nodes include account type (human account or service account), department code, etc.; the original attributes of file object nodes include character-level encoding of file path, file type (one-hot encoding of csv, xlsx, pdf, etc.), log-normalized value of file size, sine and cosine encoding of operation time, etc. Both types of nodes are concatenated according to their respective attributes and projected to 128 dimensions through independent linear projection matrices to obtain the corresponding node codes.

[0037] S4. Starting with node encoding, message passing is performed along the edges of the heterogeneous association graph in a way that uses association credibility weights for control, to obtain the fused feature vector of each data field node. Starting with node encoding, message passing is performed along the edges of the heterogeneous association graph using association confidence weights for control, resulting in the fused feature vectors of each data field node, including: For each edge, the attention weights are calculated from the corresponding learnable projection matrix based on the triplet combination of source node type, edge type, and target node type. For cross-source association edges, a gating value is calculated based on the association credibility weight of the edge, and the product of the gating value and the attention weight is used as the comprehensive transmission weight; for same-source association edges, the attention weight is directly used as the comprehensive transmission weight. The source node's current node representation is weighted by a comprehensive transmission weight and used as the transmission message. The target node updates its own node representation based on the transmission messages from all neighboring nodes. This process is repeated several times, with the number of rounds determined by the prediction accuracy of the action on the validation set. For each data field node after the update, calculate the mean of the confidence weights of all its direct cross-source incoming edges, and concatenate the mean with the updated node representation of that data field node to obtain the fused feature vector.

[0038] The gating value is calculated based on the association confidence weight of the edge, including: The source node's current node representation, the target node's current node representation, and the association credibility weight of the cross-source association edge are concatenated along the feature dimension, compressed into a scalar by a learnable linear transformation, and then mapped to the (0,1) interval to obtain the gating value.

[0039] In this embodiment, it should be noted that the node encoding of each node only reflects its own original attributes. The representation of the data field node at this time only includes content features such as field name, field type, and value range statistics, and does not yet include any cross-system operation behavior information. The role of message passing is to allow each node to gradually integrate information from its neighboring nodes along the edges of the graph, so that the representation vector of the data field node retains its own content features while accumulating operation behavior information passed from user account nodes and access event nodes. The node encoding of each node serves as the initial representation for message passing.

[0040] In each round of message passing, the attention weight of each edge in the graph is calculated. During calculation, based on the triple combination consisting of the source node type, edge type, and target node type of the edge, the current node representation of the source node is projected as a key vector and the current node representation of the target node is projected as a query vector from the pre-configured learnable projection matrix for that triple combination. The dot product of these two projection vectors is then normalized to obtain the attention weight. Different triple combinations use their own independent projection matrices, ensuring that information passing under different types of relationships calculates attention weights in a manner consistent with their respective semantics. The attention weight reflects the relevance between the information of a neighboring node and the target node under the same type of relationship. In this example, when the access event node passes a message to the credit_score data field node, the attention weight is calculated using the projection matrix corresponding to the triple combination "access event node - involved relationship edge - data field node".

[0041] There is a fundamental difference in the calculation of the comprehensive transmission weight between homologous edges and cross-source association edges. Homologous edges connect records within the same data system that are definitively related; the existence of the edge itself is reliable, and the attention weight is directly used as the comprehensive transmission weight. Cross-source association edges are dynamically inferred from temporal co-occurrence, and the inference itself is uncertain. The association credibility weight carries is precisely the quantification of this uncertainty. For cross-source association edges, a gating mechanism is introduced on the basis of the attention weight: the current node representation of the source node, the current node representation of the target node, and the association credibility weight of the edge are concatenated along the feature dimension, compressed into a scalar through a learnable linear transformation, and mapped to the (0,1) interval to obtain the gating value. The product of the gating value and the attention weight is used as the comprehensive transmission weight. When the association credibility weight is high, the gating value approaches 1, and cross-source information is fully transmitted; when the association credibility weight is low, the gating value approaches 0, and the information introduced by the cross-source association edge is suppressed, avoiding low-credibility noise associations from polluting the representation of data field nodes. In this example, the cross-source association edge between access event node E1 and file object node O1 carries a confidence weight of 0.583. The 128-dimensional current node representation of E1 and the 128-dimensional current node representation of O1 are concatenated with 0.583 to form a 257-dimensional vector. After linear transformation and interval mapping, the gating value is approximately 0.78. This value is multiplied by the attention weight to obtain the comprehensive transitive weight.

[0042] Each target node normalizes the messages transmitted from all neighboring nodes according to their respective comprehensive transmission weights, then sums them up. The sum is added to its own current node representation, and the node representation is updated using a non-linear activation function (such as ReLU), completing one round of message transmission. Normalizing the comprehensive transmission weights prevents nodes with a large number of neighbors from experiencing unstable representation vector values ​​due to excessive message accumulation. After several rounds of execution, the representation of the data field node gradually incorporates information from multi-hop paths. Taking the two rounds of message transmission in this embodiment as an example: After the first round, the representation of the credit_score data field node incorporates information from the access event node E1 directly connected via relational edges, including features such as the operation type, time, and number of returned rows corresponding to E1; after the second round, the representation of this node further incorporates information from E1's neighboring nodes, including information from the user account node analyzer_01 connected to E1 via an initiating relational edge, and information from the file object node O1 connected to E1 via a cross-source association edge (gating value 0.78), i.e., features of file export behavior. After two rounds of message passing, the 128-dimensional representation of the credit_score data field node has simultaneously encoded the content characteristics of the field and the cross-system operation behavior pattern of "file export immediately following high-frequency query".

[0043] The selection method for the number of message passing rounds is as follows: message passing is performed with 1, 2, and 3 rounds respectively, and the prediction accuracy of the corresponding action is evaluated on the validation set. The round with the highest accuracy is selected. The validation set is a snapshot of the graph constructed from the later part of the historical data, which does not overlap with the training set in time to ensure the objectivity of the evaluation results. In this example, the validation set accuracy is highest with 2 rounds, so 2 rounds are selected.

[0044] The fused feature vector consists of two concatenated parts: a 128-dimensional updated node representation after the data field node message passing ends, and a scalar reflecting the overall credibility of the cross-system association evidence for that data field. This scalar is calculated as follows: identify all access event nodes directly connected to the data field node in the graph via relational edges; then identify all cross-source association edges pointing to file object nodes on these access event nodes; and average the association credibility weights of these cross-source association edges to obtain the scalar. The implication of this calculation path is that if an access operation involving a data field is followed by a file export and the cross-source association has high credibility, it indicates sufficient evidence of cross-system operation behavior for that data field; if the credibility of such cross-source associations is generally low, it indicates insufficient cross-system evidence. In this example, the access event node E1 is directly connected to the credit_score data field node through a relational edge. E1 is connected to the file object node O1 through a cross-source association edge. The association confidence weight of this cross-source association edge is 0.583, with a mean of 0.583. When concatenated with the 128-dimensional update node representation, a 129-dimensional fused feature vector is obtained.

[0045] S5. Using the fused feature vector as input, the pre-trained action prediction model outputs action instructions.

[0046] Training methods for action prediction models include: Using the fused feature vectors of each data field node in historical data as training input, and the corresponding confirmed action categories in historical security governance records as initial training labels, the initial training labels of training samples whose scalars reflecting the overall credibility of cross-system related evidence in the fused feature vectors are lower than the preset credibility threshold are adjusted to action categories with higher security strength, and a training sample set is constructed. The training weight of each training sample is determined based on the scalar that reflects the overall credibility of cross-system correlation evidence in the fused feature vector of each training sample. The weighted sum of the classification loss of the action is calculated using the training weight of each training sample to form the confidence-weighted training loss. The action prediction model is trained using a confidence-weighted training loss to obtain the action prediction model.

[0047] In this embodiment, it should be noted that the action prediction model takes the 129-dimensional fused feature vector of the data field node as input and directly outputs the action category to be taken for that data field. The actions are categorized by security strength from low to high: no action required, restricted access, de-identification, encrypted storage, and blocked access, totaling five categories. Considering the characteristics of the input and output of this scheme, the action prediction model can adopt a multilayer perceptron structure: the input layer receives the 129-dimensional fused feature vector, which is progressively compressed through several fully connected layers. Each fully connected layer is followed by a ReLU activation function, resulting in a final output layer with a dimension of 5, corresponding to the five action categories. The probability distribution of each category is obtained through a softmax function, and the category with the highest probability is selected as the predicted action. In this example, a three-layer structure of 129-dimensional → 64-dimensional → 32-dimensional → 5-dimensional can be adopted. The first layer maps the 129-dimensional fused feature vector to 64-dimensional vectors, capturing the interaction between content features and cross-system behavioral features. The second layer compresses the 64-dimensional vectors to 32-dimensional vectors, further refining the discriminative features related to the decision-making process. The output layer maps the 32-dimensional vectors to 5-dimensional vectors and performs softmax normalization to obtain the probability of each action category. The last dimension of the input vector, the cross-system association evidence credibility scalar, is input into the model along with the first 128 dimensions of semantic features. By learning the correspondence between the credibility scalar and the action in the training data, the model naturally learns the decision-making pattern of tending to output higher security-strength action when the credibility scalar is low.

[0048] When constructing the training sample set, for samples whose scalar value reflecting the overall credibility of cross-system association evidence in the fused feature vector is lower than a preset credibility threshold, their training labels are adjusted to a higher-level action category with greater security strength. The preset credibility threshold can be set to 0.5. An example of the correspondence between two adjacent security strength levels is: no action required, access tightening, de-identification, encrypted storage, and access blocking, with adjustments made one step at each level. If the initial training label of a sample is already the highest level of access blocking, no adjustment is made, and the original label is retained. In this example, the credibility scalar value of a certain data field node's cross-source association evidence is 0.35, which is lower than 0.5, and its historical confirmed action is access tightening, so the training label is adjusted to de-identification. The reason for this is that when cross-source association evidence is insufficient, the actual risk faced by the field is uncertain. Adjusting the training label one level towards a more conservative action guides the model to tend to choose a stronger action when evidence is insufficient, reducing the risk of false negatives.

[0049] The training weights of each training sample are determined based on a scalar in the fused feature vector that reflects the overall credibility of cross-system association evidence. The higher the scalar, the greater the training weight, and the rate of increase of the weight with the scalar is higher than the linear growth rate. For example, the square of this scalar can be used as the training weight: a sample with a credibility scalar of 0.9 has a training weight of 0.81, while a sample with a credibility scalar of 0.3 has a training weight of only 0.09, a difference of 9 times, far greater than the 3-fold difference calculated using a linear ratio. Samples with sufficient cross-source association evidence contain relatively reliable cross-system behavioral signals in their fused feature vectors, and the correspondence between features and labels is relatively stable, making them suitable for being given higher training weights. Samples with insufficient cross-source association evidence contain more noise in their features, and giving them lower weights can prevent the model from learning incorrect patterns from noisy samples. The classification loss of the action for each training sample is weighted and summed according to the corresponding training weights to form a confidence-weighted training loss, which is used to train the action prediction model. The training set is constructed from the earlier part of the historical data, while the validation set is constructed from the later part of the historical data. The two sets do not overlap in time with the training set, thus avoiding the leakage of future information into the training process.

[0050] After model training is complete, the above steps are performed on new data field nodes to obtain fused feature vectors. These vectors are then input into the action prediction model. The output layer, after softmax normalization, selects the category with the highest probability as the action and generates the corresponding action instruction, which is then passed to the execution layer. In this example, after the 129-dimensional fused feature vector of the `credit_score` field is input into the model, the probability of encrypted storage is the highest among the probabilities of each category in the output layer. This generates an encrypted storage instruction, which is then passed to the execution layer. However, if only the field content features are relied upon, this field might only be judged as having tightened permissions due to the lack of obvious lexical features.

[0051] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0052] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

[0053] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-source heterogeneous data security governance method based on machine learning, characterized in that, include: Based on the pre-established cross-source identifier mapping relationship, cross-source alignment is performed on records in each data source to obtain a set of candidate association pairs; The association credibility weight is calculated based on the cross-source co-occurrence pattern of each candidate association pair. The candidate association pair set is filtered, and the retained candidate association pairs are used to generate cross-source association edges with credibility weights. The cross-source association edges are then combined with nodes of various types and homogeneous relationship edges to construct a heterogeneous association graph. Each type of node in the heterogeneous association graph is encoded to obtain the node code for each type of node; Starting with the node encoding, message passing is performed along the edges of the heterogeneous association graph in a manner that uses association confidence weights for control, thereby obtaining the fused feature vector of each data field node; Using the fused feature vector as input, a pre-trained action prediction model is used to output action instructions.

2. The multi-source heterogeneous data security governance method based on machine learning according to claim 1, characterized in that, The method for establishing the cross-source identifier mapping relationship includes: Scan the metadata of each data source and extract candidate identifier fields whose field names or field value formats conform to the entity identifier characteristics; For each candidate identifier field from each data source, calculate the percentage of the intersection of their value ranges. Two candidate identifier fields whose percentage of the intersection of their value ranges is not less than a preset threshold are identified as cross-source identifiers of the same entity. Record the conversion rules for converting the two identifier values ​​into a unified form that can be directly compared. By summarizing all the judgment results and corresponding conversion rules, the cross-source identifier mapping relationship is obtained.

3. The multi-source heterogeneous data security governance method based on machine learning according to claim 2, characterized in that, The cross-source alignment performed on records from each data source yields a candidate set of association pairs, including: The time difference distribution of confirmed cross-source related event pairs is statistically analyzed from historical data, and the first preset percentile of the time difference distribution is taken as the time window; Within the time window, after converting the entity identifier values ​​in each data source into a unified form according to the conversion rules in the cross-source identifier mapping relationship, records from different data sources with the same converted identifier value are identified as candidate association pairs. Summarize all candidate association pairs to obtain the candidate association pair set.

4. The multi-source heterogeneous data security governance method based on machine learning according to claim 3, characterized in that, The step of calculating association confidence weights based on the cross-source co-occurrence patterns of each candidate association pair, and filtering the candidate association pair set, includes: For each candidate association pair, the association credibility weight is calculated based on the frequency of co-occurrence of the two data source records within the historical statistical period and the degree to which subsequent events stably follow the occurrence of the preceding event. The filtering threshold is adaptively determined based on the association confidence weight distribution of all candidate association pairs in the current processing batch, and candidate association pairs with association confidence weights not lower than the filtering threshold are retained.

5. The multi-source heterogeneous data security governance method based on machine learning according to claim 4, characterized in that, The frequency of co-occurrence is the ratio of the number of windows in which the candidate association pair co-occurs within the time window during the historical statistical period to the total number of observation windows. The degree of stable following occurrence is the ratio of the number of times subsequent events occur within a preset following time after the preceding event to the total number of preceding events. The filtering threshold is the median of the association confidence weights of all candidate association pairs in the current processing batch.

6. The multi-source heterogeneous data security governance method based on machine learning according to claim 4, characterized in that, The step of constructing a heterogeneous association graph by combining the cross-source association edges with nodes of various types and homo-source relationship edges includes: Extract data field nodes, user account nodes, access event nodes, and file object nodes from each data source respectively; Based on the original data relationships between records in each data source, the same-source relationship edges are determined, including the initiation relationship edge between the user account node and the access event node, the involvement relationship edge between the access event node and the data field node, and the cross-source association between the access event node and the file object node represented by the cross-source association edge; Construct a heterogeneous association graph by combining all nodes, edges with the same origin, and cross-origin association edges with credibility weights.

7. The multi-source heterogeneous data security governance method based on machine learning according to claim 6, characterized in that, The process of encoding each type of node in the heterogeneous association graph to obtain the node encoding for each type of node includes: For data field nodes, the data field node is encoded into a fixed-dimensional vector based on the field name, field data type, and field value range statistical characteristics. This vector is then linearly projected onto the target dimension to obtain the node encoding of the data field node. For access event nodes, extract the key slots corresponding to the operation type, operation time, and operation result from the access log text and encode them into a fixed-dimensional vector. Then, project the vector linearly onto the target dimension to obtain the node encoding of the access event node. For user account nodes and file object nodes, their original attributes are encoded into fixed-dimensional vectors, which are then linearly projected onto the target dimension to obtain the corresponding node codes. The linear projection matrices of each type of node are independent of each other, and the target dimension is the same, thus obtaining the node encoding of each type of node.

8. The multi-source heterogeneous data security governance method based on machine learning according to claim 7, characterized in that, The process of executing message passing starting with the node encoding and using association confidence weights to control the process along the edges of the heterogeneous association graph, resulting in the fused feature vectors of each data field node, includes: For each edge, the attention weights are calculated from the corresponding learnable projection matrix based on the triplet combination of source node type, edge type, and target node type. For cross-source association edges, a gating value is calculated based on the association credibility weight of the edge, and the product of the gating value and the attention weight is used as the comprehensive transmission weight; for same-source association edges, the attention weight is directly used as the comprehensive transmission weight. The source node's current node representation is weighted by the comprehensive transmission weight and used as the transmission message. The target node updates its own node representation based on the transmission messages from all neighboring nodes. This process is repeated several times, with the number of rounds determined by the prediction accuracy of the action on the validation set. For each data field node after the update is completed, calculate the mean of the confidence weights of all its direct cross-source incoming edges, and concatenate the mean with the updated node representation of the data field node to obtain the fused feature vector.

9. The multi-source heterogeneous data security governance method based on machine learning according to claim 8, characterized in that, The calculation of the gate value based on the association credibility weight of the edge includes: The source node's current node representation, the target node's current node representation, and the association credibility weight of the cross-source association edge are concatenated along the feature dimension, compressed into a scalar by a learnable linear transformation, and then mapped to the (0,1) interval to obtain the gating value.

10. The multi-source heterogeneous data security governance method based on machine learning according to claim 8, characterized in that, The training method for the action prediction model includes: Using the fused feature vectors of each data field node in historical data as training input, and the corresponding confirmed action categories in historical security governance records as initial training labels, the initial training labels of training samples whose scalars reflecting the overall credibility of cross-system related evidence in the fused feature vectors are lower than the preset credibility threshold are adjusted to action categories with higher security strength, and a training sample set is constructed. The training weight of each training sample is determined based on the scalar that reflects the overall credibility of cross-system correlation evidence in the fused feature vector of each training sample. The weighted sum of the classification loss of the action is calculated using the training weight of each training sample to form the confidence-weighted training loss. The action prediction model is trained based on the confidence-weighted training loss to obtain the action prediction model.

Images