Data flow monitoring method and system based on large model

Abstract

The invention provides a data flow monitoring method and system based on a large model, and the method comprises the steps: obtaining a data flow record set generated by a to-be-monitored system in a continuous operation period, carrying out the correlation path construction of the data flow record set, generating a data flow topological graph containing a node interaction relation and a time sequence dependency relation, and carrying out the correlation path construction of the data flow record set; calling a pre-trained circulation behavior analysis large model to perform node sequence pattern recognition on the data circulation topological graph, and generating behavior abnormal confidence and abnormal pattern labels of each node in the data circulation topological graph; and according to the abnormal behavior confidence and the abnormal mode label, screening an abnormal interaction node cluster in the data flow topological graph. According to the method, relevance between abnormal nodes and time sequence relevance are considered, missing detection or false detection is avoided, and the reliability of the monitoring effect is improved.

Description

Technical Field

[0001] The present invention relates to the fields of data monitoring and machine learning, and in particular to a data flow monitoring method and system based on a large model. Background Art

[0002] With the rapid development of information technology, data flow monitoring can provide real-time or offline monitoring of the data flow process within a system, including transmission, processing, and storage between different processing nodes, to identify potential abnormal flow behavior and ensure the security and compliance of data transmission. Currently, this is usually done by collecting flow log data generated by the system, extracting information about the source and target nodes of the data transmission, constructing a simple node connection relationship map, and performing anomaly detection on flow behavior based on preset rules (such as illegal access rules between fixed nodes) or traditional machine learning models (such as classifiers based on statistical features). The final output is a list of abnormal nodes or alarm information. However, on the one hand, the collected flow logs are mostly unstructured data that simply record the transmission path, lacking the correlation and integration of key information such as the identity of the data subject and the specific operation behavior type, resulting in insufficient basic data dimensions for monitoring and analysis; on the other hand, the constructed node connection relationship graphs are mostly static structures, which can only represent the spatial interaction relationship between nodes, and cannot reflect the time sequence and dynamic dependency relationship of flow events, making it difficult to capture the temporal evolution law of data flow; at the same time, anomaly detection relies on manually preset rules or traditional models, making it difficult to automatically learn the implicit behavior patterns in complex flow scenarios, and the accuracy of identifying diversified and patterned anomalies is limited, and anomaly screening is mostly targeted at single isolated nodes, ignoring the correlation and temporal correlation between abnormal nodes, which can easily lead to missed detection or false detection, affecting the reliability of monitoring results. Summary of the Invention

[0003] The present invention provides a data flow monitoring method and system based on a large model.

[0004] In the first aspect, an embodiment of the present invention provides a data flow monitoring method based on a large model, comprising: obtaining a set of data flow records generated by the system to be monitored during a continuous operation period, the data flow record set comprising multiple groups of flow event units with unique time series tags; performing association path construction on the data flow record set to generate a data flow topology map comprising node interaction relationships and time series dependencies, wherein each node in the data flow topology map corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a time series tag; calling a pre-trained flow behavior analysis large model to perform node sequence pattern recognition on the data flow topology map to generate a behavior anomaly confidence and anomaly pattern label for each node in the data flow topology map; screening an abnormal interaction node cluster in the data flow topology map according to the behavior anomaly confidence and anomaly pattern label, wherein the abnormal interaction node cluster comprises an abnormal node identifier, an abnormal start time series tag and an abnormal behavior feature description.

[0005] In a second aspect, an embodiment of the present invention provides a computer system, comprising: a memory storing a computer program; and a processor for loading the computer program to implement the large model-based data flow monitoring method as described above.

[0006] The data flow monitoring method based on a large model provided by the present invention obtains a set of data flow records generated by the monitored system during a continuous operation period. The data flow record set includes multiple groups of flow event units with unique time series marks. Each flow event unit is composed of a data subject identifier, an initiating node identifier, a receiving node identifier and an operation behavior type. Through this multi-dimensional structured recording method, isolated flow events can be converted into basic data units with a data-node-behavior-time association relationship, avoiding the simple log limitation of only recording the transmission path in conventional data flow monitoring, providing comprehensive and structured input for subsequent in-depth analysis, and improving the basic data quality of data flow monitoring. After acquiring a set of data flow records, an association path is constructed and processed to generate a data flow topology map containing node interactions and temporal dependencies. Each node in this map corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a time series tag. By embedding the time series tag into the directed edge attributes of the topology map, the map not only represents the static interaction structure between nodes, but also dynamically reflects the temporal order and dependency relationships of flow events. This overcomes the design limitation of conventional static maps that cannot reflect temporal dynamics and improves the accuracy of the representation of the spatiotemporal evolution of data flow behavior. A pre-trained flow behavior analysis model is then called to perform node sequence pattern recognition on the data flow topology map, generating a behavioral anomaly confidence score and anomaly pattern label for each node. The model has the ability to deeply model complex structured data and long time series, and can automatically learn the implicit behavioral patterns of nodes in spatiotemporal interactions. Compared with conventional detection methods that rely on manual rules or traditional machine learning models, it can more accurately capture complex and diverse abnormal behavior characteristics, improving the accuracy of anomaly identification. The abnormal interaction node clusters in the data flow topology map are screened based on the behavioral anomaly confidence and abnormal pattern labels. The clusters contain abnormal node identifiers, abnormal start time sequence marks, and abnormal behavior feature descriptions. The abnormal nodes are clustered into clusters through abnormal pattern labels and associated time sequence marks. This can capture the group nature and temporal correlation of anomalies, avoid the limitations of single-point isolated detection, and reduce missed detections and false detections of patterned anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a flow chart of a data flow monitoring method based on a large model provided by an embodiment of the present invention.

[0008] Figure 2 It is a schematic diagram of the composition of a computer system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] See also Figure 1 , is a flow chart of a data flow monitoring method based on a large model provided by an embodiment of the present invention. The method can be executed by a computer system and specifically includes the following steps: Step S100: Obtain a set of data flow records generated by the monitored system during a continuous operation period. The data flow record set contains multiple groups of flow event units with unique time sequence marks. Each flow event unit consists of a data subject identifier, an initiating node identifier, a receiving node identifier, and an operation behavior type.

[0010] The system to be monitored is a system whose data flow needs to be monitored. The continuous operation period refers to the period of time when the system runs uninterruptedly (not limited). During this period, the system will generate various data flow activities. The data flow record set is a summary of the records of these data flow activities, which contains multiple groups of flow event units. The unique time sequence marker is a time-sequential identifier assigned to each flow event unit, which is used to distinguish flow events occurring at different times. The data subject identifier is used to identify the subject of the data involved in the data flow, such as a specific data set, file, etc.; the initiating node identifier refers to the node that initiates the data flow operation, which may be a server, device or program module in the system; the receiving node identifier is the target node that receives the data flow; the operation behavior type indicates the operation performed by the data flow, such as data transmission, data replication, data deletion, etc.

[0011] The method for obtaining a collection of data flow records can be determined based on the specific architecture and data logging mechanism of the system to be monitored. For example, if the system to be monitored is a distributed system where each node has its own logging function, then the log files of each node can be read to extract information related to the data flow, including the data subject identifier, the initiating node identifier, the receiving node identifier, the operation behavior type, and the corresponding timing marker. This information can then be integrated into a collection of data flow records.

[0012] Step S200: Construct an associated path for the data flow record set to generate a data flow topology map containing node interaction relationships and timing dependencies. Each node in the data flow topology map corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a timing mark.

[0013] Association path construction refers to analyzing the data flow relationship between each node based on the information in the data flow record set, and finding the path for data to flow between different nodes. The node interaction relationship reflects the data interaction between each data processing entity in the monitored system, such as which nodes have data transmission, replication and other operations. The timing dependency reflects the order in which data flow events occur, that is, the occurrence of a certain data flow event depends on certain previous events. The data flow topology map is a graphical representation, in which the nodes represent the data processing entities in the monitored system, such as servers, databases, applications, etc.; the directed edges represent the flow direction of data from the initiating node to the receiving node, and each directed edge is associated with a corresponding timing mark, which can be used to clearly identify the time when the data flow event occurred.

[0014] As an implementation manner, step S200 can be specifically implemented as the following steps S210 to S250: Step S210: Perform integrity check on the flow event units in the data flow record set, remove the flow event units with missing data subject identifiers, initiating node identifiers or receiving node identifiers, and obtain a set of flow event units that have passed the check.

[0015] The integrity check is performed to ensure the integrity of the flow event unit information in the data flow record set. This is because the data subject identifier, initiating node identifier, and receiving node identifier are key elements in determining the data flow path and related information. If this information is missing, subsequent analysis will be inaccurate. During the check, each flow event unit in the data flow record set is traversed to check whether its data subject identifier, initiating node identifier, and receiving node identifier exist. If a flow event unit is missing any of these identifiers, it will be removed from the set.

[0016] Step S220: Arrange the verified flow event unit sets in order according to the sequence of timing marks to generate a flow event sequence with continuous timing association, and the timing mark intervals of adjacent flow event units in the flow event sequence do not exceed the preset timing continuity threshold.

[0017] The order of the timing markers reflects the chronological order in which data flow events occur. Sorting the set of verified flow event units in this order allows the flow events to be arranged in chronological order. Continuous temporal association indicates temporal continuity between flow events, meaning that the time interval between adjacent flow event units cannot be too long. The preset temporal continuity threshold is a pre-set time interval value used to determine whether adjacent flow event units have a continuous temporal association.

[0018] Step S230: Extract the initiating node identifier and receiving node identifier of each flow event unit in the flow event sequence, and construct a node interaction pair list containing the initiating-receiving node correspondence relationship. Each element in the node interaction pair list consists of the initiating node identifier and the receiving node identifier and retains the timing mark of the corresponding flow event unit.

[0019] The initiating node identifier and receiving node identifier are key information describing the direction of data flow. By extracting these two identifiers from each flow event unit in the flow event sequence, we can construct interaction pairs between nodes. A node interaction pair list is a list of multiple node interaction pairs. Each node interaction pair contains the initiating node identifier and the receiving node identifier, and also retains the timing of the corresponding flow event unit, thus clearly indicating the time when the node interaction occurred. For example, in a data flow system, if there is a flow event unit in the flow event sequence with the initiating node identifier "NodeA", the receiving node identifier "NodeB", and the timing of "2025-07-10 10:00:00", then the node interaction pair list contains an element ("NodeA", "NodeB", "2025-07-10 10:00:00").

[0020] Step S240: Calculate the node association strength for the node interaction pair list, count the cumulative frequency of the same node interaction pair appearing in the flow event sequence, and generate a node interaction pair frequency distribution table. The node interaction pair frequency distribution table includes the node interaction pair identifier and the corresponding cumulative frequency.

[0021] Node association strength reflects the frequency of data flow interactions between two nodes. This association strength can be quantified by calculating the cumulative frequency of the same node interaction pair appearing in a flow event sequence. A node interaction pair identifier uniquely identifies each node interaction pair and is used to distinguish different node interaction pairs. A node interaction pair frequency distribution table is a table where each row contains a node interaction pair identifier and the cumulative frequency of that node interaction pair's occurrence in the flow event sequence. For example, if the node interaction pair ("NodeA", "NodeB") appears five times in the node interaction pair list, the node interaction pair frequency distribution table would contain a row ("NodeA-NodeB", 5), where "NodeA-NodeB" is the node interaction pair identifier and 5 is the cumulative frequency. By generating a node interaction pair frequency distribution table, you can intuitively understand the frequency of data flow between nodes.

[0022] As an implementation manner, step S240 can be specifically implemented as the following steps S241 to S245: Step S241: traverse each node interaction pair element in the node interaction pair list, and convert each node interaction pair element into a node interaction pair identifier in a character splicing format. The node interaction pair identifier is composed of an initiating node identifier and a receiving node identifier connected by a preset separator and maintains the initiating-receiving order.

[0023] The preset delimiter is a pre-defined character used to connect the initiating node identifier and the receiving node identifier to form a node interaction pair identifier. When traversing the node interaction pair list, each node interaction pair element is processed, and its initiating node identifier and receiving node identifier are connected using the preset delimiter. For example, if the preset delimiter is "-", the initiating node identifier of the node interaction pair element is "NodeA", and the receiving node identifier is "NodeB", the converted node interaction pair identifier is "NodeA-NodeB". Maintaining the initiating-receiving order ensures that the node interaction pair identifier accurately reflects the direction of data flow.

[0024] Step S242: Initialize an empty frequency statistics container, the key of the frequency statistics container is the node interaction pair identifier, and the value is an associated information group including the cumulative frequency and the time sequence marker list.

[0025] The frequency statistics container is a container for storing the frequency information of node interaction pairs, and is stored in the form of key-value pairs. The key is the node interaction pair identifier, which can be used to uniquely identify a node interaction pair; the value is an associated information group, which contains the cumulative frequency and a time stamp list. The cumulative frequency is used to record the number of times the node interaction pair appears in the flow event sequence, and the time stamp list stores the corresponding time stamp for each occurrence of the node interaction pair. Initializing an empty frequency statistics container means that there is no data in the container before statistics begin, and it will be gradually filled in according to the information in the node interaction pair list.

[0026] Step S243: Read the node interaction pair identifiers and the timing marks of the corresponding flow event units in the node interaction pair list in sequence. When the current node interaction pair identifier does not exist in the frequency statistics container, add the node interaction pair identifier to the frequency statistics container and set the cumulative frequency to 1, and set the timing mark list to a list containing the current timing mark. When the current node interaction pair identifier already exists in the frequency statistics container, increase the corresponding cumulative frequency by 1 and add the current timing mark to the timing mark list.

[0027] In an embodiment of the present application, the node interaction pair identifiers and corresponding timing marks in the node interaction pair list are read one by one. If the node interaction pair identifier does not yet exist in the frequency statistics container, it means that this is the first time that the node interaction pair has been counted, then this identifier is added to the frequency statistics container, and its cumulative frequency is initialized to 1, and the current timing mark is placed in the timing mark list as the only element. If the node interaction pair identifier already exists in the frequency statistics container, it means that the node interaction pair has appeared before, in which case its cumulative frequency is increased by 1, and the current timing mark is added to the corresponding timing mark list. For example, when processing the node interaction pair list, if the node interaction pair identifier "NodeA-NodeB" is encountered, if this identifier does not exist in the frequency statistics container, it is added and the cumulative frequency is set to 1, and the time marker list is set to ["2025-01-10 10:00:00"]; if it already exists, the cumulative frequency is increased by 1, and the time marker list is updated to ["2025-01-10 10:00:00", "2025-01-10 10:10:00"].

[0028] Step S244: After the traversal is completed, the key-value pairs in the frequency statistics container are converted into a structured table including a node interaction pair identification field, a cumulative frequency field, and a time sequence tag list field to obtain a node interaction pair frequency distribution table.

[0029] After traversing the node interaction pair list, the frequency statistics container stores the frequency information for all node interaction pairs. These key-value pairs are converted into a structured table, presenting the data in the container in the form of a table containing a node interaction pair identification field, a cumulative frequency field, and a time series marker list field. Each field corresponds to the corresponding information in the key-value pair in the container, for example, the node interaction pair identification field corresponds to the key, the cumulative frequency field corresponds to the cumulative frequency in the associated information group, and the time series marker list field corresponds to the time series marker list in the associated information group. Through this conversion, the data can be presented in a more intuitive form that is easier to process and analyze.

[0030] Step S245: Sort the node interaction pair frequency distribution table in descending order according to the cumulative frequency field to generate a sorted node interaction pair frequency distribution table. The sorted node interaction pair frequency distribution table is used to give priority to high-frequency node interaction pairs when constructing a data flow topology map to strengthen the core path representation.

[0031] High-frequency node interaction pairs indicate that the data flow between two nodes exceeds the preset number of interactions, meaning it is relatively frequent. When constructing a data flow topology map, prioritizing high-frequency node interaction pairs can highlight these core paths, allowing the topology map to better reflect the main trends and key paths of data flow. For example, in a sorted node interaction pair frequency distribution table, high-frequency node interaction pairs are ranked first. When constructing the topology map, the main directed edges and node connections are determined based on these high-frequency interaction pairs, and then low-frequency interaction pairs are considered. This strengthens the representation of core paths and helps analysts more clearly understand the key points and critical links of data flow.

[0032] Step S250: Based on the frequency distribution table of node interaction pairs and the time sequence arrangement results of the flow event sequence, construct a data flow topology map with nodes as vertices, node interaction pairs as directed edges, cumulative frequency as edge association strength, and time sequence marks as edge timing attributes. Each directed edge in the data flow topology map is also associated with a corresponding operation behavior type.

[0033] The frequency distribution table of node interaction pairs provides information on the frequency of interactions between nodes, while the chronological arrangement of flow event sequences reflects the temporal order of data flow events. Nodes in the data flow topology map are represented as vertices, corresponding to data processing entities in the monitored system. Node interaction pairs are used as directed edges to represent the direction of data flow from the initiating node to the receiving node. Edge association strength is expressed as cumulative frequency, reflecting the frequency of data flow between nodes. Edge timing attributes are represented by time tags, clarifying the time when the data flow event occurred. Each directed edge is also associated with a corresponding operation behavior type, such as data transmission or data replication, providing a more comprehensive description of data flow. For example, according to the frequency distribution table of node interaction pairs and the time series arrangement results, it is found that there is a high-frequency data transmission interaction between node "NodeA" and node "NodeB", and there are multiple data transmission operations at different times. Then there is a directed edge from "NodeA" to "NodeB" in the data flow topology map. The association strength of the edge is the cumulative frequency of the node interaction pair, the timing attribute of the edge is the timing mark corresponding to each data transmission operation, and the data transmission operation behavior type associated with the directed edge.

[0034] Step S300: Call the pre-trained flow behavior analysis model to perform node sequence pattern recognition on the data flow topology map, and generate the behavior anomaly confidence and abnormal pattern label of each node in the data flow topology map.

[0035] The pre-trained flow behavior analysis model is a pre-trained model that has learned a large number of data flow patterns and features. Node sequence pattern recognition uses this model to analyze the interaction sequences between nodes in the data flow topology map to identify possible patterns and regularities. The behavioral anomaly confidence level indicates the likelihood that a node's behavior is abnormal and is a probability value. The anomaly pattern label describes the type of abnormal behavior at that node, such as abnormal data transmission or abnormal data replication. For example, in a data flow topology map containing multiple nodes, the flow behavior analysis model analyzes the interaction sequence of each node and determines whether any anomalies exist based on the learned knowledge. If a node is determined to be anomaly, it will be assigned a behavioral anomaly confidence level, such as 0.8, indicating an 80% probability that the node's behavior is abnormal, and a corresponding anomaly pattern label, such as "abnormal data transmission."

[0036] As an implementation manner, step S300 can be specifically implemented as the following steps S310 to S350: Step S310: Perform graph structured encoding on the data flow topology graph, and convert the nodes, directed edges and corresponding edge association strengths and edge timing attributes in the data flow topology graph into low-dimensional dense structured feature vectors. The structured feature vectors include node attribute vectors, edge feature vectors and timing association vectors.

[0037] Graph structured coding converts the graphical data structure of the data flow topology graph into a vector form that can be processed by computers. A low-dimensional, dense structured feature vector is a vector representation with lower dimensionality and higher data density. This representation can reduce data redundancy and improve computational efficiency. Node attribute vectors represent node attributes, such as node type and function; edge feature vectors represent the characteristics of directed edges, such as edge correlation strength; and time series correlation vectors reflect the temporal properties of edges, such as timing labels. By converting nodes, directed edges, and their corresponding attributes into these vectors, the information in the data flow topology graph can be presented in a form more suitable for model processing. For example, the attributes of node "NodeA" in the data flow topology graph are converted into node attribute vectors, and the correlation strength and timing attributes of directed edges are converted into edge feature vectors and time series correlation vectors, respectively. These vectors are then combined into a structured feature vector.

[0038] As an implementation manner, step S310 can be specifically implemented as the following steps S311 to S317: Step S311: extract all node identifiers in the data flow topology map, construct a unique node set, and assign a globally unique node index code to each node identifier.

[0039] Node identifiers are used to uniquely identify each node in the data flow topology. Extracting all node identifiers can obtain a list containing all node identifiers. A unique node set is a set that does not contain duplicate node identifiers. By constructing this set, duplicate node identifiers can be removed to ensure that each node is processed only once. A globally unique node index code is a unique number assigned to each node identifier, which is used to conveniently reference and distinguish different nodes in subsequent processing. For example, in a data flow topology, there are node identifiers "NodeA", "NodeB", and "NodeA". The unique node set constructed after extraction is {"NodeA", "NodeB"}, with node index code 0 assigned to "NodeA" and node index code 1 assigned to "NodeB".

[0040] Step S312: Initialize the node basic embedding matrix based on the unique node set. The number of rows of the node basic embedding matrix is ​​the total number of nodes in the unique node set, and the number of columns is the preset feature embedding dimension. The matrix elements are generated by random initialization and dynamically updated in subsequent processing.

[0041] The preset feature embedding dimension is a pre-set value that determines the number of columns in the node base embedding matrix. The number of rows in the node base embedding matrix is ​​equal to the total number of nodes in the unique node set, so that each node has a corresponding row in the matrix. The matrix elements are generated by random initialization, which means that the elements in the matrix are randomly selected at the beginning. In subsequent processing, these elements will be dynamically updated according to the training and calculation of the model to better represent the characteristics of the node. For example, there are 3 nodes in the unique node set, and the preset feature embedding dimension is 10, then the node base embedding matrix is ​​a matrix with 3 rows and 10 columns. The elements in the matrix are randomly generated during initialization and will be continuously adjusted and updated during the subsequent graph structured encoding process.

[0042] Step S313: Extract all directed edges in the data flow topology graph, where each directed edge contains the source node index code, target node index code, edge association strength and edge timing attributes, and construct a directed edge feature list.

[0043] The source node index code and target node index code correspond to the node index codes of the starting and ending nodes of a directed edge, respectively. The edge association strength represents the association strength of the directed edge, and the edge timing attribute represents the timing tag corresponding to the directed edge. All directed edges are extracted and their related information is organized into a directed edge feature list to facilitate subsequent processing and analysis. For example, in the data flow topology graph, there is a directed edge from node "NodeA" to node "NodeB" with an association strength of 5 and a timing tag of "2025-01-10 10:00:00." The node index code of "NodeA" is 0, and the node index code of "NodeB" is 1. Therefore, the directed edge feature list will contain an element (0, 1, 5, "2025-01-10 10:00:00").

[0044] Step S314: normalizing and mapping the edge association strengths in the directed edge feature list, mapping the edge association strength values ​​to a preset feature value interval through linear scaling to obtain normalized edge strength values.

[0045] Normalization mapping unifies edge correlation strength values ​​across different ranges into a preset feature value range, making the correlation strengths of different edges comparable. Linear scaling scales edge correlation strength values ​​by calculating the proportional relationship between edge correlation strength values ​​and the preset feature value range. For example, if the preset feature value range is [0, 1] and the edge correlation strength values ​​in the directed edge feature list range from [1, 10], for a directed edge with an edge correlation strength of 5, linear scaling calculates a normalized edge strength of (5-1) / (10-1) ≈ 0.44, which is then mapped to the range [0, 1]. Normalization mapping can mitigate the impact of varying edge correlation strength value ranges on subsequent processing, improving model stability and accuracy.

[0046] Step S315: Extract timing features from the edge timing attributes in the directed edge feature list, and convert the timing mark into a timing feature vector containing timing interval features and timing trend features. The timing interval features are obtained by calculating the interval differences between adjacent timing marks, and the timing trend features are obtained by analyzing the change direction of continuous timing marks.

[0047] Time series tags indicate the time at which data flow events corresponding to directed edges occur. Extracting time series features from these tags can yield valuable information. Time series interval features reflect the time interval differences between adjacent data flow events and are obtained by calculating the time differences between adjacent time series tags. Time series trend features reflect the direction of change in consecutive time series tags, for example, whether time is gradually increasing or decreasing. For example, consider three consecutive time series tags: "2025-01-10 10:00:00," "2025-01-10 10:10:00," and "2025-01-10 10:20:00." Calculating the time interval differences between these tags yields a time series interval feature of [600, 600] (in seconds). Analyzing the direction of change in these consecutive time series tags reveals a gradual increase in time, resulting in an increasing time series trend feature. These features are combined into a time series feature vector for subsequent processing and analysis.

[0048] Step S316: Input the node basic embedding matrix, normalized edge strength value and time series feature vector into the graph structured coding model, perform multi-layer feature aggregation on the association relationship between nodes and directed edges through the graph convolutional network, generate an updated node attribute vector, and at the same time, splice the normalized edge strength value and time series feature vector according to the feature dimension to form an edge feature vector.

[0049] The graph structured coding model uses a graph convolutional network (GCN), a neural network designed to process graph-structured data. By inputting the node base embedding matrix, normalized edge strength values, and temporal feature vectors into the graph structured coding model, the GCN performs multi-layer feature aggregation on the associations between nodes and directed edges. Multi-layer feature aggregation involves extracting and integrating node and edge feature information through multiple convolution operations, enabling the model to better understand the structure and characteristics of the graph. During this process, the node base embedding matrix is ​​updated to generate an updated node attribute vector that more accurately represents the node's characteristics. Simultaneously, the normalized edge strength values ​​and temporal feature vectors are concatenated along the feature dimension to form an edge feature vector, integrating the edge association strength and temporal feature information. For example, the GCN performs multiple convolutions and updates on the node base embedding matrix based on the connection relationship between the node and the directed edge, ultimately generating an updated node attribute vector. The normalized edge strength values ​​and temporal feature vector are concatenated along the feature dimension to form a new vector, serving as the edge feature vector.

[0050] Step S317: Integrate the updated node attribute vector, edge feature vector, and time series feature vector into a feature representation of unified dimension to obtain a structured feature vector including the node attribute vector, edge feature vector, and time series association vector.

[0051] The goal of integrating the feature representation into a unified dimension is to represent and process the updated node attribute vectors, edge feature vectors, and time series feature vectors in the same dimension. This integration forms a structured feature vector containing the node attribute vectors, edge feature vectors, and time series association vectors. This allows the nodes, edges, and time series information of the data flow topology graph to be presented in a unified manner, facilitating the processing of large models for flow behavior analysis. For example, by concatenating the updated node attribute vectors, edge feature vectors, and time series feature vectors in a specific order and dimension, a new vector, called a structured feature vector, is generated. This vector contains comprehensive information about the data flow topology graph.

[0052] Step S320: extract the incoming edge feature sequence and outgoing edge feature sequence of each node in the continuous running period from the structured feature vector, and construct the behavior pattern sequence of each node. The behavior pattern sequence is composed of the incoming edge feature vector sequence and the outgoing edge feature vector sequence spliced ​​in the order of time sequence marks.

[0053] The incoming edge feature sequence is the feature vector sequence of all directed edges with a particular node as the target node, while the outgoing edge feature sequence is the feature vector sequence of all directed edges with that node as the source node. Extracting each node's incoming and outgoing edge feature sequences from the structured feature vector provides insights into the node's input and output during data flow. The behavioral pattern sequence, concatenated by the incoming and outgoing edge feature vector sequences in time-series order, reflects the node's behavioral pattern over a continuous period of time.

[0054] As an implementation manner, step S320 may be specifically implemented as the following steps S321 to S326: Step S321: Separate the edge feature vector set from the structured feature vector, and divide the edge feature vector set into an input edge feature subset and an output edge feature subset according to the source node index code and the target node index code of the directed edge. The input edge feature subset contains all edge feature vectors with the current node as the target node, and the output edge feature subset contains all edge feature vectors with the current node as the source node.

[0055] The source node index code and the target node index code are used to determine the direction and connection relationship of the directed edge. By dividing the edge feature vector set according to these two index codes, it can be divided into an input edge feature subset and an output edge feature subset. For a current node, the input edge feature subset contains the edge feature vectors of all directed edges pointing to the node, and the output edge feature subset contains the edge feature vectors of all directed edges starting from the node. For example, for the node "NodeA", its node index code is 0. In the edge feature vector set, according to the source node index code and the target node index code of the directed edge, all edge feature vectors with the target node index code of 0 are divided into the input edge feature subset, and all edge feature vectors with the source node index code of 0 are divided into the output edge feature subset. In this way, the input edge feature and output edge feature information of "NodeA" can be obtained respectively.

[0056] Step S322: adding an operation behavior type label to each edge feature vector in the input edge feature subset and the output edge feature subset, where the operation behavior type label is determined based on the operation behavior type in the flow event unit.

[0057] The operation behavior type label is used to identify the operation behavior type corresponding to the edge feature vector, such as data transmission, data replication, etc. According to the operation behavior type in the flow event unit, a corresponding label is added to each edge feature vector in the input edge feature subset and the output edge feature subset. For example, in the flow event unit, if the operation behavior type corresponding to a directed edge is data transmission, then the edge feature vector of the edge in the input edge feature subset or the output edge feature subset will be added with the operation behavior type label of "data transmission". By adding the operation behavior type label, you can more clearly understand the data flow operation represented by each directed edge, providing more information for the subsequent analysis of the node's behavior pattern.

[0058] Step S323: Preset the correspondence between the operation behavior type and the feature weight. The feature weight is configured according to the risk correlation of the operation behavior in the data flow. The edge feature vectors in the input edge feature subset and the output edge feature subset are matched with the feature weight according to the operation behavior type and weighted to generate weighted input edge feature vectors and weighted output edge feature vectors.

[0059] The preset correspondence between operation behavior types and feature weights is a pre-defined mapping table, where each operation behavior type corresponds to a feature weight. Feature weights are assigned based on the risk relevance of the operation behavior in the data flow, with operations with high risk relevance corresponding to larger feature weights. For edge feature vectors in the inbound and outbound feature subsets, a matching feature weight is found from the preset correspondence based on their operation behavior type label. The edge feature vector is then multiplied by the feature weight to obtain the weighted inbound and outbound feature vectors.

[0060] Step S324: Arrange the weighted incoming edge feature vectors according to the timing mark order of the edge timing attribute to generate an incoming edge feature vector sequence; arrange the weighted outgoing edge feature vectors according to the timing mark order of the edge timing attribute to generate an outgoing edge feature vector sequence.

[0061] The time series tags of edge timing attributes reflect the chronological order of the data flow events represented by the directed edge. Arranging the weighted incoming edge feature vectors and weighted outgoing edge feature vectors in this order allows the incoming and outgoing edge feature vector sequences to be arranged in chronological order. For example, consider three weighted incoming edge feature vectors with corresponding time series tags of "2025-01-10 10:00:00," "2025-01-10 10:10:00," and "2025-01-10 10:20:00." Arranging them in this order yields a sequence of incoming edge feature vectors, which more clearly illustrates the incoming edge characteristics of nodes at different times. Similarly, performing the same processing on the weighted outgoing edge feature vectors yields a sequence of outgoing edge feature vectors.

[0062] Step S325: perform time alignment on the incoming edge feature vector sequence and the outgoing edge feature vector sequence, and perform zero vector interpolation processing on the positions where the timing markers are missing.

[0063] Timing alignment ensures that the incoming edge feature vector sequence and the outgoing edge feature vector sequence are consistent in time. In practice, some timing markers may be missing. Zero vector interpolation is performed at these locations. Zero vector interpolation inserts a zero vector at the location where a timing marker is missing to ensure temporal continuity of the sequence. For example, if the incoming edge feature vector sequence does not have a corresponding feature vector at a certain point in time, a zero vector is inserted at that location to ensure sequence integrity and continuity. Timing alignment and zero vector interpolation make the incoming and outgoing edge feature vector sequences comparable in time, more accurately reflecting the node's behavior.

[0064] Step S326: The time-aligned incoming edge feature vector sequence and the outgoing edge feature vector sequence are sequentially concatenated according to the feature dimension to generate a behavior pattern sequence for each node.

[0065] The time-aligned incoming and outgoing feature vector sequences are concatenated along the feature dimension, forming a new sequence. This new sequence contains the incoming and outgoing feature information for the node during its continuous operation, arranged in chronological order to form a behavioral pattern sequence for each node. For example, concatenating the incoming and outgoing feature vector sequences along the feature dimension yields a longer vector sequence that comprehensively reflects the node's behavioral characteristics and data flow patterns at different times.

[0066] Step S330: Input the behavior pattern sequence into the graph attention encoder of the flow behavior analysis model, model the node interaction dependency in the behavior pattern sequence through the graph structure attention mechanism, and generate a node association pattern vector containing node context interaction information.

[0067] The graph attention encoder uses a graph-structured attention mechanism, which automatically learns the interaction dependencies between nodes and assigns different attention weights based on the importance of each node. After the behavior pattern sequence is input into the graph attention encoder, the mechanism models the node interaction dependencies in the behavior pattern sequence. By analyzing the connections and interactions between nodes, it identifies important relationships between nodes. This process generates a node association pattern vector containing node contextual interaction information. This vector not only includes the node's own feature information, but also its interaction information with neighboring nodes and contextual information. For example, for node "NodeA", the graph attention encoder generates a node association pattern vector based on its behavior pattern sequence and its interaction relationships with other nodes. This vector reflects the contextual interactions of "NodeA" in the entire data flow topology graph, helping the model better understand node behavior and anomalies.

[0068] As an implementation manner, step S330 may be specifically implemented as the following steps S331 to S338: Step S331: grouping and dividing the behavior pattern sequence according to the node index code to obtain a behavior pattern subsequence corresponding to each node. The behavior pattern subsequence includes all incoming edge feature vectors and outgoing edge feature vectors of the node in a continuous running period.

[0069] Node index coding is used to uniquely identify each node. By grouping and dividing the behavior pattern sequence according to the node index coding, the information in the behavior pattern sequence can be allocated to the subsequence corresponding to each node. The behavior pattern subsequence corresponding to each node contains all the incoming edge feature vectors and outgoing edge feature vectors of the node during the continuous running period, which can fully reflect the behavior pattern of the node. For example, the behavior pattern sequence contains the incoming and outgoing edge feature vector information of multiple nodes. By grouping them using node index coding, the behavior pattern subsequence corresponding to the node "NodeA" is obtained. This subsequence contains all the incoming and outgoing edge feature vectors of "NodeA" during the continuous running period, which facilitates the subsequent separate analysis and processing of each node.

[0070] Step S332: Standardize the length of the behavior pattern subsequence of each node. When the subsequence length exceeds the preset upper limit of the sequence length, truncate the head. When the subsequence length is lower than the preset lower limit of the sequence length, fill the tail with zero vectors to obtain a standardized behavior pattern subsequence with consistent length.

[0071] The preset upper and lower limits of the sequence length are pre-set length values ​​used to standardize the length of the behavior pattern subsequence. Length normalization is to ensure that the behavior pattern subsequences of each node have the same length, facilitating model processing and comparison. When the subsequence length exceeds the upper limit of the sequence length, its head is truncated, that is, some of the feature vectors with a preset number of features are removed to make the subsequence length reach the upper limit; when the subsequence length is lower than the lower limit of the sequence length, a zero vector is added to the tail of the subsequence to fill it up to the lower limit. For example, if the preset upper limit of the sequence length is 10 and the lower limit is 5, the length of the behavior pattern subsequence of node "NodeA" is 12, and its head will be truncated to retain the following 10 feature vectors; the length of the behavior pattern subsequence of node "NodeB" is 3, and 2 zero vectors are added to its tail to make its length reach 5, and finally a standardized behavior pattern subsequence with consistent length is obtained.

[0072] Step S333: Initialize the multi-layer attention parameter matrix of the graph attention encoder. The dimension of the attention parameter matrix matches the feature dimension of the standardized behavior pattern subsequence and each layer of the parameter matrix is ​​initialized independently.

[0073] Multi-layer attention parameter matrices are used to calculate attention weights between nodes. The dimensions of the attention parameter matrices match the feature dimensions of the standardized behavior pattern subsequence, ensuring that the feature vectors are correctly processed when calculating the attention weights. Independent initialization of each layer of parameter matrices means that the parameter matrices of each layer are randomly generated at the beginning and are not correlated with each other. For example, if the feature dimension of the standardized behavior pattern subsequence is 10, and the graph attention encoder has three layers of attention parameter matrices, then the dimensions of each layer of parameter matrices match 10, and each layer of parameter matrices is independently randomly initialized and continuously adjusted and optimized based on the input behavior pattern subsequence during subsequent training.

[0074] Step S334: For each node, extract its directly adjacent node set from the data flow topology map, where the directly adjacent node set includes all source nodes and target nodes that are directly connected to the node through directed edges.

[0075] The directly adjacent node set reflects the nodes that a node is directly connected to in the data flow topology. For each node, all source and target nodes directly connected to the node through directed edges are found in the data flow topology, and these nodes are combined into a directly adjacent node set. For example, in the data flow topology, node "NodeA" has two directed edges connecting to nodes "NodeB" and "NodeC", and there is a directed edge from node "NodeD" to "NodeA". Then the directly adjacent node set of "NodeA" is {"NodeB", "NodeC", "NodeD"}. By extracting the directly adjacent node set, we can understand the direct interaction relationship between nodes.

[0076] Step S335: concatenate the standardized behavior pattern subsequence of the current node with the standardized behavior pattern subsequence of each directly adjacent node in terms of feature dimension to generate an adjacency-related feature vector.

[0077] Concatenating the standardized behavior pattern subsequence of the current node with the standardized behavior pattern subsequences of each directly adjacent node along the feature dimension creates a new vector. This new vector contains the feature information of the current node and its directly adjacent nodes, reflecting the relationship between them. For example, concatenating the standardized behavior pattern subsequence of the current node "NodeA" with the standardized behavior pattern subsequence of its directly adjacent node "NodeB" along the feature dimension yields a longer vector, the adjacency association feature vector, which reflects the interaction and association between "NodeA" and "NodeB."

[0078] Step S336: Perform multi-level linear transformation on the adjacent associated feature vector through the multi-layer attention parameter matrix to generate a multi-layer attention score vector, and normalize each layer of the attention score vector through the normalized exponential function to obtain the adjacent node attention weight corresponding to each layer.

[0079] Multi-level linear transformation involves applying multiple linear transformations to the adjacent correlation feature vectors using a multi-level attention parameter matrix. Each transformation results in a new vector, which together form the multi-level attention score vector. A normalized exponential function (such as the Softmax function) is used to normalize the attention score vectors at each layer, ensuring that the values ​​of each score vector are within the range [0, 1] and that the sum of all elements is 1. This normalization process yields attention weights for adjacent nodes at each layer. These weights reflect the degree of attention the current node pays to its direct neighbors. For example, for the current node "NodeA" and its direct neighbors "NodeB" and "NodeC," multi-level linear transformations are applied to the adjacent correlation feature vectors using a multi-level attention parameter matrix to yield multi-level attention score vectors. Each score vector is then normalized using a normalized exponential function to obtain the attention weights for "NodeB" and "NodeC" at each layer.

[0080] Step S337: Perform weighted summation on the standardized behavior pattern subsequence of each directly adjacent node and the attention weight of the adjacent nodes of the corresponding layer to generate a multi-layer adjacent aggregated feature vector.

[0081] The standardized behavior pattern subsequence of each directly adjacent node is weighted and summed with the attention weight of the adjacent nodes in the corresponding layer. This means that the standardized behavior pattern subsequence of each directly adjacent node is multiplied by its corresponding attention weight, and then these results are added together to obtain a multi-layer adjacent aggregate feature vector. The multi-layer adjacent aggregate feature vector contains information about the current node's direct adjacent nodes. The weighted summation based on the attention weight can highlight the influence of important adjacent nodes. For example, for the current node "NodeA", its direct adjacent nodes are "NodeB" and "NodeC". The standardized behavior pattern subsequences of "NodeB" and "NodeC" are respectively multiplied by the attention weight of the corresponding layer, and the results are added together to obtain a multi-layer adjacent aggregate feature vector. This vector can comprehensively reflect the information of "NodeA"'s direct adjacent nodes and the importance between them.

[0082] Step S338: The standardized behavior pattern subsequence of the current node is fused element by element with the multi-layer adjacent aggregation feature vector, and the feature is nonlinearly transformed through a nonlinear activation function to generate a node association pattern vector containing node context interaction information.

[0083] The standardized behavior pattern subsequence of the current node is fused with the multi-layer adjacent aggregate feature vector element-by-element, that is, they are added at the element level to obtain a new vector. Non-linear activation functions (such as the ReLU function) are used to perform nonlinear feature transformation on this new vector, introducing nonlinear factors, enabling the model to learn more complex features and patterns. The node association pattern vector generated in this way contains the node's own feature information as well as its contextual interaction information with directly adjacent nodes, which can more comprehensively reflect the node's position and interaction in the graph. For example, the standardized behavior pattern subsequence of the current node "NodeA" is added element-by-element with the multi-layer adjacent aggregate feature vector, and then nonlinearly transformed using the ReLU function to obtain a node association pattern vector containing the contextual interaction information of "NodeA".

[0084] Step S340: Input the node association pattern vector into the time series modeling layer of the flow behavior analysis model, capture the dynamic change rules of the node association pattern vector in the time series dimension, and generate a time series enhancement pattern vector that strengthens the dynamic characteristics of the time series.

[0085] The time series modeling layer processes node association pattern vectors in the time series dimension. While node association pattern vectors contain contextual interaction information about nodes, further analysis is required to determine their dynamic temporal changes. The time series modeling layer models node association pattern vectors in the time series dimension, analyzing changes in the vectors at different time points to capture their dynamic patterns. For example, time series modeling methods such as recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) are used to process node association pattern vectors and learn their temporal dependencies and changing trends. This process generates enhanced time series pattern vectors that enhance temporal dynamics and better reflect the behavioral characteristics and changing patterns of nodes in the time dimension.

[0086] Step S350: Input the time series enhancement pattern vector into the anomaly recognition layer of the flow behavior analysis model, calculate the behavioral anomaly confidence of each node through the multi-layer perceptron network and probability normalization function, and determine the corresponding anomaly pattern label based on the confidence distribution result.

[0087] The anomaly identification layer determines whether a node exhibits abnormal behavior. The multilayer perceptron (MLP) network is a feedforward neural network composed of multiple layers of neurons that can learn complex nonlinear relationships. After the time series enhancement pattern vector is input into the MLP network, the network processes and analyzes it, outputting a prediction result. A probabilistic normalization function (such as the Softmax function) converts the MLP network's output into a probability value, resulting in a confidence score for each node's abnormal behavior. This confidence score indicates the likelihood that the node's behavior is abnormal. Based on the resulting confidence score distribution, a corresponding abnormal pattern label is determined. For example, if a node's abnormal behavior confidence score exceeds a preset threshold, it is assigned a corresponding abnormal pattern label, such as "abnormal data transmission" or "abnormal data replication," to clearly identify the type of abnormal behavior.

[0088] Step S400: Filter abnormal interaction node clusters in the data flow topology map according to the behavior anomaly confidence and abnormal pattern labels. The abnormal interaction node clusters include abnormal node identifiers, abnormal start time sequence markers, and abnormal behavior feature descriptions.

[0089] The confidence level of behavioral anomalies and the abnormal pattern label are the basis for determining whether a node is abnormal and the type of anomaly. Based on this information, the nodes in the data flow topology map are screened to find abnormal interaction node clusters. The abnormal interaction node cluster is a collection of multiple abnormal nodes, which contains an abnormal node identifier, which is used to uniquely identify each abnormal node; the abnormal start timing mark indicates the time when the abnormal node cluster begins to exhibit abnormal behavior; the abnormal behavior feature description is matched from the preset abnormal pattern feature library based on the abnormal pattern label, and describes the characteristics and manifestations of the abnormal behavior in detail. For example, by comparing the node's behavioral anomaly confidence level with the preset threshold, nodes with a confidence level exceeding the threshold are screened out, and nodes with the same abnormal pattern are grouped into an abnormal interaction node cluster based on their abnormal pattern labels. At the same time, the corresponding abnormal behavior feature description is obtained from the abnormal pattern feature library to provide analysts with comprehensive abnormal information.

[0090] As an implementation manner, step S400 can be specifically implemented as the following steps S410 to S460: Step S410: Obtain a preset abnormality confidence threshold, traverse each node in the data flow topology map, compare the node's behavior abnormality confidence with the abnormality confidence threshold, and when the behavior abnormality confidence is greater than or equal to the abnormality confidence threshold, mark the node as a potential abnormal node.

[0091] The preset anomaly confidence threshold is a pre-set probability value used to determine whether a node is likely to exhibit abnormal behavior. When traversing each node in the data flow topology, its behavior anomaly confidence score is compared with this threshold. If a node's behavior anomaly confidence score is greater than or equal to the anomaly confidence threshold, the node is likely to exhibit abnormal behavior and is marked as a potential anomaly node.

[0092] Step S420: collecting the node identifications, behavior anomaly confidences, and anomaly pattern labels of all potential abnormal nodes, and constructing a candidate list of potential abnormal nodes.

[0093] The node identifiers, behavior anomaly confidence scores, and anomaly pattern labels of all nodes marked as potential anomaly nodes are collected to form a list, known as the potential anomaly node candidate list. This list contains information about all nodes that may have abnormal behavior. For example, the node identifiers, behavior anomaly confidence scores, and anomaly pattern labels of all potential anomaly nodes are organized into a list according to a specific format, with each element in the list corresponding to the relevant information of a potential anomaly node.

[0094] Step S430: classify the potential abnormal nodes in the potential abnormal node candidate list into abnormal patterns, classify the potential abnormal nodes with the same abnormal pattern label into the same abnormal pattern category group, and generate an abnormal pattern category grouping set.

[0095] Abnormal pattern classification involves categorizing potential abnormal nodes according to their abnormal pattern labels, grouping nodes with the same abnormal pattern label. An abnormal pattern category grouping set is a set consisting of multiple abnormal pattern category groups, each of which contains potential abnormal nodes with the same abnormal pattern label. For example, in the list of potential abnormal node candidates, some nodes have the abnormal pattern label "abnormal data transmission" and are grouped into one abnormal pattern category group; others have the abnormal pattern label "abnormal data replication" and are grouped into another abnormal pattern category group. Ultimately, an abnormal pattern category grouping set containing multiple abnormal pattern category groups is generated.

[0096] As an implementation manner, step S430 may be specifically implemented as the following steps S431 to S437: Step S431: extracting abnormal pattern labels of all potential abnormal nodes in the potential abnormal node candidate list, and constructing an abnormal pattern label set, which includes all non-repeated abnormal pattern labels.

[0097] Extract the abnormal pattern labels of all potential abnormal nodes in the candidate list of potential abnormal nodes, collect these labels, remove duplicate labels, and construct an abnormal pattern label set. An abnormal pattern label set is a set without duplicate elements, where each label represents an abnormal pattern. For example, if the candidate list of potential abnormal nodes has multiple nodes with the abnormal pattern labels "abnormal data transmission" and "abnormal data replication", the abnormal pattern label set constructed after extracting these labels is {"abnormal data transmission", "abnormal data replication"}, which facilitates subsequent analysis of the correlation between abnormal patterns.

[0098] Step S432: Based on the preset abnormal pattern association rule library, analyze the association relationship between each abnormal pattern label in the abnormal pattern label set. The association relationship includes a causal association relationship and a concurrent association relationship. The causal association relationship indicates that the occurrence of one abnormal pattern may lead to the occurrence of another abnormal pattern. The concurrent association relationship indicates that the probability of two abnormal patterns occurring simultaneously in the same time window is higher than the preset probability threshold.

[0099] The preset anomaly pattern association rule base is a predefined set of rules that contains information about the associations between different anomaly patterns. Based on this rule base, the associations between the anomaly pattern labels in the anomaly pattern label set are analyzed. A causal relationship indicates that the occurrence of one anomaly pattern may trigger the occurrence of another, for example, "abnormal data transmission" may lead to "data integrity loss." A concurrent relationship indicates that the probability of two anomaly patterns occurring simultaneously within the same time window is high, exceeding a preset probability threshold.

[0100] Step S433: Construct an abnormal pattern association map based on the association relationship. Each node in the abnormal pattern association map corresponds to an abnormal pattern label, each edge corresponds to an association relationship, and the edge weight is the association strength value. The association strength value is calculated by analyzing the co-occurrence frequency of abnormal pattern labels in the historical data flow monitoring records.

[0101] The anomaly pattern association graph is used to display the associations between anomaly pattern labels. Each node corresponds to an anomaly pattern label, and each edge corresponds to an association, such as a causal relationship or a concurrent relationship. Edge weights represent the strength of the association, calculated by analyzing the co-occurrence frequency of anomaly pattern labels in historical data flow monitoring records. For example, if "abnormal data transmission" and "abnormal data replication" appear frequently together in historical data flow monitoring records, the edge weight between them will be larger, indicating a stronger association. By constructing an anomaly pattern association graph, you can intuitively visualize the associations and strengths between anomaly patterns.

[0102] Step S434: Perform community discovery on the abnormal pattern association graph to identify abnormal pattern label clusters with strong correlations, and divide the abnormal pattern labels in each cluster into a main abnormal pattern label and multiple sub-abnormal pattern labels. The main abnormal pattern label is the abnormal pattern label with the largest total correlation strength in the cluster.

[0103] For the anomaly pattern association graph, a community discovery algorithm (such as the Louvain algorithm) is used to identify clusters of anomaly pattern labels with strong correlations. Each cluster contains multiple anomaly pattern labels. The anomaly pattern label with the highest sum of correlation strengths is classified as the primary anomaly pattern label, and the remaining labels are classified as sub-anomaly pattern labels. For example, in an anomaly pattern label cluster, "abnormal data transmission" has the highest sum of correlation strengths, so it is classified as the primary anomaly pattern label. Other labels, such as "abnormal data replication," are classified as sub-anomaly pattern labels. This division provides a clearer understanding of the primary and secondary relationships and correlation structure between anomaly patterns.

[0104] Step S435: traverse each potential abnormal node in the potential abnormal node candidate list, read its abnormal pattern label, and determine whether the abnormal pattern label belongs to a main abnormal pattern label or a sub-abnormal pattern label in the abnormal pattern association graph.

[0105] When traversing the candidate list of potential abnormal nodes, the abnormal pattern label of each potential abnormal node is checked. The classification of the abnormal pattern label and the sub-abnormal pattern label in the abnormal pattern association map is used to determine which category the label belongs to. For example, for the potential abnormal node "NodeA", its abnormal pattern label is "abnormal data replication". This label is queried in the abnormal pattern association map to determine whether it is a main abnormal pattern label or a sub-abnormal pattern label.

[0106] Step S436: When the abnormal pattern label is a main abnormal pattern label, directly assign the corresponding potential abnormal node to the abnormal pattern category group named after the main abnormal pattern label; when the abnormal pattern label is a sub-abnormal pattern label, query the main abnormal pattern label to which it belongs, and assign the corresponding potential abnormal node to the abnormal pattern category group corresponding to the main abnormal pattern label.

[0107] Based on the judgment result, if the abnormal pattern label is a main abnormal pattern label, the corresponding potential abnormal node will be directly assigned to the abnormal pattern category group named after the main abnormal pattern label; if the abnormal pattern label is a sub-abnormal pattern label, the main abnormal pattern label to which it belongs will be queried, and then the potential abnormal node will be assigned to the abnormal pattern category group corresponding to the main abnormal pattern label. For example, the abnormal pattern label of the potential abnormal node "NodeA" is "abnormal data replication". After judging that it is a sub-abnormal pattern label and the main abnormal pattern label to which it belongs is "abnormal data transmission", "NodeA" will be assigned to the abnormal pattern category group named "abnormal data transmission". In this way, potential abnormal nodes can be accurately classified according to the association relationship of abnormal patterns, forming a reasonable abnormal pattern category grouping set.

[0108] Step S437: Collect all abnormal pattern category groups, each category group includes a main abnormal pattern label, a sub-abnormal pattern label list and a corresponding potential abnormal node list, and generate an abnormal pattern category grouping set.

[0109] Collect all classified and assigned abnormal pattern category groups. Each category group contains a main abnormal pattern label, a list of sub-abnormal pattern labels, and a corresponding list of potential abnormal nodes. For example, an abnormal pattern category group might have "abnormal data transmission" as the main abnormal pattern label, a list of sub-abnormal pattern labels like {"abnormal data replication"}, and a corresponding list of potential abnormal nodes like {"NodeA", "NodeB"}. Combining all such category groups creates an abnormal pattern category grouping set, which clearly demonstrates the distribution of potential abnormal nodes under different abnormal patterns.

[0110] Step S440: For each abnormal pattern category group, extract the abnormal start time sequence marks of all potential abnormal nodes in the group, arrange them in ascending order according to the sequence of the time sequence marks, and generate an abnormal time sequence arrangement sequence.

[0111] For each anomaly pattern category group, extract the anomaly start time series markers of all potential anomaly nodes within it. These markers record the time when each potential anomaly node began exhibiting abnormal behavior. Arrange these markers in ascending order, sorting them from smallest to largest, to generate an anomaly time series sequence. For example, in a category group with "abnormal data transmission" as the primary anomaly pattern marker, the anomaly start time series marker for the potential anomaly node "NodeA" is "2025-01-10 10:00:00," and the anomaly start time series marker for "NodeB" is "2025-01-10 10:10:00." Arranging these markers in ascending order generates the anomaly time series sequence ["2025-01-10 10:00:00," "2025-01-10 10:10:00"].

[0112] Step S450: Analyze the timing intervals of adjacent abnormal start timing marks in the abnormal timing arrangement sequence. When the timing interval is less than the preset cluster association threshold, determine that the corresponding potential abnormal node is a timing-associated abnormal node, and merge the timing-associated abnormal nodes into an abnormal node cluster. The abnormal node cluster contains node identifiers of multiple timing-associated abnormal nodes and cluster start timing marks and cluster end timing marks.

[0113] The timing interval refers to the time difference between two adjacent abnormal start timing marks in the abnormal timing arrangement sequence. The preset cluster association threshold is a pre-set time value used to determine whether adjacent potential abnormal nodes have temporal correlation. If the timing interval of adjacent abnormal start timing marks is less than the threshold, the corresponding potential abnormal node is considered to be a timing-correlated abnormal node, and there may be some kind of intrinsic connection between them, such as a series of abnormal behaviors caused by the same abnormal event. These timing-correlated abnormal nodes are merged into an abnormal node cluster, which contains the node identifiers of multiple timing-correlated abnormal nodes, as well as the cluster start timing mark (that is, the abnormal start timing mark of the node that has the earliest abnormality in the cluster) and the cluster end timing mark (that is, the abnormal start timing mark of the node that has the latest abnormality in the cluster).

[0114] As an implementation manner, step S450 can be specifically implemented as the following steps S451 to S457: Step S451: Initialize an empty abnormal node cluster container, set the current cluster sequence number to an initial value, and initialize the current cluster element list to empty.

[0115] The abnormal node cluster container stores the final abnormal node cluster information and is initially empty. The current cluster sequence number identifies different abnormal node clusters and is typically initialized to a value such as 0 or 1. The current cluster element list temporarily stores potential abnormal nodes in the abnormal node cluster being processed and is also initially empty. For example, before processing an abnormal timing sequence, an empty list is created as the abnormal node cluster container, the current cluster sequence number is set to 1, and an empty list is created as the current cluster element list to prepare for subsequent cluster partitioning operations.

[0116] Step S452: Read the first abnormal start timing mark in the abnormal timing arrangement sequence as the reference timing mark of the current cluster, and add the corresponding potential abnormal node to the current cluster element list.

[0117] The first anomaly start time marker is the earliest anomaly start time in the anomaly time sequence. It serves as the baseline time marker for the current cluster and is subsequently compared with other anomaly start time markers. The corresponding potential anomaly node is the potential anomaly node corresponding to this anomaly start time marker. This node is added to the current cluster element list as the first element. For example, if the anomaly time sequence is ["2025-01-10 10:00:00", "2025-01-10 10:10:00", "2025-01-10 10:20:00"], the first anomaly start time marker is "2025-01-10 10:00:00", and the corresponding potential anomaly node is "NodeA", then "2025-01-10 10:00:00" is used as the baseline time marker for the current cluster and "NodeA" is added to the current cluster element list.

[0118] Step S453: Read the next abnormal start timing mark in the abnormal timing arrangement sequence, and calculate the timing interval difference between the timing mark and the current cluster reference timing mark.

[0119] Arrange the sequence of abnormal timings, read the next abnormal start timing mark in sequence, and then calculate the time difference between the timing mark and the current cluster reference timing mark to obtain the timing interval difference.

[0120] Step S454: when the time series interval difference is less than a preset cluster association threshold, the corresponding potential abnormal node is added to the current cluster element list, and the baseline time series mark of the current cluster is updated to the time series mark.

[0121] If the calculated timing interval difference is less than the preset cluster correlation threshold, the potential anomaly node is temporally correlated with the nodes in the current cluster and is added to the current cluster element list. At the same time, the current cluster's baseline timing marker is updated to the anomaly start timing marker, allowing for subsequent comparisons with other anomaly start timing markers.

[0122] Step S455: When the timing interval difference is greater than or equal to the preset cluster association threshold, the current cluster element list is added to the abnormal node cluster container, the current cluster sequence number is incremented, a new current cluster element list is initialized, the potential abnormal node is added to the new current cluster element list, and the new current cluster baseline timing mark is set to the timing mark.

[0123] If the timing interval difference is greater than or equal to the preset cluster association threshold, the potential abnormal node has no clear timing association with the nodes in the current cluster. The current cluster element list is added to the abnormal node cluster container as a complete abnormal node cluster. The current cluster sequence number is incremented, and a new sequence number is assigned to the new abnormal node cluster. A new current cluster element list is initialized, the potential abnormal node is added to the new current cluster element list, and the abnormal start timing marker is used as the new current cluster baseline timing marker. For example, if the preset cluster association threshold is 900 seconds, the calculated timing interval difference is 1200 seconds, and the current cluster element list is ["NodeA", "NodeB"], this list is added to the abnormal node cluster container, the current cluster sequence number is incremented from 1 to 2, a new, empty current cluster element list is created, the corresponding potential abnormal node "NodeC" is added to the new current cluster element list, and "2025-01-10 10:30:00" is used as the new current cluster baseline timing marker.

[0124] Step S456: Repeat the above timing interval analysis and cluster division steps until all abnormal starting timing markers in the abnormal timing arrangement sequence are traversed, and add the last current cluster element list to the abnormal node cluster container.

[0125] Repeat steps S453-S455 continuously, continuously read the next abnormal start timing mark in the abnormal timing arrangement sequence, calculate the timing interval difference, and perform cluster division operations based on the difference result. Until all abnormal start timing marks in the abnormal timing arrangement sequence are traversed. Finally, add the last current cluster element list as an abnormal node cluster to the abnormal node cluster container. For example, when all abnormal start timing marks in the abnormal timing arrangement sequence have been processed, add the last current cluster element list to the abnormal node cluster container. At this time, the abnormal node cluster container stores all the divided abnormal node cluster information.

[0126] Step S457: For each abnormal node cluster in the abnormal node cluster container, extract all node identifiers in the cluster element list, calculate the average of the behavioral anomaly confidence within the cluster as the cluster anomaly confidence, record the earliest abnormal start timing mark in the cluster as the cluster start timing mark, and the latest abnormal start timing mark as the cluster end timing mark, and generate an abnormal node cluster descriptor containing a node identifier list, cluster anomaly confidence, cluster start timing mark, and cluster end timing mark.

[0127] For each abnormal node cluster in the abnormal node cluster container, a series of information extraction and calculation operations are performed. First, the node identifiers of all potential abnormal nodes are extracted from the cluster element list to form a node identifier list. Then, the average behavioral anomaly confidence of all potential abnormal nodes in the cluster is calculated and used as the cluster anomaly confidence. This value can reflect the degree of abnormality of the entire abnormal node cluster. Next, the earliest abnormal start time sequence mark in the cluster is found as the cluster start time sequence mark, and the latest abnormal start time sequence mark is used as the cluster end time sequence mark. These two marks can clearly define the time range of the abnormal occurrence of the abnormal node cluster. Finally, the node identifier list, cluster anomaly confidence, cluster start time sequence mark, and cluster end time sequence mark are combined to generate an abnormal node cluster descriptor, which is used to describe the information of each abnormal node cluster in detail.

[0128] Step S460: Extract the node identifier, abnormal pattern label, cluster start timing mark, cluster end timing mark and corresponding behavior anomaly confidence in each abnormal node cluster to generate an abnormal interaction node cluster. Each cluster element in the abnormal interaction node cluster contains an abnormal node identifier, an abnormal start timing mark and an abnormal behavior feature description. The abnormal behavior feature description is matched from a preset abnormal pattern feature library based on the abnormal pattern label.

[0129] For each abnormal node cluster, information such as the node identifier, abnormal pattern label, cluster start timing marker, cluster end timing marker, and corresponding behavioral anomaly confidence score is extracted. This information is organized and combined to generate an abnormal interaction node cluster. Each cluster element in the abnormal interaction node cluster contains the abnormal node identifier, abnormal start timing marker, and abnormal behavior feature description. The abnormal behavior feature description is matched based on the abnormal pattern label from a preset abnormal pattern feature library, which stores detailed feature descriptions corresponding to different abnormal patterns. For example, if the abnormal pattern label is "abnormal data transmission," the corresponding abnormal behavior feature description from the preset abnormal pattern feature library is "abnormal data transmission rate fluctuations, and the amount of data transmitted exceeds the normal range." This information is combined into a cluster element. Ultimately, all cluster elements are combined to form an abnormal interaction node cluster, which comprehensively displays the abnormal interactions in the data flow topology map.

[0130] As an implementation manner, the method provided in the embodiment of the present invention further includes the step of generating a monitoring and analysis report including the node abnormality degree and flow risk situation based on the abnormal interaction node cluster, which can be specifically implemented as the following steps S500 to S1100: Step S500: obtaining a preset abnormality level classification rule, wherein the abnormality level classification rule includes a correspondence between a behavior abnormality confidence interval and an abnormality level. The abnormality level is dynamically classified according to the relative size of the behavior abnormality confidence.

[0131] The preset abnormality level classification rules are a set of pre-set rules used to classify abnormality levels based on the confidence level of behavioral anomalies. The behavioral anomaly confidence interval refers to different confidence ranges, and each interval corresponds to an abnormality level. The abnormality level is dynamically divided according to the relative size of the behavioral anomaly confidence, that is, the abnormal situation is divided into different levels according to the level of confidence, such as mild abnormality, moderate abnormality, severe abnormality, etc. For example, the preset abnormality level classification rules are: behavioral anomaly confidence between 0-0.3 corresponds to mild abnormality level, between 0.3-0.7 corresponds to moderate abnormality level, and between 0.7-1 corresponds to severe abnormality level. By obtaining these rules, the abnormality level of the node in the abnormal interaction node cluster can be determined based on its behavioral anomaly confidence.

[0132] Step S600: traverse each cluster element in the abnormal interaction node cluster, extract the behavior abnormality confidence, and determine the corresponding node abnormality level according to the abnormality level classification rule.

[0133] For each cluster element in the abnormal interaction node cluster, the system extracts its behavioral anomaly confidence score. Then, based on the pre-defined anomaly classification rules, it finds the behavioral anomaly confidence interval within which the confidence score falls, thereby determining the corresponding node anomaly level. For example, if a cluster element has a behavioral anomaly confidence score of 0.8, the node's anomaly level would be considered severe according to the aforementioned anomaly classification rules. By traversing all cluster elements, the corresponding anomaly level can be determined for each node, providing a foundation for subsequent statistics and analysis.

[0134] Step S700: Collect the node abnormality levels of all cluster elements, count the number of nodes with different abnormality levels, and generate abnormality level distribution statistics.

[0135] The node anomaly levels for all cluster elements in the abnormal interaction node cluster are collected, and the number of nodes at each anomaly level is counted. For example, the number of nodes with mild, moderate, and severe anomalies is counted. These statistical results are organized into a table or chart to generate an anomaly level distribution. This result can intuitively display the distribution of nodes at different anomaly levels, helping analysts understand the overall distribution of abnormal nodes.

[0136] Step S800: According to the abnormal behavior feature description in the abnormal interaction node cluster, corresponding risk impact feature parameters are matched from a preset risk impact feature library. The risk impact feature parameters include data flow sensitivity features and node interaction range features.

[0137] The preset risk impact feature library stores risk impact feature parameters corresponding to different abnormal behavior feature descriptions. These parameters include data flow sensitivity features and node interaction range features. Data flow sensitivity features reflect the sensitivity of the data flow process, such as the frequency of sensitive operations and the degree of exposure of sensitive fields. Node interaction range features reflect the potential impact coverage of abnormal behavior, such as the number of receiving nodes when the abnormal node acts as the initiating node. Based on the abnormal behavior feature descriptions in the abnormal interaction node cluster, matching risk impact feature parameters are searched from the risk impact feature library. For example, if the abnormal behavior feature description is "abnormal data transmission," the corresponding data flow sensitivity features matched in the risk impact feature library are a high frequency of sensitive operations and a high degree of exposure of sensitive fields, and the node interaction range features are a high number of receiving nodes when the abnormal node acts as the initiating node.

[0138] Step S900: Based on the node anomaly level, data flow sensitivity characteristics and node interaction range characteristics, multi-dimensional risk fusion is performed, and a comprehensive flow risk value is obtained through a weighted combination method. The weighted combination method dynamically adjusts the weight distribution according to the relative importance of each feature parameter.

[0139] Multidimensional risk fusion comprehensively considers information from multiple dimensions, including the node anomaly level, data flow sensitivity, and node interaction range characteristics, and fuses these information to calculate a comprehensive flow risk value. Weighted combination assigns a weight to each characteristic parameter, then multiplies the characteristic parameter by its corresponding weight and sums them to obtain the comprehensive flow risk value. The relative importance of each characteristic parameter varies, and the weighted combination method dynamically adjusts the weight assignment based on the current values ​​of these characteristic parameters. For example, when the value of a data flow sensitivity characteristic is high, its corresponding weight is appropriately increased to highlight its impact on the comprehensive flow risk value.

[0140] As an implementation method, step S900 performs multi-dimensional risk fusion based on the node anomaly level, data flow sensitivity characteristics, and node interaction range characteristics, and obtains a comprehensive flow risk value through a weighted combination method. The weighted combination method dynamically adjusts the weight distribution according to the relative importance of each characteristic parameter. Specifically, the following steps S910 to S960 can be implemented: Step S910: A mapping relationship between node abnormality levels and level importance coefficients is preset. The level importance coefficients are set in a step-by-step manner according to the abnormality levels. The higher the abnormality level, the greater the corresponding level importance coefficient.

[0141] The mapping relationship between the preset node abnormality level and the level importance coefficient is a pre-set correspondence table, in which each abnormality level corresponds to a level importance coefficient. The level importance coefficient is set in a stepped manner according to the abnormality level. The higher the abnormality level, the larger the corresponding level importance coefficient. For example, the level importance coefficient corresponding to the mild abnormality level is 0.2, the level importance coefficient corresponding to the moderate abnormality level is 0.5, and the level importance coefficient corresponding to the severe abnormality level is 0.8. Through this setting, the impact of nodes with high abnormality levels on risk can be highlighted when calculating the comprehensive flow risk value.

[0142] Step S920: Read the data flow sensitive features corresponding to the data subject identifier in the abnormal interaction node cluster from the preset data sensitive feature table. The data sensitive feature table contains sensitive feature parameters corresponding to different data subject types. The sensitive feature parameters are determined comprehensively based on the frequency of sensitive operations and the degree of exposure of sensitive fields during the data flow process.

[0143] The pre-set data sensitivity feature table stores sensitive feature parameters corresponding to different data subject types. These parameters are determined by the frequency of sensitive operations and the degree of exposure of sensitive fields during data flow. Based on the data subject identifier in the abnormal interaction node cluster, the corresponding sensitive feature parameters are read from the data sensitivity feature table. For example, if the data subject identifier is "Important Business Data," the corresponding data flow sensitive features read from the data sensitivity feature table include a high frequency of sensitive operations and a high degree of exposure of sensitive fields. These sensitive feature parameters are used in the subsequent calculation of the comprehensive flow risk value.

[0144] Step S930: Based on the abnormal node identifier in the abnormal interaction node cluster, query the number of all receiving nodes in the data flow topology map where the node acts as an initiating node during the continuous operation period, and use the ratio of the number of receiving nodes to the total number of nodes in the system to be monitored as the node interaction range feature. The node interaction range feature reflects the potential impact coverage of the abnormal behavior.

[0145] Based on the abnormal node identifier in the abnormal interaction node cluster, the data flow topology map is used to find the number of all receiving nodes for which the node served as the initiating node during the continuous operation period. This number is divided by the total number of nodes in the monitored system, and the resulting ratio is used as the node interaction range feature. This feature reflects the potential impact coverage of the abnormal behavior. A larger ratio indicates a wider impact range.

[0146] Step S940: Initialize the dynamic weight allocation model. The dynamic weight allocation model dynamically adjusts the fusion weight of each feature parameter according to the current values ​​of the node anomaly level, data flow sensitivity characteristics and node interaction range characteristics. When the value of a feature parameter exceeds the preset feature threshold, its corresponding fusion weight is increased.

[0147] The dynamic weight allocation model dynamically adjusts the fusion weights of various feature parameters. This model analyzes and determines the current values ​​of node anomaly levels, data flow sensitivity characteristics, and node interaction range characteristics. A preset feature threshold is a value set for each feature parameter. When a feature parameter exceeds this threshold, its corresponding fusion weight is increased to highlight its impact on the overall flow risk value.

[0148] Step S950: The weight of the hierarchical importance coefficient, the weight of the data flow sensitive feature and the weight of the node interaction range feature are calculated through the dynamic weight allocation model, and each feature parameter is multiplied by its corresponding weight and then summed to obtain a comprehensive flow risk value.

[0149] The dynamic weight allocation model calculates the weights of the grade importance coefficient, the weights of the data flow sensitivity feature, and the weights of the node interaction range feature based on the current values ​​of each feature parameter. The grade importance coefficient, data flow sensitivity feature, and node interaction range feature corresponding to the node anomaly level are then multiplied by their corresponding weights. Finally, these products are added together to obtain the comprehensive flow risk value.

[0150] Step S960: Perform feature scaling on the calculated comprehensive flow risk value, and map the comprehensive flow risk value to a preset risk assessment interval. The feature scaling process is implemented through linear transformation to obtain a standardized flow risk assessment value used to characterize the risk level of abnormal nodes. The standardized flow risk assessment value is used to quantitatively identify the risk situation of abnormal nodes in the monitoring and analysis report.

[0151] The preset risk assessment interval is a predefined range of values ​​used to uniformly assess the risk level of abnormal nodes. Feature scaling uses a linear transformation to map the calculated comprehensive flow risk value to this preset risk assessment interval. For example, if the preset risk assessment interval is [0, 1], the comprehensive flow risk value is converted using a linear transformation formula to obtain a standardized flow risk assessment value. This value can be used to quantitatively identify the risk status of abnormal nodes in monitoring and analysis reports.

[0152] Step S1000: Calculate the comprehensive flow risk value of each cluster element, and sort the cluster elements in descending order according to the numerical values ​​of the comprehensive flow risk values ​​to generate a risk priority sorting list.

[0153] For each cluster element in the abnormal interaction node cluster, its comprehensive flow risk value is calculated according to the method of steps S900-S960 above. Then, the cluster elements are sorted in descending order according to the size of the comprehensive flow risk value, that is, the cluster elements with larger comprehensive flow risk values ​​are ranked higher. The generated risk priority ranking list can help quickly identify high-risk abnormal node clusters so that corresponding measures can be taken to deal with them first. For example, there are three cluster elements in an abnormal interaction node cluster, and their comprehensive flow risk values ​​are 0.8, 0.6, and 0.4 respectively. The risk priority ranking list obtained after descending sorting is [(cluster element 1, 0.8), (cluster element 2, 0.6), (cluster element 3, 0.4)].

[0154] Step S1100: Integrate the statistical results of the abnormality level distribution, the risk priority ranking list, the abnormal start time sequence mark in the abnormal interaction node cluster, and the abnormal behavior feature description into structured data, and generate a monitoring and analysis report containing the node abnormality level distribution, high-risk node ranking and abnormal detail description. The document format of the monitoring and analysis report complies with the preset structured report template specification.

[0155] The system integrates statistical results of anomaly level distribution, risk priority rankings, anomaly onset timing markers within clusters of abnormally interacting nodes, and descriptions of abnormal behavior characteristics to form structured data. This data includes node anomaly level distribution, ranking information for high-risk nodes, and detailed descriptions of anomalies. This structured data is organized into a monitoring and analysis report based on pre-defined structured report template specifications. This report can be presented as a document.

[0156] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 may be connected via a bus or other means. The processor 101 (also known as the Central Processing Unit (CPU)) is the computing and control core of the computer system, capable of parsing various instructions within the computer system and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), which can be used to send and receive data under the control of the processor 101. The communication interface 102 may also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system for storing programs and data. It is understood that the memory 103 herein may include both the built-in memory of the computer system and, of course, the extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system, but this is not limited to this in the present invention.

[0157] In one embodiment, the processor 101 executes the data flow monitoring method based on the large model provided in the above embodiment of the present invention by running the computer program in the memory 103.

Claims

1. A data flow monitoring method based on a large model, characterized in that: include: Acquire a data flow record set generated by the system to be monitored during a continuous operation period, wherein the data flow record set includes multiple groups of flow event units with unique time sequence tags; Constructing an associated path for the data flow record set to generate a data flow topology map containing node interaction relationships and timing dependencies, wherein each node in the data flow topology map corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a timing marker; Calling a pre-trained flow behavior analysis model to perform node sequence pattern recognition on the data flow topology map, and generating a behavior anomaly confidence and anomaly pattern label for each node in the data flow topology map; The abnormal interaction node cluster in the data flow topology map is screened according to the behavior anomaly confidence and the abnormal pattern label, and the abnormal interaction node cluster includes an abnormal node identifier, an abnormal starting time sequence mark and an abnormal behavior feature description.

2. The method according to claim 1, characterized in that The step of constructing an association path for the data flow record set to generate a data flow topology graph including node interaction relationships and timing dependency relationships includes: Performing integrity check on the flow event units in the data flow record set, removing flow event units with missing data subject identifiers, initiating node identifiers, or receiving node identifiers, and obtaining a set of flow event units that pass the check; The verified flow event units are arranged in order according to the sequence of timing marks to generate a flow event sequence with continuous timing association, wherein the interval between the timing marks of adjacent flow event units in the flow event sequence does not exceed a preset timing continuity threshold; Extracting the initiating node identifier and the receiving node identifier of each flow event unit in the flow event sequence, and constructing a node interaction pair list containing the corresponding relationship between the initiating node and the receiving node, wherein each element in the node interaction pair list consists of the initiating node identifier and the receiving node identifier and retains the timing mark of the corresponding flow event unit; Calculating node association strength for the node interaction pair list, counting the cumulative frequency of the same node interaction pair appearing in the flow event sequence, and generating a node interaction pair frequency distribution table, wherein the node interaction pair frequency distribution table includes node interaction pair identifiers and corresponding cumulative frequencies; According to the frequency distribution table of node interaction pairs and the time sequence arrangement results of the flow event sequence, a data flow topology map is constructed with nodes as vertices, node interaction pairs as directed edges, cumulative frequency as edge association strength, and time sequence marks as edge time sequence attributes. Each directed edge in the data flow topology map is also associated with a corresponding operation behavior type.

3. The method according to claim 2, characterized in that The node association strength calculation is performed on the node interaction pair list, and the cumulative frequency of the same node interaction pair appearing in the flow event sequence is counted to generate a node interaction pair frequency distribution table, including: Traversing each node interaction pair element in the node interaction pair list, converting each node interaction pair element into a node interaction pair identifier in a character splicing format, wherein the node interaction pair identifier is composed of an initiating node identifier and a receiving node identifier connected by a preset separator and maintaining an initiating-receiving order; Initialize an empty frequency statistics container, where the key of the frequency statistics container is the node interaction pair identifier and the value is an associated information group including the cumulative frequency and the time sequence marker list; Sequentially reading the node interaction pair identifiers and the timing marks of the corresponding flow event units in the node interaction pair list; when the current node interaction pair identifier does not exist in the frequency statistics container, adding the node interaction pair identifier to the frequency statistics container and setting the cumulative frequency to 1, and setting the timing mark list to a list containing the current timing mark; when the current node interaction pair identifier already exists in the frequency statistics container, increasing the corresponding cumulative frequency by 1 and adding the current timing mark to the timing mark list; After the traversal is completed, the key-value pairs in the frequency statistics container are converted into a structured table including a node interaction pair identification field, a cumulative frequency field, and a time sequence tag list field to obtain a node interaction pair frequency distribution table; The node interaction pair frequency distribution table is sorted in descending order according to the cumulative frequency field to generate a sorted node interaction pair frequency distribution table. The sorted node interaction pair frequency distribution table is used to give priority to high-frequency node interaction pairs when constructing a data flow topology map to strengthen the core path representation.

4. The method according to claim 1, wherein The calling of the pre-trained flow behavior analysis model to perform node sequence pattern recognition on the data flow topology map, and generating the behavior anomaly confidence and anomaly pattern label of each node in the data flow topology map, includes: Performing graph structured encoding on the data flow topology graph, converting the nodes, directed edges, and corresponding edge association strengths and edge timing attributes in the data flow topology graph into a low-dimensional dense structured feature vector, wherein the structured feature vector includes a node attribute vector, an edge feature vector, and a timing association vector; Extracting an incoming edge feature sequence and an outgoing edge feature sequence of each node in a continuous running period from the structured feature vector, and constructing a behavior pattern sequence of each node, wherein the behavior pattern sequence is composed of the incoming edge feature vector sequence and the outgoing edge feature vector sequence spliced ​​in a time sequence mark order; Inputting the behavior pattern sequence into the graph attention encoder of the flow behavior analysis model, modeling the node interaction dependency in the behavior pattern sequence through the graph structure attention mechanism, and generating a node association pattern vector containing node context interaction information; Inputting the node association pattern vector into the time series modeling layer of the flow behavior analysis model, capturing the dynamic change rules of the node association pattern vector in the time series dimension, and generating a time series enhancement pattern vector that strengthens the dynamic characteristics of the time series; The timing enhancement pattern vector is input into the anomaly recognition layer of the flow behavior analysis model, and the behavioral anomaly confidence of each node is calculated through a multi-layer perceptron network and a probability normalization function, and the corresponding anomaly pattern label is determined according to the confidence distribution result.

5. The method according to claim 4, characterized in that The graph structured encoding of the data flow topology graph is performed to convert the nodes, directed edges, and corresponding edge association strengths and edge timing attributes in the data flow topology graph into low-dimensional dense structured feature vectors, including: Extract all node identifiers in the data flow topology map, construct a unique node set, and assign a globally unique node index code to each node identifier; Initializing a node basic embedding matrix based on the unique node set, wherein the number of rows of the node basic embedding matrix is ​​the total number of nodes in the unique node set, the number of columns is the preset feature embedding dimension, and the matrix elements are generated by random initialization and dynamically updated in subsequent processing; Extract all directed edges in the data flow topology graph, each directed edge including a source node index code, a target node index code, an edge association strength, and an edge timing attribute, and construct a directed edge feature list; Normalizing the edge association strengths in the directed edge feature list, mapping the edge association strength values ​​to a preset feature value interval by linear scaling to obtain normalized edge strength values; Performing time series feature extraction on the edge time series attributes in the directed edge feature list, converting the time series mark into a time series feature vector including a time series interval feature and a time series trend feature, wherein the time series interval feature is obtained by calculating the interval difference between adjacent time series marks, and the time series trend feature is obtained by analyzing the change direction of continuous time series marks; The node basic embedding matrix, normalized edge strength value and time series feature vector are input into the graph structured coding model, and the association relationship between the node and the directed edge is aggregated by the graph convolutional network to generate an updated node attribute vector. At the same time, the normalized edge strength value and time series feature vector are concatenated according to the feature dimension to form an edge feature vector. The updated node attribute vector, edge feature vector and time series feature vector are integrated into a feature representation of a unified dimension to obtain a structured feature vector including the node attribute vector, the edge feature vector and the time series association vector.

6. The method according to claim 4, characterized in that The step of inputting the behavior pattern sequence into the graph attention encoder of the flow behavior analysis model, modeling the node interaction dependency relationship in the behavior pattern sequence through the graph structure attention mechanism, and generating a node association pattern vector containing node context interaction information includes: The behavior pattern sequence is divided into groups according to the node index code to obtain a behavior pattern subsequence corresponding to each node, wherein the behavior pattern subsequence includes all incoming edge feature vectors and outgoing edge feature vectors of the node in a continuous running period; The length of each node's behavior pattern subsequence is normalized. When the subsequence length exceeds the preset upper limit of the sequence length, the head is truncated. When the subsequence length is lower than the preset lower limit of the sequence length, the tail is padded with zero vectors to obtain a standardized behavior pattern subsequence with consistent length. Initialize the multi-layer attention parameter matrix of the graph attention encoder, where the dimension of the attention parameter matrix matches the feature dimension of the standardized behavior pattern subsequence and each layer of the parameter matrix is ​​initialized independently; For each node, extract its directly adjacent node set from the data flow topology map, wherein the directly adjacent node set includes all source nodes and target nodes directly connected to the node through directed edges; The standardized behavior pattern subsequence of the current node is concatenated with the standardized behavior pattern subsequence of each directly adjacent node in terms of feature dimension to generate an adjacency-related feature vector; Performing a multi-level linear transformation on the adjacent associated feature vector using the multi-layer attention parameter matrix to generate a multi-layer attention score vector, and normalizing each layer of the attention score vector using a normalized exponential function to obtain the adjacent node attention weight corresponding to each layer; The standardized behavior pattern subsequence of each directly adjacent node is weighted summed with the attention weight of the adjacent nodes in the corresponding layer to generate a multi-layer adjacent aggregate feature vector; The standardized behavior pattern subsequence of the current node is fused element by element with the multi-layer adjacency aggregation feature vector, and the features are nonlinearly transformed through a nonlinear activation function to generate a node association pattern vector containing node context interaction information.

7. The method according to claim 1, characterized in that The filtering of abnormal interaction node clusters in the data flow topology map according to the abnormal behavior confidence and the abnormal pattern label includes: Obtain a preset abnormality confidence threshold, traverse each node in the data flow topology map, compare the node's behavior abnormality confidence with the abnormality confidence threshold, and mark the node as a potential abnormal node when the behavior abnormality confidence is greater than or equal to the abnormality confidence threshold; Collect the node identification, behavior anomaly confidence, and anomaly pattern labels of all potential abnormal nodes to build a candidate list of potential abnormal nodes; Classifying the potential abnormal nodes in the potential abnormal node candidate list into abnormal patterns, classifying the potential abnormal nodes with the same abnormal pattern label into the same abnormal pattern category group, and generating an abnormal pattern category grouping set; For each abnormal pattern category group, extract the abnormal start time series marks of all potential abnormal nodes in the group, arrange them in ascending order according to the order of the time series marks, and generate an abnormal time series arrangement sequence; Analyze the time intervals of adjacent abnormal start time series marks in the abnormal time series arrangement sequence. When the time interval is less than a preset cluster association threshold, determine that the corresponding potential abnormal node is a time series associated abnormal node, and merge the time series associated abnormal nodes into an abnormal node cluster. The abnormal node cluster includes node identifiers of multiple time series associated abnormal nodes and cluster start time series marks and cluster end time series marks; The node identifier, abnormal pattern label, cluster start timing mark, cluster end timing mark and corresponding behavior anomaly confidence in each abnormal node cluster are extracted to generate an abnormal interaction node cluster. Each cluster element in the abnormal interaction node cluster contains an abnormal node identifier, an abnormal start timing mark and an abnormal behavior feature description. The abnormal behavior feature description is matched from a preset abnormal pattern feature library based on the abnormal pattern label.

8. The method according to claim 7, characterized in that The analyzing the time intervals of adjacent abnormal start time series marks in the abnormal time series arrangement sequence, determining that the corresponding potential abnormal node is a time series-correlated abnormal node when the time interval is less than a preset cluster correlation threshold, and merging the time series-correlated abnormal nodes into an abnormal node cluster, includes: Initialize an empty abnormal node cluster container, set the current cluster sequence number to the initial value, and initialize the current cluster element list to empty; Reading the first abnormal starting timing mark in the abnormal timing arrangement sequence as the reference timing mark of the current cluster, and adding the corresponding potential abnormal node to the current cluster element list; Reading the next abnormal start timing mark in the abnormal timing arrangement sequence, and calculating the timing interval difference between the timing mark and the current cluster reference timing mark; When the timing interval difference is less than a preset cluster association threshold, the corresponding potential abnormal node is added to the current cluster element list, and the baseline timing mark of the current cluster is updated to the timing mark; When the timing interval difference is greater than or equal to a preset cluster association threshold, the current cluster element list is added to the abnormal node cluster container, the current cluster sequence number is incremented, a new current cluster element list is initialized, the potential abnormal node is added to the new current cluster element list, and the new current cluster baseline timing mark is set to the timing mark; Repeat the above timing interval analysis and cluster division steps until all abnormal starting timing markers in the abnormal timing arrangement sequence are traversed, and add the last current cluster element list to the abnormal node cluster container; For each abnormal node cluster in the abnormal node cluster container, extract all node identifiers in the cluster element list, calculate the average of the behavioral anomaly confidence within the cluster as the cluster anomaly confidence, record the earliest abnormal start timing mark in the cluster as the cluster start timing mark, and the latest abnormal start timing mark as the cluster end timing mark, and generate an abnormal node cluster descriptor including the node identifier list, cluster anomaly confidence, cluster start timing mark, and cluster end timing mark.

9. The method according to claim 4, characterized in that Extracting the incoming edge feature sequence and the outgoing edge feature sequence of each node in the continuous running period from the structured feature vector to construct the behavior pattern sequence of each node includes: Separating an edge feature vector set from the structured feature vector, and dividing the edge feature vector set into an input edge feature subset and an output edge feature subset according to the source node index code and the target node index code of the directed edge, wherein the input edge feature subset includes all edge feature vectors with the current node as the target node, and the output edge feature subset includes all edge feature vectors with the current node as the source node; Add an operation behavior type label to each edge feature vector in the input edge feature subset and the output edge feature subset, where the operation behavior type label is determined based on the operation behavior type in the flow event unit; The correspondence between operation behavior types and feature weights is preset. Feature weights are configured based on the risk association of the operation behavior in the data flow. The edge feature vectors in the input edge feature subset and the output edge feature subset are matched with feature weights according to the operation behavior type and weighted to generate weighted input edge feature vectors and weighted output edge feature vectors. Arrange the weighted input edge feature vectors according to the time sequence mark order of the edge timing attribute to generate an input edge feature vector sequence; arrange the weighted output edge feature vectors according to the time sequence mark order of the edge timing attribute to generate an output edge feature vector sequence; Perform time alignment on the input edge feature vector sequence and the output edge feature vector sequence, and perform zero vector interpolation processing on the positions where the timing markers are missing; The time-aligned incoming edge feature vector sequence and outgoing edge feature vector sequence are concatenated in sequence according to the feature dimension to generate the behavior pattern sequence of each node.

10. A computer system, characterized in that: include: a memory, wherein the computer program is stored in the memory; A processor is used to load the computer program to implement the data flow monitoring method based on a large model as described in any one of claims 1 to 9.

Images