Data Flow Monitoring Method and System Based on Large Model

Abstract

The application provides a kind of data flow conversion monitoring method and system based on large model, by obtaining the data flow conversion record set generated by the system to be monitored in continuous operation period, the data flow conversion record set is associated path construction, generate the data flow conversion topological atlas containing node interaction relationship and timing dependency, call the pre-trained flow conversion behavior analysis large model to the data flow conversion topological atlas is node sequence pattern recognition, generate the behavior abnormal confidence and abnormal mode label of each node in the data flow conversion topological atlas;According to the behavior abnormal confidence and abnormal mode label, filter the abnormal interaction node cluster in the data flow conversion topological atlas.The application considers the correlation and timing correlation between abnormal nodes, avoids missing or false detection, improves the reliability of monitoring effect.

Description

Technical Field

[0001] This invention relates to the fields of data monitoring and machine learning, and more specifically, to a data flow monitoring method and system based on a large model. Background Technology

[0002] With the rapid development of information technology, data flow monitoring can monitor the transmission, processing, and storage of data between different processing nodes within a system in real time or offline. This helps identify potential abnormal flow behaviors and ensures the security and compliance of data transmission. Currently, this is typically achieved by collecting system-generated flow log data, extracting information about the source and target nodes of data transmission, constructing a simple node connection graph, and then detecting anomalies in flow behavior based on preset rules (such as illegal access rules between fixed nodes) or traditional machine learning models (such as classifiers based on statistical features). The final output is a list of abnormal nodes or alarm information. However, on the one hand, the collected flow logs are mostly unstructured data that simply records the transmission path, lacking the association and integration of key information such as the identity of the data subject and the specific operation behavior type, resulting in insufficient basic data dimensions for monitoring and analysis; on the other hand, the constructed node connection relationship graphs are mostly static structures, which can only represent the spatial interaction relationship between nodes and cannot reflect the temporal sequence and dynamic dependency relationship of flow events, making it difficult to capture the temporal evolution pattern of data flow; at the same time, anomaly detection relies on manually preset rules or traditional models, making it difficult to automatically learn the implicit behavioral patterns in complex flow scenarios, and the accuracy of identifying diverse and patterned anomalies is limited. Moreover, anomaly screening is mostly aimed at a single isolated node, ignoring the correlation and temporal correlation between abnormal nodes, which can easily lead to missed detections or false detections, affecting the reliability of monitoring results. Summary of the Invention

[0003] This invention provides a data flow monitoring method and system based on a large model.

[0004] In a first aspect, embodiments of the present invention provide a data flow monitoring method based on a large model, comprising: acquiring a set of data flow records generated by a system under monitoring during continuous operation, wherein the set of data flow records contains multiple sets of flow event units with unique temporal markers; constructing associated paths on the set of data flow records to generate a data flow topology graph containing node interaction relationships and temporal dependencies, wherein each node in the data flow topology graph corresponds to a data processing entity in the system under monitoring, and each directed edge corresponds to one flow event unit and is associated with a temporal marker; calling a pre-trained large model for flow behavior analysis to perform node sequence pattern recognition on the data flow topology graph to generate a behavior anomaly confidence score and anomaly pattern label for each node in the data flow topology graph; and filtering abnormal interaction node clusters in the data flow topology graph based on the behavior anomaly confidence score and anomaly pattern label, wherein the abnormal interaction node cluster contains anomaly node identifiers, anomaly start temporal markers, and anomaly behavior feature descriptions.

[0005] Secondly, embodiments of the present invention provide a computer system, including: a memory storing a computer program; and a processor for loading the computer program to implement the data flow monitoring method based on a large model as described above.

[0006] The data flow monitoring method based on a large model provided by this invention acquires a set of data flow records generated by the system under monitoring during continuous operation. This set of data flow records contains multiple sets of flow event units with unique time-series markers. Each flow event unit consists of a data subject identifier, an initiating node identifier, a receiving node identifier, and an operation behavior type. Through this multi-dimensional and structured recording method, isolated flow events can be transformed into basic data units with data-node-behavior-time correlation relationships. This avoids the limitations of simple logs that only record transmission paths in conventional data flow monitoring, providing comprehensive and structured input for subsequent in-depth analysis and improving the quality of basic data for data flow monitoring. After obtaining the data flow record set, the associated paths are constructed to generate a data flow topology graph containing node interaction relationships and temporal dependencies. In this graph, each node corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a temporal marker. By embedding the temporal marker into the directed edge attributes of the topology graph, the graph can not only represent the static interaction structure between nodes, but also dynamically reflect the temporal order and dependencies of flow events. This overcomes the design limitations of conventional static graphs, which cannot reflect temporal dynamics, and improves the accuracy of representing the spatiotemporal evolution of data flow behavior. Subsequently, a pre-trained flow behavior analysis model is called to perform node sequence pattern recognition on the data flow topology graph, generating behavioral anomaly confidence and anomaly pattern labels for each node. The model has the ability to deeply model complex structured data and long time series, and can automatically learn the implicit behavioral patterns of nodes in spatiotemporal interactions. Compared with conventional detection methods or traditional machine learning models that rely on manual rules, it can more accurately capture complex and diverse abnormal behavioral features and improve the accuracy of anomaly identification. Based on the confidence level of behavioral anomalies and the labels of anomalies, clusters of abnormal interaction nodes in the data flow topology are filtered. These clusters include anomaly node identifiers, anomaly start time sequence markers, and descriptions of abnormal behavior characteristics. By clustering anomaly nodes into clusters and associating them with time sequence markers through anomaly pattern labels, the collective nature and temporal correlation of anomalies can be captured, avoiding the limitations of single-point isolated detection and reducing missed and false detections of patterned anomalies. Attached Figure Description

[0007] Figure 1 This is a flowchart of a data flow monitoring method based on a large model provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of the composition of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0009] Please see Figure 1 This is a flowchart of a data flow monitoring method based on a large model provided by an embodiment of the present invention. The method can be executed by a computer system and specifically includes the following steps:

[0010] Step S100: Obtain the set of data flow records generated by the system under monitoring during continuous operation. The set of data flow records contains multiple sets of flow event units with unique time sequence markers. Each flow event unit consists of a data subject identifier, an initiating node identifier, a receiving node identifier, and an operation behavior type.

[0011] The system to be monitored is one whose data flow needs to be monitored. A continuous runtime period refers to the uninterrupted time period during which the system generates various data flow activities. The data flow record set is a summary of these data flow activities, containing multiple sets of flow event units. A unique time sequence marker is a time-ordered identifier assigned to each flow event unit to distinguish flow events occurring at different times. The data subject identifier identifies the subject of the data participating in the data flow, such as a specific dataset or file; the initiating node identifier refers to the node that initiates the data flow operation, which in the system may be a server, device, or program module; the receiving node identifier is the target node that receives the data flow; and the operation behavior type indicates the operation performed during the data flow, such as data transmission, data copying, or data deletion.

[0012] The method for obtaining the data flow record set can be determined based on the specific architecture and data recording mechanism of the system to be monitored. For example, if the system to be monitored is a distributed system, and each node has its own log recording function, then information related to data flow can be extracted by reading the log files of each node, including the data subject identifier, the initiating node identifier, the receiving node identifier, the operation behavior type, and the corresponding time sequence marker. This information can then be integrated into a data flow record set.

[0013] Step S200: Construct associated paths for the data flow record set to generate a data flow topology graph containing node interaction relationships and temporal dependencies. Each node in the data flow topology graph corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a temporal marker.

[0014] Association path construction refers to analyzing the data flow relationships between nodes based on information in the data flow record set, and identifying the paths through which data flows between different nodes. Node interaction relationships reflect the data interaction between various data processing entities in the monitored system, such as which nodes perform data transmission, replication, or other operations. Temporal dependencies reflect the chronological order of data flow events, i.e., the occurrence of a data flow event depends on certain previous events. The data flow topology graph is a graphical representation where nodes represent data processing entities in the monitored system, such as servers, databases, and applications; directed edges represent the direction of data flow from the initiating node to the receiving node, and each directed edge is associated with a corresponding temporal marker, which indicates when the data flow event occurred.

[0015] As one implementation method, step S200 can be specifically implemented as the following steps S210~S250:

[0016] Step S210: Perform integrity verification on the flow event units in the data flow record set, remove flow event units that are missing data subject identifier, initiating node identifier or receiving node identifier, and obtain a set of flow event units that have passed the verification.

[0017] Integrity verification ensures the completeness of information for each event unit in the data flow record set. This is because the data subject identifier, initiating node identifier, and receiving node identifier are crucial elements for determining the data flow path and related information; missing information can lead to inaccurate subsequent analysis. During verification, each event unit in the data flow record set is traversed, checking for the presence of its data subject identifier, initiating node identifier, and receiving node identifier. If any event unit is missing any of these identifiers, it is removed from the set.

[0018] Step S220: Arrange the verified flow event units in order according to the timing markers to generate a flow event sequence with continuous timing association. The timing marker interval between adjacent flow event units in the flow event sequence does not exceed a preset timing continuity threshold.

[0019] The order of time stamps reflects the chronological sequence of data flow events. Sort the validated set of flow event units according to this order to arrange the flow events chronologically. Continuous temporal correlation indicates that flow events are continuous in time; that is, the time interval between adjacent flow event units cannot be too long. The preset temporal continuity threshold is a pre-defined time interval value used to determine whether adjacent flow event units have a continuous temporal correlation.

[0020] Step S230: Extract the initiating node identifier and receiving node identifier of each flow event unit in the flow event sequence, and construct a node interaction pair list containing the correspondence between initiating and receiving nodes. Each element in the node interaction pair list consists of an initiating node identifier and a receiving node identifier and retains the timing mark of the corresponding flow event unit.

[0021] The initiating node identifier and the receiving node identifier are key information describing the direction of data flow. By extracting these two identifiers from each flow event unit in the flow event sequence, interaction pairs between nodes can be constructed. The node interaction pair list is a list of multiple node interaction pairs. Each node interaction pair contains the initiating node identifier and the receiving node identifier, and retains the time sequence mark of the corresponding flow event unit, thus clearly indicating when the node interaction occurred. For example, in a data flow system, there is a flow event unit in the flow event sequence with the initiating node identifier "NodeA", the receiving node identifier "NodeB", and the time sequence mark "2025-07-10 10:00:00". Then, the node interaction pair list contains an element that is ("NodeA", "NodeB", "2025-07-10 10:00:00").

[0022] Step S240: Calculate the node association strength of the node interaction pair list, count the cumulative frequency of the same node interaction pair in the flow event sequence, and generate a node interaction pair frequency distribution table. The node interaction pair frequency distribution table includes the node interaction pair identifier and the corresponding cumulative frequency.

[0023] Node association strength reflects the frequency of data flow and interaction between two nodes. This association strength can be quantified by calculating the cumulative frequency of identical node interaction pairs appearing in a sequence of events. A node interaction pair identifier is a unique identifier for each node interaction pair, used to distinguish different pairs. The node interaction pair frequency distribution table is a table where each row contains a node interaction pair identifier and the cumulative frequency of that node interaction pair appearing in a sequence of events. For example, if a node interaction pair ("NodeA", "NodeB") appears 5 times in the node interaction pair list, then the node interaction pair frequency distribution table will have a record of ("NodeA-NodeB", 5), where "NodeA-NodeB" is the node interaction pair identifier and 5 is the cumulative frequency. By generating a node interaction pair frequency distribution table, the frequency of data flow between various nodes can be intuitively understood.

[0024] As one implementation method, step S240 can be specifically implemented as the following steps S241~S245:

[0025] Step S241: Traverse each node interaction pair element in the node interaction pair list and convert each node interaction pair element into a node interaction pair identifier in character concatenation format. The node interaction pair identifier is composed of the initiating node identifier and the receiving node identifier connected by a preset separator and maintains the initiating-receiving order.

[0026] The preset separator is a pre-defined character used to connect the initiating node identifier and the receiving node identifier, forming a node interaction pair identifier. When traversing the list of node interaction pairs, each node interaction pair element is processed, connecting its initiating node identifier and receiving node identifier using the preset separator. For example, if the preset separator is "-", and the initiating node identifier of a node interaction pair element is "NodeA" and the receiving node identifier is "NodeB", then the transformed node interaction pair identifier is "NodeA-NodeB". Maintaining the initiating-receiving order ensures that the node interaction pair identifier accurately reflects the direction of data flow.

[0027] Step S242: Initialize an empty frequency statistics container. The key of the frequency statistics container is the node interaction pair identifier, and the value is an association information group containing the cumulative frequency and time series tag list.

[0028] The frequency statistics container is used to store frequency information of node interaction pairs, using key-value pairs. The key is the node interaction pair identifier, which uniquely identifies a node interaction pair; the value is a group of associated information, including cumulative frequency and a time-series marker list. The cumulative frequency records the number of times the node interaction pair appears in the flow event sequence, while the time-series marker list stores the time-series marker corresponding to each occurrence of the node interaction pair. Initializing an empty frequency statistics container means that the container contains no data before statistics begin; it will be gradually filled with information from the node interaction pair list.

[0029] Step S243: Read the node interaction pair identifier and the timing mark of the corresponding flow event unit in the node interaction pair list in sequence. When the current node interaction pair identifier does not exist in the frequency statistics container, add the node interaction pair identifier to the frequency statistics container and set the cumulative frequency to 1. Set the timing mark list to a list containing the current timing mark. When the current node interaction pair identifier already exists in the frequency statistics container, increase the corresponding cumulative frequency by 1 and add the current timing mark to the timing mark list.

[0030] In this embodiment, the node interaction pair identifiers and corresponding timing markers in the node interaction pair list are read one by one. If the node interaction pair identifier is not yet in the frequency statistics container, it means that this is the first time the node interaction pair has been counted. In this case, the identifier is added to the frequency statistics container, its cumulative frequency is initialized to 1, and the current timing marker is added as a unique element to the timing marker list. If the node interaction pair identifier already exists in the frequency statistics container, it means that the node interaction pair has appeared before. In this case, its cumulative frequency is incremented by 1, and the current timing marker is added to the corresponding timing marker list. For example, when processing the list of node interaction pairs, if the node interaction pair identifier "NodeA-NodeB" is encountered, if this identifier is not in the frequency statistics container, it is added and the cumulative frequency is set to 1, and the time series marker list is set to ["2025-01-10 10:00:00"]. If it already exists, the cumulative frequency is incremented by 1, and the time series marker list is updated to ["2025-01-10 10:00:00", "2025-01-10 10:10:00"].

[0031] Step S244: After the traversal is completed, the key-value pairs in the frequency statistics container are converted into a structured table containing a node interaction pair identifier field, a cumulative frequency field, and a time series tag list field, to obtain a node interaction pair frequency distribution table.

[0032] After traversing the list of node interaction pairs, the frequency statistics container stores the frequency information for all node interaction pairs. These key-value pairs are then converted into a structured table, presenting the data in the container in tabular form. The table includes a node interaction pair identifier field, a cumulative frequency field, and a time-series marker list field. Each field corresponds to the relevant information in the key-value pair within the container; for example, the node interaction pair identifier field corresponds to the key, the cumulative frequency field corresponds to the cumulative frequency in the associated information group, and the time-series marker list field corresponds to the time-series marker list in the associated information group. This conversion allows the data to be displayed in a more intuitive and easier-to-process and analyze format.

[0033] Step S245: Sort the node interaction pair frequency distribution table in descending order according to the cumulative frequency field to generate a sorted node interaction pair frequency distribution table. The sorted node interaction pair frequency distribution table is used to prioritize high-frequency node interaction pairs when constructing the data flow topology map to strengthen the representation of the core path.

[0034] High-frequency node interaction pairs indicate that the data flow between two nodes exceeds the preset number of interactions, meaning it is relatively frequent. When constructing a data flow topology graph, prioritizing high-frequency node interaction pairs can highlight these core paths, making the topology graph better reflect the main trends and critical paths of data flow. For example, in a sorted node interaction pair frequency distribution table, high-frequency node interaction pairs will be ranked first. When constructing the topology graph, the main directed edges and node connections will be determined first based on these high-frequency interaction pairs, and then low-frequency interaction pairs will be considered. This strengthens the representation of core paths and helps analysts more clearly understand the focus and key links of data flow.

[0035] Step S250: Based on the frequency distribution table of node interaction pairs and the temporal arrangement results of the flow event sequence, construct a data flow topology graph with nodes as vertices, node interaction pairs as directed edges, cumulative frequency as edge association strength, and temporal marker as edge temporal attribute. Each directed edge in the data flow topology graph is also associated with a corresponding operation behavior type.

[0036] The node interaction frequency distribution table provides information on the frequency of interactions between nodes, while the temporal arrangement of the data flow event sequence reflects the temporal order of data flow events. Nodes are represented as vertices in the data flow topology graph, corresponding to the data processing entities in the monitored system. Node interaction pairs are represented as directed edges, indicating the direction of data flow from the initiating node to the receiving node. Edge association strength is represented by cumulative frequency, reflecting the frequency of data flow between nodes. Edge temporal attributes are represented by temporal tags, clarifying the time when the data flow event occurred. Each directed edge is also associated with a corresponding operation type, such as data transmission or data replication, thus providing a more comprehensive description of the data flow. For example, based on the frequency distribution table and temporal arrangement results of node interaction pairs, it is found that there is a high frequency of data transmission interaction between node "NodeA" and node "NodeB", with multiple data transmission operations at different times. Then, in the data flow topology graph, there is a directed edge from "NodeA" to "NodeB". The association strength of the edge is the cumulative frequency of the node interaction pair, the temporal attribute of the edge is the temporal marker corresponding to each data transmission operation, and the data transmission operation behavior type associated with the directed edge.

[0037] Step S300: Call the pre-trained large-scale model for analyzing data flow behavior to perform node sequence pattern recognition on the data flow topology map, and generate the behavioral anomaly confidence and anomaly pattern label of each node in the data flow topology map.

[0038] The pre-trained large-scale model for data flow behavior analysis is a pre-trained model that has learned a large number of data flow patterns and features. Node sequence pattern recognition refers to using this model to analyze the interaction sequences between nodes in the data flow topology graph to find possible patterns and regularities. Behavioral anomaly confidence represents the probability that a node's behavior is abnormal; it is a probability value. Anomaly pattern labels describe the type of abnormal behavior of the node, such as abnormal data transmission or abnormal data replication. For example, in a data flow topology graph containing multiple nodes, the large-scale model for flow behavior analysis analyzes the interaction sequence of each node, determines whether there is an anomaly based on the learned knowledge, and if a node is determined to be abnormal, it will give the node's behavioral anomaly confidence, such as 0.8, indicating an 80% probability that the node's behavior is abnormal, and simultaneously provide a corresponding anomaly pattern label, such as "abnormal data transmission".

[0039] As one implementation method, step S300 can be specifically implemented as the following steps S310~S350:

[0040] Step S310: Perform graph structure encoding on the data flow topology graph, converting the nodes, directed edges and their corresponding edge association strengths and temporal attributes in the data flow topology graph into low-dimensional dense structured feature vectors. The structured feature vectors include node attribute vectors, edge feature vectors and temporal association vectors.

[0041] Graph structured encoding transforms the graphical data structure of a data flow topology graph into a vector form that computers can process. Low-dimensional, dense structured feature vectors are a vector representation with low dimensionality and high data density, reducing data redundancy and improving computational efficiency. Node attribute vectors represent the relevant attributes of nodes, such as node type and function; edge feature vectors represent the characteristics of directed edges, such as edge association strength; and temporal association vectors reflect the temporal attributes of edges, such as temporal markers. By converting nodes, directed edges, and their corresponding attributes into these vectors, the information of the data flow topology graph can be presented in a form more suitable for model processing. For example, the relevant attributes of node "NodeA" in the data flow topology graph can be converted into node attribute vectors, and the association strength and temporal attributes of directed edges can be converted into edge feature vectors and temporal association vectors, respectively. These vectors are then integrated into a structured feature vector.

[0042] As one implementation method, step S310 can be specifically implemented as the following steps S311~S317:

[0043] Step S311: Extract all node identifiers in the data flow topology graph, construct a unique node set, and assign a globally unique node index code to each node identifier.

[0044] Node identifiers are used to uniquely identify each node in the data flow topology graph. Extracting all node identifiers yields a list containing all node identifiers. A unique node set is a collection that does not contain duplicate node identifiers. By constructing this set, duplicate node identifiers are removed, ensuring that each node is processed only once. A globally unique node index code is a unique number assigned to each node identifier, used for convenient referencing and differentiation of different nodes in subsequent processing. For example, in a data flow topology graph with node identifiers "NodeA", "NodeB", and "NodeA", the extracted unique node set is {"NodeA", "NodeB"}, with node index code 0 assigned to "NodeA" and node index code 1 assigned to "NodeB".

[0045] Step S312: Initialize the node basic embedding matrix based on the unique node set. The number of rows in the node basic embedding matrix is ​​the total number of nodes in the unique node set, and the number of columns is the preset feature embedding dimension. The matrix elements are generated through random initialization and dynamically updated in subsequent processing.

[0046] The preset feature embedding dimension is a pre-defined value used to determine the number of columns in the node basic embedding matrix. The number of rows in the node basic embedding matrix equals the total number of nodes in the unique node set, so each node has a corresponding row in the matrix. Matrix elements are generated through random initialization, meaning that the elements in the matrix are randomly selected at the beginning. In subsequent processing, these elements are dynamically updated based on model training and computation to better represent the node features. For example, if there are 3 unique nodes in the set and the preset feature embedding dimension is 10, then the node basic embedding matrix will be a 3x10 matrix, with elements randomly generated during initialization and continuously adjusted and updated during subsequent graph structured encoding.

[0047] Step S313: Extract all directed edges in the data flow topology graph. Each directed edge contains the source node index code, the target node index code, the edge association strength, and the edge temporal attributes, and construct a directed edge feature list.

[0048] The source node index code and the target node index code correspond to the node index codes of the start and end nodes of the directed edge, respectively. The edge association strength represents the association strength of the directed edge, and the edge temporal attribute represents the temporal marker corresponding to the directed edge. All directed edges are extracted and their relevant information is organized into a directed edge feature list for easier subsequent processing and analysis. For example, in the data flow topology graph, there is a directed edge from node "NodeA" to node "NodeB" with an association strength of 5 and a temporal marker of "2025-01-10 10:00:00". The node index code of "NodeA" is 0, and the node index code of "NodeB" is 1. Therefore, the directed edge feature list would contain an element (0, 1, 5, "2025-01-10 10:00:00").

[0049] Step S314: Normalize the edge association strength in the directed edge feature list by mapping the edge association strength value to a preset feature value range through linear scaling to obtain the normalized edge strength value.

[0050] Normalization mapping unifies edge association strength values ​​from different ranges into a preset feature value interval, thus making the association strengths of different edges comparable. Linear scaling scales the edge association strength value by calculating the ratio between the edge association strength value and the preset feature value interval. For example, if the preset feature value interval is [0, 1], and the range of edge association strength values ​​in the directed edge feature list is [1, 10], for a directed edge with an association strength value of 5, the normalized edge strength value calculated through linear scaling is (5-1) / (10-1) ≈ 0.44, mapping it to the interval [0, 1]. Normalization mapping avoids the impact of different edge association strength value ranges on subsequent processing, improving the stability and accuracy of the model.

[0051] Step S315: Extract time-series features from the edge time-series attributes in the directed edge feature list, and convert the time-series markers into time-series feature vectors containing time-series interval features and time-series trend features. The time-series interval features are obtained by calculating the interval difference between adjacent time-series markers, and the time-series trend features are obtained by analyzing the changing direction of continuous time-series markers.

[0052] Time sequence markers represent the occurrence time of data flow events corresponding to directed edges. Extracting time sequence features from them can yield more valuable information. Time sequence interval features reflect the time interval differences between adjacent data flow events, obtained by calculating the time difference between adjacent time sequence markers. Time sequence trend features reflect the direction of change of consecutive time sequence markers, such as whether time gradually increases or decreases. For example, given three consecutive time sequence markers "2025-01-10 10:00:00", "2025-01-10 10:10:00", and "2025-01-10 10:20:00", calculating the interval difference between adjacent time sequence markers yields a time sequence interval feature of [600, 600] (in seconds). Analyzing the direction of change of consecutive time sequence markers shows that time gradually increases, resulting in an increasing time sequence trend feature. These features are combined into a time sequence feature vector for subsequent processing and analysis.

[0053] Step S316: Input the node basic embedding matrix, normalized edge strength value and time series feature vector into the graph structured coding model. Perform multi-layer feature aggregation on the relationship between nodes and directed edges through graph convolutional network to generate updated node attribute vector. At the same time, concatenate the normalized edge strength value and time series feature vector according to feature dimension to form edge feature vector.

[0054] The graph structured coding model employs a graph convolutional network (GCN), a neural network used to process graph-structured data. By inputting the node basic embedding matrix, normalized edge strength values, and temporal feature vectors into the graph structured coding model, the GCN performs multi-layer feature aggregation on the relationships between nodes and directed edges. Multi-layer feature aggregation refers to continuously extracting and integrating the feature information of nodes and edges through multiple convolution operations, enabling the model to better understand the structure and features of the graph. In this process, the node basic embedding matrix is ​​updated, generating an updated node attribute vector that more accurately represents the features of the nodes. Simultaneously, the normalized edge strength values ​​and temporal feature vectors are concatenated along the feature dimension to form an edge feature vector, integrating the association strength and temporal feature information of the edges. For example, the GCN performs multiple convolutions and updates on the node basic embedding matrix based on the connection relationship between nodes and directed edges, ultimately obtaining an updated node attribute vector, while simultaneously concatenating the normalized edge strength values ​​and temporal feature vectors along the feature dimension to form a new vector as the edge feature vector.

[0055] Step S317: Integrate the updated node attribute vector, edge feature vector, and time-series feature vector into a unified dimension feature representation to obtain a structured feature vector containing node attribute vector, edge feature vector, and time-series association vector.

[0056] Integrating the features into a unified dimension enables the updated node attribute vectors, edge feature vectors, and temporal feature vectors to be represented and processed within the same dimension. This integration creates a structured feature vector containing node attribute vectors, edge feature vectors, and temporal correlation vectors. This allows for the unified presentation of node, edge, and temporal information in the data flow topology, facilitating processing by large-scale models analyzing flow behavior. For example, concatenating the updated node attribute vectors, edge feature vectors, and temporal feature vectors in a specific order and dimension yields a new vector as the structured feature vector, which contains comprehensive information about the data flow topology.

[0057] Step S320: Extract the incoming edge feature sequence and outgoing edge feature sequence of each node in the continuous runtime from the structured feature vector, and construct the behavior pattern sequence of each node. The behavior pattern sequence is composed of the incoming edge feature vector sequence and the outgoing edge feature vector sequence concatenated in the order of time markers.

[0058] The incoming edge feature sequence refers to the sequence of feature vectors of all directed edges with a given node as the target node, while the outgoing edge feature sequence refers to the sequence of feature vectors of all directed edges with that node as the source node. Extracting the incoming and outgoing edge feature sequences of each node from the structured feature vectors allows us to understand the node's input and output during data flow. The behavior pattern sequence is formed by concatenating the incoming and outgoing edge feature vector sequences according to their temporal markers; it reflects the node's behavior patterns over consecutive runtime periods.

[0059] As one implementation method, step S320 can be specifically implemented as the following steps S321~S326:

[0060] Step S321: Separate the edge feature vector set from the structured feature vector. Based on the source node index encoding and target node index encoding of the directed edge, divide the edge feature vector set into an inbound feature subset and an outbound feature subset. The inbound feature subset contains all edge feature vectors with the current node as the target node, and the outbound feature subset contains all edge feature vectors with the current node as the source node.

[0061] Source node index encoding and target node index encoding are used to determine the direction and connectivity of directed edges. By partitioning the set of edge feature vectors according to these two index encodings, it can be divided into an incoming edge feature subset and an outgoing edge feature subset. For a given node, the incoming edge feature subset contains the edge feature vectors of all directed edges pointing to that node, and the outgoing edge feature subset contains the edge feature vectors of all directed edges originating from that node. For example, for node "NodeA", its node index encoding is 0. In the set of edge feature vectors, based on the source node index encoding and target node index encoding of the directed edges, all edge feature vectors with a target node index encoding of 0 are assigned to the incoming edge feature subset, and all edge feature vectors with a source node index encoding of 0 are assigned to the outgoing edge feature subset. This allows us to obtain the incoming and outgoing edge feature information for "NodeA".

[0062] Step S322: Add an operation behavior type label to each edge feature vector in the inbound and outbound feature subsets. The operation behavior type label is determined based on the operation behavior type in the flow event unit.

[0063] Operation behavior type labels are used to identify the operation behavior type corresponding to the edge feature vector, such as data transmission or data copying. Based on the operation behavior type in the flow event unit, a corresponding label is added to each edge feature vector in the incoming and outgoing edge feature subsets. For example, in a flow event unit, if the operation behavior type corresponding to a directed edge is data transmission, then the edge feature vector of that edge in the incoming or outgoing edge feature subsets will be labeled with the operation behavior type "data transmission". By adding operation behavior type labels, we can more clearly understand the data flow operation represented by each directed edge, providing more information for subsequent analysis of node behavior patterns.

[0064] Step S323: Preset the correspondence between operation behavior types and feature weights. The feature weights are configured according to the risk correlation of operation behavior in data flow. For the edge feature vectors in the input edge feature subset and output edge feature subset, the feature weights are matched according to the operation behavior type and weighted to generate weighted input edge feature vectors and weighted output edge feature vectors.

[0065] The preset mapping between operation behavior types and feature weights is a pre-defined mapping table, where each operation behavior type corresponds to a feature weight. Feature weights are configured based on the risk correlation of the operation behavior in data flow; operations with higher risk correlation receive larger feature weights. For edge feature vectors in the input and output feature subsets, a matching feature weight is found from the preset mapping table based on their operation behavior type label. Then, the edge feature vector is multiplied by this feature weight to obtain the weighted input and output feature vectors.

[0066] Step S324: Arrange the weighted incoming edge feature vectors according to the temporal label order of the edge temporal attributes to generate the incoming edge feature vector sequence; arrange the weighted outgoing edge feature vectors according to the temporal label order of the edge temporal attributes to generate the outgoing edge feature vector sequence.

[0067] The temporal markers of edge time attributes reflect the chronological order of the data flow events represented by the directed edges. Arranging the weighted incoming and outgoing edge feature vectors in this order allows the sequences of incoming and outgoing edge feature vectors to be arranged chronologically. For example, given three weighted incoming edge feature vectors with corresponding temporal markers of "2025-01-10 10:00:00", "2025-01-10 10:10:00", and "2025-01-10 10:20:00", arranging them in this order yields the incoming edge feature vector sequence, which more clearly shows the incoming edge characteristics of nodes at different times. Similarly, the same processing is applied to the weighted outgoing edge feature vectors to generate the outgoing edge feature vector sequence.

[0068] Step S325: Perform temporal alignment on the incoming edge feature vector sequence and the outgoing edge feature vector sequence, and perform zero vector interpolation on the positions where the temporal markers are missing.

[0069] Temporal alignment ensures that the incoming and outgoing feature vector sequences are consistent over time. In practice, some temporal markers may be missing. Zero-vector interpolation is performed at these locations. Zero-vector interpolation inserts a zero vector at the missing temporal marker position to maintain temporal continuity. For example, if the incoming feature vector sequence lacks a corresponding feature vector at a certain time point, a zero vector is inserted at that position to guarantee the integrity and continuity of the sequence. Through temporal alignment and zero-vector interpolation, the incoming and outgoing feature vector sequences become comparable in the temporal dimension, more accurately reflecting the behavioral patterns of nodes.

[0070] Step S326: Concatenate the temporally aligned incoming edge feature vector sequence and outgoing edge feature vector sequence sequentially according to the feature dimension to generate the behavior pattern sequence of each node.

[0071] The temporally aligned incoming and outgoing feature vector sequences are concatenated sequentially along their feature dimensions to form a new sequence. This new sequence contains the incoming and outgoing edge feature information of a node during consecutive runtime periods, arranged chronologically to constitute a sequence of behavioral patterns for each node. For example, concatenating the incoming and outgoing feature vector sequences along their feature dimensions yields a longer vector sequence that comprehensively reflects the node's behavioral characteristics and data flow patterns at different times.

[0072] Step S330: Input the behavior pattern sequence into the graph attention encoder of the large-scale behavior analysis model, and model the node interaction dependencies in the behavior pattern sequence through the graph structure attention mechanism to generate a node association pattern vector containing node context interaction information.

[0073] The graph attention encoder employs a graph-based attention mechanism, which automatically learns the interaction dependencies between nodes and assigns different attention weights based on the importance of each node. After inputting a sequence of behavioral patterns into the encoder, the mechanism models the interaction dependencies between nodes within the sequence. By analyzing the connections and interactions between nodes, it identifies important relationships between them. This process generates a node association pattern vector containing contextual interaction information. This vector includes not only the node's own feature information but also its interaction information with neighboring nodes and contextual information. For example, for node "NodeA", the graph attention encoder generates a node association pattern vector based on its behavioral pattern sequence and its interactions with other nodes. This vector reflects the contextual interactions of "NodeA" within the entire data flow topology graph, helping the model better understand node behavior and anomalies.

[0074] As one implementation method, step S330 can be specifically implemented as the following steps S331~S338:

[0075] Step S331: Divide the behavior pattern sequence into groups according to the node index encoding to obtain the behavior pattern subsequence corresponding to each node. The behavior pattern subsequence contains all the incoming edge feature vectors and outgoing edge feature vectors of the node in the continuous running segment.

[0076] Node index encoding is used to uniquely identify each node. Grouping the behavior pattern sequence according to the node index encoding allows the information in the behavior pattern sequence to be distributed into subsequences corresponding to each node. Each node's behavior pattern subsequence contains all incoming and outgoing edge feature vectors for that node during its continuous runtime, comprehensively reflecting its behavior pattern. For example, if the behavior pattern sequence contains incoming and outgoing edge feature vectors for multiple nodes, grouping them using node index encoding yields the behavior pattern subsequence corresponding to node "NodeA." This subsequence contains all incoming and outgoing edge feature vectors for "NodeA" during its continuous runtime, facilitating subsequent individual analysis and processing of each node.

[0077] Step S332: Standardize the length of the behavior pattern subsequence for each node. When the subsequence length exceeds the preset upper limit of sequence length, truncate the head. When the subsequence length is lower than the preset lower limit of sequence length, fill the tail with zero vectors to obtain standardized behavior pattern subsequences with consistent length.

[0078] The preset upper and lower limits of sequence length are pre-defined length values ​​used to standardize the length of behavioral pattern subsequences. Length standardization ensures that the behavioral pattern subsequences of each node have the same length, facilitating model processing and comparison. When the subsequence length exceeds the upper limit, it is truncated at the beginning, i.e., a preset number of feature vectors are removed to bring the subsequence length to the upper limit. When the subsequence length is below the lower limit, zero vectors are added to the end of the subsequence to bring its length to the lower limit. For example, with a preset upper limit of 10 and a lower limit of 5, the behavioral pattern subsequence of node "NodeA" has a length of 12, so its head will be truncated, retaining the last 10 feature vectors; the behavioral pattern subsequence of node "NodeB" has a length of 3, so 2 zero vectors are added to the end to bring its length to 5, ultimately resulting in standardized behavioral pattern subsequences of consistent length.

[0079] Step S333: Initialize the multi-layer attention parameter matrix of the graph attention encoder. The dimension of the attention parameter matrix matches the feature dimension of the standardized behavioral pattern subsequence, and each layer parameter matrix is ​​initialized independently.

[0080] Multi-layer attention parameter matrices are used to calculate attention weights between nodes. The dimension of the attention parameter matrix matches the feature dimension of the normalized behavior pattern subsequence, ensuring that the feature vector is correctly processed when calculating attention weights. Independent initialization of each layer's parameter matrix means that the parameter matrix of each layer is randomly generated at the beginning and is unrelated to each other. For example, if the feature dimension of the normalized behavior pattern subsequence is 10, and the graph attention encoder has 3 layers of attention parameter matrices, then the dimension of each layer's parameter matrix matches 10, and each layer's parameter matrix is ​​independently and randomly initialized, continuously adjusted and optimized based on the input behavior pattern subsequence during subsequent training.

[0081] Step S334: For each node, extract its set of directly adjacent nodes from the data flow topology graph. The set of directly adjacent nodes includes all source nodes and target nodes that are directly connected to the node through directed edges.

[0082] The set of directly adjacent nodes reflects the nodes directly connected to a given node in the data flow topology graph. For each node, all source and target nodes directly connected to it via directed edges are searched in the data flow topology graph, and these nodes are grouped into the set of directly adjacent nodes. For example, in the data flow topology graph, node "NodeA" has two directed edges connecting to nodes "NodeB" and "NodeC" respectively, and one directed edge connecting node "NodeD" to "NodeA". Therefore, the set of directly adjacent nodes for "NodeA" is {"NodeB", "NodeC", "NodeD"}. By extracting the set of directly adjacent nodes, we can understand the direct interaction relationships between nodes.

[0083] Step S335: Concatenate the standardized behavior pattern subsequence of the current node with the standardized behavior pattern subsequence of each directly adjacent node according to the feature dimension to generate an adjacency association feature vector.

[0084] Concatenating the normalized behavior pattern subsequence of the current node with the normalized behavior pattern subsequences of each of its direct neighbors along the feature dimension results in a new vector. This new vector contains feature information about the current node and its direct neighbors, reflecting the relationship between them. For example, concatenating the normalized behavior pattern subsequence of the current node "NodeA" with the normalized behavior pattern subsequence of its direct neighbor "NodeB" along the feature dimension yields a longer vector as the adjacency association feature vector, which reflects the interaction and association between "NodeA" and "NodeB".

[0085] Step S336: Perform multi-level linear transformation on the adjacency association feature vector through the multi-level attention parameter matrix to generate multi-level attention score vectors, and normalize the attention score vector of each layer through the normalized exponential function to obtain the attention weight of the adjacent node corresponding to each layer.

[0086] Multi-level linear transformation refers to performing multiple linear transformations on the adjacency-related feature vectors through a multi-level attention parameter matrix. Each transformation yields a new vector, and these vectors constitute the multi-level attention score vectors. A normalization exponential function (such as the Softmax function) is used to normalize the attention score vectors of each level, ensuring that the element values ​​in each level's score vector are all within the range [0, 1] and that the sum of all elements is 1. Through normalization, the attention weights of the adjacent nodes corresponding to each level are obtained. These weights reflect the degree of attention the current node pays to its direct neighbors. For example, for the current node "NodeA" and its direct neighbors "NodeB" and "NodeC", a multi-level linear transformation is performed on the adjacency-related feature vectors through a multi-level attention parameter matrix to obtain multi-level attention score vectors. Then, a normalization exponential function is used to normalize each level's score vector to obtain the attention weights for "NodeB" and "NodeC" at each level.

[0087] Step S337: The standardized behavior pattern subsequence of each directly adjacent node is weighted and summed with the attention weights of the adjacent nodes in the corresponding layer to generate a multi-layer adjacency aggregation feature vector.

[0088] The standardized behavioral pattern subsequence of each directly adjacent node is weighted and summed with the attention weights of the adjacent nodes in the corresponding layer. This involves multiplying the standardized behavioral pattern subsequence of each directly adjacent node by its corresponding attention weight, and then summing the results to obtain a multi-layer adjacency aggregation feature vector. This vector contains information about the current node's directly adjacent nodes, and the weighted summation based on attention weights highlights the influence of important adjacent nodes. For example, for the current node "NodeA", its directly adjacent nodes are "NodeB" and "NodeC". Multiplying the standardized behavioral pattern subsequences of "NodeB" and "NodeC" by their respective attention weights and then summing the results yields the multi-layer adjacency aggregation feature vector. This vector comprehensively reflects the information about "NodeA's" directly adjacent nodes and their relative importance.

[0089] Step S338: The standardized behavior pattern subsequence of the current node is fused with the multi-level adjacency aggregation feature vector element by element, and the feature is non-linearly transformed by a non-linear activation function to generate a node association pattern vector containing node context interaction information.

[0090] The normalized behavioral pattern subsequence of the current node is fused with the multi-level adjacency aggregation feature vector element-wise, resulting in a new vector. A non-linear activation function (such as ReLU) is used to perform a feature non-linear transformation on this new vector, introducing non-linearity and enabling the model to learn more complex features and patterns. The node association pattern vector generated in this way contains the node's own feature information as well as its contextual interaction information with its direct neighbors, providing a more comprehensive reflection of the node's position and interactions in the graph. For example, by element-wise adding the normalized behavioral pattern subsequence of the current node "NodeA" with the multi-level adjacency aggregation feature vector and then performing a non-linear transformation using the ReLU function, a node association pattern vector containing the contextual interaction information of "NodeA" is obtained.

[0091] Step S340: Input the node association pattern vector into the temporal modeling layer of the large-scale flow behavior analysis model to capture the dynamic change pattern of the node association pattern vector in the temporal dimension and generate a temporal enhancement pattern vector that strengthens the temporal dynamic characteristics.

[0092] The temporal modeling layer processes the temporal information of node association pattern vectors. While node association pattern vectors contain contextual interaction information about nodes, their dynamic changes over time require further analysis. The temporal modeling layer models these vectors along the temporal dimension, capturing their dynamic patterns by analyzing vector changes at different points in time. For example, temporal modeling methods such as Recurrent Neural Networks (RNNs) or Long Short-Term Memory Networks (LSTMs) are used to process the node association pattern vectors, learning their temporal dependencies and trends. This process generates enhanced temporal pattern vectors that better reflect the behavioral characteristics and change patterns of nodes over time.

[0093] Step S350: Input the temporal enhancement pattern vector into the anomaly recognition layer of the large-scale behavior analysis model, calculate the behavioral anomaly confidence of each node through a multilayer perceptron network and a probability normalization function, and determine the corresponding anomaly pattern label based on the confidence distribution results.

[0094] The anomaly detection layer is used to determine whether nodes exhibit abnormal behavior. A Multilayer Perceptron (MLP) network is a feedforward neural network composed of multiple neuron layers capable of learning complex nonlinear relationships. After the temporal augmentation pattern vector is input into the MLP, the network processes and analyzes it, outputting a prediction result. A probability normalization function (such as the Softmax function) is used to convert the output of the MLP into probability values, obtaining the anomaly confidence score for each node. This confidence score represents the likelihood of abnormal node behavior. Based on the confidence score distribution, a corresponding anomaly pattern label is determined. For example, if the anomaly confidence score of a node exceeds a preset threshold, a corresponding anomaly pattern label is assigned, such as "abnormal data transmission" or "abnormal data duplication," clearly defining the type of abnormal behavior of that node.

[0095] Step S400: Filter the abnormal interaction node clusters in the data flow topology map according to the behavioral anomaly confidence and abnormal pattern label. The abnormal interaction node cluster includes abnormal node identifier, abnormal start time sequence mark and abnormal behavior feature description.

[0096] Behavioral anomaly confidence and anomaly pattern labels are the basis for determining whether a node is abnormal and the type of anomaly. Based on this information, nodes in the data flow topology map are filtered to identify clusters of abnormal interaction nodes. An abnormal interaction node cluster is a set of multiple abnormal nodes, containing an abnormal node identifier to uniquely identify each abnormal node; an anomaly start time sequence marker indicating when the cluster began exhibiting abnormal behavior; and an abnormal behavior feature description, obtained by matching the anomaly pattern label from a pre-defined anomaly pattern feature library, detailing the characteristics and manifestations of the abnormal behavior. For example, by comparing the behavioral anomaly confidence of a node with a pre-defined threshold, nodes with confidence exceeding the threshold are filtered out. Nodes with the same anomaly pattern are grouped into an abnormal interaction node cluster based on their anomaly pattern label, and corresponding abnormal behavior feature descriptions are obtained from the anomaly pattern feature library, providing analysts with comprehensive anomaly information.

[0097] As one implementation method, step S400 can be specifically implemented as the following steps S410~S460:

[0098] Step S410: Obtain the preset anomaly confidence threshold, traverse each node in the data flow topology graph, compare the behavior anomaly confidence of the node with the anomaly confidence threshold, and mark the node as a potential anomaly node when the behavior anomaly confidence is greater than or equal to the anomaly confidence threshold.

[0099] The preset anomaly confidence threshold is a pre-defined probability value used to determine whether a node might exhibit abnormal behavior. When traversing each node in the data flow topology graph, its anomaly confidence is compared to this threshold. If a node's anomaly confidence is greater than or equal to the threshold, it indicates a high probability of abnormal behavior and is marked as a potential anomaly node.

[0100] Step S420: Collect the node identifier, behavioral anomaly confidence, and anomaly pattern label of all potential abnormal nodes to construct a candidate list of potential abnormal nodes.

[0101] The node identifiers, behavioral anomaly confidence scores, and anomaly pattern labels of all nodes marked as potential anomalous nodes are collected to form a list, namely the candidate list of potential anomalous nodes. This list contains relevant information about all nodes that may exhibit anomalous behavior. For example, the node identifiers, behavioral anomaly confidence scores, and anomaly pattern labels of all potential anomalous nodes are organized into a list according to a certain format, and each element in the list corresponds to the relevant information of a potential anomalous node.

[0102] Step S430: Classify the potential abnormal nodes in the potential abnormal node candidate list according to their abnormal patterns, and group potential abnormal nodes with the same abnormal pattern label into the same abnormal pattern category group to generate an abnormal pattern category group set.

[0103] Anomaly pattern classification categorizes potentially anomalous nodes according to their anomaly pattern labels, grouping nodes with the same label together. An anomaly pattern category grouping set is a collection of multiple anomaly pattern category groups, each containing potentially anomalous nodes with the same label. For example, in a candidate list of potential anomalous nodes, some nodes might have the label "abnormal data transmission," which would be grouped into one category; others might have the label "abnormal data replication," which would be grouped into another, ultimately generating an anomaly pattern category grouping set containing multiple category groups.

[0104] As one implementation method, step S430 can be specifically implemented as the following steps S431~S437:

[0105] Step S431: Extract the abnormal pattern labels of all potential abnormal nodes in the potential abnormal node candidate list, and construct an abnormal pattern label set containing all non-repeating abnormal pattern labels.

[0106] Extract the anomaly pattern labels from all potential anomaly node candidate lists, collect these labels, remove duplicate labels, and construct an anomaly pattern label set. The anomaly pattern label set is a set without duplicate elements, where each label represents an anomaly pattern. For example, if multiple nodes in the potential anomaly node candidate list have the anomaly pattern labels "abnormal data transmission" and "abnormal data replication," the extracted anomaly pattern label set would be {"abnormal data transmission", "abnormal data replication"}, facilitating subsequent analysis of the relationships between anomaly patterns.

[0107] Step S432: Based on the preset abnormal pattern association rule base, analyze the association relationship between each abnormal pattern label in the abnormal pattern label set. The association relationship includes causal association relationship and concurrent association relationship. Causal association relationship means that the occurrence of one abnormal pattern may lead to the occurrence of another abnormal pattern. Concurrent association relationship means that the probability of two abnormal patterns occurring at the same time within the same time window is higher than the preset probability threshold.

[0108] The pre-defined rule base for anomaly patterns is a pre-defined set of rules containing information on the relationships between different anomaly patterns. Based on this rule base, the relationships between the anomaly pattern tags in the anomaly pattern tag set are analyzed. Causal relationships indicate that the occurrence of one anomaly pattern may trigger the occurrence of another; for example, "abnormal data transmission" may lead to "data integrity corruption." Concurrency relationships indicate that the probability of two anomaly patterns occurring simultaneously within the same time window is high, exceeding a pre-defined probability threshold.

[0109] Step S433: Construct an anomaly pattern association graph based on the association relationships. Each node in the anomaly pattern association graph corresponds to an anomaly pattern label, and each edge corresponds to an association relationship with an edge weight that is the association strength value. The association strength value is calculated by analyzing the co-occurrence frequency of anomaly pattern labels in historical data flow monitoring records.

[0110] Anomaly pattern association graphs are used to display the relationships between anomaly pattern labels. Each node corresponds to an anomaly pattern label, and each edge corresponds to a relationship, such as causal or concurrent relationships. The edge weight is the association strength value, which is calculated by analyzing the co-occurrence frequency of anomaly pattern labels in historical data flow monitoring records. For example, if "abnormal data transmission" and "abnormal data replication" occur frequently in historical data flow monitoring records, then the edge weight between them will be larger, indicating a stronger association between them. By constructing anomaly pattern association graphs, the relationships and strengths between anomaly patterns can be visually observed.

[0111] Step S434: Perform community detection on the abnormal pattern association graph, identify abnormal pattern label clusters with strong associations, and divide the abnormal pattern labels in each cluster into a main abnormal pattern label and multiple sub-abnormal pattern labels. The main abnormal pattern label is the abnormal pattern label with the largest sum of association strength in the cluster.

[0112] For anomaly pattern association graphs, community detection algorithms (such as the Louvain algorithm) are used to identify clusters of anomaly pattern labels with strong associations. Each cluster contains multiple anomaly pattern labels, and the anomaly pattern label with the highest total association strength is designated as the primary anomaly pattern label, while the remaining labels are designated as secondary anomaly pattern labels. For example, in an anomaly pattern label cluster, "abnormal data transmission" has the highest total association strength, so it is designated as the primary anomaly pattern label, while other labels such as "abnormal data replication" are designated as secondary anomaly pattern labels. This division provides a clearer understanding of the primary and secondary relationships and association structure among anomaly patterns.

[0113] Step S435: Traverse each potential abnormal node in the candidate list of potential abnormal nodes, read its abnormal pattern label, and determine whether the abnormal pattern label belongs to the main abnormal pattern label or the sub-abnormal pattern label in the abnormal pattern association graph.

[0114] When traversing the candidate list of potential anomalous nodes, the anomalous pattern label of each potential anomalous node is checked. Based on the classification results of the main anomalous pattern label and the sub-annomalous pattern label in the anomalous pattern association graph, it is determined which category the label belongs to. For example, for the potential anomalous node "NodeA", its anomalous pattern label is "abnormal data replication". The label is queried in the anomalous pattern association graph to determine whether it is a main anomalous pattern label or a sub-annomalous pattern label.

[0115] Step S436: When the abnormal pattern label is the main abnormal pattern label, directly assign the corresponding potential abnormal node to the abnormal pattern category group named after the main abnormal pattern label; when the abnormal pattern label is a sub-abnormal pattern label, query the main abnormal pattern label to which it belongs, and assign the corresponding potential abnormal node to the abnormal pattern category group corresponding to the main abnormal pattern label.

[0116] Based on the judgment results, if the anomaly pattern label is the primary anomaly pattern label, the corresponding potential anomaly node is directly assigned to the anomaly pattern category group named after that primary anomaly pattern label. If the anomaly pattern label is a sub-anomaly pattern label, its primary anomaly pattern label is queried, and then the potential anomaly node is assigned to the anomaly pattern category group corresponding to that primary anomaly pattern label. For example, if the anomaly pattern label of the potential anomaly node "NodeA" is "abnormal data replication," and it is determined to be a sub-anomaly pattern label, its primary anomaly pattern label is "abnormal data transmission," then "NodeA" will be assigned to the anomaly pattern category group named "abnormal data transmission." In this way, potential anomaly nodes can be accurately classified according to the association relationship of anomaly patterns, forming a reasonable set of anomaly pattern category groups.

[0117] Step S437: Collect all anomaly pattern category groups. Each category group contains a main anomaly pattern label, a list of sub-anomaly pattern labels, and a list of corresponding potential anomaly nodes, generating an anomaly pattern category group set.

[0118] All categorized and assigned anomaly pattern groups are collected. Each group contains a main anomaly pattern label, a list of sub-anomaly pattern labels, and a corresponding list of potential anomaly nodes. For example, an anomaly pattern group might have "abnormal data transmission" as the main anomaly pattern label, a list of sub-anomaly pattern labels of {"abnormal data replication"}, and a list of potential anomaly nodes of {"NodeA", "NodeB"}. Combining all such groupings generates an anomaly pattern category set, which clearly displays the distribution of potential anomaly nodes under different anomaly patterns.

[0119] Step S440: For each abnormal pattern category group, extract the abnormal start time sequence markers of all potential abnormal nodes in the group, and arrange them in ascending order according to the order of the time sequence markers to generate an abnormal time sequence arrangement sequence.

[0120] For each anomaly pattern category group, the anomaly start time sequence markers of all potential anomaly nodes are extracted. These markers record the time when each potential anomaly node begins to exhibit abnormal behavior. The time sequence markers are then sorted in ascending order to generate an anomaly time sequence arrangement. For example, in a category group with "abnormal data transmission" as the main anomaly pattern label, the anomaly start time sequence marker for potential anomaly node "NodeA" is "2025-01-10 10:00:00", and the anomaly start time sequence marker for "NodeB" is "2025-01-10 10:10:00". The resulting anomaly time sequence arrangement after sorting these markers in ascending order is ["2025-01-10 10:00:00", "2025-01-10 10:10:00"].

[0121] Step S450: Analyze the temporal interval of adjacent abnormal start time sequence markers in the abnormal temporal sequence. When the temporal interval is less than the preset cluster association threshold, determine the corresponding potential abnormal node as a temporally associated abnormal node, and merge the temporally associated abnormal nodes into an abnormal node cluster. The abnormal node cluster contains the node identifiers of multiple temporally associated abnormal nodes, as well as the cluster start time sequence marker and the cluster end time sequence marker.

[0122] The temporal interval refers to the time difference between two adjacent anomaly start time markers in an anomaly temporal sequence. A preset cluster association threshold is a pre-defined time value used to determine whether adjacent potential anomaly nodes have a temporal correlation. If the temporal interval between adjacent anomaly start time markers is less than this threshold, the corresponding potential anomaly node is considered a temporally correlated anomaly node; they may have some inherent connection, such as a series of abnormal behaviors triggered by the same anomaly event. These temporally correlated anomaly nodes are merged into an anomaly node cluster. This cluster contains node identifiers for multiple temporally correlated anomaly nodes, as well as a cluster start time marker (i.e., the anomaly start time marker of the earliest node in the cluster to exhibit an anomaly) and a cluster end time marker (i.e., the anomaly start time marker of the latest node in the cluster to exhibit an anomaly).

[0123] As one implementation method, step S450 can be specifically implemented as the following steps S451~S457:

[0124] Step S451: Initialize an empty abnormal node cluster container, set the current cluster sequence number to the initial value and initialize the current cluster element list to empty.

[0125] The abnormal node cluster container stores the final generated abnormal node cluster information; initially, this container is empty. The current cluster index identifies different abnormal node clusters; its initial value is typically set to 0 or 1. The current cluster element list temporarily stores potential abnormal nodes in the currently processed abnormal node cluster; this list is also initially empty. For example, before starting to process the abnormal sequence arrangement, an empty list is created as the abnormal node cluster container, the current cluster index is set to 1, and an empty list is created as the current cluster element list, preparing for subsequent cluster partitioning operations.

[0126] Step S452: Read the first abnormal start time sequence marker in the abnormal time sequence arrangement sequence, use it as the base time sequence marker of the current cluster, and add the corresponding potential abnormal node to the current cluster element list.

[0127] The first anomaly start time marker is the earliest anomaly start time in the anomaly time sequence. It is used as the baseline time marker for the current cluster and is used for subsequent comparisons with other anomaly start time markers. The corresponding potential anomaly node refers to the potential anomaly node corresponding to this anomaly start time marker. It is added to the current cluster element list as the first element of the current cluster. For example, if the anomaly time sequence is ["2025-01-10 10:00:00", "2025-01-10 10:10:00", "2025-01-10 10:20:00"], the first anomaly start time marker is "2025-01-10 10:00:00", and the corresponding potential anomaly node is "NodeA", then "2025-01-10 10:00:00" is used as the baseline time marker for the current cluster, and "NodeA" is added to the current cluster element list.

[0128] Step S453: Read the next abnormal start time sequence marker in the abnormal time sequence arrangement sequence, and calculate the time interval difference between the time sequence marker and the current cluster baseline time sequence marker.

[0129] Following the order of the abnormal timing sequence, the next abnormal start timing marker is read sequentially, and then the time difference between the timing marker and the current cluster baseline timing marker is calculated to obtain the timing interval difference value.

[0130] Step S454: When the time interval difference is less than the preset cluster association threshold, add the corresponding potential abnormal node to the current cluster element list and update the current cluster's baseline time sequence marker to the time sequence marker.

[0131] If the calculated time interval difference is less than the preset cluster association threshold, it indicates that the potentially anomalous node has a temporal correlation with the nodes in the current cluster, and it is added to the current cluster element list. Simultaneously, the baseline time sequence marker of the current cluster is updated to the anomalous start time sequence marker, so that it can be compared with other anomalous start time sequence markers subsequently.

[0132] Step S455: When the time interval difference is greater than or equal to the preset cluster association threshold, add the current cluster element list to the abnormal node cluster container, increment the current cluster number, initialize a new current cluster element list, add the potential abnormal node to the new current cluster element list, and set the new current cluster baseline time sequence marker to the time sequence marker.

[0133] If the time interval difference is greater than or equal to the preset cluster association threshold, it indicates that the potential anomalous node has no obvious time correlation with the nodes in the current cluster. In this case, the current cluster element list is added to the anomalous node cluster container as a complete anomalous node cluster. The current cluster index is incremented, and a new index is assigned to the new anomalous node cluster. A new current cluster element list is initialized, the potential anomalous node is added to the new current cluster element list, and the anomalous start time marker is used as the new current cluster baseline time marker. For example, if the preset cluster association threshold is 900 seconds, the calculated time interval difference is 1200 seconds, and the current cluster element list is ["NodeA", "NodeB"], this list is added to the anomalous node cluster container. The current cluster index is incremented from 1 to 2, a new empty current cluster element list is created, the corresponding potential anomalous node "NodeC" is added to the new current cluster element list, and "2025-01-10 10:30:00" is used as the new current cluster baseline time marker.

[0134] Step S456: Repeat the above time interval analysis and cluster partitioning steps until all abnormal start time sequence markers in the abnormal time sequence arrangement are traversed, and add the last current cluster element list to the abnormal node cluster container.

[0135] Steps S453-S455 are repeated continuously, reading the next abnormal start time marker in the abnormal time sequence, calculating the time interval difference, and performing cluster partitioning based on the difference result. This process continues until all abnormal start time markers in the abnormal time sequence have been traversed. Finally, the last current cluster element list is added as an abnormal node cluster to the abnormal node cluster container. For example, after all abnormal start time markers in the abnormal time sequence have been processed, the last current cluster element list is added to the abnormal node cluster container, at which point the abnormal node cluster container stores all the partitioned abnormal node cluster information.

[0136] Step S457: For each abnormal node cluster in the abnormal node cluster container, extract all node identifiers from the cluster element list, calculate the mean of the abnormal behavior confidence score within the cluster as the cluster abnormal confidence score, record the earliest abnormal start time sequence marker within the cluster as the cluster start time sequence marker, and the latest abnormal start time sequence marker as the cluster end time sequence marker, and generate an abnormal node cluster descriptor containing a list of node identifiers, cluster abnormal confidence score, cluster start time sequence marker, and cluster end time sequence marker.

[0137] For each cluster of abnormal nodes within the abnormal node cluster container, a series of information extraction and calculation operations are performed. First, the node identifiers of all potential abnormal nodes are extracted from the cluster element list to form a node identifier list. Then, the average confidence score of the behavior of all potential abnormal nodes within the cluster is calculated and used as the cluster abnormality confidence score, which reflects the degree of abnormality of the entire abnormal node cluster. Next, the earliest abnormality start time marker within the cluster is identified as the cluster start time marker, and the latest abnormality start time marker is identified as the cluster end time marker. These two markers clearly define the time range of the abnormality occurrence in the abnormal node cluster. Finally, the node identifier list, the cluster abnormality confidence score, the cluster start time marker, and the cluster end time marker are combined to generate an abnormal node cluster descriptor, which is used to describe the information of each abnormal node cluster in detail.

[0138] Step S460: Extract the node identifier, abnormal pattern label, cluster start time sequence mark, cluster end time sequence mark and corresponding behavioral anomaly confidence score from each abnormal node cluster to generate an abnormal interaction node cluster. Each cluster element in the abnormal interaction node cluster contains an abnormal node identifier, an abnormal start time sequence mark and an abnormal behavior feature description. The abnormal behavior feature description is obtained by matching the abnormal pattern label from the preset abnormal pattern feature library.

[0139] For each abnormal node cluster, information such as node identifier, abnormal pattern label, cluster start time sequence marker, cluster end time sequence marker, and corresponding behavioral anomaly confidence level are extracted. This information is then organized and combined to generate an abnormal interaction node cluster. Each cluster element in the abnormal interaction node cluster contains an abnormal node identifier, an abnormal start time sequence marker, and an abnormal behavior feature description. The abnormal behavior feature description is obtained by matching the abnormal pattern label from a pre-defined abnormal pattern feature library, which stores detailed feature descriptions corresponding to different abnormal patterns. For example, if the abnormal pattern label is "abnormal data transmission," the corresponding abnormal behavior feature description matched from the pre-defined abnormal pattern feature library is "abnormal fluctuations in data transmission rate, and the amount of transmitted data exceeds the normal range." This information is combined into a cluster element, and finally, all cluster elements are combined to form an abnormal interaction node cluster, which can comprehensively display abnormal interactions in the data flow topology map.

[0140] As one implementation method, the method provided in this embodiment of the invention further includes a step of generating a monitoring and analysis report containing the degree of node anomaly and the flow risk status based on the abnormal interaction node cluster, which can be implemented as the following steps S500~S1100:

[0141] Step S500: Obtain the preset anomaly degree classification rules. The anomaly degree classification rules include the correspondence between the behavioral anomaly confidence interval and the anomaly degree level. The anomaly degree level is dynamically classified by the relative size of the behavioral anomaly confidence.

[0142] The preset anomaly severity classification rules are a pre-defined set of rules used to classify anomaly severity levels based on the confidence level of behavioral anomalies. A behavioral anomaly confidence level range refers to different confidence levels, with each range corresponding to an anomaly severity level. The anomaly severity level is dynamically classified based on the relative magnitude of the behavioral anomaly confidence level; that is, anomalies are categorized into different levels according to their confidence level, such as mild anomaly, moderate anomaly, and severe anomaly. For example, the preset anomaly severity classification rules are: a behavioral anomaly confidence level between 0 and 0.3 corresponds to a mild anomaly level, between 0.3 and 0.7 corresponds to a moderate anomaly level, and between 0.7 and 1 corresponds to a severe anomaly level. By obtaining these rules, the anomaly severity level of nodes in an anomaly interaction node cluster can be determined based on the behavioral anomaly confidence level.

[0143] Step S600: Traverse each cluster element in the abnormal interaction node cluster, extract the behavioral anomaly confidence, and determine the corresponding node anomaly level according to the anomaly degree classification rules.

[0144] For each cluster element in the abnormal interaction node cluster, its behavioral anomaly confidence score is extracted. Then, according to a preset anomaly severity classification rule, the behavioral anomaly confidence score interval to which that confidence score falls is found, thereby determining the corresponding node anomaly severity level. For example, if a cluster element has a behavioral anomaly confidence score of 0.8, according to the above anomaly severity classification rule, the node's anomaly severity level is severe anomaly. By traversing all cluster elements, the corresponding anomaly severity level for each node can be determined, providing a foundation for subsequent statistics and analysis.

[0145] Step S700: Collect the node anomaly level of all cluster elements, count the distribution of the number of nodes at different anomaly levels, and generate statistical results of the anomaly level distribution.

[0146] The system collects the anomaly severity levels of all cluster elements in the abnormal interaction node cluster, and then counts the number of nodes under each anomaly severity level. For example, it counts the number of nodes under the mild, moderate, and severe anomaly levels. These statistical results are then compiled into a table or chart to generate an anomaly severity level distribution statistical result. This result can visually display the distribution of the number of nodes at different anomaly severity levels, helping analysts understand the overall distribution trend of abnormal nodes.

[0147] Step S800: Based on the description of abnormal behavior characteristics in the abnormal interaction node cluster, match the corresponding risk impact characteristic parameters from the preset risk impact characteristic library. The risk impact characteristic parameters include data flow sensitivity characteristics and node interaction range characteristics.

[0148] The pre-defined risk impact feature library stores risk impact feature parameters corresponding to different abnormal behavior characteristics. These parameters include data flow sensitivity features and node interaction range features. Data flow sensitivity features reflect the sensitivity of the data flow process, such as the frequency of sensitive operations and the degree of exposure of sensitive fields; node interaction range features reflect the potential impact coverage of abnormal behavior, such as the number of receiving nodes when the abnormal node acts as the initiating node. Based on the abnormal behavior feature description in the abnormal interaction node cluster, matching risk impact feature parameters are searched from the risk impact feature library. For example, if the abnormal behavior feature is described as "abnormal data transmission," the corresponding data flow sensitivity features matched from the risk impact feature library are high frequency of sensitive operations and high degree of exposure of sensitive fields, and the node interaction range feature is a large number of receiving nodes when the abnormal node acts as the initiating node.

[0149] Step S900: Perform multi-dimensional risk fusion based on node anomaly level, data flow sensitivity features, and node interaction range features. Obtain a comprehensive flow risk value through a weighted combination method. The weighted combination method dynamically adjusts the weight allocation according to the relative importance of each feature parameter.

[0150] Multi-dimensional risk fusion refers to comprehensively considering information from multiple dimensions, such as node anomaly level, data flow sensitivity characteristics, and node interaction range characteristics, and then fusing these dimensions to calculate a comprehensive flow risk value. The weighted combination method assigns a weight to each feature parameter, then multiplies the feature parameter by its corresponding weight and sums the results to obtain the comprehensive flow risk value. Since the relative importance of each feature parameter varies, the weighted combination method dynamically adjusts the weight allocation based on the current values ​​of these feature parameters. For example, when the value of a data flow sensitivity feature is high, its corresponding weight will be appropriately increased to highlight the impact of that feature on the comprehensive flow risk value.

[0151] As one implementation method, step S900 involves multi-dimensional risk fusion based on node anomaly level, data flow sensitivity characteristics, and node interaction range characteristics. A comprehensive flow risk value is obtained through a weighted combination method. The weighted combination method dynamically adjusts the weight allocation according to the relative importance of each feature parameter. Specifically, this can be implemented as follows: steps S910~S960:

[0152] Step S910: Preset the mapping relationship between the node anomaly level and the level importance coefficient. The level importance coefficient is set in a step-by-step manner according to the anomaly level. The higher the anomaly level, the greater the corresponding level importance coefficient.

[0153] The mapping relationship between the predefined node anomaly severity levels and their importance coefficients is a pre-defined table, where each anomaly severity level corresponds to a specific importance coefficient. The importance coefficients are set in a tiered manner based on the anomaly severity level; the higher the anomaly severity level, the greater the corresponding importance coefficient. For example, a mild anomaly level corresponds to an importance coefficient of 0.2, a moderate anomaly level to 0.5, and a severe anomaly level to 0.8. This setting highlights the impact of nodes with high anomaly severity levels on risk when calculating the overall circulation risk value.

[0154] Step S920: Read the data flow sensitive features corresponding to the data subject identifier in the abnormal interaction node cluster from the preset data sensitive feature table. The data sensitive feature table contains sensitive feature parameters corresponding to different data subject types. The sensitive feature parameters are determined by comprehensively considering the frequency of sensitive operations and the degree of exposure of sensitive fields during the data flow process.

[0155] A pre-defined data sensitivity feature table stores sensitivity feature parameters corresponding to different data subject types. These parameters are determined by combining the frequency of sensitive operations and the degree of exposure of sensitive fields during data flow. Based on the data subject identifier in the abnormal interaction node cluster, the corresponding sensitivity feature parameters are read from the data sensitivity feature table. For example, if the data subject identifier is "important business data," the corresponding data flow sensitivity features read from the data sensitivity feature table would be information such as high frequency of sensitive operations and high degree of exposure of sensitive fields. These sensitivity feature parameters will be used for subsequent comprehensive flow risk value calculation.

[0156] Step S930: Based on the abnormal node identifier in the abnormal interaction node cluster, query the number of all receiving nodes that the node acts as the initiating node in the data flow topology map during continuous operation. Use the ratio of the number of receiving nodes to the total number of nodes in the system to be monitored as the node interaction range feature. The node interaction range feature reflects the potential impact coverage of abnormal behavior.

[0157] Based on the abnormal node identifier in the abnormal interaction node cluster, the number of all receiving nodes when that node acted as the initiating node during a continuous runtime period is found in the data flow topology graph. This number is divided by the total number of nodes in the system to be monitored, and the resulting ratio is used as the node interaction range feature. This feature reflects the potential impact coverage of abnormal behavior; the larger the ratio, the wider the impact range of the abnormal behavior.

[0158] Step S940: Initialize the dynamic weight allocation model. The dynamic weight allocation model dynamically adjusts the fusion weight of each feature parameter according to the current values ​​of the node anomaly level, data flow sensitivity features, and node interaction range features. When the value of a certain feature parameter exceeds the preset feature threshold, its corresponding fusion weight is increased.

[0159] The dynamic weight allocation model is used to dynamically adjust the fusion weights of various feature parameters. This model analyzes and judges based on the current values ​​of node anomaly severity, data flow sensitivity features, and node interaction range features. A preset feature threshold is a value pre-set for each feature parameter. When the value of a feature parameter exceeds this threshold, its corresponding fusion weight is increased to highlight the impact of that feature on the overall flow risk value.

[0160] Step S950: Calculate the weights of the importance coefficient of the level, the data flow sensitivity features, and the node interaction range features through the dynamic weight allocation model. Multiply each feature parameter with its corresponding weight and sum them to obtain the comprehensive flow risk value.

[0161] The dynamic weight allocation model calculates the weights of the importance coefficient, data flow sensitivity feature, and node interaction range feature based on the current values ​​of each feature parameter. Then, it multiplies the importance coefficient, data flow sensitivity feature, and node interaction range feature corresponding to the node anomaly level with their respective weights, and finally adds these products together to obtain the comprehensive flow risk value.

[0162] Step S960: Perform feature scaling on the calculated comprehensive circulation risk value to map the comprehensive circulation risk value to a preset risk assessment range. The feature scaling process is achieved through linear transformation to obtain a standardized circulation risk assessment value used to characterize the risk level of abnormal nodes. The standardized circulation risk assessment value is used to quantitatively identify the risk status of abnormal nodes in the monitoring and analysis report.

[0163] The preset risk assessment interval is a pre-defined numerical range used to uniformly assess the risk level of abnormal nodes. Feature scaling processing maps the calculated comprehensive circulation risk value to this preset risk assessment interval through a linear transformation. For example, if the preset risk assessment interval is [0, 1], the comprehensive circulation risk value is transformed using a linear transformation formula to obtain a standardized circulation risk assessment value. This value can quantify the risk status of abnormal nodes in the monitoring and analysis report.

[0164] Step S1000: Calculate the comprehensive flow risk value of each cluster element, and sort the cluster elements in descending order according to the value of the comprehensive flow risk value to generate a risk priority sorting list.

[0165] For each cluster element in an abnormal interaction node cluster, its comprehensive flow risk value is calculated according to steps S900-S960 above. Then, the cluster elements are sorted in descending order according to the magnitude of the comprehensive flow risk value, that is, the cluster element with the larger comprehensive flow risk value is ranked first. The generated risk priority ranking list can help quickly identify high-risk abnormal node clusters so that corresponding measures can be taken first. For example, if there are three cluster elements in an abnormal interaction node cluster, and their comprehensive flow risk values ​​are 0.8, 0.6, and 0.4 respectively, the risk priority ranking list obtained after sorting in descending order is [(cluster element 1, 0.8), (cluster element 2, 0.6), (cluster element 3, 0.4)].

[0166] Step S1100: Integrate the statistical results of the distribution of abnormality levels, the risk priority ranking list, and the abnormal start time sequence markers and abnormal behavior feature descriptions in the abnormal interaction node cluster into structured data, and generate a monitoring and analysis report containing the distribution of node abnormality levels, the ranking of high-risk nodes, and a description of abnormal details. The document format of the monitoring and analysis report conforms to the preset structured report template specification.

[0167] The statistical results of anomaly severity distribution, the risk priority ranking list, and information such as the anomaly initiation time sequence markers and anomalous behavior characteristic descriptions in the anomaly interaction node cluster are integrated to form structured data. This data includes the distribution of node anomaly severity, the ranking information of high-risk nodes, and detailed descriptions of anomalies. Based on a pre-defined structured report template, this structured data is compiled into a monitoring and analysis report. This report can be presented in document format.

[0168] Please see Figure 2 , Figure 2 This is a schematic diagram of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computing and control core of the computer system, capable of parsing various instructions and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system used to store programs and data. It is understood that the memory 103 here can include the computer system's built-in memory, or it can include extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system; this invention does not limit this storage space.

[0169] In one embodiment, the processor 101 executes the data flow monitoring method based on a large model provided in the above embodiments of the present invention by running a computer program in the memory 103.

Claims

1. A method for monitoring data flow conversion based on a large model, characterized in that, The method comprises the following steps: acquire a set of data flow record generated by the system to be monitored in a continuous running period, the set of data flow record contains multiple groups of flow event units with unique time markers; perform association path construction on the set of data flow record to generate a data flow topology graph containing node interaction relationship and time sequence dependency, including: performing integrity check on the flow event units in the set of data flow record, eliminating flow event units with missing data subject identifier, initiating node identifier or receiving node identifier, obtaining a set of flow event units passing the check; arranging the set of flow event units passing the check in order according to the sequence of time markers, generating a sequence of flow events with continuous time sequence association, the time marker interval between adjacent flow event units in the sequence of flow events does not exceed a preset time sequence continuity threshold; extracting the initiating node identifier and receiving node identifier of each flow event unit in the sequence of flow events, constructing a list of node interaction pairs containing initiating-receiving node correspondence, each element in the list of node interaction pairs consists of initiating node identifier and receiving node identifier and retains the time marker of the corresponding flow event unit; performing node association strength calculation on the list of node interaction pairs, counting the cumulative frequency of the same node interaction pair appearing in the sequence of flow events, generating a node interaction pair frequency distribution table, the node interaction pair frequency distribution table contains node interaction pair identifier and corresponding cumulative frequency; constructing a data flow topology graph with nodes as vertices, node interaction pairs as directed edges, cumulative frequency as edge association strength, and time markers as edge time sequence attributes according to the node interaction pair frequency distribution table and the time sequence arrangement result of the sequence of flow events, each directed edge in the data flow topology graph is also associated with the corresponding operation behavior type; each node in the data flow topology graph corresponds to a data processing entity in the system to be monitored, and each directed edge corresponds to a flow event unit and is associated with a time marker; invoke a pre-trained flow behavior analysis large model to perform node sequence pattern recognition on the data flow topology graph, generate behavior anomaly confidence and abnormal pattern label of each node in the data flow topology graph; screen abnormal interaction node clusters in the data flow topology graph according to the behavior anomaly confidence and abnormal pattern label; extract node identifier, abnormal pattern label, cluster start time marker, cluster end time marker and corresponding behavior anomaly confidence in each abnormal node cluster to generate abnormal interaction node cluster, each cluster element in the abnormal interaction node cluster contains abnormal node identifier, abnormal start time marker and abnormal behavior feature description, the abnormal behavior feature description is matched from a preset abnormal pattern feature library based on the abnormal pattern label.

2. The method of claim 1, wherein, The node association strength calculation on the list of node interaction pairs, counting the cumulative frequency of the same node interaction pair appearing in the sequence of flow events, generating a node interaction pair frequency distribution table, includes: Traverse each node interaction pair element in the node interaction pair list, convert each node interaction pair element into a node interaction pair identifier in character concatenation format, the node interaction pair identifier being composed of an initiating node identifier and a receiving node identifier connected by a preset separator and maintaining an initiating-receiving order; Initialize an empty frequency statistics container, the key of the frequency statistics container being a node interaction pair identifier and the value being associated information including an accumulated frequency and a time sequence marker list; Read the node interaction pair identifier in the node interaction pair list and the time sequence marker of the corresponding flow conversion event element in sequence, when the current node interaction pair identifier does not exist in the frequency statistics container, add the node interaction pair identifier in the frequency statistics container and set the accumulated frequency to 1 and the time sequence marker list to a list containing the current time sequence marker, when the current node interaction pair identifier already exists in the frequency statistics container, increase the corresponding accumulated frequency by 1 and add the current time sequence marker to the time sequence marker list; After the traversal is completed, convert the key-value pairs in the frequency statistics container into a structured table including a node interaction pair identifier field, an accumulated frequency field and a time sequence marker list field, to obtain a node interaction pair frequency distribution table; Sort the node interaction pair frequency distribution table in descending order according to the accumulated frequency field to generate a sorted node interaction pair frequency distribution table, the sorted node interaction pair frequency distribution table being used to preferentially process high-frequency node interaction pairs to strengthen core path representation when constructing a data flow conversion topology graph.

3. The method of claim 1, wherein, The pre-trained flow conversion behavior analysis large model is called to perform node sequence pattern recognition on the data flow conversion topology graph, to generate behavior anomaly confidence and abnormal pattern labels of each node in the data flow conversion topology graph, including: The data flow conversion topology graph is structured and coded to convert the nodes, directed edges and corresponding edge association strength and edge time sequence attributes in the data flow conversion topology graph into low-dimensional dense structured feature vectors, the structured feature vectors including node attribute vectors, edge feature vectors and time sequence association vectors; The in-edge feature sequence and out-edge feature sequence of each node within a continuous running period are extracted from the structured feature vectors to construct a behavior pattern sequence of each node, the behavior pattern sequence being composed of an in-edge feature vector sequence and an out-edge feature vector sequence spliced in time sequence marker order; The behavior pattern sequence is input into the graph attention encoder of the flow conversion behavior analysis large model, the node interaction dependency relationship in the behavior pattern sequence is modeled through a graph structure attention mechanism to generate a node association pattern vector containing node context interaction information; The node association pattern vector is input into the time sequence modeling layer of the flow conversion behavior analysis large model to capture the dynamic change law of the node association pattern vector in the time sequence dimension to generate a time sequence enhanced pattern vector strengthening time sequence dynamic characteristics; The time sequence enhancement mode vector is input into an anomaly identification layer of the flow conversion behavior analysis large model, a behavior anomaly confidence of each node is calculated through a multi-layer perception network and a probability normalization function, and a corresponding anomaly mode label is determined according to a confidence distribution result.

4. The method of claim 3, wherein, The data flow conversion topology graph is graph-structured coded, the nodes, directed edges and corresponding edge correlation strengths and edge time sequence attributes in the data flow conversion topology graph are converted into low-dimensional dense structured feature vectors, and the method comprises the following steps: All node identifiers in the data flow conversion topology graph are extracted, a unique node set is constructed, and a globally unique node index code is allocated to each node identifier; A node basis embedding matrix is initialized based on the unique node set, the number of rows of the node basis embedding matrix is the total number of nodes in the unique node set, the number of columns is a preset feature embedding dimension, and matrix elements are generated through a random initialization mode and dynamically updated in subsequent processing; All directed edges in the data flow conversion topology graph are extracted, each directed edge contains a source node index code, a target node index code, an edge correlation strength and an edge time sequence attribute, and a directed edge feature list is constructed; The edge correlation strength in the directed edge feature list is normalized and mapped, the edge correlation strength value is mapped to a preset feature value interval through a linear scaling mode, and a normalized edge strength value is obtained; The edge time sequence attribute in the directed edge feature list is subjected to time sequence feature extraction, the time sequence label is converted into a time sequence feature vector containing a time sequence interval feature and a time sequence trend feature, the time sequence interval feature is obtained by calculating the interval difference of adjacent time sequence labels, and the time sequence trend feature is obtained by analyzing the change direction of continuous time sequence labels; The node basis embedding matrix, the normalized edge strength value and the time sequence feature vector are input into a graph-structured coding model, the association relationship between nodes and directed edges is subjected to multi-layer feature aggregation through a graph convolution network, an updated node attribute vector is generated, and the normalized edge strength value and the time sequence feature vector are spliced into an edge feature vector according to the feature dimension; The updated node attribute vector, the edge feature vector and the time sequence feature vector are integrated into a unified dimension feature representation, and a structured feature vector containing a node attribute vector, an edge feature vector and a time sequence association vector is obtained.

5. The method of claim 3, wherein, The behavior mode sequence is input into the graph attention encoder of the flow conversion behavior analysis large model, the node interaction dependency relationship in the behavior mode sequence is modeled through a graph structure attention mechanism, a node association mode vector containing node context interaction information is generated, and the method comprises the following steps: The behavior mode sequence is grouped and divided according to the node index code, a behavior mode subsequence corresponding to each node is obtained, and the behavior mode subsequence contains all in-edge feature vectors and out-edge feature vectors of the node in a continuous running period; The behavior mode subsequence of each node is subjected to length standardization, when the subsequence length exceeds a preset upper limit of sequence length, the head is truncated, when the subsequence length is lower than a preset lower limit of sequence length, the tail is filled with a zero vector, and a standardized behavior mode subsequence with consistent length is obtained; initializing a multi-layer attention parameter matrix of the graph attention encoder, wherein a dimension of the attention parameter matrix matches a feature dimension of the normalized behavior pattern subsequence, and each layer of the parameter matrix is independently initialized; for each node, extracting a set of direct adjacent nodes from the data flow transition topology graph, wherein the set of direct adjacent nodes includes all source nodes and target nodes directly connected to the node through directed edges; concatenating the normalized behavior pattern subsequence of the current node with the normalized behavior pattern subsequences of each direct adjacent node in a feature dimension to generate an adjacent correlation feature vector; performing multi-level linear transformation on the adjacent correlation feature vector through the multi-layer attention parameter matrix to generate a multi-layer attention score vector, and normalizing each layer of the attention score vector through a normalization exponential function to obtain an adjacent node attention weight corresponding to each layer; performing weighted summation on the normalized behavior pattern subsequence of each direct adjacent node and the adjacent node attention weight of the corresponding layer to generate a multi-layer adjacent aggregation feature vector; performing element-wise addition fusion on the normalized behavior pattern subsequence of the current node and the multi-layer adjacent aggregation feature vector, and performing feature nonlinear conversion through a nonlinear activation function to generate a node correlation pattern vector containing node context interaction information.

6. The method of claim 1, wherein, The abnormal interaction node cluster in the data flow transition topology graph is filtered according to the behavior anomaly confidence and the abnormal pattern label, comprising: obtaining a preset abnormal confidence threshold, traversing each node in the data flow transition topology graph, comparing the behavior anomaly confidence of the node with the abnormal confidence threshold, and marking the node as a potential abnormal node when the behavior anomaly confidence is greater than or equal to the abnormal confidence threshold; collecting the node identifier, behavior anomaly confidence and abnormal pattern label of all potential abnormal nodes to construct a potential abnormal node candidate list; classifying the potential abnormal nodes in the potential abnormal node candidate list according to the abnormal pattern, grouping the potential abnormal nodes with the same abnormal pattern label into the same abnormal pattern category group to generate an abnormal pattern category grouping set; for each abnormal pattern category group, extracting the abnormal start time markers of all potential abnormal nodes in the group, and arranging the abnormal start time markers in ascending order according to the time sequence to generate an abnormal time sequence arrangement sequence; analyzing the time interval of adjacent abnormal start time markers in the abnormal time sequence arrangement sequence, and determining that the corresponding potential abnormal node is a time sequence correlation abnormal node when the time interval is less than a preset cluster correlation threshold, and merging the time sequence correlation abnormal nodes into an abnormal node cluster, wherein the abnormal node cluster includes the node identifier and the cluster start time marker and the cluster end time marker of multiple time sequence correlation abnormal nodes.

7. The method of claim 6, wherein, The analysis of the time interval of adjacent abnormal start time markers in the abnormal time sequence arrangement sequence, and the determination that the corresponding potential abnormal node is a time sequence correlation abnormal node when the time interval is less than a preset cluster correlation threshold, and the merging of the time sequence correlation abnormal nodes into an abnormal node cluster, comprises: initializing an empty abnormal node cluster container, setting the current cluster serial number to an initial value and initializing the current cluster element list as empty; reading a first abnormal start time marker in the abnormal time sequence arrangement sequence as a reference time marker of a current cluster, and adding a corresponding potential abnormal node to a current cluster element list; reading a next abnormal start time marker in the abnormal time sequence arrangement sequence, and calculating a time interval difference between the time marker and the reference time marker of the current cluster; when the time interval difference is less than a preset cluster association threshold, adding the corresponding potential abnormal node to the current cluster element list, and updating the reference time marker of the current cluster to the time marker; when the time interval difference is greater than or equal to the preset cluster association threshold, adding the current cluster element list to an abnormal node cluster container, incrementing a current cluster serial number, initializing a new current cluster element list, adding the potential abnormal node to the new current cluster element list, and setting a new current cluster reference time marker to the time marker; repeating the time interval analysis and cluster division steps until all abnormal start time markers in the abnormal time sequence arrangement sequence are traversed, and adding a last current cluster element list to the abnormal node cluster container; for each abnormal node cluster in the abnormal node cluster container, extracting all node identifiers in the cluster element list, calculating a mean value of abnormal behavior confidence in the cluster as cluster abnormal confidence, recording an earliest abnormal start time marker in the cluster as a cluster start time marker and a latest abnormal start time marker in the cluster as a cluster end time marker, and generating an abnormal node cluster descriptor containing the node identifier list, the cluster abnormal confidence, the cluster start time marker and the cluster end time marker.

8. The method of claim 3, wherein, The abnormal start time marker in the abnormal time sequence arrangement sequence is read as a reference time marker of a current cluster, and a corresponding potential abnormal node is added to a current cluster element list; The abnormal start time marker in the abnormal time sequence arrangement sequence is read as a reference time marker of a current cluster, and a corresponding potential abnormal node is added to a current cluster element list; when the time interval difference is less than a preset cluster association threshold, adding the corresponding potential abnormal node to the current cluster element list, and updating the reference time marker of the current cluster to the time marker; when the time interval difference is greater than or equal to the preset cluster association threshold, adding the current cluster element list to an abnormal node cluster container, incrementing a current cluster serial number, initializing a new current cluster element list, adding the potential abnormal node to the new current cluster element list, and setting a new current cluster reference time marker to the time marker; repeating the time interval analysis and cluster division steps until all abnormal start time markers in the abnormal time sequence arrangement sequence are traversed, and adding a last current cluster element list to the abnormal node cluster container; for each abnormal node cluster in the abnormal node cluster container, extracting all node identifiers in the cluster element list, calculating a mean value of abnormal behavior confidence in the cluster as cluster abnormal confidence, recording an earliest abnormal start time marker in the cluster as a cluster start time marker and a latest abnormal start time marker in the cluster as a cluster end time marker, and generating an abnormal node cluster descriptor containing the node identifier list, the cluster abnormal confidence, the cluster start time marker and the cluster end time marker. The abnormal start time marker in the abnormal time sequence arrangement sequence is read as a reference time marker of a current cluster, and a corresponding potential abnormal node is added to a current cluster element list; The abnormal start time marker in the abnormal time sequence arrangement sequence is read as a reference time marker of a current cluster, and a corresponding potential abnormal node is added to a current cluster element list; when the time interval difference is less than a preset cluster association threshold, adding the corresponding potential abnormal node to the current cluster element list, and updating the reference time marker of the current cluster to the time marker; when the time interval difference is greater than or equal to the preset cluster association threshold, adding the current cluster element list to an abnormal node cluster container, incrementing a current cluster serial number, initializing a new current cluster element list, adding the potential abnormal node to the new current cluster element list, and setting a new current cluster reference time marker to the time marker; repeating the time interval analysis and cluster division steps until all abnormal start time markers in the abnormal time sequence arrangement sequence are traversed, and adding a last current cluster element list to the abnormal node cluster container; for each abnormal node cluster in the abnormal node cluster container, extracting all node identifiers in the cluster element list, calculating a mean value of abnormal behavior confidence in the cluster as cluster abnormal confidence, recording an earliest abnormal start time marker in the cluster as a cluster start time marker and a latest abnormal start time marker in the cluster as a cluster end time marker, and generating an abnormal node cluster descriptor containing the node identifier list, the cluster abnormal confidence, the cluster start time marker and the cluster end time marker. The time-aligned in-edge feature vector sequence and the out-edge feature vector sequence are sequentially spliced according to the feature dimensions to generate a behavior pattern sequence of each node.

9. A computer system, characterized by Comprise: A memory, wherein a computer program is stored; A processor for loading the computer program to realize the big model-based data flow conversion monitoring method according to any one of claims 1-8.

Images