Power dispatch automation multi-source data fusion collection method, system, device and medium

Abstract

The application discloses a power dispatching automation multi-source data fusion collection method, system, equipment and medium, and the method comprises the following steps: acquiring multi-source network protocol data, preprocessing the multi-source network protocol data to obtain standardized structure data; constructing the association relationship between network nodes to generate a unified network entity model; constructing a device node relationship graph according to the device node adjacency relationship in the unified network entity model, and generating a network topology relationship based on multi-protocol association data; constructing a topology subgraph for a device node corresponding to a detected network state change event, and performing local topology updating on the topology subgraph to obtain an updated network topology relationship; acquiring a network entity, establishing a network semantic association relationship for the network entity, and generating a network knowledge graph; and generating a hierarchical network topology view according to the unified network entity model, the updated network topology relationship and the network knowledge graph, so that the data association conflict problem between heterogeneous protocol data is reduced.

Description

Technical Field

[0001] This invention relates to the field of power dispatch automation technology, and in particular to a method, system, equipment and medium for multi-source data fusion acquisition in power dispatch automation. Background Technology

[0002] As the scale of power dispatch automation systems increases, the number of switching devices, routing devices, and server devices in the dispatch network continues to increase. Different devices exchange data through Simple Network Management Protocol (SNMP), Link Layer Discovery Protocol (LLDP), Address Resolution Protocol (ARP), Forwarding Table Protocol (FDB), System Log Protocol (Syslog), and Trap Alarm Protocol.

[0003] In existing technologies, independent protocol acquisition methods are typically used to obtain data corresponding to different protocols. Specifically, SNMP is used to obtain device operating status data, LLDP is used to obtain device neighbor relationship data, ARP is used to obtain IP address-MAC address mappings, FDB is used to obtain switch port-MAC address mappings, Syslog is used to obtain device log data, and Trap is used to obtain device status change events. Because the data structures, field formats, and time formats of different protocols differ, the device identifier field, port field, and status field in different protocols lack a unified mapping relationship, making it difficult to perform unified association processing between data from different protocols.

[0004] Current technologies lack a unified standardized processing mechanism for data corresponding to different protocols, which easily leads to data association conflicts between heterogeneous protocol data. Therefore, this invention proposes a method, system, equipment, and medium for multi-source data fusion acquisition in power dispatch automation to solve the above problems. Summary of the Invention

[0005] In view of the above-mentioned existing problems, the present invention provides a method, system, equipment and medium for multi-source data fusion acquisition in power dispatch automation to solve the problem that the existing technology lacks a unified standardized processing mechanism for data corresponding to different protocols, which easily leads to data association conflicts between heterogeneous protocol data.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for multi-source data fusion and acquisition in power dispatch automation, comprising: Acquire multi-source network protocol data, preprocess the multi-source network protocol data to obtain standardized structured data; Based on the standardized structural data, the relationships between network nodes are constructed to generate a unified network entity model; A device node relationship graph is constructed based on the device node adjacency relationship in the unified network entity model, and a network topology relationship is generated based on multi-protocol association data; A topology subgraph is constructed for the device nodes corresponding to the detected network state change events, and the topology subgraph is locally updated to obtain the updated network topology relationship. Obtain network entities, establish network semantic relationships between the network entities, and generate a network knowledge graph; Based on the unified network entity model, the updated network topology and network knowledge graph generate a hierarchical network topology view, enabling the fusion and collection of multi-source data for automated power dispatching.

[0007] As a preferred embodiment of the multi-source data fusion acquisition method for power dispatch automation described in this invention, the preprocessing of the multi-source network protocol data to obtain standardized structured data includes: Based on the protocol data packets sent by different network devices, the corresponding data format is identified according to the protocol type, the corresponding protocol parsing rules are called to parse the core fields in the packets, and the parsed protocol data is converted into unified structure data. Nodes are deployed in different network areas, and different protocols are used to obtain data collected by the devices. Based on the unified structure data and the data collected by the devices, a mapping relationship between different protocol fields and unified fields is established. The basic fields of the devices are uniformly converted, the data timestamps of different devices are uniformly calibrated, and abnormal data is filtered to obtain standardized structure data.

[0008] As a preferred embodiment of the multi-source data fusion acquisition method for power dispatch automation described in this invention, the following steps are included: constructing the association relationships between network nodes and generating a unified network entity model: Device nodes are generated based on device identification information in standardized structural data; Generate port nodes based on port identification information; Generate link nodes using data from at least two protocols; Establish the relationships between device nodes, port nodes, and link nodes, and generate a node adjacency matrix; A unified network entity model is generated based on the device nodes, port nodes, link nodes, and node adjacency matrix.

[0009] The beneficial effects of this preferred technical solution are that it constructs three types of nodes and an association matrix to form a unified model and eliminates data heterogeneity conflicts.

[0010] As a preferred embodiment of the multi-source data fusion and acquisition method for power dispatch automation described in this invention, the following steps are included: constructing a device node relationship diagram and generating network topology relationships based on multi-protocol association data: Extract device node data and link node data from the unified network entity model; Extract device relationship features from at least three types of protocol data to generate device feature vectors; The adjacency relationships between device nodes are constructed based on the device feature vectors, and a device node adjacency relationship matrix is ​​generated. Construct a device node relationship graph based on the device node adjacency matrix; Perform feature propagation on the device node relationship graph and update the device node features; Based on the updated device node features, generate device node embedding vectors and calculate the association values ​​between device nodes; The device connection relationships are determined based on the association values, and the network topology is generated.

[0011] The beneficial effect of this preferred technical solution is that it improves the accuracy and completeness of connection identification by integrating multi-protocol inference topology.

[0012] As a preferred embodiment of the power dispatch automation multi-source data fusion acquisition method described in this invention, the method includes: constructing a topology subgraph and performing local topology updates on the topology subgraph. A network status change event was detected, and the device identification information in the event data was parsed. The affected device nodes are identified based on the device identification information, and the associated device nodes adjacent to the affected device nodes are identified to generate a set of affected device nodes; Construct a topology subgraph based on the set of affected device nodes and their corresponding link relationships; The connectivity relationships in the topology subgraph are recalculated, and the network topology relationships are updated based on the recalculated connectivity relationships.

[0013] The beneficial effects of this preferred technical solution are that it reduces the amount of computation and efficiently responds to state changes by updating the topology in a local range.

[0014] As a preferred embodiment of the multi-source data fusion and acquisition method for power dispatch automation described in this invention, the generation of the network knowledge graph includes: Extract device entities, port entities, and business system entities; Establish the connection relationships between device entities, the mapping relationships between port entities and device entities, and the association relationships between business system entities and device entities respectively; A network knowledge graph is generated based on the connection, mapping, and association relationships between device entities.

[0015] As a preferred embodiment of the multi-source data fusion acquisition method for power dispatch automation described in this invention, generating a hierarchical network topology view includes: The network nodes are hierarchically divided according to the device type, and a node hierarchy relationship is generated. Based on the connection relationships and node hierarchy relationships between device nodes, generate the topology layout relationship and node layout coordinates; Distinguish and identify primary links and redundant links; Based on the unified network entity model, network topology relationships, and network knowledge graph, combined with topology layout relationships and link identifiers, a hierarchical network topology view is generated.

[0016] Secondly, this invention provides a power dispatch automation multi-source data fusion acquisition system, comprising: The data preprocessing module is used to acquire multi-source network protocol data, preprocess the multi-source network protocol data, and obtain standardized structured data. The unified modeling module is used to construct the relationships between network nodes based on the standardized structure data and generate a unified network entity model. The topology reasoning module is used to construct a device node relationship graph based on the device node adjacency relationship in the unified network entity model, and generate network topology relationships based on multi-protocol association data. The dynamic topology update module is used to construct a topology subgraph for the device nodes corresponding to the detected network state change event, and to perform local topology updates on the topology subgraph to obtain the updated network topology relationship. The knowledge graph fusion module is used to acquire network entities, establish network semantic relationships between the network entities, and generate a network knowledge graph. The topology visualization module is used to generate a hierarchical network topology view based on the unified network entity model, the updated network topology relationships, and the network knowledge graph, thereby realizing the fusion and collection of multi-source data for power dispatch automation.

[0017] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the power dispatch automation multi-source data fusion acquisition method.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the power dispatch automation multi-source data fusion acquisition method.

[0019] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention unifies and standardizes the access to data from Simple Network Management Protocol (SNMP), Link Layer Discovery Protocol (LLDP), Address Resolution Protocol (ARP), Forwarding Table Protocol (FDB), System Log Protocol (Syslog), and Trap Alarm Protocol (TAP), enabling data from different protocols to form a unified data structure, thereby reducing data association conflicts between heterogeneous protocol data. This invention constructs a device node relationship graph and generates network topology relationships based on multi-protocol association data, enabling unified analysis of physical and logical connection relationships between network devices, thereby improving the completeness of topology identification in complex network environments. This invention locally updates the topology subgraphs corresponding to affected device nodes and establishes association relationships between device entities, port entities, and service entities, enabling the network topology structure to be updated synchronously with changes in network status, thereby reducing the problem of topology data update lag. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the overall process logic of a multi-source data fusion and acquisition method for power dispatch automation provided in one embodiment of the present invention; Figure 2 A flowchart illustrating the unified network entity model construction process of a multi-source data fusion and acquisition method for power dispatch automation provided in one embodiment of the present invention; Figure 3 The flowchart illustrates the dynamic local topology update process of a multi-source data fusion acquisition method for power dispatch automation provided in one embodiment of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0023] Example 1, referring to Figures 1-3 As an embodiment of the present invention, a method for multi-source data fusion acquisition in power dispatch automation is provided, comprising: S100: Acquire multi-source network protocol data, preprocess the multi-source network protocol data, and obtain standardized structured data; S200: Construct the relationships between network nodes based on standardized structural data to generate a unified network entity model; S300: Constructs a device node relationship graph based on the device node adjacency relationship in the unified network entity model, and generates network topology relationships based on multi-protocol association data; S400: Construct a topology subgraph for the device nodes corresponding to the detected network state change event, and perform local topology updates on the topology subgraph to obtain the updated network topology relationship; S500: Obtain network entities, establish network semantic relationships between network entities, and generate a network knowledge graph; S600: Based on the unified network entity model, the updated network topology relationships and network knowledge graph, a hierarchical network topology view is generated to realize the fusion and collection of multi-source data for power dispatch automation.

[0024] It should be noted that this invention forms standardized data through multi-source protocol data preprocessing, constructs a unified network entity model, and accurately generates a complete network topology; it supports local dynamic updates under state changes, reduces computational redundancy, achieves semantic association by combining knowledge graphs, and finally presents it in a hierarchical view for visualization, effectively solving problems such as heterogeneous data conflicts, incomplete topology identification, and update lag, and significantly improving the efficiency and reliability of power dispatch automation data collection.

[0025] In this embodiment of the invention, step S100 includes the following sub-steps A1-A3; In A1: Based on the protocol data packets sent by different network devices, the corresponding data format is identified according to the protocol type, the corresponding protocol parsing rules are called to parse the core fields in the packet, and the parsed protocol data is converted into unified structure data; In A2: Nodes are deployed in different network areas, and different protocols are used to obtain data collected by the devices; In A3: Based on the unified structure data and the data collected by the devices, a mapping relationship between different protocol fields and unified fields is established, the basic fields of the devices are uniformly converted, the data timestamps of different devices are uniformly calibrated, and abnormal data is filtered to obtain standardized structure data.

[0026] In one optional embodiment, parsing the core fields in the message can be a parsing method based on key field location, which involves decoding the protocol message in binary or text format; locating key fields such as device identifier, port, and status according to a preset offset or delimiter; directly extracting the field content to complete the parsing; and outputting standardized field values. In another optional embodiment, parsing the core fields in the message can also be done using a regular expression-based message parsing method, configuring a corresponding regular expression template for each protocol; using regular expressions to match core fields such as device identifier, port, and status in the message; extracting the matching results and completing field formatting; and outputting the standardized field data. In this embodiment of the invention, parsing the core fields in the message includes a rule-matching parsing method based on protocol type; Specifically, it listens to the SNMP port, Syslog port and Trap port in the network, receives the corresponding protocol data packets, classifies and stores the received protocol data packets according to the preset protocol type, and generates the corresponding protocol cache queue. The protocol header fields in the protocol data packets are read, and the protocol type is identified based on these fields. Specifically, SNMP protocol data packets are identified by the Object Identifier (OID) field, LLDP protocol data packets by the TLV field, ARP protocol data packets by both the hardware address and protocol address fields, FDB protocol data packets by both the MAC address and switch port fields, Syslog protocol data packets by the log level field, and Trap protocol data packets by the event identifier field.

[0027] Based on the protocol type, the corresponding protocol parsing rules are invoked to extract the device identifier field, port field, and status field from the protocol data packet. The device identifier field includes the device IP address, device MAC address, and device name; the port field includes the port number, port type, and port status; and the status field includes the device operating status, interface status, and link status.

[0028] After parsing the protocol data message, a protocol structure data set is generated, which is represented as follows: in, Indicates the first The protocol structure data set corresponding to each device; This represents the SNMP protocol structure data; This represents the LLDP protocol structure data; This represents ARP protocol structure data; This represents the FDB protocol structure data; This represents the Syslog protocol structure data; This represents the structure data of the Trap protocol.

[0029] Fields from different protocols are uniformly converted, and data with a unified structure is generated. This includes establishing field mapping relationships: ;in, Indicates a field mapping function; Represents the original protocol field space; This represents a unified field space. Based on the field mapping relationship, the device name field in the SNMP protocol, the device identifier field in the LLDP protocol, and the IP address field in the ARP protocol are converted into a unified device identifier field.

[0030] Further, the time fields in different protocols are uniformly calibrated, and the calibrated time is represented as follows: in, Indicates the time after calibration; Indicates the original time; This indicates the device time offset.

[0031] The device time offset is generated based on the Network Time Protocol (NTP) synchronization results.

[0032] After generating the unified structure data, the unified structure data will undergo data standardization processing.

[0033] It should be noted that by uniformly accessing, parsing fields, and converting the data of Simple Network Management Protocol (SNMP), Link Layer Discovery Protocol (LLDP), Address Resolution Protocol (ARP), Forwarding Table Protocol (FDB), System Log Protocol (Syslog), and Trap Alarm Protocol (Trap), data corresponding to different protocols can form a unified data structure, thereby reducing the problem of data format differences between heterogeneous protocol data.

[0034] Edge acquisition nodes are deployed within different network areas to collect data from network devices in their respective areas. The central acquisition node receives, aggregates, and stores the data uploaded by the edge acquisition nodes, and schedules acquisition tasks for different protocols. It generates acquisition tasks based on the protocol type and issues acquisition instructions to the edge acquisition nodes according to a preset acquisition cycle.

[0035] The data collection period is expressed as: in, Indicates the average acquisition period; Indicates the number of protocol types; Indicates the first The collection time corresponding to the protocol class.

[0036] After receiving a data collection command, the edge data acquisition node obtains network device operational status data via the SNMP protocol. This data includes device CPU utilization, memory utilization, interface status, port traffic, and device uptime. The edge data acquisition node uses the Object Identifier (OID) in the SNMP protocol to retrieve the corresponding operational status data and establishes a data association based on the device's IP address.

[0037] Edge acquisition nodes obtain device neighbor relationship data through the LLDP protocol, the IP address-MAC address mapping through the ARP protocol, and the MAC address-switch port mapping through the FDB protocol. The LLDP protocol data includes local device identifier, remote device identifier, local port number, and remote port number; the ARP protocol data includes IP address and MAC address fields; and the FDB protocol data includes switch port and MAC address fields.

[0038] Edge acquisition nodes construct a device link relationship data set based on LLDP protocol data, ARP protocol data, and FDB protocol data. The link relationship data set is represented as follows: in, Indicates the first A collection of link relationship data corresponding to each device; This represents LLDP protocol link data; This represents ARP protocol mapping data; This indicates FDB protocol forwarding table data.

[0039] Edge acquisition nodes obtain network alarm event data through the Trap and Syslog protocols. The Trap protocol receives device status change events, while the Syslog protocol receives device log events. Trap protocol event data includes port down events, device offline events, and link status change events, while Syslog protocol event data includes device log level, log type, and log generation time.

[0040] The edge acquisition node encapsulates the acquired data and generates corresponding device acquisition data, which is represented as follows: in, Indicates the first Each device collects data; This indicates the equipment's operating status data; Represents link relationship data; This indicates alarm event data.

[0041] After the edge acquisition node generates the data collected by the device, it sends it to the central acquisition node through the network communication interface. The central acquisition node then performs data standardization processing on the data collected by the device.

[0042] It should be noted that by deploying collection nodes in different network areas and classifying and collecting network device operating status data, link connection data, and alarm event data, device data in different network areas can be uniformly summarized, thereby reducing the problem of abnormal data correlation caused by the dispersion of network device data.

[0043] In one optional embodiment, filtering abnormal data can be a threshold-based filtering method, which presets normal threshold ranges for device status, port traffic, and log frequency; compares real-time data with the thresholds, and determines data exceeding the threshold range as abnormal; directly discards data exceeding the limits and retains valid data that meets the thresholds; and completes the filtering and outputs the results. In another optional embodiment, filtering of abnormal data can also be based on data verification and legality validation. The message is verified and validated, and verification errors are directly determined as abnormal data; the device identifier, port number, and timestamp format are checked for legality; data with incorrect format, empty values, and garbled characters are directly filtered out; and valid data with legal format and passed verification is output. In this embodiment of the invention, filtering of abnormal data includes a filtering method based on time windows and hash verification; Specifically, a mapping relationship between different protocol fields and a unified field is established, and the device identification field, port field, and status field in different protocols are uniformly converted according to the field mapping relationship; a protocol field mapping table is pre-established, which includes the correspondence between SNMP protocol fields, LLDP protocol fields, ARP protocol fields, FDB protocol fields, Syslog protocol fields, and Trap protocol fields and the unified field.

[0044] The field mapping relationship is represented as follows: ;in: Indicates a field mapping function; Represents the original protocol field space; This represents a unified field space.

[0045] Based on the field mapping function, the device name field in the SNMP protocol, the device identifier field in the LLDP protocol, and the IP address field in the ARP protocol are uniformly converted into the device identifier field; the switch port field in the FDB protocol and the port number field in the LLDP protocol are uniformly converted into the port identifier field; and the event status field in the Trap protocol and the log status field in the Syslog protocol are uniformly converted into the device status field.

[0046] The timestamps of data generated by different devices are uniformly calibrated.

[0047] Specifically, a device time offset is generated based on the Network Time Protocol (NTP) synchronization result, and the original time is calibrated based on the device time offset. The calibrated time is expressed as follows: in, Indicates the time after calibration; Indicates the original time; This indicates the device time offset.

[0048] The data from different protocols are further arranged in a unified time sequence to generate unified time series data.

[0049] Filter duplicate log data, abnormal fluctuation data, and conflict state data.

[0050] Specifically, the Syslog log data is compared by hash value, and duplicate log data is filtered when the hash values ​​of logs in consecutive time windows are the same; the device operating status data is detected by sliding window, and abnormal fluctuation data is filtered when the device status data deviates from the average status value in the current time window; and multiple sets of status data of the same device in the same time window are compared by time, and the status data with the newest timestamp is retained.

[0051] The average equipment status is expressed as: in, This represents the average value of the equipment status; This indicates the number of status data points in the current time window; Indicates the first Each status data value.

[0052] Standardized structured data is generated based on unified fields, unified time series, and filtered status data.

[0053] Standardized structured data is represented as follows: in, Indicates the first Standardized structural data corresponding to each device; This represents the device identification data; Indicates port data; Represents time series data; This indicates the device status data.

[0054] It should be noted that by establishing a field mapping relationship between different protocol fields and a unified field, and by uniformly calibrating and filtering the data timestamps generated by different devices, data corresponding to different protocols can form a unified structure data, thereby reducing the data format differences and state conflicts between heterogeneous protocol data.

[0055] In this embodiment of the invention, step S200 includes the following sub-steps B1-B5; In B1: Device nodes are generated based on device identification information in the standardized structure data; In B2: Port nodes are generated based on port identification information; In B3: Link nodes are generated using data from at least two protocols; In B4: Establish the association relationships between device nodes, port nodes, and link nodes, and generate a node adjacency matrix; In B5: A unified network entity model is generated based on device nodes, port nodes, link nodes, and node adjacency matrix.

[0056] In one optional embodiment, link nodes can be generated using at least two protocol data, namely SNMP protocol data and ARP protocol data. The SNMP protocol is used to obtain device interface status, port index, and neighbor node information; the ARP protocol is used to obtain the mapping relationship between IP address and MAC address; the physical connection relationship between devices is determined based on the matching of port index and MAC address; and link nodes between devices are generated. In another optional embodiment, the link node can be generated using at least two protocol data, namely LLDP protocol data and SNMP protocol data. The LLDP protocol is used to obtain device adjacency relationships, local port and remote port information; the SNMP protocol is used to obtain port running status, traffic and interface index information; the link authenticity is verified by combining the neighbor relationship and port status; and a stable and reliable link node is generated. In this embodiment of the invention, link nodes are generated using at least two protocol data, including LLDP protocol data, ARP protocol data, and FDB protocol data. Specifically, such as Figure 2 As shown, the device identification data and port data in the standardized structure data are read, and the corresponding device nodes and port nodes are generated.

[0057] Device nodes are generated based on the device's IP address, MAC address, name, and type, and each device node is assigned a unique identifier. A device node is represented as follows: in, Indicates the first One device node; Indicates the equipment category; Indicate the equipment model; Indicates the device address; Indicates the device status.

[0058] Port nodes are further generated based on port number, port type, and port status, and the attribution relationship between port nodes and device nodes is established.

[0059] Link nodes are generated based on LLDP protocol data, ARP protocol data, and FDB protocol data.

[0060] Specifically, the physical connection relationship between devices is established based on the local device identifier, remote device identifier, local port number, and remote port number in the LLDP protocol; the network address mapping relationship is established based on the IP address to MAC address correspondence in the ARP protocol; and the switching link relationship is established based on the switching port field and MAC address field in the FDB protocol. Link nodes are represented as follows: in, Indicates the link nodes between device nodes; Indicates the starting device node; Indicates the target device node; This indicates the node connection relationship.

[0061] Establish the relationships between device nodes, port nodes, and link nodes. Specifically, establish device port mapping relationships based on the affiliation between device nodes and port nodes, establish device connection relationships based on the link nodes between device nodes, and establish network address association relationships based on the IP address mapping relationships in the ARP protocol.

[0062] Generate a node adjacency matrix based on the connection relationships between device nodes: in: A unified network entity model is generated based on device nodes, port nodes, link nodes, and node adjacency matrix.

[0063] The unified network entity model is represented as: ;in: Represents a unified network entity model; Represents a set of device nodes; Represents the set of link nodes.

[0064] It should be noted that by constructing the relationships between device nodes, port nodes, and link nodes based on standardized structural data and generating a unified network entity model, device relationship data corresponding to different protocols can form a unified network structure, thereby reducing the problem of incomplete topology caused by scattered device connection relationships.

[0065] In this embodiment of the invention, step S300 includes the following sub-steps C1-C7; In C1: Extract device node data and link node data from the unified network entity model; In C2: Extract device relationship features from at least three protocol data to generate device feature vectors; In C3: the adjacency relationships between device nodes are constructed based on the device feature vectors, and a device node adjacency relationship matrix is ​​generated; In C4: Construct a device node relationship graph based on the device node adjacency matrix; In C5: Feature propagation is performed on the device node relationship graph to update the device node features; In C6: Generate device node embedding vectors based on the updated device node features, and calculate the association values ​​between device nodes; In C7: Device connection relationships are determined based on association values, and network topology relationships are generated.

[0066] In this embodiment of the invention, device node data and link node data in the unified network entity model are read, and device relationship features in SNMP data, LLDP data, ARP data and FDB data are extracted.

[0067] The device relationship characteristics in SNMP data include device operating status, interface status, and port traffic; the device relationship characteristics in LLDP data include local device identifier, remote device identifier, local port number, and remote port number; the device relationship characteristics in ARP data include the correspondence between IP address and MAC address; and the device relationship characteristics in FDB data include the correspondence between MAC address and switch port.

[0068] Based on the equipment relationship characteristics, a device feature vector is generated. The device feature vector is represented as follows: in, Indicates the first The device feature vector corresponding to each device node; Indicates the characteristics of the equipment's operating status; Indicates port connection characteristics; Indicates the link adjacency characteristics; This indicates the characteristics of network address mapping.

[0069] The adjacency relationships between device nodes are constructed based on device feature vectors. Specifically, physical adjacency relationships are established based on device adjacency relationships in the LLDP protocol, network address association relationships are established based on IP address mapping relationships in the ARP protocol, and port connection relationships are established based on switch port mapping relationships in the FDB protocol.

[0070] Generate a device node adjacency matrix: in: A device node relationship graph is constructed based on the device node adjacency matrix. In this graph, nodes represent network devices, and edges represent the connections between devices.

[0071] In one optional embodiment, feature propagation on the device node relationship graph can be performed using a graph convolution feature propagation method based on the adjacency matrix. This involves constructing an adjacency matrix and a degree matrix based on the adjacency relationships of device nodes; normalizing the adjacency matrix and constructing a graph convolution kernel; using device status, port, and link features as initial features, performing matrix multiplication propagation layer by layer; aggregating the features of adjacent nodes and updating the feature vector of the current node; and outputting node features used to determine device connection relationships. In another optional embodiment, feature propagation on the device node relationship graph can also be a node feature propagation method based on random walk. The method involves selecting a target node as the starting point from the device node relationship graph; performing a random walk among adjacent nodes at a preset number of steps; statistically analyzing and weighting the node features on the walk path; fusing the path features into the propagated features of the current node; and completing the update of the node features of the entire graph for topological relationship reasoning. In this embodiment of the invention, feature propagation of the device node relationship graph includes using a graph neural network to propagate features of the device node relationship graph; Specifically, based on the set of adjacent nodes in the device node relationship graph, the device node features are iteratively updated. The device node feature update process is represented as follows: in, Indicates the first Device node characteristics in the layer; Indicates the relationship with the first The set of adjacent nodes corresponding to each device node; and Represents the feature weight matrix; This represents the activation function.

[0072] Based on the updated device node features, a device node embedding vector is generated, which is represented as follows: ;in: Indicates the first The device node embedding vector corresponding to each device node; This indicates the number of propagation layers in the graph neural network model.

[0073] The association relationships between device nodes are calculated based on the device node embedding vectors. The device node association relationships are represented as follows: in, Indicates the first The device node and the first The association values ​​between device nodes; Indicates the first The device node embedding vector corresponding to each device node; This represents the device node embedding vector corresponding to the j-th device node; This indicates the total number of device nodes.

[0074] It should be noted that device connectivity is determined based on the association values ​​between device nodes, and network topology is generated based on these connectivity values. By constructing a device node relationship graph based on the adjacency relationships between device nodes and generating network topology based on multi-protocol association data, unified association analysis of device relationship data corresponding to different protocols can be performed, thereby reducing the problem of missing topology relationships in complex network environments.

[0075] In this embodiment of the invention, step S400 includes the following sub-steps D1-D4; In D1: A network status change event is detected, and the device identification information in the event data is parsed; In D2: The affected device nodes are determined based on the device identification information, and the associated device nodes adjacent to the affected device nodes are determined, generating a set of affected device nodes; In D3: Construct a topology subgraph based on the set of affected device nodes and their corresponding link relationships; In D4: The connections in the topology subgraph are recalculated, and the network topology is updated based on the recalculated connections.

[0076] In embodiments of the present invention, such as Figure 3 As shown, the system monitors Trap protocol events, Syslog log events, and device status change events in real time and parses the event data. Trap protocol events include port down events, port up events, device offline events, and link status change events; Syslog log events include link exception logs, interface exception logs, and device operation exception logs.

[0077] Based on the device identifier field, port number field, and event type field in the event data, a corresponding event data set is generated. The event data set is represented as follows: in, Indicates the first The event data set corresponding to each device; This represents Trap protocol event data; This represents Syslog log event data; This indicates data on changes in device status.

[0078] The affected device nodes are determined based on the device identification information corresponding to the event. Specifically, the corresponding device node is retrieved from the unified network entity model based on the device identification field, and the associated device nodes connected to the current device node are determined based on the adjacency relationship between device nodes.

[0079] A set of affected device nodes is generated based on the adjacency relationships between device nodes. The set of affected device nodes is represented as follows: ; This represents the set of affected device nodes; Indicates the first One affected device node.

[0080] Construct a corresponding topology subgraph based on the set of affected device nodes.

[0081] The topological subgraph is represented as: ;in: Represents a topological subgraph; This represents the set of affected device nodes; This represents the set of affected links. Specifically, based on the link relationships between device nodes, the link nodes corresponding to the affected device nodes are added to the affected link set.

[0082] The connectivity relationships in the topology subgraph are recalculated. Based on the neighbor relationships in the LLDP protocol data, the IP address mapping relationships in the ARP protocol data, and the switch port mapping relationships in the FDB protocol data, the connectivity relationships between device nodes are recalculated, and link weight values ​​are generated based on the connectivity status between device nodes.

[0083] The link weight value is represented as: ;in: This represents the link weight value between device nodes; This represents the neighbor relationship value corresponding to the LLDP protocol; This represents the address mapping value corresponding to the ARP protocol; This represents the port mapping value corresponding to the FDB protocol; , as well as This represents the corresponding weighting coefficient.

[0084] The connection relationships in the topology subgraph are recalculated based on the link weight values. The objective of the connection relationship calculation is expressed as: ,in, For nodes and The link edge between them, This is the set of all candidate links.

[0085] The network topology is updated based on the recalculated connectivity relationships. This involves replacing the original link relationships in the unified network entity model and synchronously updating the adjacency matrix between the corresponding device nodes.

[0086] It should be noted that by locally updating the topology subgraph corresponding to the affected device node when a change in link state, device state, or neighbor relationship is detected, the network topology relationship can be synchronously adjusted according to the change in device state, thereby reducing the computational redundancy problem generated during the full topology reconstruction process.

[0087] In this embodiment of the invention, step S500 includes the following sub-steps E1-E3; In E1: Extract device entities, port entities, and business system entities; In E2: Establish the connection relationships between device entities, the mapping relationships between port entities and device entities, and the association relationships between business system entities and device entities; In E3: A network knowledge graph is generated based on the connection, mapping, and association relationships between device entities.

[0088] In this embodiment of the invention, device entities, port entities, and business system entities are extracted from the unified network entity model and business system data. Device entities include device IP address, device name, device type, and device status; port entities include port number, port type, and port status; and business system entities include business system identifier, business system name, and information about the device to which the business system belongs.

[0089] An entity data set is generated based on the device identifier field, port identifier field, and business system identifier field. The entity data set is represented as follows: in, Indicates the first A collection of entity data; Represents device entity data; Represents port entity data; This represents entity data in the business system.

[0090] Establish connections between device entities. This involves establishing physical connections between device entities based on link nodes in the unified network entity model, and establishing logical connections between device entities based on adjacency relationships between device nodes.

[0091] The connection relationships between device entities are represented as follows: in, Indicates the connection relationships between device entities; Indicates the starting device entity; Represents the target device entity; Indicates the type of relationship between device entities.

[0092] Establish a mapping relationship between port entities and device entities, and establish an association relationship between business system entities and device entities. Specifically, establish the attribution relationship between port entities and device entities based on the device identifier to which the port belongs, and establish the deployment relationship between business system entities and device entities based on business system deployment information.

[0093] The mapping relationship between port entities and device entities is represented as follows: in, This indicates the mapping relationship between port entities and device entities; Indicates a port entity; Represents the physical device.

[0094] The relationship between business system entities and device entities is represented as follows: ;in: This indicates the relationship between business system entities and device entities; Represents a business system entity; Represents the physical device.

[0095] A network knowledge graph is generated based on the connection relationships between device entities, the mapping relationships between port entities and device entities, and the association relationships between business system entities and device entities.

[0096] The network knowledge graph is represented as: ;in: Represents a network knowledge graph; Represents a set of entities; Represents a set of entity relations; Represents a set of entity mapping relationships.

[0097] It should be noted that by establishing network semantic relationships based on device entities, port entities, and business system entities, and generating a network knowledge graph, network device relationships, port relationships, and business deployment relationships can be uniformly associated, thereby reducing the problem of scattered associations between network device data and business system data.

[0098] In this embodiment of the invention, step S600 includes the following sub-steps F1-F4; In F1: Network nodes are hierarchically divided according to device type, generating node hierarchy relationships; In F2: Based on the connection relationships between device nodes and the node hierarchy, the topology layout relationship and node layout coordinates are generated; In F3: Primary links and redundant links are distinguished and identified; In F4: Based on the unified network entity model, network topology relationships, and network knowledge graph, combined with topology layout relationships and link identifiers, a hierarchical network topology view is generated.

[0099] In this embodiment of the invention, device node data from the unified network entity model is read, and network nodes are hierarchically divided according to device type. The device types include core layer devices, aggregation layer devices, and access layer devices.

[0100] The corresponding node hierarchy is generated based on the device type. The node hierarchy is represented as follows: in, Indicates the first The hierarchical value corresponding to each device node; Indicates the first Each device node.

[0101] The topology layout is generated based on the connection relationships between device nodes. Specifically, the node connection relationships are determined based on the adjacency matrix between device nodes, and the node layout coordinates are generated based on the node hierarchy.

[0102] The device node layout coordinates are represented as follows: ;in: Indicates the first The layout coordinates of each device node; Indicates the horizontal coordinate of the device node; This represents the vertical coordinates of a device node. The vertical coordinates are generated based on the node's hierarchy value, while the horizontal coordinates are arranged according to the number of nodes at the same hierarchy. Furthermore, a topology layout is generated based on the connections between device nodes, and a visual connection relationship is established between device nodes and link nodes.

[0103] Primary and redundant links are distinguished and identified. Specifically, primary and redundant links are determined based on link weight values ​​and the connection status between device nodes.

[0104] The link weight value is represented as: in, This represents the link weight value between device nodes; Indicates the link connection status value; Indicates the link's stable state value; and This represents the corresponding weighting coefficient.

[0105] Specifically, when the link weight value is greater than the preset main link threshold, the corresponding link is determined as the main link; when the link weight value is less than the preset main link threshold, the corresponding link is determined as a redundant link.

[0106] Generate corresponding link display identifiers based on the types of primary and redundant links. Generate a visual network topology view based on the unified network entity model, network topology relationships, and network knowledge graph.

[0107] A device node display area is generated based on the device node layout coordinates, a link display area is generated based on the connection relationships between device nodes, and a business association display area is generated based on the association relationships between business system entities and device entities in the network knowledge graph. Furthermore, the status of the corresponding device nodes and link nodes is updated based on device status data, link status data, and business system association data, and the corresponding visualized network topology view is updated synchronously.

[0108] It should be noted that by generating a hierarchical network topology view based on a unified network entity model, network topology relationships, and network knowledge graph, device connection relationships, link status relationships, and service association relationships can be uniformly displayed, thereby reducing the problem of display separation between network topology relationships and service deployment relationships.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0110] Example 2: The above is an illustrative scheme of the power dispatch automation multi-source data fusion acquisition method of this embodiment. It should be noted that the technical solution of this power dispatch automation multi-source data fusion acquisition system belongs to the same concept as the technical solution of the power dispatch automation multi-source data fusion acquisition method described above. Details not described in detail in the technical solution of the power dispatch automation multi-source data fusion acquisition system in this embodiment can be found in the description of the technical solution of the power dispatch automation multi-source data fusion acquisition method described above.

[0111] The power dispatch automation multi-source data fusion acquisition system in this embodiment includes: The data preprocessing module is used to acquire multi-source network protocol data, preprocess the multi-source network protocol data, and obtain standardized structured data. The unified modeling module is used to construct the relationships between network nodes based on the standardized structure data and generate a unified network entity model. The topology reasoning module is used to construct a device node relationship graph based on the device node adjacency relationship in the unified network entity model, and generate network topology relationships based on multi-protocol association data. The dynamic topology update module is used to construct a topology subgraph for the device nodes corresponding to the detected network state change event, and to perform local topology updates on the topology subgraph to obtain the updated network topology relationship. The knowledge graph fusion module is used to acquire network entities, establish network semantic relationships between the network entities, and generate a network knowledge graph. The topology visualization module is used to generate a hierarchical network topology view based on the unified network entity model, the updated network topology relationships, and the network knowledge graph, thereby realizing the fusion and collection of multi-source data for power dispatch automation.

[0112] This embodiment also provides a computer device applicable to the multi-source data fusion acquisition method for power dispatch automation, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the power dispatch automation multi-source data fusion acquisition method proposed in the above embodiments.

[0113] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the power dispatch automation multi-source data fusion acquisition method proposed in the above embodiments.

[0114] The storage medium proposed in this embodiment and the multi-source data fusion acquisition method for realizing power dispatch automation proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0115] Based on the above description of the implementation methods, those skilled in the art will clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computing device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

Claims

1. A method for multi-source data fusion and acquisition in power dispatch automation, characterized in that, include: Acquire multi-source network protocol data, preprocess the multi-source network protocol data to obtain standardized structured data; Based on the standardized structural data, the relationships between network nodes are constructed to generate a unified network entity model; A device node relationship graph is constructed based on the adjacency relationships of device nodes in the unified network entity model, and a network topology relationship is generated based on multi-protocol association data; A topology subgraph is constructed for the device nodes corresponding to the detected network state change events, and the topology subgraph is locally updated to obtain the updated network topology relationship; Obtain network entities, establish network semantic relationships between the network entities, and generate a network knowledge graph; Based on the unified network entity model, the updated network topology and network knowledge graph generate a hierarchical network topology view, enabling the fusion and collection of multi-source data for automated power dispatching.

2. The power dispatch automation multi-source data fusion acquisition method as described in claim 1, characterized in that, The preprocessing of the multi-source network protocol data yields standardized structured data, including: Based on the protocol data packets sent by different network devices, the corresponding data format is identified according to the protocol type, the corresponding protocol parsing rules are called to parse the core fields in the packets, and the parsed protocol data is converted into unified structure data. Nodes are deployed in different network areas, and different protocols are used to obtain data collected by the devices. Based on the unified structure data and the data collected by the devices, a mapping relationship between different protocol fields and unified fields is established. The basic fields of the devices are uniformly converted, the data timestamps of different devices are uniformly calibrated, and abnormal data is filtered to obtain standardized structure data.

3. The power dispatch automation multi-source data fusion acquisition method as described in claim 2, characterized in that, Building the relationships between network nodes and generating a unified network entity model includes: Device nodes are generated based on device identification information in standardized structural data; Generate port nodes based on port identification information; Generate link nodes using data from at least two protocols; Establish the relationships between device nodes, port nodes, and link nodes, and generate a node adjacency matrix; A unified network entity model is generated based on the device nodes, port nodes, link nodes, and node adjacency matrix.

4. The power dispatch automation multi-source data fusion acquisition method as described in claim 3, characterized in that, Constructing a device node relationship graph and generating network topology relationships based on multi-protocol association data includes: Extract device node data and link node data from the unified network entity model; Extract device relationship features from at least three types of protocol data to generate device feature vectors; The adjacency relationships between device nodes are constructed based on the device feature vectors, and a device node adjacency relationship matrix is ​​generated. Construct a device node relationship graph based on the device node adjacency matrix; Perform feature propagation on the device node relationship graph and update the device node features; Based on the updated device node features, generate device node embedding vectors and calculate the association values ​​between device nodes; The device connection relationships are determined based on the association values, and the network topology is generated.

5. The power dispatch automation multi-source data fusion acquisition method as described in claim 4, characterized in that, Constructing a topological subgraph and performing local topological updates on the topological subgraph includes: A network status change event was detected, and the device identification information in the event data was parsed. The affected device nodes are identified based on the device identification information, and the associated device nodes adjacent to the affected device nodes are identified to generate a set of affected device nodes; Construct a topology subgraph based on the set of affected device nodes and their corresponding link relationships; The connectivity relationships in the topology subgraph are recalculated, and the network topology relationships are updated based on the recalculated connectivity relationships.

6. The power dispatch automation multi-source data fusion acquisition method as described in claim 5, characterized in that, Generative network knowledge graphs include: Extract device entities, port entities, and business system entities; Establish the connection relationships between device entities, the mapping relationships between port entities and device entities, and the association relationships between business system entities and device entities respectively; A network knowledge graph is generated based on the connection, mapping, and association relationships between device entities.

7. The power dispatch automation multi-source data fusion acquisition method as described in claim 1 or 6, characterized in that, Generating a hierarchical network topology view includes: The network nodes are hierarchically divided according to the device type, and a node hierarchy relationship is generated. Based on the connection relationships and node hierarchy relationships between device nodes, generate the topology layout relationship and node layout coordinates; Distinguish and identify primary links and redundant links; Based on the unified network entity model, network topology relationships, and network knowledge graph, combined with topology layout relationships and link identifiers, a hierarchical network topology view is generated.

8. A power dispatch automation multi-source data fusion acquisition system, employing the power dispatch automation multi-source data fusion acquisition method as described in any one of claims 1-7, characterized in that, include: The data preprocessing module is used to acquire multi-source network protocol data, preprocess the multi-source network protocol data, and obtain standardized structured data. The unified modeling module is used to construct the relationships between network nodes based on the standardized structure data and generate a unified network entity model. The topology reasoning module is used to construct a device node relationship graph based on the device node adjacency relationship in the unified network entity model, and generate network topology relationships based on multi-protocol association data. The dynamic topology update module is used to construct a topology subgraph for the device nodes corresponding to the detected network state change event, and to perform local topology updates on the topology subgraph to obtain the updated network topology relationship. The knowledge graph fusion module is used to acquire network entities, establish network semantic relationships between the network entities, and generate a network knowledge graph. The topology visualization module is used to generate a hierarchical network topology view based on the unified network entity model, the updated network topology relationships, and the network knowledge graph, thereby realizing the fusion and collection of multi-source data for power dispatch automation.

9. A computer device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the power dispatch automation multi-source data fusion acquisition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the power dispatch automation multi-source data fusion acquisition method as described in any one of claims 1 to 7.

Images