Network equipment data acquisition method and system combined with large model
By using large language models and alarm knowledge graph technology, the problem of maintaining parsing rules in multi-device scenarios has been solved, enabling efficient and accurate network device data collection and fault root cause location, thus improving operation and maintenance efficiency and accuracy.
Patent Information
- Application Number
- CN202511274000.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-08
- Publication Date
- 2025-12-19
AI Technical Summary
Existing technologies require manual writing of parsing rules when dealing with multiple network devices, resulting in high maintenance costs, delayed rule updates, and difficulty in effectively locating the root cause of cross-device faults, especially in the case of new devices and multi-device scenarios where efficiency is low.
The system uses a large language model to generate device-adaptive parsing instructions, transforms unstructured data into structured indicators using NLP technology, constructs an alarm knowledge graph, combines temporal relationships and device topology, uses correlation algorithms to locate the root cause of the fault, and dynamically optimizes the module to adjust the sampling frequency and model parameters.
It achieves efficient parsing with multi-device compatibility, improves parsing accuracy by 40%, increases root cause localization efficiency by 20 times, and continuously evolves system capabilities, reducing plugin development and improving operation and maintenance efficiency.
Smart Images
Figure CN121168643A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of network operation and artificial intelligence, in particular to a network device data acquisition method and system combined with a large model. BACKGROUND
[0002] A typical data center contains servers (supporting Redfish / IPMI), switches (SNMPv2 / v3), dynamic environment devices (Modbus / HTTP API), etc. The traditional solution needs to customize collection scripts for each device. For example, a large cloud data center needs to maintain more than 200 device collection plug-ins, with high annual maintenance costs; device log formats are diverse (such as Cisco device logs using the "%SYS-5-CONFIG_I" format, and Huawei devices using the "ERROR:0x1001" encoding), and traditional regular expression parsing requires manual writing of more than 100,000 lines of rules, and the rule maintenance period is as long as 2 months when new device types are added; a certain IDC room once had "server CPU overload" and "switch port packet loss" alarms at the same time, and the traditional system independently analyzed and led to a false judgment of hardware failure, the actual root cause was device overheating caused by air conditioning system abnormalities, and manual troubleshooting was time-consuming.
[0003] The existing solution relies on manual writing of parsing rules, and when facing new devices such as 5G networks and edge computing, the rule update lag problem is prominent. For example, when a certain operator deploys a 5G base station, the traditional collection system needs to redevelop plug-ins due to the upgrade of the dynamic environment device protocol, resulting in a 3-day delay in the operation window. SUMMARY
[0004] The purpose of the present application is to provide a network device data acquisition method and system combined with a large model to solve the problems raised in the background.
[0005] To achieve the above purpose, the present application provides the following technical solution: a network device data acquisition method combined with a large model, comprising the following steps:
[0006] A universal adaptation layer is used to interface multiple source devices, supporting SSH, SNMP, Redfish, dynamic environment systems, databases, and message queue protocols, and mapping raw data to a unified data model;
[0007] A large language model is used to generate device-adapted parsing instructions, and NLP technology is used to convert unstructured performance / alarm data into structured indicators, and the parsing instructions are dynamically optimized according to device type, protocol version, and historical data characteristics;
[0008] An alarm knowledge graph is constructed based on the time sequence relationship and device topology, and a correlation degree algorithm is used to locate the cross-device fault root cause, and the knowledge graph nodes include performance fluctuations, resource threshold overruns, and hardware failure events.
[0009] Preferably, the process of dynamically generating the analysis instruction by the large model comprises: using a templated Prompt engineering to predefine an analysis instruction framework for switch and server device types; and continuously correcting the instruction template based on operation and maintenance feedback data by a reinforcement learning algorithm to improve the analysis accuracy to more than 95%.
[0010] Preferably, the construction of the alarm knowledge graph comprises: generating a device connection relationship graph by analyzing a network topology configuration file through an LLM; and calculating an alarm event correlation degree by using a sliding window time series analysis, and establishing a graph edge connection when the correlation degree R is greater than 0.7.
[0011] Preferably, the system further comprises a dynamic optimization module for dynamically adjusting the device sampling frequency: starting high-frequency sampling for a device with continuous N (N≥3) index abnormalities, and the sampling frequency is increased by 2-5 times the baseline frequency.
[0012] Preferably, the system further comprises a dynamic optimization module for fine-tuning the large model: fine-tuning the large model using the analysis results labeled by operation and maintenance personnel, and the fine-tuning period is less than or equal to 7 days.
[0013] A system applied to a network device data acquisition method combined with a large model, comprising:
[0014] A data interfacing and mapping module interfaces multiple source devices through a general adaptation layer, supports SSH, SNMP, Redfish, dynamic environment system, database and message queue protocols, and maps raw data to a unified data model;
[0015] A data analysis module generates device-adapted analysis instructions using a large language model, converts unstructured performance / alarm data into structured indicators through NLP technology, and the analysis instructions are dynamically optimized according to device type, protocol version and historical data characteristics;
[0016] A fault location module constructs an alarm knowledge graph based on time series relationship and device topology, locates cross-device fault root causes through a correlation degree algorithm, and the knowledge graph nodes include performance fluctuations, resource threshold overruns and hardware fault events.
[0017] Preferably, the process of dynamically generating the analysis instruction by the large model in the data analysis module comprises: an instruction framework predefinition unit that uses a templated Prompt engineering to predefine an analysis instruction framework for switch and server device types; and an instruction template correction unit that continuously corrects the instruction template based on operation and maintenance feedback data by a reinforcement learning algorithm to improve the analysis accuracy to more than 95%.
[0018] Preferably, the construction of the alarm knowledge graph in the fault positioning module comprises: a device connection relationship generation unit generates a device connection relationship graph by analyzing a network topology configuration file through an LLM; and a graph edge connection establishment unit calculates an alarm event correlation degree by using sliding window time series analysis, and establishes a graph edge connection when the correlation degree R is greater than 0.7.
[0019] Preferably, the dynamic optimization module further comprises a sampling frequency adjustment unit: the sampling frequency adjustment unit starts high-frequency sampling on a device with continuous N (N is greater than or equal to 3) index abnormalities, and the sampling frequency is increased by 2-5 times of the baseline frequency.
[0020] Preferably, the dynamic optimization module further comprises a large model fine-tuning unit: the large model fine-tuning unit fine-tunes the large model by using the analysis results labeled by the operation and maintenance personnel, and the fine-tuning period is less than or equal to 7 days.
[0021] Compared with the prior art, the present application has the following beneficial effects:
[0022] The network device data acquisition method and system combining a large model provided by the present application realize "one development, multiple device adaptation" through a general adaptation layer and a dynamic Prompt mechanism, and reduce the plug-in development amount by 40% compared with the traditional scheme; introduce NLP technology into device log processing, and improve the analysis accuracy by 40% compared with regular expressions; realize the continuous evolution of system capability through the two-dimensional feedback of acquisition strategy and model parameters; and break through the single-device analysis limitation of the traditional rule engine by combining time series analysis and topology graph, and improve the root cause positioning efficiency by 20 times. BRIEF DESCRIPTION OF DRAWINGS
[0023] Figure 1 The system flowchart of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical scheme of the present application clear, complete and more clear and understandable, the embodiments of the present application are further described in detail below in combination with the drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present application, not all embodiments, and are only used to explain the embodiments of the present application, and do not limit the embodiments of the present application. All other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0025] Embodiment one, the present application provides a technical scheme: a network device data acquisition method combining a large model, comprising the following steps:
[0026] Through the general adaptation layer, multi-source devices are connected, SSH, SNMP, Redfish, dynamic environment system, database and message queue protocols are supported, and original data is mapped to a unified data model.
[0027] A large language model is used to generate device adaptation analysis instructions, and unstructured performance / alarm data is converted into structured indicators through NLP technology. The analysis instructions are dynamically optimized according to device types, protocol versions and historical data characteristics. The process of dynamically generating analysis instructions by a large model includes: using a templated Prompt engineering to predefine analysis instruction frameworks for switch and server device types; continuously correcting instruction templates based on operation and maintenance feedback data through reinforcement learning algorithms to improve analysis accuracy to more than 95%.
[0028] An alarm knowledge graph is constructed based on time sequence relationship and device topology, and a cross-device fault root cause is located through a correlation degree algorithm. The knowledge graph nodes include performance fluctuations, resource threshold overruns and hardware failure events. The construction of the alarm knowledge graph includes: generating a device connection relationship graph by analyzing network topology configuration files through a LLM; using a sliding window time sequence analysis to calculate alarm event correlation degree, and establishing a graph edge connection when the correlation degree R>0.7.
[0029] A dynamic optimization module dynamically adjusts the sampling frequency of a device: high-frequency sampling is started for a device with continuous N times (N≥3) of abnormal indicators, and the sampling frequency is increased by 2-5 times the baseline frequency.
[0030] A dynamic optimization module fine-tunes a large model: the large model is fine-tuned using the analysis results labeled by operation and maintenance personnel, and the fine-tuning period is ≤7 days.
[0031] In the first embodiment, a system applied to a network device data acquisition method combined with a large model includes:
[0032] 1. General adaptation layer
[0033] The general adaptation layer includes a protocol plug-in library and a data standardization component. The protocol plug-in library includes an SNMP plug-in (supporting OID automatic discovery and MIB file analysis), a Redfish plug-in (generating a data model based on JSONSchema), and a dynamic environment API plug-in (adapting Modbus / TCP and HTTPRESTful interfaces); and the data standardization component is responsible for mapping data of different protocols to a unified model.
[0034] 2. Large model intelligent analysis engine
[0035] The unstructured data processing flow first is log segmentation, long text logs are processed through an NLP segmentation model; second is entity recognition, a fine-tuned BERT model is used to extract index values (such as "65%" in "CPU load: 65%") and alarm levels (Critical / Warning); and finally is relation extraction, the causal relationship in "fan failure causes power overload" is identified.
[0036] The multi-source alarm correlation algorithm is based on a time sequence sliding window (default 5 minutes) and a device topology graph (generated by parsing a network topology configuration file through LLM), and constructs an alarm correlation rule: when the correlation threshold R>0.7, it is determined as a related alarm.
[0037] The dynamic optimization module collects abnormal devices (such as CPU utilization>80% for three consecutive times) to start high-frequency sampling (from 5 minutes / time to 1 minute / time) by the policy optimization part; the large model continuous learning uses the reinforcement learning (RLHF) to fine-tune the model parameters through the analysis results (such as correct / incorrect labels) labeled by the operation and maintenance personnel, and automatically updates the analysis ability every week.
[0038] Although embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A network device data collection method combined with a large model, characterized by: The method comprises the following steps: Through a general adaptation layer, multi-source devices are connected, SSH, SNMP, Redfish, dynamic environment system, database and message queue protocols are supported, and original data is mapped to a unified data model; A large language model is used to generate device adaptation analysis instructions, unstructured performance / alarm data is converted into structured indicators through NLP technology, and the analysis instructions are dynamically optimized according to device types, protocol versions and historical data characteristics; An alarm knowledge graph is constructed based on a time sequence relationship and a device topology, a cross-device fault root cause is located through a correlation degree algorithm, and the knowledge graph nodes include performance fluctuations, resource threshold overruns and hardware fault events.
2. The network device data collection method combined with a large model according to claim 1, characterized in that: The process of dynamically generating analysis instructions by the large model includes: using a templated Prompt engineering, and predefining analysis instruction frameworks for switch and server device types; continuously correcting instruction templates based on operation and maintenance feedback data through a reinforcement learning algorithm, and improving the analysis accuracy to more than 95%.
3. The network device data collection method incorporating a large model according to claim 2, characterized in that: The construction of the alarm knowledge graph includes: analyzing network topology configuration files through an LLM to generate a device connection relationship graph; and using a sliding window time sequence analysis to calculate alarm event correlation degrees, and establishing graph edge connections when the correlation degree R is greater than 0.
7.
4. The network device data collection method incorporating a large model according to claim 3, characterized in that: The dynamic optimization module also dynamically adjusts device sampling frequencies: high-frequency sampling is started for devices with continuous N (N is greater than or equal to 3) index abnormalities, and the sampling frequency is increased by 2-5 times the baseline frequency.
5. The network device data collection method incorporating a large model according to claim 4, characterized in that: The dynamic optimization module also fine-tunes the large model: the large model is fine-tuned using operation and maintenance personnel labeled analysis results, and the fine-tuning period is less than or equal to 7 days.
6. A system applied to the network device data acquisition method of claim 5, wherein: The method comprises the following steps: A data connection and mapping module connects multi-source devices through a general adaptation layer, supports SSH, SNMP, Redfish, dynamic environment system, database and message queue protocols, and maps original data to a unified data model; A data analysis module uses a large language model to generate device adaptation analysis instructions, converts unstructured performance / alarm data into structured indicators through NLP technology, and dynamically optimizes the analysis instructions according to device types, protocol versions and historical data characteristics; A fault location module constructs an alarm knowledge graph based on a time sequence relationship and a device topology, locates a cross-device fault root cause through a correlation degree algorithm, and the knowledge graph nodes include performance fluctuations, resource threshold overruns and hardware fault events.
7. The system of claim 6, wherein: The process of dynamically generating analysis instructions by the large model in the data analysis module includes: an instruction framework predefinition unit that uses a templated Prompt engineering to predefine analysis instruction frameworks for switch and server device types; and an instruction template correction unit that continuously corrects instruction templates based on operation and maintenance feedback data through a reinforcement learning algorithm to improve the analysis accuracy to more than 95%.
8. The system of claim 7, wherein: The construction of the alarm knowledge graph in the fault location module includes: a device connection relationship generation unit that analyzes network topology configuration files through an LLM to generate a device connection relationship graph; and a graph edge connection establishment unit that uses a sliding window time sequence analysis to calculate alarm event correlation degrees, and establishes graph edge connections when the correlation degree R is greater than 0.
7.
9. The system of claim 8, wherein: Also include a dynamic optimization module, the dynamic optimization module includes a sampling frequency adjustment unit: the sampling frequency adjustment unit starts high-frequency sampling for the equipment with N (N≥3) consecutive index abnormalities, and the sampling frequency is increased by 2-5 times of the reference frequency.
10. The system of claim 9, wherein: Also include a dynamic optimization module, the dynamic optimization module includes a large model fine-tuning unit: the large model fine-tuning unit fine-tunes the large model by using the analysis results labeled by the operation and maintenance personnel, and the fine-tuning period is ≤7 days.