A network device configuration file automatic classification and analysis method and system
By employing a multi-level template extraction mechanism and a YAML rule engine, combined with a lightweight machine learning model, the automation and scalability issues in network device configuration file parsing are resolved. This enables accurate device type identification and cross-platform parsing, reducing maintenance costs and improving the system's versatility and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN FIBERHOME TECHNICAL SERVICES CO LTD
- Filing Date
- 2026-01-20
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies for parsing network device configuration files suffer from problems such as high dependence on manual intervention, hard-coded parsing logic, lack of anomaly detection, single output format, and poor scalability, making it impossible to achieve automated batch processing and joint parsing across device types.
It employs a multi-level template extraction mechanism and nested rule definition, combined with a lightweight machine learning model and YAML rule engine, to generate parsing files through multi-level pattern matching and state machine-driven line-by-line parsing, and performs multi-dimensional quality monitoring and anomaly detection.
It achieves automated classification and parsing of network device configuration files, improves the accuracy and availability of parsing results, reduces maintenance costs, supports dynamic adaptation and cross-platform parsing of devices from multiple vendors, and has high scalability and self-healing capabilities.
Smart Images
Figure CN121541922B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of network file parsing technology, and in particular relates to a method and system for automated classification and parsing of network device configuration files. Background Technology
[0002] As enterprise networks continue to expand, the variety of network devices is increasing, including routers, switches, firewalls, load balancers, etc. The configuration syntax and formats of devices from different vendors (such as Cisco, Juniper, Huawei, H3C, etc.) vary significantly. In traditional network operations and maintenance, configuration file parsing mainly relies on operations personnel manually identifying the device type and writing dedicated parsing scripts (such as Python or Perl scripts) for each device type. This has the following technical drawbacks:
[0003] 1. High dependence on manual intervention: Equipment type must be specified in advance, making automated batch processing impossible;
[0004] 2. Hard-coded parsing logic: Every time a new device type is added or the configuration syntax changes, the code needs to be modified and redeployed, resulting in high maintenance costs;
[0005] 3. Lack of anomaly detection mechanism: It cannot automatically identify syntax errors, semantic conflicts, or security violations in the configuration file;
[0006] 4. Limited output format: Most scripts only support raw text output, making it difficult to integrate with data analysis platforms and configuration management systems;
[0007] 5. Poor scalability: Single-layer template structures cannot handle complex nested configurations (such as address families under routing protocols or sub-interfaces under interfaces), and do not support joint resolution across device types.
[0008] Existing technologies such as Ansible and Puppet configuration management tools offer some parsing capabilities, but they primarily focus on configuration generation rather than parsing. Some network analysis platforms use fixed regular expression libraries for parsing, but lack flexible rule definition languages and intelligent classification capabilities. Therefore, there is an urgent need for an automated solution that can automatically identify device types, flexibly adapt to multi-vendor syntax, support complex-level configuration parsing, and has high scalability. Summary of the Invention
[0009] To address the shortcomings of existing technologies, this invention provides an automated classification and parsing method for network device configuration files.
[0010] Firstly, the method includes,
[0011] Obtain the original configuration file from a heterogeneous data source;
[0012] Convert the obtained raw configuration into a parsing-friendly standardized format;
[0013] The standardized format after parsing and conversion is generated into a parsed file, and the parsed file is subjected to multi-dimensional quality monitoring.
[0014] The monitored and parsed results are converted into a format usable by downstream systems for subsequent data analysis and integration.
[0015] Furthermore, obtaining the original configuration file from the heterogeneous data source specifically includes obtaining the original configuration file from the local file system, network shared storage, and device real-time interface.
[0016] Furthermore, the process of obtaining the original configuration file from the heterogeneous data source also includes verifying the original configuration file using a three-level verification system, generating a unique identifier for the original configuration file, and associating it with data lineage metadata.
[0017] Furthermore, the process of converting the acquired raw configuration into a parsing-friendly standardized format specifically includes: removing comments using multi-level pattern matching; merging continuation lines for vendor-differentiated syntax; performing hierarchical parsing using indentation standardization; processing interactive CLI logs using configuration pattern stripping; constructing a line number mapping table using a dual-dictionary structure; and adding metadata tags to configuration block boundary lines after cleaning.
[0018] Furthermore, the standardized format after parsing and conversion generates a parsing file, specifically including:
[0019] The original configuration file is divided into a multi-level structure, including global configuration, interface configuration, and route configuration, and the matching rules for each level are defined by nested templates.
[0020] By combining a keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device types, and template trees of different device types are organized into a unified template set.
[0021] The logic for extracting fields is dynamically defined using YAML rules, and a general data model is built to mask device differences.
[0022] A state machine is used to parse the text line by line, and regular expressions are used to achieve line-by-line or block-by-block matching to generate a parsed file.
[0023] Secondly, the system includes: a data acquisition module, a standardization module, a parsing module, and a conversion module;
[0024] The data acquisition module is used to obtain the original configuration file from the heterogeneous data source;
[0025] The standardization module is used to convert the acquired raw configuration into a parsing-friendly standardized format;
[0026] The parsing module is used to parse the converted standardized format, generate a parsed file, and perform multi-dimensional quality monitoring on the parsed file;
[0027] The conversion module is used to convert the monitored and parsed results into a format usable by downstream systems for subsequent data analysis and integration.
[0028] Furthermore, the data acquisition module is specifically used to obtain the original configuration file from the local file system, network shared storage, and the device's real-time interface.
[0029] Furthermore, the data acquisition module is also used to verify the original configuration file using a three-level verification system, and to generate a unique identifier for the original configuration file and associate data lineage metadata.
[0030] Furthermore, the standardization module is specifically used for: removing comments using multi-level pattern matching; merging continuation lines for vendor-differentiated syntax; performing hierarchical parsing using indentation standardization; processing interactive CLI logs using configuration pattern stripping; constructing a line number mapping table using a dual-dictionary structure; and adding metadata tags to configuration block boundary lines after cleaning.
[0031] Furthermore, the parsing module is specifically used for,
[0032] The original configuration file is divided into a multi-level structure, including global configuration, interface configuration, and route configuration, and the matching rules for each level are defined by nested templates.
[0033] By combining a keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device types, and template trees of different device types are organized into a unified template set.
[0034] The logic for extracting fields is dynamically defined using YAML rules, and a general data model is built to mask device differences.
[0035] A state machine is used to parse the text line by line, and regular expressions are used to achieve line-by-line or block-by-block matching to generate a parsed file.
[0036] Thirdly, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for automatically classifying and parsing network device configuration files.
[0037] Fourthly, an electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.
[0038] Memory, used to store computer programs;
[0039] When the processor executes a program stored in memory, it implements the steps of the automated classification and parsing method for network device configuration files described above.
[0040] Compared with the prior art, the present invention has the following advantages:
[0041] 1. This invention proposes a multi-level template extraction mechanism and nested rule definition, which enables the system to accurately restore the semantic hierarchy of configuration files (such as global configurations, interface configurations, etc.), avoiding the structural flattening and semantic loss problems common in traditional parsing methods, and greatly improving the accuracy and usability of the parsing results. With the help of the "template forest" architecture and automated classification algorithm, the system can dynamically identify and adapt to the configurations of devices from different manufacturers and models, without the need to develop a separate parser for each type of device, significantly improving the system's versatility and scalability, and effectively reducing the integration complexity in multi-vendor environments.
[0042] 2. The system adopts a YAML-based rule engine, which supports dynamic configuration of field extraction logic and template reuse. No code modification is required when adding new device types or configuration syntax, which greatly reduces the system's maintenance costs and iteration cycle. Through a three-level verification mechanism, data lineage metadata association, and anomaly detection and repair prompts, the system can detect and record data anomalies in real time during the parsing process, provide repair suggestions, and ensure data quality and the reliability and auditability of the processing process.
[0043] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 A schematic diagram of the process for an automated classification and parsing method for network device configuration files according to the present invention is shown.
[0046] Figure 2 A schematic diagram of a network device configuration file automated classification and parsing system module according to the present invention is shown. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0048] In one embodiment of the present invention, to achieve automated, intelligent, and structured parsing of configurations for large-scale heterogeneous network devices, significantly reduce operation and maintenance costs, and improve parsing accuracy and system scalability, such as... Figure 1 As shown, this invention proposes an automated classification and parsing method for network device configuration files, the steps of which include:
[0049] S1. Obtain the original configuration file from the heterogeneous data source to ensure the integrity and reliability of the data source.
[0050] In this embodiment, the present invention supports seamless access and collection from three types of data sources, including local file systems, network shared storage, and real-time device interfaces. Each data source adopts a differentiated collection strategy and fault tolerance mechanism.
[0051] Local file system data collection: A distributed file listener is deployed, implementing event-driven data collection based on the operating system kernel event notification mechanism. A collection task is triggered when a file creation or modification event occurs in the monitored directory. To avoid half-state reading during file writing, a delayed reading strategy is adopted. After detecting a stable file size for several seconds, a checksum is calculated and compared with historical records, processing only changed files. For large configuration files, block-based streaming reading is implemented, using memory mapping technology to reduce I / O overhead. File filters support regular expression configuration and can automatically categorize by device type subdirectories.
[0052] Network shared storage data acquisition: Supports mainstream network file protocols, with built-in connection pool management for persistent sessions. Each session maintains an independent transmission connection and authentication context. For SMB protocol, encrypted transmission is implemented, supporting Kerberos and NTLM authentication. During acquisition, file metadata is first obtained; if the modification time is later than the last acquisition time, it is retrieved via the file transfer interface. In NFS scenarios, client-side caching is used to reduce server query calls. To cope with network jitter, an exponential backoff retry strategy is configured, with strict control over total timeout. The breakpoint resume function is implemented by recording the offset of transmitted bytes; after interruption, transmission resumes from the breakpoint.
[0053] Real-time device interface data acquisition: As the core acquisition channel, it supports three southbound protocols: SSH, Telnet, and SNMP. The SSH acquisition engine establishes encrypted sessions, executes a predefined command queue, automatically detects page break prompts and sends control characters, and sets the terminal width to avoid line break truncation. Acquisition results are compressed before transmission, significantly reducing bandwidth consumption. SNMP acquisition uses batch operations to poll the standard configuration management information base, with each protocol data unit requesting multiple variable bindings and dynamically adjusting the timeout. In SNMPv3 scenarios, a user security model is configured, enabling authentication and encryption functions.
[0054] Integrity assurance mechanism: A three-level verification system ensures data quality. Transport layer verification uses message authentication codes or checksums; syntax layer verification pre-scans configuration text to check bracket matching and indentation validity; semantic layer verification checks the existence of necessary subcommands in critical configuration blocks. Data collection failures are categorized into retryable and non-retryable types; retryable errors are placed in a delay queue for rescheduling. Data collection metrics are exposed through a monitoring system with configured alarm rules.
[0055] Metadata Management and Traceability: Each configuration file generates a unique identifier, associated with data lineage metadata, including collection timestamp, data source path, transmission protocol, file size, checksum, and change history. Metadata is stored in a relational database, supporting traceability queries based on time ranges. To ensure audit compliance, a write-once, read-many storage strategy is adopted, with metadata appended to an immutable log.
[0056] S2. Convert the obtained raw configuration into a parsing-friendly standardized format. Specifically, this includes:
[0057] a. Multi-level pattern matching is used for comment removal: regular expressions are used for full-line comments, lookahead assertions are used for inline comments to retain valid commands; for nested comments, a state machine is implemented to scan character by character, count the occurrence of comment symbols, and determine the comment state based on parity. The processing efficiency reaches 100,000 lines per second.
[0058] b. Perform line continuation merging for vendor-specific syntax (e.g., Cisco uses a backslash \ or ^ at the end of a line, Juniper uses indentation, and H3C uses an ellipsis). Specifically, this includes maintaining a line continuation buffer, appending the next line to the buffer when a continuation character is detected, until a non-continuation end character is encountered or there is no continuation marker at the end of the line; recalculating column positions after merging and updating the line number mapping table; for scenarios where brackets are not closed (e.g., ACL rules span multiple lines), a stack structure is used to count the bracket depth, and the command is considered to have ended when the depth reaches zero.
[0059] c. Use indentation normalization for hierarchical parsing, specifically including:
[0060] Use K-Means clustering (k=2-4) to group the number of leading spaces; where the cluster center value is the candidate indentation unit (e.g., 4 or 8 spaces).
[0061] Alternatively, clustering quality can be evaluated using the silhouette coefficient, and if the quality is below 0.5, it can fall back to the default 4-space strategy.
[0062] Replace all tab characters \t with 4 spaces, and record non-standard indentation as IndentationWarnings. After conversion, use an edit distance algorithm to align similar commands, ensuring that child commands under the same parent configuration block have the same indentation level.
[0063] An indentation stack `indent_stack` is constructed, recording the starting indentation value configured for each level. When an indentation fallback exceeding the current level is detected, an `IndentationError` is triggered. This error is considered fatal and prevents further parsing.
[0064] d. Use configuration mode to process interactive CLI logs. Identify prompts using regular expressions and track mode transitions (user mode → privileged mode → configuration mode), retaining only commands from configuration mode. Filter non-configuration content such as `show` command output using a blacklist. Simultaneously extract device hostname and software version information and add them to the metadata.
[0065] e. A double-dictionary structure is used to construct the line number mapping table. The double-dictionary structure `orig_to_cleaned` and `cleaned_to_orig` record the precise mapping of deletion and merging operations. For example, if the original lines 10-12 are merged into the cleaned line 8, the mapping table will record this as {10:8, 11:8, 12:8}. Furthermore, during anomaly detection, the original file position can be precisely traced back, enabling operations personnel to quickly locate problems.
[0066] f. After cleaning, add metadata tags to the configuration block boundary lines, such as...<NESTED_BLOCK_START type="interface" name="GigabitEthernet0 / 0"> These tags accelerate block identification during subsequent parsing and avoid repeated regular expression matching.
[0067] S3. Parse the standardized format after conversion, generate a parsed file, and perform multi-dimensional quality monitoring on the parsed file.
[0068] In this embodiment, technologies such as multi-level template extraction, template forest construction, rule engine parsing, automated classification, text matching and extraction, and anomaly detection and repair prompts are used to perform in-depth analysis of standardized configurations and monitor the parsing quality in real time. Specifically, this includes...
[0069] 1. Multi-level template extraction mechanism: The device configuration is divided into a multi-level structure such as global configuration, interface configuration, and routing configuration. By defining the matching rules of each level through nested templates, the accurate preservation and reuse of configuration semantic levels are ensured.
[0070] 2. Template Forest Construction Mechanism: The template trees of different device types are organized into a unified template set, supporting joint parsing across device types; when adding a new device type, only the corresponding template tree needs to be added, without modifying the code, and dynamic loading is supported.
[0071] 3. Flexible parsing mechanism based on rule engine: Field extraction logic is dynamically defined through YAML rules, supporting nested structures, conditional judgments and template reuse, achieving device type adaptation without code modification.
[0072] 4. Unified modeling of multi-source heterogeneous configuration files: Construct a general data model to shield device differences and achieve cross-platform parsing capabilities.
[0073] 5. Automated classification algorithm: Combining keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device type, improving the accuracy and efficiency of type determination.
[0074] 6. Highly efficient text matching and extraction technology: It adopts a state machine-driven approach to parse text line by line, and combines regular expressions to achieve line-by-line or block-by-block matching. It also supports future expansion to other matching modes through a plugin mechanism.
[0075] 7. Anomaly detection and repair prompt mechanism: During the parsing process, non-compliant configurations are detected in real time, detailed logs are recorded and repair suggestions are provided to enhance the system's fault tolerance and self-healing capabilities.
[0076] S4. Convert the monitored and parsed results into a format usable by downstream systems for subsequent data analysis and integration.
[0077] In this embodiment, the present invention converts the parsing results into formats such as JSON, XML, database records, and Excel as needed, for use by downstream systems such as network management systems and data analysis platforms.
[0078] like Figure 2 As shown, an automated classification and parsing system for network device configuration files includes: a data acquisition module, a standardization module, a parsing module, and a conversion module.
[0079] 1. The data acquisition module is used to obtain the original configuration file from heterogeneous data sources to ensure the integrity and reliability of the data sources.
[0080] In this embodiment, the data acquisition module supports seamless access to three types of data sources, including local file systems, network shared storage, and real-time device interfaces. Each data source adopts a differentiated acquisition strategy and fault tolerance mechanism.
[0081] Local file system data collection: A distributed file listener is deployed, implementing event-driven data collection based on the operating system kernel event notification mechanism. A collection task is triggered when a file creation or modification event occurs in the monitored directory. To avoid half-state reading during file writing, a delayed reading strategy is adopted. After detecting a stable file size for several seconds, a checksum is calculated and compared with historical records, processing only changed files. For large configuration files, block-based streaming reading is implemented, using memory mapping technology to reduce I / O overhead. File filters support regular expression configuration and can automatically categorize by device type subdirectories.
[0082] Network shared storage data acquisition: Supports mainstream network file protocols, with built-in connection pool management for persistent sessions. Each session maintains an independent transmission connection and authentication context. For SMB protocol, encrypted transmission is implemented, supporting Kerberos and NTLM authentication. During acquisition, file metadata is first obtained; if the modification time is later than the last acquisition time, it is retrieved via the file transfer interface. In NFS scenarios, client-side caching is used to reduce server query calls. To cope with network jitter, an exponential backoff retry strategy is configured, with strict control over total timeout. The breakpoint resume function is implemented by recording the offset of transmitted bytes; after interruption, transmission resumes from the breakpoint.
[0083] Real-time device interface data acquisition: As the core acquisition channel, it supports three southbound protocols: SSH, Telnet, and SNMP. The SSH acquisition engine establishes encrypted sessions, executes a predefined command queue, automatically detects page break prompts and sends control characters, and sets the terminal width to avoid line break truncation. Acquisition results are compressed before transmission, significantly reducing bandwidth consumption. SNMP acquisition uses batch operations to poll the standard configuration management information base, with each protocol data unit requesting multiple variable bindings and dynamically adjusting the timeout. In SNMPv3 scenarios, a user security model is configured, enabling authentication and encryption functions.
[0084] In this embodiment, the data acquisition module utilizes a three-level verification system to ensure data quality. Transport layer verification uses message authentication codes or checksums; syntax layer verification pre-scans the configuration text to check for bracket matching and indentation validity; semantic layer verification checks the existence of necessary subcommands in key configuration blocks. Data acquisition failures are categorized into retryable and non-retryable types; retryable errors are placed in a delay queue for rescheduling. Acquisition metrics are exposed through a monitoring system, and alarm rules are set.
[0085] In this embodiment, the data acquisition module also generates a unique identifier for each original configuration file and associates it with data lineage metadata, including the collection timestamp, data source path, transmission protocol, file size, checksum, and change history. The metadata is stored in a relational database, supporting time-range-based traceability queries. To ensure audit compliance, a write-once, read-many storage strategy is adopted, and metadata is appended to an immutable log.
[0086] 2. The standardization module is used to convert the acquired raw configuration into a parsing-friendly standardized format, specifically including:
[0087] a. Multi-level pattern matching is used for comment removal: regular expressions are used for full-line comments, lookahead assertions are used for inline comments to retain valid commands; for nested comments, a state machine is implemented to scan character by character, count the occurrence of comment symbols, and determine the comment state based on parity. The processing efficiency reaches 100,000 lines per second.
[0088] b. Perform line continuation merging for vendor-specific syntax (e.g., Cisco uses a backslash \ or ^ at the end of a line, Juniper uses indentation, and H3C uses an ellipsis). Specifically, this includes maintaining a line continuation buffer, appending the next line to the buffer when a continuation character is detected, until a non-continuation end character is encountered or there is no continuation marker at the end of the line; recalculating column positions after merging and updating the line number mapping table; for scenarios where brackets are not closed (e.g., ACL rules span multiple lines), a stack structure is used to count the bracket depth, and the command is considered to have ended when the depth reaches zero.
[0089] c. Use indentation normalization for hierarchical parsing, specifically including:
[0090] Use K-Means clustering (k=2-4) to group the number of leading spaces; where the cluster center value is the candidate indentation unit (e.g., 4 or 8 spaces).
[0091] Alternatively, clustering quality can be evaluated using the silhouette coefficient, and if the quality is below 0.5, it can fall back to the default 4-space strategy.
[0092] Replace all tab characters \t with 4 spaces, and record non-standard indentation as IndentationWarnings. After conversion, use an edit distance algorithm to align similar commands, ensuring that child commands under the same parent configuration block have the same indentation level.
[0093] An indentation stack `indent_stack` is constructed, recording the starting indentation value configured for each level. When an indentation fallback exceeding the current level is detected, an `IndentationError` is triggered. This error is considered fatal and prevents further parsing.
[0094] d. Use configuration mode to process interactive CLI logs. Identify prompts using regular expressions and track mode transitions (user mode → privileged mode → configuration mode), retaining only commands from configuration mode. Filter non-configuration content such as `show` command output using a blacklist. Simultaneously extract device hostname and software version information and add them to the metadata.
[0095] e. A double-dictionary structure is used to construct the line number mapping table. The double-dictionary structure `orig_to_cleaned` and `cleaned_to_orig` record the precise mapping of deletion and merging operations. For example, if the original lines 10-12 are merged into the cleaned line 8, the mapping table will record this as {10:8, 11:8, 12:8}. Furthermore, during anomaly detection, the original file position can be precisely traced back, enabling operations personnel to quickly locate problems.
[0096] f. After cleaning, add metadata tags to the configuration block boundary lines, such as...<NESTED_BLOCK_START type="interface" name="GigabitEthernet0 / 0"> These tags accelerate block identification during subsequent parsing and avoid repeated regular expression matching.
[0097] 3. The parsing module is used to parse the converted standardized format, generate parsed files, and perform multi-dimensional quality monitoring on the parsed files.
[0098] In this embodiment, the parsing module utilizes technologies such as multi-level template extraction, template forest construction, rule engine parsing, automated classification, text matching and extraction, and anomaly detection and repair prompts to perform in-depth parsing of standardized configurations and monitor the parsing quality in real time. Specifically, this includes...
[0099] (1) Multi-level template extraction mechanism: The original configuration file is divided into a multi-level structure such as global configuration, interface configuration, and routing configuration. By defining the matching rules of each level through nested templates, the accurate preservation and reuse of the configuration semantic level is ensured.
[0100] (2) Template forest construction mechanism: The template trees of different device types are organized into a unified template set, which supports joint parsing across device types; when adding a new device type, only the corresponding template tree needs to be added, without modifying the code, and dynamic loading is supported.
[0101] (3) Flexible parsing mechanism based on rule engine: Field extraction logic is dynamically defined through YAML rules, supporting nested structure, condition judgment and template reuse, and realizing device type adaptation without code modification.
[0102] (4) Unified modeling of multi-source heterogeneous configuration files: Construct a general data model to shield device differences and realize cross-platform parsing capabilities.
[0103] (5) Automated classification algorithm: Combining keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device type, thereby improving the accuracy and efficiency of type determination.
[0104] (6) Efficient text matching and extraction technology: The text is parsed line by line using a state machine and regular expressions are combined to achieve line-by-line or block-by-block matching. It can be extended to other matching modes in the future through a plug-in mechanism.
[0105] (7) Anomaly detection and repair prompt mechanism: During the parsing process, non-compliant configurations are detected in real time, detailed logs are recorded and repair suggestions are provided to enhance the system's fault tolerance and self-healing capabilities.
[0106] 4. The conversion module is used to convert the monitored and parsed results into a format usable by downstream systems for subsequent data analysis and integration.
[0107] In this embodiment, the conversion module converts the parsed results into formats such as JSON, XML, database records, and Excel as needed, for use by downstream systems such as network management systems and data analysis platforms.
[0108] Based on the above disclosure, the present invention also provides an electronic device. The electronic device of this embodiment includes at least one processor and at least one storage medium electrically connected to the processor. The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.
[0109] Based on the same inventive concept, the present invention also provides a storage medium storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.
[0110] The foregoing description and accompanying drawings fully illustrate embodiments of the invention to enable those skilled in the art to practice them. Other embodiments may include structural and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Some portions and features of some embodiments may be included or substituted for portions and features of other embodiments. Embodiments of the invention are not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from their scope. The scope of the invention is limited only by the appended claims.
Claims
1. A method for automated classification and parsing of network device configuration files, characterized in that, The method includes, The system obtains raw configuration files from heterogeneous data sources. Specifically, this includes obtaining raw configuration files from local file systems, network shared storage, and device real-time interfaces. This process further includes deploying a distributed file listener and implementing event-driven data collection based on the operating system kernel event notification mechanism. The system supports mainstream network file protocols, has a built-in connection pool to manage persistent sessions, and each session maintains an independent transmission connection and authentication context. It also supports three southbound protocols: SSH, Telnet, and SNMP. The acquired raw configuration is converted into a parsing-friendly standardized format. Specifically, this conversion includes: removing comments using multi-level pattern matching; merging continuation lines for vendor-specific syntax; performing hierarchical parsing using indentation standardization; processing interactive CLI logs using configuration pattern stripping; constructing a line number mapping table using a dual-dictionary structure; and adding metadata tags to configuration block boundary lines after cleaning. The hierarchical parsing using indentation standardization includes: grouping leading spaces using K-Means clustering and evaluating clustering quality using silhouette coefficients. The dual dictionary structure includes orig_to_cleaned and cleaned_to_orig, which are used to record the precise mapping of deletion and merging operations; The standardized format after parsing and conversion is generated into a parsed file, and the parsed file is subjected to multi-dimensional quality monitoring. The monitored and parsed results are converted into a format usable by downstream systems for subsequent data analysis and integration.
2. The method for automated classification and parsing of network device configuration files according to claim 1, characterized in that, The process of obtaining the original configuration file from the heterogeneous data source also includes verifying the original configuration file using a three-level verification system, generating a unique identifier for the original configuration file, and associating data lineage metadata.
3. The method for automated classification and parsing of network device configuration files according to claim 1, characterized in that, The standardized format after parsing and conversion generates a parsing file, specifically including... The original configuration file is divided into a multi-level structure of global configuration, interface configuration, and route configuration, and the matching rules of each level are defined by nested templates; By combining a keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device types, and template trees of different device types are organized into a unified template set. The logic for extracting fields is dynamically defined using YAML rules, and a general data model is built to mask device differences. A state machine is used to parse the text line by line, and regular expressions are used to achieve line-by-line or block-by-block matching to generate a parsed file.
4. A network device configuration file automated classification and parsing system, characterized in that, The system includes: a data acquisition module, a standardization module, a parsing module, and a conversion module; The data acquisition module is used to obtain the original configuration file from heterogeneous data sources. Specifically, the data acquisition module is used to obtain the original configuration file from the local file system, network shared storage, and device real-time interface. The process of obtaining the original configuration file from the local file system, network shared storage, and device real-time interface specifically includes deploying a distributed file listener, implementing event-driven collection based on the operating system kernel event notification mechanism, supporting mainstream network file protocols, managing persistent sessions with a built-in connection pool, and maintaining an independent transmission connection and authentication context for each session; and supporting three southbound protocols: SSH, Telnet, and SNMP. The standardization module is used to convert the acquired raw configuration into a parsing-friendly standardized format. Specifically, the standardization module includes: removing comments using multi-level pattern matching; merging continuation lines for vendor-specific syntax; performing hierarchical parsing using indentation standardization; processing interactive CLI logs using configuration pattern stripping; constructing a line number mapping table using a dual-dictionary structure; and adding metadata tags to configuration block boundary lines after cleaning. The hierarchical parsing using indentation standardization includes grouping leading spaces using K-Means clustering and evaluating clustering quality using silhouette coefficients. The dual-dictionary structure includes `orig_to_cleaned` and `cleaned_to_orig`, used to record the precise mapping of deletion and merging operations. The parsing module is used to parse the converted standardized format, generate a parsed file, and perform multi-dimensional quality monitoring on the parsed file; The conversion module is used to convert the monitored and parsed results into a format usable by downstream systems for subsequent data analysis and integration.
5. The automated classification and parsing system for network device configuration files according to claim 4, characterized in that, The data acquisition module is also used to verify the original configuration file using a three-level verification system, and to generate a unique identifier for the original configuration file and associate data lineage metadata.
6. The automated classification and parsing system for network device configuration files according to claim 4, characterized in that, The parsing module is specifically used for, The original configuration file is divided into a multi-level structure of global configuration, interface configuration, and route configuration, and the matching rules of each level are defined by nested templates; By combining a keyword library with TF-IDF or word vector feature extraction, a lightweight machine learning model is used to identify device types, and template trees of different device types are organized into a unified template set. The logic for extracting fields is dynamically defined using YAML rules, and a general data model is built to mask device differences. A state machine is used to parse the text line by line, and regular expressions are used to achieve line-by-line or block-by-block matching to generate a parsed file.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the automated classification and parsing method for network device configuration files according to any one of claims 1-3.
8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the automated classification and parsing method for network device configuration files as described in any one of claims 1-3.