High-concurrency SNMP (Simple Network Management Protocol) data acquisition method based on virtual thread
By replacing platform threads with virtual threads, a one-to-one mapping is established between SNMP protocol data acquisition tasks and virtual threads. This solves the resource waste and latency problems of traditional platform thread concurrent acquisition, achieves efficient and timely data acquisition, reduces hardware costs, and improves the response speed of device monitoring.
Patent Information
- Application Number
- CN202511026074.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-24
- Publication Date
- 2025-11-14
AI Technical Summary
Traditional platform thread-based concurrent SNMP protocol data acquisition suffers from problems such as excessive memory consumption, CPU waste, I/O blocking leading to resource idleness, and high overall processing latency.
Virtual threads are used to replace platform threads, and a one-to-one mapping processing method between SNMP protocol data acquisition tasks and virtual threads is constructed. By leveraging the high concurrency and low resource consumption characteristics of virtual threads, timely and efficient data acquisition from thousands of devices can be achieved, and the underlying platform threads are automatically suspended to handle other tasks when I/O is waiting.
The overall processing latency has been optimized from minutes to seconds, the number of hardware components has been reduced by 50%, the efficiency of machine utilization and data collection has been improved, and timely support has been provided for business scenarios such as equipment anomaly warnings.
Smart Images
Figure CN120956577A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a high-concurrency SNMP protocol data acquisition method based on virtual threads. Background Technology
[0002] 1) SNMP (Simple Network Management Protocol) is an application-layer protocol for network device management and monitoring, based on UDP transmission. Common use cases include real-time monitoring of network devices such as routers, switches, and firewalls, detecting port status, traffic, packet loss rate, etc. However, traditional platform-based concurrent SNMP protocol data acquisition suffers from problems such as excessive memory consumption, CPU waste due to platform thread switching, resource idleness caused by I / O blocking, and high overall processing latency.
[0003] 2) Virtual Threads is a lightweight user-mode thread model introduced in JDK 21. This model has two main advantages: First, it has extremely low resource consumption: Virtual threads are directly scheduled by the JVM, without relying on operating system kernel threads. A single thread only occupies a few hundred bytes of memory (compared to about 1-2MB for traditional platform threads), and a single machine can easily create millions of threads. Second, it features non-blocking scheduling: When a thread performs I / O operations (such as network requests, file read / write), the JVM automatically suspends and releases the underlying platform thread, avoiding resource waste and significantly improving throughput. Summary of the Invention
[0004] This invention, starting from the high concurrency and low resource consumption characteristics of virtual threads, reconstructs the overall processing flow, replaces platform threads with virtual threads, and provides a high-concurrency SNMP protocol data acquisition method based on virtual threads. It is highly practical and has significant application results.
[0005] The technical solution of this invention is:
[0006] A high-concurrency SNMP protocol data acquisition method based on virtual threads reconstructs the traditional SNMP protocol data acquisition process by replacing the platform thread with a virtual thread. By leveraging the high concurrency and low resource consumption characteristics of virtual threads, the overall latency of data acquisition via SNMP protocol for thousands of devices is optimized from minutes to seconds.
[0007] A one-to-one mapping method between SNMP protocol data acquisition tasks and virtual threads was constructed to enable timely data acquisition from thousands of devices.
[0008] When a virtual thread is automatically suspended while waiting for I / O, the underlying platform thread immediately releases its resources and begins processing other tasks.
[0009] Construct a mapping model of tables, devices, and metrics. For a device in a table, which involves several metrics, use a one-time connection and batch metric collection method.
[0010] Furthermore,
[0011] Configure device metric collection information. Based on business needs, configure device metric collection information. Required information includes device IP, port number, password, metric identifier, metric name, and the table to which it belongs.
[0012] Data collection task generation involves iterating through the configuration device indicator collection information, grouping by the table to which it belongs and the device IP, generating data collection tasks, and generating task formats.
[0013] Each task is assigned to a virtual thread for processing. Each table involves a device table, and each device only needs to be connected once for batch indicator collection.
[0014] Data collection tasks are distributed according to a list of tasks needed; data collection involves connecting to specific devices and ports, logging in with a password, traversing the indicator tree, retrieving the corresponding table based on the indicator identifier, and filling in the value corresponding to the indicator name.
[0015] Metrics merging involves organizing different metrics obtained from the same table into a single row of data.
[0016] This invention is based on the virtual thread model introduced in JDK 21, which breaks through the limitation of the limited number of platform threads and constructs a one-to-one mapping processing method between SNMP protocol data acquisition tasks and virtual threads, enabling timely and efficient acquisition of data from thousands of devices and fully ensuring the timeliness of data acquisition.
[0017] When virtual threads are automatically suspended while waiting for I / O (such as SNMP request and response), the underlying platform threads are immediately released and can handle other tasks, greatly reducing context switching and reducing CPU utilization by 60%+ (from 80% to 30%).
[0018] An innovative mapping model for tables, devices, and metrics is constructed. For multiple metrics involved in a single device in a table, a batch metric collection method with a single connection is adopted, reducing the number of connections, avoiding connection pressure on the data collection device, and achieving fast device response and reducing processing latency.
[0019] The beneficial effects of this invention are
[0020] For SNMP protocol data acquisition scenarios, this algorithm, compared with the traditional platform thread model-based concurrent acquisition implementation scheme, optimizes the overall processing latency from minutes to seconds, reduces the overall number of hardware components by 50%, lowers the overall cost, improves machine utilization and acquisition efficiency, and greatly enhances timeliness for subsequent business scenarios such as device anomaly warning. Attached Figure Description
[0021] Figure 1 This is a schematic diagram of the workflow of the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0023] The invention discloses a high-concurrency SNMP protocol data acquisition method based on virtual threads. By reconstructing the traditional SNMP protocol data acquisition process by replacing the platform thread with a virtual thread, the high concurrency and low resource consumption characteristics of the virtual thread are utilized to optimize the overall latency of data acquisition via SNMP protocol for thousands of devices from minutes to seconds.
[0024] This invention is results-oriented and is explained through configuration of equipment indicator collection information, collection task generation, collection task distribution, indicator merging, and other parts.
[0025] a) Configure equipment indicator collection information
[0026] Configure device metric collection information according to business needs. Required information includes device IP, port number, password, metric identifier, metric name, and the table to which it belongs, as follows:
[0027]
[0028] b) Data Acquisition Task Generation
[0029] Iterate through the configuration device indicator collection information, group it by its table and device IP, and generate collection tasks. The generated task format is as follows:
[0030]
[0031] Each task is assigned to a virtual thread for processing. Although there are multiple device tables involved in each table, each device only needs to connect once to collect batch indicators. This greatly reduces the connection pressure on the data collection devices, helps the devices respond quickly, and reduces the overall processing latency.
[0032] c) Data collection task issuance
[0033] The list of data collection tasks to be distributed as needed is as follows:
[0034]
[0035] Based on the data collection task list, each task is assigned a virtual thread to perform concurrent data collection on thousands of devices. Specifically, it connects to a specific device and port, logs in using a password, traverses the metric tree, retrieves the corresponding table based on the metric identifier, and populates the value corresponding to the metric name.
[0036] d) Indicator merging
[0037] To organize different metrics retrieved from the same table into a single row of data, the following format is used:
[0038]
[0039] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A high-concurrency SNMP protocol data acquisition method based on virtual threads, characterized in that, By reconstructing the traditional SNMP protocol data acquisition process by replacing platform threads with virtual threads, and leveraging the high concurrency and low resource consumption characteristics of virtual threads, the overall latency for data acquisition via the SNMP protocol for thousands of devices is optimized from minutes to seconds.
2. The method according to claim 1, characterized in that, A one-to-one mapping method between SNMP protocol data acquisition tasks and virtual threads was constructed to enable timely data acquisition from thousands of devices.
3. The method according to claim 2, characterized in that, When a virtual thread is automatically suspended while waiting for I / O, the underlying platform thread immediately releases its resources and begins processing other tasks.
4. The method according to claim 3, characterized in that, Construct a mapping model of tables, devices, and metrics. For a device in a table, which involves several metrics, use a one-time connection and batch metric collection method.
5. The method according to claim 4, characterized in that, Configure device metric collection information. Based on business needs, configure device metric collection information. Required information includes device IP, port number, password, metric identifier, metric name, and the table to which it belongs.
6. The method according to claim 5, characterized in that, Data collection task generation involves iterating through the configuration device indicator collection information, grouping by the table to which it belongs and the device IP, generating data collection tasks, and generating task formats.
7. The method according to claim 6, characterized in that, Each task is assigned to a virtual thread for processing. Each table involves a device table, and each device only needs to be connected once for batch indicator collection.
8. The method according to claim 7, characterized in that, Data collection tasks are distributed according to a list of tasks needed; data collection involves connecting to specific devices and ports, logging in with a password, traversing the indicator tree, retrieving the corresponding table based on the indicator identifier, and filling in the value corresponding to the indicator name.
9. The method according to claim 8, characterized in that, Metrics merging involves organizing different metrics obtained from the same table into a single row of data.