Method, device, equipment, medium and program product for collecting network device resources

Abstract

The present disclosure provides a network device resource collection method, which can be applied to the fields of information security and big data technology. The method comprises the following steps: collecting device resources in a target network in multiple threads, and sending a set of collectable device resources in the target network to a worker node; classifying and saving the set of collectable device resources received by the worker node by using a created resource task table; performing single-thread scanning on the resource task table by using a master worker node, and issuing a resource discovery message queue to a slave worker node when the scanning result meets a preset condition; controlling the slave worker node to perform periodic resource collection on the target network based on a preset resource threshold and a collection interval; and dynamically updating the resource task table by using a set of collectable device resources obtained by the periodic resource collection of the slave worker node. The present disclosure also provides a network device resource collection device, equipment, medium and program product.

Description

Technical Field

[0001] This disclosure relates to the fields of information security and big data technology, specifically the field of network device management technology, and more specifically to a method, apparatus, device, medium, and program product for collecting network device resources. Background Technology

[0002] With the rapid development of information technology and big data, the number of network devices has also increased dramatically. How to effectively manage these massive amounts of network device resources has become a key research focus for technical personnel. Network device resource management typically includes resource acquisition, resource configuration, resource monitoring, resource maintenance, and resource updates. Effective management of network device resources can not only improve network performance and reduce operating costs, but also enhance the security and availability of data running on or stored within these resources.

[0003] To effectively manage network device resources, it is necessary to first collect these resources so that they can be processed in a targeted manner later. However, when faced with massive amounts of network device resources, existing technical solutions suffer from problems such as synchronization blocking and data inconsistency during the resource collection process. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a method, apparatus, equipment, medium and program product for collecting network device resources to improve the efficiency of collecting device resources in a target network.

[0005] According to a first aspect of this disclosure, a method for collecting network device resources is provided, comprising:

[0006] Multi-threaded acquisition of device resources in the target network is performed, and the set of collectable device resources in the target network is sent to the worker node;

[0007] Create a resource task table and use the resource task table to classify and save the collection of collectable device resources received by the work node;

[0008] The master worker node performs a single-threaded scan of the resource task table. If the scan results meet the preset conditions, the resource discovery message queue is sent to the slave worker node.

[0009] Based on preset resource thresholds and collection intervals, the resource discovery message queue is used to control the periodic collection of resources from the target network from the working nodes.

[0010] The resource task table is dynamically updated using the set of collectable device resources obtained from the periodic resource collection of the working nodes.

[0011] According to embodiments of this disclosure, the above-mentioned multi-threaded collection of device resources in the target network includes:

[0012] Perform connectivity tests on the target network and send the device information to be collected and the predefined resource discovery formula to the distributed collector;

[0013] Based on the device information to be collected and the predefined resource discovery formula, a distributed collector is used to scan the target network and collect the collectable device resources in the target network in a multi-threaded manner to obtain a set of collectable device resources.

[0014] According to embodiments of this disclosure, the above-mentioned connectivity test of the target network includes:

[0015] Perform connectivity tests on devices in the target network and on the network protocols used during scanning and data collection.

[0016] According to embodiments of this disclosure, the above-mentioned method, based on the device information to be collected and a predefined resource discovery formula, utilizes a distributed collector to scan the target network and performs multi-threaded collection of collectable device resources in the target network, resulting in a set of collectable device resources including:

[0017] Access to the target network is achieved by sending SNMP-based requests to the target network using a distributed data collector.

[0018] Based on the SNMP protocol, a distributed data collector is used to scan the target network to determine the collectable device resources in the target network;

[0019] Based on the SNMP protocol, a distributed collector is used to collect collectable device resources in the target network through multi-threaded collection, resulting in a collection set of collectable device resources.

[0020] According to embodiments of this disclosure, the creation of a resource task table and the categorization and storage of the set of collectable device resources received by the working node using the resource task table include:

[0021] Based on object identifiers, the device resources in the set of collectable device resources are classified using worker nodes;

[0022] Based on the classification results, a resource task table is created for each type of device resource, and the identifier of each type of device resource is stored in the resource task table;

[0023] Create a resource task sub-table and use it to store the message information returned during the creation process of the resource task table.

[0024] According to embodiments of this disclosure, the resource task table includes a task identifier, a device resource identifier, the number of currently received resources, and the total number of resources.

[0025] The resource task sub-table includes the task sub-table identifier, the task identifier of the resource task table, and message information.

[0026] According to embodiments of this disclosure, the above-described method of using the master worker node to perform a single-threaded scan of the resource task table, and then, if the scan results meet preset conditions, sending the resource discovery message queue to the slave worker node, includes:

[0027] The master worker node performs a single-threaded scan of the currently received resource count and the total number of resources in the resource task table. If the currently received resource count and the total number of resources are equal, the resource discovery message queue is sent to the slave worker node.

[0028] According to embodiments of this disclosure, the above-mentioned method of periodically collecting resources from the target network from the worker node based on a preset resource threshold and collection interval, using a resource discovery message queue, includes:

[0029] Select target device resources that meet the preset resource threshold from the set of collectable device resources, and determine the scanning cycle based on the collection interval;

[0030] A resource discovery message queue is used to control the acquisition of resource discovery messages by multiple distributed worker nodes in a distributed lock manner;

[0031] Upon receiving a resource discovery message, the system periodically collects resources from the target device by controlling the collector from the working node, based on the scanning cycle.

[0032] According to a second aspect of this disclosure, a network device resource acquisition device is provided, comprising:

[0033] The multi-threaded acquisition module is used to perform multi-threaded acquisition of device resources in the target network and send the set of collectable device resources in the target network to the worker node;

[0034] The task table creation module is used to create resource task tables and use the resource task tables to classify and save the collection of collectable device resources received by the work nodes.

[0035] The single-threaded scanning module is used to perform a single-threaded scan of the resource task table using the main worker node. If the scan results meet the preset conditions, the resource discovery message queue is sent to the slave worker node.

[0036] The periodic resource acquisition module is used to periodically acquire resources from the target network from the worker node based on preset resource thresholds and acquisition intervals, using a resource discovery message queue.

[0037] The task table dynamic update module is used to dynamically update the resource task table using the set of collectable device resources obtained from periodic scanning of the working nodes.

[0038] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0039] A fourth aspect of this disclosure also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0040] The fifth aspect of this disclosure also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0041] The network device resource acquisition method disclosed herein improves the acquisition efficiency by performing multi-threaded resource acquisition on the target network and saving the acquired device resources through a resource task table. Simultaneously, it utilizes a master worker node to perform a single-threaded scan of the resource task table and uses a generated resource discovery message queue to control the periodic acquisition of collectable device resources in the target network by the worker nodes, thereby avoiding technical problems such as synchronization blocking and data inconsistency. Furthermore, those skilled in the art can use the generated resource task table to perform real-time monitoring or management of collectable device resources in the target network, reducing operation and maintenance costs. Attached Figure Description

[0042] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0043] Figure 1 The diagram illustrates an application scenario of a method for collecting network device resources according to an embodiment of the present disclosure.

[0044] Figure 2 A flowchart illustrating a method for collecting network device resources according to an embodiment of the present disclosure is shown schematically.

[0045] Figure 3 The flowcharts are illustrated schematically in multiple instances according to embodiments of the present disclosure;

[0046] Figure 4 This illustration schematically depicts a data processing flow for multiple instances or a single instance according to embodiments of the present disclosure;

[0047] Figure 5 This is a schematic diagram illustrating the network device resource discovery process according to an embodiment of the present disclosure;

[0048] Figure 6 The flowchart illustrating the discovery and collection of network device resources by an AP cluster according to an embodiment of the present disclosure is shown in the illustration.

[0049] Figure 7 This schematic diagram illustrates the structure of a network device resource acquisition apparatus according to an embodiment of the present disclosure;

[0050] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for collecting network device resources according to an embodiment of the present disclosure. Detailed Implementation

[0051] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0052] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0053] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0054] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0055] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0056] In the technical solution disclosed herein, the information (including but not limited to the hardware information, software information, configuration information, and address information of the target network) and data (including but not limited to data used for analysis, data stored, and data displayed) of the target network are all information and data authorized by the owner of the target network or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for the owner of the target network to choose to authorize or refuse.

[0057] During network device resource collection or discovery, existing technical solutions, in multi-instance scenarios, simultaneously query the database, leading to duplicate queries and data duplication. This results in technical issues such as synchronization blocking and data inconsistency. Synchronization blocking involves several processes (e.g., the Coordinator waiting for all participants to vote) that are synchronously blocked. All logic involved in the transaction is blocked, and each participant is unable to perform any other operations while waiting for responses from other participants. In practical applications, this problem is addressed through timeout mechanisms, but these do not completely resolve synchronization blocking, deadlock, and database table locking issues. Data inconsistency arises when data arrives at the same time during the query phase. This leads to updates being completed, data being queried, and row locks being unlocked at the same point in time. Due to the database's consistency, atomicity, and durability, the query results are identical, causing data redundancy.

[0058] Figure 1 The diagram illustrates an application scenario of a method for collecting network device resources according to an embodiment of the present disclosure.

[0059] like Figure 1 As shown, application scenario 100 according to this embodiment may include the fields of information security and big data technology. Network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0060] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0061] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0062] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0063] It should be noted that the network device resource acquisition method provided in this embodiment can generally be executed by server 105. Correspondingly, the network device resource acquisition device provided in this embodiment can generally be located in server 105. The network device resource acquisition method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the network device resource acquisition device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0064] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0065] The following will be based on Figure 1 The described scene, through Figures 2-6 The method for collecting network device resources according to the disclosed embodiments is described in detail.

[0066] Figure 2 A flowchart illustrating a method for collecting network device resources according to an embodiment of the present disclosure is shown.

[0067] like Figure 2 As shown, the network device resource acquisition method of this embodiment includes operations S210 to S250.

[0068] During operation S210, multi-threaded acquisition of device resources in the target network is performed, and the set of collectable device resources in the target network is sent to the worker node.

[0069] The target network mentioned above can be a local area network (LAN) within the same geographical location, a LAN distributed in different geographical locations, or a wide area network (WAN) that has been trusted and authorized.

[0070] Before performing operation S210, authorization from the target network administrator is required. The aforementioned set of collectable device resources includes shared device resources, externally visible device resources, and device resources that have been authorized and made visible by the target network administrator.

[0071] The aforementioned equipment resources include the equipment's hardware and software resources, such as CPU resource information, hard disk resource information, motherboard resource information, and graphics card resource information. Those skilled in the art can configure the equipment resources to be collected according to actual management needs.

[0072] For example, before operating S210, a request for information collection can be sent to the target network administrator. The operation is performed if the target network administrator agrees or authorizes access to device resources.

[0073] In the embodiments of this disclosure, a corresponding operation entry point can be provided to the target network administrator, allowing the administrator to choose to agree to or reject the automated decision result. That is, before collecting data from the device resources, the target network administrator can input an instruction to agree to or reject the data collection through the corresponding operation entry point. If the target network administrator agrees to the data collection, the device resources are collected, i.e., step S210 is executed. If the target network administrator refuses to collect data, the expert decision-making process is initiated.

[0074] In operation S220, a resource task table is created and the collection of collectable device resources received by the working node is categorized and saved using the resource task table.

[0075] The resource task table is used to store the ID of the device resource, the status information of the device resource, etc. Those skilled in the art can set the device information to be stored according to actual management needs.

[0076] When operating S230, the master worker node performs a single-threaded scan of the resource task table. If the scan results meet the preset conditions, the resource discovery message queue is sent to the slave worker node.

[0077] One worker node can be randomly selected from multiple distributed worker nodes to serve as the master worker node. The master worker node is used to control periodic resource acquisition.

[0078] In operation S240, based on preset resource thresholds and collection intervals, the resource discovery message queue is used to control the periodic collection of resources from the target network from the working node.

[0079] The aforementioned resource thresholds may include, for example, threshold temperatures for the CPU, threshold rotational speeds, threshold temperatures for the hard drive, threshold storage capacities, peak read / write speeds, etc. Those skilled in the art can set these resource thresholds according to actual management needs. Furthermore, those skilled in the art can also set the aforementioned data collection intervals according to actual management needs.

[0080] When operating S250, the resource task table is dynamically updated using the set of collectable device resources obtained from the periodic resource collection of the working nodes.

[0081] During the real-time operation of the target network, device resources are dynamically changing. Therefore, it is necessary to dynamically update the collectable device resources in the target network. The dynamically updated device resources can provide relevant maintenance personnel with real-time device resource data, facilitating subsequent maintenance, thereby reducing maintenance costs and improving maintenance efficiency.

[0082] The network device resource acquisition method disclosed herein improves the acquisition efficiency by performing multi-threaded resource acquisition on the target network and saving the acquired device resources through a resource task table. Simultaneously, it utilizes a master worker node to perform a single-threaded scan of the resource task table and uses a generated resource discovery message queue to control the periodic acquisition of collectable device resources in the target network by the worker nodes, thereby avoiding technical problems such as synchronization blocking and data inconsistency. Furthermore, those skilled in the art can use the generated resource task table to perform real-time monitoring or management of collectable device resources in the target network, reducing operation and maintenance costs.

[0083] According to embodiments of this disclosure, the above-mentioned multi-threaded collection of device resources in the target network includes: performing connectivity tests on the target network and sending the device information to be collected and the predefined resource discovery formula to a distributed collector; based on the device information to be collected and the predefined resource discovery formula, using the distributed collector to scan the target network and performing multi-threaded collection of collectable device resources in the target network to obtain a set of collectable device resources.

[0084] The device information to be collected includes both hardware and software information, such as CPU and hard drive details. The collected device resources can then be used by maintenance personnel to manage the target network.

[0085] A predefined resource discovery formula can be used, for example, to set importance weights for the above information based on the type, location, quantity, and status of the device resources. By calculating the scores of the device resources, the resources with scores higher than the threshold are included in the list of device resources to be collected.

[0086] Before collecting device resources, connectivity tests are required to ensure smooth network operation during the subsequent collection process, thereby improving collection efficiency and reducing data error rates.

[0087] According to embodiments of this disclosure, the above-mentioned connectivity test of the target network includes: conducting connectivity tests on devices in the target network, and conducting connectivity tests on network protocols used in the scanning and acquisition process.

[0088] Connectivity testing is the process of evaluating whether different devices in a target network can communicate effectively. This testing is crucial for ensuring the stability and reliability of the network.

[0089] Those skilled in the art can adopt appropriate network protocols based on actual management needs.

[0090] In this embodiment, the optional network protocol is the SNMP protocol. SNMP connectivity testing is an important method for network performance evaluation, primarily used to test the connectivity and communication speed of the SNMP protocol between the host and the SNMP Agent. This testing process can be based on UDP packets, evaluating network performance by sending and receiving packets. The implementation principle of SNMP testing involves sending a request packet and calculating the time difference between receiving the response packet and sending the request packet, thereby reflecting the connectivity and performance status of the network's SNMP protocol. SNMP protocol connectivity testing generally includes several key steps: sending an SNMP request packet, receiving a response packet, and calculating the communication time.

[0091] The connectivity test ensured the smooth transmission of information in the target network, facilitating the rapid transfer of the collected device resources to the working nodes.

[0092] According to embodiments of this disclosure, the above-mentioned method of scanning the target network using a distributed collector based on the device information to be collected and a predefined resource discovery formula, and performing multi-threaded collection of collectable device resources in the target network to obtain a set of collectable device resources includes: sending a request based on the SNMP protocol to the target network using a distributed collector to access the target network; scanning the target network using a distributed collector based on the SNMP protocol to determine collectable device resources in the target network; and performing multi-threaded collection of collectable device resources in the target network using a distributed collector based on the SNMP protocol to obtain a set of collectable device resources.

[0093] SNMP (Simple Network Management Protocol) is a standard protocol specifically designed for managing network nodes (servers, workstations, HUBS, etc.) in IP networks. It is an application layer protocol.

[0094] SNMP enables network administrators to manage network performance, identify and resolve network problems, and plan network growth. By receiving random messages via SNMP, the network management system becomes aware of network issues.

[0095] Technical personnel can configure the network protocols used according to actual data collection needs.

[0096] Distributed data collectors, employing multi-threaded device resource acquisition, can significantly improve resource acquisition efficiency. These distributed collectors can be deployed within a distributed system, which refers to a system where different service modules are deployed across multiple servers, collaborating remotely to provide services. Furthermore, distributed systems can form clusters, where the same application or service modules are deployed across multiple servers, creating a cluster that provides services through load balancing devices.

[0097] According to embodiments of this disclosure, the creation of a resource task table and the classification and storage of the set of collectable device resources received by the working node using the resource task table include: classifying the device resources in the set of collectable device resources based on object identifiers using the working node; creating a resource task table for each type of device resource based on the classification results, and storing the identifier of each type of device resource in the resource task table; creating a resource task sub-table, and storing the message information returned during the creation of the resource task table using the resource task sub-table.

[0098] Both the resource task table and resource task sub-tables mentioned above can be created using a database. Databases offer numerous advantages, making them suitable for storing large amounts of data, especially when there are numerous collectable device resources within the target network. Database-based resource task tables and sub-tables can centrally store large amounts of data, while also facilitating data organization, updating, querying, and deletion, avoiding data fragmentation and duplication. Databases leverage data consistency and integrity to ensure data integrity. For example, primary keys (i.e., task identifiers), foreign keys, and unique constraints can be defined to avoid data redundancy and inconsistency. Because databases provide strict security mechanisms, including user authentication and access control, only authorized users can access and manipulate data. This protects data security and privacy. Databases also feature efficient index structures, enabling fast data retrieval and querying. Furthermore, regarding data backup and recovery, databases allow for regular backups and provide corresponding recovery mechanisms to prevent data loss or corruption. This is a crucial means of ensuring data reliability and business continuity.

[0099] The resource task table (hereinafter referred to as the task table) provided in this disclosure can not only be used to store the collected device resources, but also to control the periodic resource collection process in real time.

[0100] The aforementioned resource task sub-table (item table, hereinafter the same) can store message information to indicate whether the relevant device resources have been successfully saved. Relevant maintenance personnel can check the item table to verify whether the device resources have been successfully saved.

[0101] According to embodiments of this disclosure, the resource task table includes a task identifier, a device resource identifier, the number of currently received resources, and the total number of resources; wherein, the resource task sub-table includes a task sub-table identifier, a task identifier of the resource task table, and message information.

[0102] The aforementioned resource task table and resource task sub-table can be created using data tables in the database, and can be distinguished by task identifiers and task sub-table identifiers.

[0103] The device resource identifier serves as a unique identifier for the collected device, distinguishing it from other devices. The total resource count represents the total number of device resources collected across multiple collection cycles; the currently received resource count represents the device resources acquired in the current collection cycle.

[0104] The resource task table mentioned above enables efficient control of the periodic resource acquisition process. Together with the resource task sub-table, it forms a cross-reference to verify whether the relevant equipment resources have been successfully saved or acquired.

[0105] The attributes or information to be saved in the above resource task table and resource task sub-table are exemplary. Those skilled in the art can set other attribute information or information to be saved according to actual testing needs.

[0106] According to the embodiments of this disclosure, the above-mentioned method of using the master worker node to perform a single-threaded scan of the resource task table and sending the resource discovery message queue to the slave worker node when the scan result meets the preset conditions includes: using the master worker node to perform a single-threaded scan of the currently received number of resources and the total number of resources in the resource task table, and sending the resource discovery message queue to the slave worker node when the currently received number of resources and the total number of resources are equal.

[0107] In the field of network device resource management, multiple distributed worker nodes simultaneously reading the same information or table can lead to deadlock. Therefore, locking of related resources is necessary, allowing only one worker node to read or write the relevant information or table at a time. Locks are categorized into pessimistic and optimistic locks. Pessimistic locking, based on a pessimistic attitude, aims to prevent all data conflicts. It takes a preventative approach by locking the data before modification, preventing any operation until the lock is released. Database locking mechanisms are generally based on pessimistic locking. Optimistic locking, on the other hand, takes an optimistic approach to data conflicts. It doesn't lock the data during operation (allowing multiple tasks to operate on the data concurrently). Only upon data submission is a mechanism used to verify for conflicts (typically implemented by adding version numbers and comparing them).

[0108] This disclosure utilizes a resource discovery message queue to achieve single-threaded control, thereby avoiding problems such as multiple distributed worker nodes competing for resources and deadlocks caused by reading the resource task table simultaneously.

[0109] According to embodiments of this disclosure, the above-mentioned method of controlling periodic resource collection from the target network by using a resource discovery message queue based on a preset resource threshold and collection interval includes: selecting target device resources that meet the preset resource threshold from the set of collectable device resources, and determining the scanning period based on the collection interval; using a resource discovery message queue to control multiple distributed slave nodes to obtain resource discovery messages in a distributed lock manner; and, when a resource discovery message is obtained, controlling the collector from the slave node to periodically collect resources from the target device based on the scanning period.

[0110] For example, during periodic equipment resource collection, users (such as maintenance personnel) can set up collection intervals to collect CPU temperature resources. If the CPU temperature exceeds a specified threshold during a certain collection cycle, an alarm message will be sent to the maintenance personnel, who can then promptly detect and handle the issue.

[0111] The aforementioned periodic resource collection ensures that maintenance personnel can monitor the status of relevant equipment resources in real time and promptly handle equipment resources in abnormal states (i.e., exceeding preset resource thresholds).

[0112] The method for collecting network device resources provided by the present invention will be further described in detail below through specific implementation and accompanying drawings.

[0113] Figure 3 The flowcharts illustrate multiple instances of embodiments according to this disclosure.

[0114] like Figure 3 As shown, existing technical solutions, when handling multiple instances, query the database simultaneously, leading to queries returning the same state, resulting in data duplication and inconsistency. In distributed systems, the CPA principle (Consistency, Availability, and Partition tolerance) often applies, and all three cannot be simultaneously achieved.

[0115] Consistency (C): Whether all data backups in a distributed system have the same value at the same time (equivalent to all nodes accessing the same latest data copy); Availability (A): Guaranteeing that every request receives a response regardless of success or failure; Partition tolerance (P): The loss or failure of any information in the system will not affect the continued operation of the system.

[0116] The aforementioned consistency can be categorized into strong consistency, weak consistency, and eventual consistency.

[0117] Strong consistency means that once an update operation is complete, any subsequent access by multiple processes or threads will return the latest updated value, until that data is updated by other data. However, this implementation has a significant impact on performance because it means that users cannot read data until the previous operation has been completed.

[0118] Weak consistency means the system does not guarantee that access by a process or thread will return the latest updated value. After data is successfully written, the system does not promise that the latest written value can be read immediately, nor does it specify a timeframe after which it can be read. It cannot even guarantee that the value will be accessible.

[0119] Eventual consistency, a type of weak consistency, cannot guarantee that all subsequent accesses will see the latest value after a data update. Instead, it requires a period of time before consistency is guaranteed (that is, after a certain period, the data between nodes will eventually reach a consistent state). During this time, the data may be inconsistent; this period of inconsistency is called the "inconsistency window." The length of the inconsistency window depends on many factors, such as the number of backups, network transmission latency, and system load.

[0120] The availability mentioned above refers to the service always being available with normal response times. Good availability mainly means that the system can serve users well, without user operation failures or access timeouts, resulting in a poor user experience.

[0121] The aforementioned partition tolerance refers to the ability of a distributed system to still provide services that meet consistency and availability when encountering network failures, unless the entire network environment fails.

[0122] Figure 4 The illustration schematically depicts a data processing flow for multiple instances or a single instance according to embodiments of the present disclosure.

[0123] like Figure 4 As shown, the multi / single instance processing utilizes database table locks. This ensures that inserts and updates to the `task` table (i.e., the resource task table) are queued and executed. Once `count` (the current number of resources) equals `total` (the total number of resources), subsequent business processes are triggered. The process involves first locking the table, updating the `count`, performing an atomic increment operation, releasing the table lock, querying the `count`, and if `count` equals `total`, executing subsequent operations, creating a "task", and setting the threshold and collection interval.

[0124] Figure 5 This is a schematic diagram illustrating the network device resource discovery process according to an embodiment of the present disclosure.

[0125] like Figure 5 As shown, this disclosure first uses multiple threads to receive messages, saves device resources, and updates the current state to 1. Then, a single thread scans the state; upon reaching a state threshold, the current state is updated to 2. The task is then sent to the slave node queue (i.e., the resource message queue) for execution. Next, the slave node receives the queue task from the master node, ensuring that only one slave node can preempt the task, and updates the state to 3. After executing the subsequent tasks, the state is updated to 4. Finally, when the process encounters a situation where multiple slave nodes need to ensure that only one node can handle the task, the single-threaded scanning operation and task preemption operation are repeated.

[0126] Taking a distributed system, specifically a wireless access point cluster (AP cluster), as an example, and combining it with the attached... Figure 6 The above-mentioned network device resource discovery and collection methods provided in this disclosure are further explained in detail.

[0127] Figure 6 The flowchart illustrating the discovery and collection of network device resources by an AP cluster according to an embodiment of the present disclosure is shown in the illustration.

[0128] First, connectivity tests are performed on the target network where the distributed system resides. These tests include, for example, device management testing, device connectivity testing, and SNMP protocol connectivity testing. If the connectivity tests are successful, device information and discovery formulas are then distributed. Second, such as... Figure 6 As shown, the collectors in the AP cluster send SNMP requests and return the OID (Object Identifier) ​​of collectable resource components in the target network. The AP or distributed worker nodes receive the resource information of the collected device and save / update the resource task table, while simultaneously saving the returned message to the resource task sub-table. The master node scans the resource task table, and when count=total is reached, it sends resource discovery information (Message Queue, MQ). The worker nodes receive the message, create resource records in the resource table, create tasks in the task table, update the threshold information and collection interval information in the task table, and send it to the corresponding collectors, which then generate periodic tasks. Those skilled in the art can set the resource task table and resource task sub-table according to actual management needs. The collectors access the "collected device" via the SNMP protocol, and the "collected device" returns a set of collectable resources. The collectors send this information to the worker nodes, which collect each type of resource set, save it to the task table, and save the returned message information to the item sub-table. The master node scans for tasks that meet the conditions, sends them to the worker nodes, and continues the subsequent business process.

[0129] from Figure 6 As can be seen, the network device resource collection method provided in this disclosure fundamentally eliminates database waiting issues and avoids database deadlocks by removing the database for update operation. Simultaneously, in the multiple instances of this disclosure, the master node polls the database in round-robin fashion, retrieves records indicating the "status" of the next operation, and sends them to the worker nodes. After completion, the business process ensures data consistency and reduces data redundancy.

[0130] Figure 7 The diagram illustrates the structure of a network device resource acquisition apparatus according to an embodiment of the present disclosure.

[0131] like Figure 7As shown, the network device resource acquisition device 700 includes a multi-threaded acquisition module 710, a task table creation module 720, a single-threaded scanning module 730, a periodic resource acquisition module 740, and a task table dynamic update module 750.

[0132] The multi-threaded acquisition module 710 is used to perform multi-threaded acquisition of device resources in the target network and send the set of collectable device resources in the target network to the working node. In one embodiment, the multi-threaded acquisition module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0133] In the embodiments of this disclosure, the multi-threaded acquisition module 710 can provide a corresponding operation entry point for the target network administrator, allowing the administrator to choose to agree or reject the automated decision-making result. That is, before acquiring data from the device resources, the system can obtain the target network administrator's instruction to agree or reject the acquisition through the corresponding operation entry point. If the target network administrator agrees to the acquisition, the device resources are acquired. If the target network administrator refuses to acquire data, the expert decision-making process is initiated.

[0134] The task table creation module 720 is used to create a resource task table and use the resource task table to classify and save the collection of collectable device resources received by the working node; in one embodiment, the task table creation module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0135] The single-threaded scanning module 730 is used to perform a single-threaded scan of the resource task table using the master worker node. If the scan result meets the preset conditions, the resource discovery message queue is sent to the slave worker node. In one embodiment, the single-threaded scanning module 730 can be used to execute the operation S230 described above, which will not be repeated here.

[0136] The periodic resource acquisition module 740 is used to periodically acquire resources from the target network from the working node based on a preset resource threshold and acquisition interval using a resource discovery message queue. In one embodiment, the periodic resource acquisition module 740 can be used to execute the operation S240 described above, which will not be repeated here.

[0137] The task table dynamic update module 750 is used to dynamically update the resource task table using the set of collectable device resources obtained from periodic scanning of the working nodes. In one embodiment, the task table dynamic update module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0138] According to embodiments of this disclosure, any multiple modules among the multi-threaded acquisition module 710, task table creation module 720, single-threaded scanning module 730, periodic resource acquisition module 740, and task table dynamic update module 750 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the multi-threaded acquisition module 710, task table creation module 720, single-threaded scanning module 730, periodic resource acquisition module 740, and task table dynamic update module 750 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the multi-threaded acquisition module 710, task table creation module 720, single-threaded scanning module 730, periodic resource acquisition module 740, and task table dynamic update module 750 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0139] According to embodiments of this disclosure, the multi-threaded acquisition module 710 includes a connectivity testing submodule and a scanning and acquisition submodule.

[0140] The connectivity testing submodule is used to perform connectivity tests on the target network and distribute the device information to be collected and the predefined resource discovery formula to the distributed collector.

[0141] The scanning and acquisition submodule is used to scan the target network using a distributed collector based on the device information to be acquired and the predefined resource discovery formula, and to acquire the collectable device resources in the target network through multi-threaded acquisition to obtain a set of collectable device resources.

[0142] According to embodiments of this disclosure, the connectivity testing submodule includes a network and protocol testing unit.

[0143] The network and protocol testing unit is used to perform connectivity tests on devices in the target network and to test the network protocols used in the scanning and acquisition process.

[0144] According to embodiments of this disclosure, the scanning and acquisition submodule includes an access unit, a scanning unit, and an acquisition unit.

[0145] The access unit is used to send SNMP-based requests to the target network using a distributed collector, thereby enabling access to the target network.

[0146] The scanning unit is used to scan the target network based on the SNMP protocol using a distributed collector to determine the collectable device resources in the target network.

[0147] The acquisition unit is used to collect data on collectable device resources in the target network using a distributed collector based on the SNMP protocol, thereby obtaining a set of collectable device resources.

[0148] According to embodiments of this disclosure, the task table creation module 720 includes a classification submodule, a storage submodule, and a subtable creation submodule.

[0149] The classification submodule is used to classify device resources in the set of collectable device resources based on object identifiers and using worker nodes.

[0150] The storage submodule is used to create a resource task table for each type of device resource based on the classification results, and to store the identifier of each type of device resource in the resource task table.

[0151] The sub-table creation sub-module is used to create resource task sub-tables and store the message information returned during the creation process of resource task tables.

[0152] According to embodiments of this disclosure, the single-threaded scanning module 730 described above includes a discrimination submodule.

[0153] The discrimination submodule is used to perform a single-threaded scan of the currently received resource count and the total number of resources in the resource task table using the master worker node. If the currently received resource count and the total number of resources are equal, the resource discovery message queue is sent to the slave worker node.

[0154] According to embodiments of this disclosure, the periodic resource acquisition module 740 includes: a filtering and determination submodule, a resource acquisition submodule, and a periodic acquisition submodule.

[0155] The filtering and determination submodule is used to filter target device resources that meet the preset resource threshold from the set of collectable device resources, and determine the scanning cycle based on the collection interval.

[0156] The resource acquisition submodule is used to control multiple distributed worker nodes to acquire resource discovery messages using a distributed lock via a resource discovery message queue.

[0157] The periodic acquisition submodule is used to periodically acquire resources from the target device by controlling the collector from the working node based on the scanning cycle when a resource discovery message is received.

[0158] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for collecting network device resources according to an embodiment of the present disclosure.

[0159] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0160] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0161] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0162] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0163] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0164] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the network device resource acquisition method provided in the embodiments of this disclosure.

[0165] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0166] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0167] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0168] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0170] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0171] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for collecting network device resources, characterized in that, The method includes: The device resources in the target network are collected in a multi-threaded manner, and the collection of collectable device resources in the target network is sent to the working node; Create a resource task table and use the resource task table to classify and save the set of collectable device resources received by the working node; The resource task table is scanned in a single thread using the master worker node. If the scan results meet the preset conditions, the resource discovery message queue is sent to the slave worker node. Based on preset resource thresholds and collection intervals, the resource discovery message queue is used to control the slave node to periodically collect resources from the target network. The resource task table is dynamically updated using the set of collectable device resources obtained from the periodic resource collection of the working nodes.

2. The method according to claim 1, characterized in that, Multi-threaded acquisition of device resources in the target network includes: The target network is tested for connectivity, and the device information to be collected and the predefined resource discovery formula are sent to the distributed collector. Based on the device information to be collected and the predefined resource discovery formula, the distributed collector is used to scan the target network and collect the collectable device resources in the target network in a multi-threaded manner to obtain the set of collectable device resources.

3. The method according to claim 2, characterized in that, The connectivity test of the target network includes: Connectivity tests are performed on devices in the target network, and connectivity tests are also performed on the network protocols used in the scanning and data collection process.

4. The method according to claim 2, characterized in that, Based on the device information to be collected and the predefined resource discovery formula, the distributed collector scans the target network and performs multi-threaded collection of collectable device resources in the target network, resulting in a set of collectable device resources including: The distributed collector sends a request based on the SNMP protocol to the target network to achieve access to the target network; Based on the SNMP protocol, the distributed collector scans the target network to determine the collectable device resources in the target network; Based on the SNMP protocol, the distributed collector is used to collect collectable device resources in the target network in a multi-threaded manner to obtain the set of collectable device resources.

5. The method according to claim 1, characterized in that, Creating a resource task table and using the resource task table to classify and store the set of collectable device resources received by the working node includes: Based on object identifiers, the working nodes are used to classify the device resources in the set of collectable device resources; Based on the classification results, a resource task table is created for each type of device resource, and the identifier of each type of device resource is stored in the resource task table; Create a resource task sub-table and use the resource task sub-table to store the message information returned during the creation process of the resource task table.

6. The method according to claim 5, characterized in that, The resource task table includes a task identifier, a device resource identifier, the number of currently received resources, and the total number of resources. The resource task sub-table includes a task sub-table identifier, a resource task table task identifier, and message information.

7. The method according to claim 1, characterized in that, The resource task table is scanned in a single thread using the master worker node. If the scan results meet preset conditions, the resource discovery message queue is sent to the slave worker nodes, including: The master worker node performs a single-threaded scan of the currently received resource count and the total number of resources in the resource task table. If the currently received resource count and the total number of resources are equal, the resource discovery message queue is sent to the slave worker node.

8. The method according to claim 1, characterized in that, Based on preset resource thresholds and collection intervals, controlling the periodic resource collection of the target network by the slave node using the resource discovery message queue includes: Target device resources that meet the preset resource threshold are selected from the set of collectable device resources, and the scanning cycle is determined based on the collection interval; The resource discovery message queue is used to control the acquisition of resource discovery messages from multiple distributed worker nodes in a distributed lock manner. Upon receiving the resource discovery message, the resource collector from the slave node periodically collects resources from the target device based on the scanning cycle.

9. A network device resource acquisition device, characterized in that, The device includes: The multi-threaded acquisition module is used to perform multi-threaded acquisition of device resources in the target network and send the set of collectable device resources in the target network to the working node; The task table creation module is used to create a resource task table and use the resource task table to classify and save the set of collectable device resources received by the working node. The single-threaded scanning module is used to perform a single-threaded scan of the resource task table using the main working node. If the scan result meets the preset conditions, the resource discovery message queue is sent to the slave working node. A periodic resource acquisition module is used to control the slave node to periodically acquire resources from the target network based on a preset resource threshold and acquisition interval using the resource discovery message queue. The task table dynamic update module is used to dynamically update the resource task table using the set of collectable device resources obtained from the periodic resource collection of the working nodes.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images