Network device resource acquisition method, device, equipment, medium and program product

Abstract

The invention provides a network equipment resource acquisition method which can be applied to the technical field of information security and big data. The method comprises the following steps: performing multi-thread acquisition on equipment resources in a target network, and sending an acquirable equipment resource set in the target network to a working node; utilizing the created resource task table to store an acquirable equipment resource set received by the working node in a classified manner; performing single-thread scanning on the resource task table by using the master working node, and issuing the resource discovery message queue to the slave working node under the condition that a scanning result meets a preset condition; based on a preset resource threshold and an acquisition interval, using the resource discovery message queue to control the slave working node to perform periodic resource acquisition on the target network; and dynamically updating the resource task table by utilizing an acquirable equipment resource set obtained by periodic resource acquisition of the working node. The invention also provides a network equipment resource acquisition device, equipment, a medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the fields of information security and big data technology, specifically the field of network device management technology, and more specifically to a method, device, equipment, medium and program product for collecting network device resources. Background Art

[0002] With the rapid development of informatization and big data, the number of network equipment resources has increased. How to effectively manage the massive network equipment resources has become the focus of research by relevant technical personnel. Network equipment resource management usually includes resource acquisition, resource configuration, resource monitoring, resource maintenance, and resource update. Effective management of network equipment resources can not only improve network performance and reduce operating costs, but also improve the security and availability of data running or stored in network equipment resources.

[0003] In order to effectively manage network equipment resources, it is necessary to first collect network equipment resources so as to subsequently process the collected network equipment resources in a targeted manner. However, in the face of massive network equipment resources, existing technical solutions have technical problems such as synchronization blocking and data inconsistency during the network equipment resource collection process. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a method, apparatus, device, medium and program product for collecting network device resources to improve the efficiency of collecting device resources in a target network.

[0005] According to a first aspect of the present disclosure, a method for collecting network device resources is provided, comprising:

[0006] Perform multi-threaded collection of device resources in the target network, and send the collection of device resources that can be collected in the target network to the working node;

[0007] Create a resource task table and use the resource task table to classify and save the collection of collectible device resources received by the working node;

[0008] Use the master working node to perform a single-thread scan on the resource task table. If the scan result meets the preset conditions, send the resource discovery message queue to the slave working node;

[0009] Based on the preset resource threshold and collection interval, the resource discovery message queue is used to control the periodic resource collection of the target network from the working node;

[0010] The resource task table is dynamically updated using the collectible device resource set obtained from the periodic resource collection of the working node.

[0011] According to an embodiment of the present disclosure, the multi-threaded acquisition of device resources in the target network includes:

[0012] Perform connectivity tests on the target network and send the device information to be collected and the predefined resource discovery formula to the distributed collector;

[0013] Based on the device information to be collected and the predefined resource discovery formula, the target network is scanned using a distributed collector, and the collectible device resources in the target network are collected through multi-threading to obtain a set of collectible device resources.

[0014] According to an embodiment of the present disclosure, the above-mentioned connectivity test on the target network includes:

[0015] Perform connectivity tests on devices in the target network and on the network protocols used during scanning and acquisition.

[0016] According to an embodiment of the present disclosure, based on the device information to be collected and the predefined resource discovery formula, the target network is scanned using a distributed collector, and the device resources that can be collected in the target network are collected by multi-threading, and the device resource set that can be collected includes:

[0017] Use the distributed collector to send SNMP protocol-based requests to the target network to access the target network;

[0018] Based on the SNMP protocol, the target network is scanned using distributed collectors to determine the device resources that can be collected in the target network;

[0019] Based on the SNMP protocol, distributed collectors are used to perform multi-threaded collection of collectible device resources in the target network to obtain a collection of collectible device resources.

[0020] According to an embodiment of the present disclosure, the above-mentioned creation of a resource task table and using the resource task table to classify and save the set of collectible device resources received by the working node include:

[0021] Based on the object identifier, the device resources in the collectible device resource set are classified using the working node;

[0022] Based on the classification result, a resource task table is created for each type of device resource, and an identifier of each type of device resource is stored in the resource task table;

[0023] Create a resource task subtable and use the resource task subtable to save the message information returned during the resource task table creation process.

[0024] According to an embodiment of the present disclosure, the resource task table includes a task identifier, a device resource identifier, a number of currently received resources, and a total number of resources;

[0025] The resource-task sub-table includes a task sub-table identifier, a task identifier of the resource-task table, and message information.

[0026] According to an embodiment of the present disclosure, the above-mentioned single-threaded scanning of the resource task table by the master working node, when the scanning result meets the preset conditions, sending the resource discovery message queue to the slave working node includes:

[0027] The master working node is used to perform a single-threaded scan on the number of currently received resources and the total number of resources in the resource task table. When the number of currently received resources and the total number of resources are equal, the resource discovery message queue is sent to the slave working node.

[0028] According to an embodiment of the present disclosure, the above-mentioned method of controlling the periodic resource collection of the target network from the working node by using the resource discovery message queue based on the preset resource threshold and collection interval includes:

[0029] Filter target device resources that meet a preset resource threshold from a collection of device resources that can be collected, and determine a scanning period based on a collection interval;

[0030] Use the resource discovery message queue to control multiple distributed slave working nodes to obtain resource discovery messages in a distributed lock manner;

[0031] When a resource discovery message is obtained, periodic resource collection of target device resources is performed by controlling the collector from the working node based on the scanning cycle.

[0032] According to a second aspect of the present disclosure, a network device resource collection device is provided, comprising:

[0033] A multi-threaded acquisition module is used to perform multi-threaded acquisition of device resources in the target network and send the set of device resources that can be acquired in the target network to the working node;

[0034] A task table creation module is used to create a resource task table and use the resource task table to classify and save the collection of collectible device resources received by the working node;

[0035] The single-thread scanning module is used to use the master working node to perform a single-thread scan on the resource task table. When the scanning result meets the preset conditions, the resource discovery message queue is sent to the slave working node;

[0036] A periodic resource collection module is used to control the periodic resource collection of the target network from the working node based on the preset resource threshold and collection interval by using the resource discovery message queue;

[0037] The task table dynamic update module is used to dynamically update the resource task table using the collectible device resource set obtained from the periodic scan of the working node.

[0038] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0039] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0040] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0041] The above-mentioned network device resource collection method provided by the present invention improves the collection efficiency of network device resources by performing multi-threaded resource collection on the target network and saving the collected device resources through a resource task table; at the same time, the main working node is used to perform a single-threaded scan on the resource task table, and a resource discovery message queue is generated to control the slave working node to periodically collect the collectible device resources in the target network, thereby avoiding technical problems such as synchronization blocking and data inconsistency; in addition, technical personnel in this field can use the generated resource task table to monitor or manage the collectible device resources in the target network in real time, reducing operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0043] Figure 1 The following schematically shows an application scenario diagram of the method for collecting network device resources according to an embodiment of the present disclosure;

[0044] Figure 2 A flowchart of a method for collecting network device resources according to an embodiment of the present disclosure is schematically shown;

[0045] Figure 3 Schematically shows a flowchart in a multi-instance situation according to an embodiment of the present disclosure;

[0046] Figure 4 The data processing flow of multiple instances or a single instance according to an embodiment of the present disclosure is schematically shown;

[0047] Figure 5 is a diagram schematically showing a process of discovering network device resources according to an embodiment of the present disclosure;

[0048] Figure 6 The flowchart of discovering and collecting network device resources by an AP cluster according to an embodiment of the present disclosure is schematically shown;

[0049] Figure 7 The structure diagram of the network device resource collection device according to the embodiment of the present disclosure is schematically shown;

[0050] Figure 8 A block diagram of an electronic device suitable for implementing a method for collecting network device resources according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0051] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0052] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0053] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0054] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0055] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0056] In the technical solution of the present disclosure, the information of the target network involved (including but not limited to the hardware information of the target network, the software information of the target network, the configuration information of the target network, the address information of the target network, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the owner of the target network or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for the owner of the target network to choose to authorize or refuse.

[0057] In the process of collecting or discovering network equipment resources, the existing technical solutions query the database at the same time in multiple instances, resulting in the same state being queried, resulting in data duplication, and thus leading to technical problems such as synchronization blocking and data inconsistency. Among them, synchronization blocking: including several processes (such as the process of the Coordinator waiting for all participants to vote) are all synchronously blocked, and all logic involved in the transaction operation is in a blocked state. Each participant will not be able to perform any other operations while waiting for other participants to respond. In actual applications, this problem is solved by a timeout judgment mechanism, but it cannot completely solve the synchronization blocking problem, deadlock problem, and database table lock problem; among them, data inconsistency: if the received data arrives at the same time during the query phase. As a result, the update is completed, and the query data is unlocked at the same time point, and the row lock of the table is unlocked. Due to the consistency, atomicity, and persistence of the database, the query results are the same, resulting in data redundancy.

[0058] Figure 1 The application scenario diagram of the network device resource collection method according to an embodiment of the present disclosure is schematically shown.

[0059] like Figure 1 As shown, the application scenario 100 according to this embodiment may include the fields of information security and big data technology. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0060] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0061] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0062] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0063] It should be noted that the network device resource collection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the network device resource collection device provided in the embodiment of the present disclosure can generally be set in the server 105. The network device resource collection method provided in the embodiment of the present disclosure can also be performed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the network device resource collection device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0064] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0065] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The method for collecting network device resources of the disclosed embodiment is described in detail.

[0066] Figure 2 The flowchart of the method for collecting network device resources according to an embodiment of the present disclosure is schematically shown.

[0067] like Figure 2 As shown, the method for collecting network device resources in this embodiment includes operations S210 to S250.

[0068] In operation S210, device resources in the target network are collected through multi-threading, and a set of collectible device resources in the target network is sent to a working node.

[0069] The target network may be a local area network in the same geographical location, or local area networks distributed in different geographical locations, or a wide area network that has been authorized with trust.

[0070] Before performing operation S210, it is necessary to obtain authorization from the target network administrator. The above-mentioned device resource set that can be collected includes shared device resources, externally visible device resources, and device resources that are visible with authorization from the target network administrator.

[0071] The above-mentioned device resources include the software and hardware resources of the device, for example, the device's CPU resource information, hard disk resource information, motherboard resource information, graphics card resource information, etc. Those skilled in the art can set the device resources to be collected according to actual management requirements.

[0072] For example, before operation S210, a request for acquiring information collection may be sent to the target network manager. If the target network manager agrees or authorizes the acquisition of device resources, the operation is performed.

[0073] In the embodiment of the present disclosure, a corresponding operation entry can be provided for the target network manager to choose to agree or reject the automated decision result. That is, before the device resources are collected, the instruction of the target network manager to agree or reject the collection can be obtained through the corresponding operation entry. If the target network manager agrees to collect, the device resources are collected, that is, step S210 is executed. If the target network manager refuses to collect, the expert decision process is entered.

[0074] In operation S220, a resource task table is created and the resource task table is used to classify and save the set of collectible device resources received by the working node.

[0075] The resource task table is used to store the ID of the device resource, the status information of the device resource, etc. Those skilled in the art can set the device information to be stored according to actual management requirements.

[0076] In operation S230, the master working node is used to perform a single-thread scan on the resource task table, and when the scan result meets the preset conditions, the resource discovery message queue is sent to the slave working node.

[0077] A worker node can be randomly designated as the master worker node from multiple distributed worker nodes. The master worker node is used to control periodic resource collection.

[0078] In operation S240, based on a preset resource threshold and collection interval, a resource discovery message queue is used to control the slave working node to perform periodic resource collection on the target network.

[0079] The resource thresholds may be, for example, a CPU temperature threshold, a hard disk speed threshold, a temperature threshold, a storage capacity threshold, a read / write peak value, etc. Those skilled in the art may set the resource thresholds according to actual management needs. In addition, those skilled in the art may also set the collection intervals according to actual management needs.

[0080] In operation S250, the resource task table is dynamically updated using the collectible device resource set obtained from the periodic resource collection of the working node.

[0081] During the real-time operation of the target network, the equipment resources are changing dynamically. Therefore, it is necessary to dynamically update the collectible equipment resources in the target network. The dynamically updated equipment resources can provide real-time equipment resource data to the relevant operation and maintenance personnel, facilitate subsequent operation and maintenance, and thus reduce operation and maintenance costs and improve operation and maintenance efficiency.

[0082] The above-mentioned network device resource collection method provided by the present invention improves the collection efficiency of network device resources by performing multi-threaded resource collection on the target network and saving the collected device resources through a resource task table; at the same time, the main working node is used to perform a single-threaded scan on the resource task table, and a resource discovery message queue is generated to control the slave working node to periodically collect the collectible device resources in the target network, thereby avoiding technical problems such as synchronization blocking and data inconsistency; in addition, technical personnel in this field can use the generated resource task table to monitor or manage the collectible device resources in the target network in real time, reducing operation and maintenance costs.

[0083] According to an embodiment of the present disclosure, the above-mentioned multi-threaded collection of device resources in the target network includes: performing a connectivity test on the target network, and sending the device information to be collected and the predefined resource discovery formula to the distributed collector; based on the device information to be collected and the predefined resource discovery formula, using the distributed collector to scan the target network, and performing multi-threaded collection on the device resources that can be collected in the target network to obtain a set of device resources that can be collected.

[0084] The device information to be collected includes the device hardware information and device software information, such as the device CPU information, hard disk information, etc. The collected device resources can be used by subsequent operation and maintenance personnel to manage the target network.

[0085] The predefined resource discovery formula, for example, may be to set the importance weight of the above information according to the type, location, quantity, status, etc. of the device resources, calculate the scores of the device resources, sort them, and include the device resources with scores higher than the threshold in the list of device resources to be collected.

[0086] Before collecting device resources, a connectivity test is required to ensure smooth network access during the subsequent collection process, improve collection efficiency and reduce data error rate.

[0087] According to an embodiment of the present disclosure, the above-mentioned connectivity test on the target network includes: performing connectivity test on devices in the target network, and performing connectivity test on network protocols used in the scanning and acquisition process.

[0088] Connectivity testing is the process of evaluating whether different devices in a target network can communicate effectively with each other. This type of testing is critical to ensuring the stability and reliability of the network.

[0089] Those skilled in the art can adopt appropriate network protocols according to actual management requirements.

[0090] In the disclosed embodiment, the optional network protocol is the SNMP protocol. SNMP connectivity test is an important network performance detection method, which is mainly used to detect the connectivity and communication speed of the SNMP protocol between the host and the SNMP Agent. This test process can be performed based on UDP messages, and the network performance is evaluated by sending and receiving messages. The implementation principle of the SNMP test involves sending a request message and calculating the time difference between receiving a response message and sending a request message, thereby reflecting the connectivity and performance status of the network SNMP protocol. The SNMP protocol connectivity test generally includes several key steps: sending an SNMP request message, receiving a response message, and calculating the communication time.

[0091] The above connectivity test ensures the smooth transmission of target network information, making it convenient for the subsequent rapid transfer of collected device resources to the working node.

[0092] According to an embodiment of the present disclosure, based on the device information to be collected and the predefined resource discovery formula, the target network is scanned by a distributed collector, and the device resources that can be collected in the target network are collected by multi-threading to obtain a set of device resources that can be collected, including: using a distributed collector to send a request based on the SNMP protocol to the target network to achieve access to the target network; based on the SNMP protocol, the target network is scanned by a distributed collector to determine the device resources that can be collected in the target network; based on the SNMP protocol, the device resources that can be collected in the target network are collected by a distributed collector in multi-threading to obtain a set of device resources that can be collected.

[0093] SNMP (Simple Network Management Protocol) is a standard protocol specially designed for managing network nodes (servers, workstations, HUBS, etc.) in IP networks. It is an application layer protocol.

[0094] SNMP enables network administrators to manage network performance, discover and solve network problems, and plan network growth. By receiving random messages through SNMP, the network management system is informed of network problems.

[0095] Skilled technicians can set the network protocol used according to actual collection needs.

[0096] The distributed collector can greatly improve the efficiency of resource collection by performing multi-threaded device resource collection. Among them, the distributed collector can be set in a distributed system. A distributed system (distributed) refers to deploying different service modules in multiple different servers, working together through remote calls, and providing services to the outside. A distributed system can form a cluster, which refers to deploying the same application or service module in multiple different servers to form a cluster, and providing services to the outside through a load balancing device.

[0097] According to an embodiment of the present disclosure, the above-mentioned creation of a resource task table and use of the resource task table to classify and save the set of collectible device resources received by the working node include: based on the object identifier, using the working node to classify the device resources in the set of collectible device resources; based on the classification result, creating a resource task table for each type of device resource, and saving the identifier of each type of device resource in the resource task table; creating a resource task sub-table, and using the resource task sub-table to save the message information returned during the resource task table creation process.

[0098] The resource task table and resource task subtable mentioned above can be created through a database. Databases have many advantages and are suitable for storing large amounts of data, especially when there are a large number of device resources that can be collected in the target network. Among them, the resource task table and resource task subtable based on the database can centrally store a large amount of data, and at the same time, it can easily organize, update, query and delete data to avoid data dispersion and duplication. The consistency and integrity of the database data are ensured. For example, primary keys (i.e., task identifiers), foreign keys, unique constraints, etc. can be defined to avoid data redundancy and inconsistency. Since the database provides strict security mechanisms, including user authentication and authority management, it ensures that only authorized users can access and operate data. This protects the security and privacy of the data. The database is equipped with an efficient index structure, which allows for quick retrieval and query of data. In addition, in terms of data backup and recovery, the database allows regular data backup and provides a corresponding recovery mechanism to prevent data loss or damage. This is an important means to ensure data reliability and business continuity.

[0099] The resource task table (task table, the same below) provided by the present disclosure can not only be used to save the collected device resources, but also be used to control the periodic resource collection process in real time.

[0100] The resource task sub-table (item table, the same below) can store message information to show whether the relevant device resources are successfully saved. Relevant operation and maintenance personnel can verify whether the device resources are successfully saved by checking the item table.

[0101] According to an embodiment of the present disclosure, the above-mentioned resource task table includes a task identifier, a device resource identifier, the number of currently received resources and the total number of resources; wherein the resource task subtable includes a task subtable identifier, a task identifier of the resource task table and message information.

[0102] The resource-task table and resource-task sub-table can be created using a data table in a database and can be distinguished by a task identifier and a task sub-table identifier.

[0103] The device resource identifier is used as a unique identifier of the collected device to distinguish it from other devices. The total number of resources indicates the total number of device resources collected in multiple collection cycles; the number of resources currently received indicates the device resources obtained in the current collection cycle.

[0104] The resource task table can realize efficient control of the periodic resource collection process, and form a cross-reference with the resource task sub-table to verify whether the relevant equipment resources have been successfully saved or collected.

[0105] The attributes or information to be saved in the above resource task table and resource task sub-table are exemplary, and those skilled in the art may set other attribute information or information to be saved according to actual test requirements.

[0106] According to an embodiment of the present disclosure, the above-mentioned use of the main working node to perform a single-threaded scan on the resource task table, and when the scan result meets the preset conditions, sending the resource discovery message queue to the slave working node includes: using the main working node to perform a single-threaded scan on the currently received number of resources and the total number of resources in the resource task table, and when the currently received number of resources is equal to the total number of resources, sending the resource discovery message queue to the slave working node.

[0107] In the field of network equipment resource management, multiple distributed working nodes reading a certain information or information table at the same time will cause deadlock problems. Therefore, it is necessary to lock the relevant resources, and only one working node is allowed to read and write relevant information or information tables at the same time. Among them, locks are divided into pessimistic locks and optimistic locks. Pessimistic locks are based on a pessimistic attitude to prevent all data conflicts. It locks the data before modifying the data in a preventive manner, and then reads and writes the data. Before it releases the lock, no one can operate on its data. Only after the previous person releases the lock and the next person locks the data can the data be locked, and then the data can be operated. Generally, the lock mechanism of the database itself is implemented based on the pessimistic lock mechanism; optimistic locks maintain an optimistic attitude towards data conflicts. When operating data, the data will not be locked (this allows multiple tasks to operate on the data in parallel). Only when the data is submitted, a mechanism is used to verify whether there is a conflict in the data (the general implementation method is to add a version number and then compare the version numbers).

[0108] The present disclosure utilizes a resource discovery message queue to implement single-thread control, thereby avoiding problems such as multiple distributed slave working nodes grabbing resources and simultaneously reading a resource task table causing deadlock.

[0109] According to an embodiment of the present disclosure, the above-mentioned use of a resource discovery message queue to control the periodic resource collection of a target network from a working node based on a preset resource threshold and collection interval includes: screening target device resources that meet a preset resource threshold from a set of collectible device resources, and determining a scanning period based on the collection interval; using a resource discovery message queue to control multiple distributed working nodes to obtain resource discovery messages in a distributed lock manner; and when a resource discovery message is obtained, based on the scanning period, periodic resource collection of the target device resources is performed by controlling the collector from the working node.

[0110] For example, during the periodic equipment resource collection process, users (such as operation and maintenance personnel) can set the collection interval to collect the CPU temperature resources. When the CPU temperature exceeds the specified threshold within a certain collection cycle, an alarm message is sent to the operation and maintenance personnel, who can then discover and handle it in time.

[0111] The above-mentioned periodic resource collection can ensure that the operation and maintenance personnel can monitor the status of relevant equipment resources in real time and handle equipment resources that are in abnormal status (i.e., exceeding the preset resource threshold) in a timely manner.

[0112] The above-mentioned network device resource collection method provided by the present invention is further described in detail below through specific implementation and drawings.

[0113] Figure 3 The flowchart in a multi-instance situation according to an embodiment of the present disclosure is schematically shown.

[0114] like Figure 3 As shown in the figure, the existing technical solutions will query the database at the same time when processing multiple instances, resulting in the same state being queried, data duplication and data inconsistency. In a distributed system, the CPA principle constraints are often faced, namely, consistency, availability, and partition tolerance, and the three cannot be achieved at the same time.

[0115] Among them, consistency (C): whether all data backups in a distributed system have the same value at the same time. (Equivalent to all nodes accessing the same latest copy of data); among them, availability (A): ensure that each request has a response regardless of success or failure; among them, partition fault tolerance (P): the loss or failure of any information in the system will not affect the continued operation of the system.

[0116] The above consistency is divided into strong consistency, weak consistency and eventual consistency.

[0117] Among them, strong consistency: when the update operation is completed, any subsequent access by multiple processes or threads will return the latest updated value until the data is updated by other data. However, this implementation has a greater impact on performance, because it means that users cannot read data as long as the last operation is not completed.

[0118] Among them, weak consistency: the system does not guarantee that the access of a process or thread will return the latest updated value. After the data is successfully written, the system does not promise that the latest written value can be read immediately, nor does it promise a specific time to read it. It cannot even guarantee that it can be accessed.

[0119] Among them, eventual consistency: eventual consistency is also a kind of weak consistency. It cannot guarantee that after the data is updated, all subsequent accesses can see the latest value. Instead, it takes a period of time to guarantee this (that is, after a period of time, the data between nodes will eventually reach a consistent state). During this period of time, the data may be inconsistent. The time segment in which the system cannot guarantee strong consistency is called the "inconsistency window". The length of the inconsistency window depends on many factors, such as the number of backup data, network transmission delay speed, system load, etc.

[0120] The above availability means that the service is always available and the response time is normal. Good availability mainly means that the system can serve users well without user operation failures or access timeouts and other poor user experience.

[0121] The above-mentioned partition fault tolerance means that when a distributed system encounters a network failure, it can still provide services that meet consistency and availability to the outside world, unless the entire network environment fails.

[0122] Figure 4 The data processing flow of multiple instances or a single instance according to an embodiment of the present disclosure is schematically shown.

[0123] like Figure 4 As shown, multi- / single-instance processing uses database table locks. Ensure that the task table (i.e., resource task table) is inserted and updated, queued for execution, and reaches count (current number of resources) equal to total (total resource tree), triggering subsequent business processes. First lock the table, update the count, atomic +1 operation, unlock the table lock, query the count, if count is equal to total, perform subsequent operations, create a "task", set the threshold and collection interval.

[0124] Figure 5 The diagram schematically shows a process diagram of network device resource discovery according to an embodiment of the present disclosure.

[0125] like Figure 5 As shown, the disclosure first receives messages in multiple threads, saves device resources, and updates the current state to 1; secondly, the single-thread scans the state, reaches the state threshold point, and the current state is updated to 2. And it is sent to the slave node queue (i.e., resource message queue) to execute the task; then, the slave node receives the queue task sent by the master node, ensures that only one slave node preempts the task, and updates to state 3. After executing the subsequent tasks, it updates to state 4. Finally, the process encounters a situation where multiple slave nodes are required to ensure that only one node is processed, and the single-thread scanning operation and task preemption operation are repeated.

[0126] Take the wireless access point cluster (AP Cluster) in a distributed system as an example and combine it with the attached Figure 6 The above-mentioned network device resource discovery and collection method provided by the present disclosure is further explained in detail.

[0127] Figure 6 The flowchart of discovering and collecting network device resources by an AP cluster according to an embodiment of the present disclosure is schematically shown.

[0128] First, the target network where the distributed system is located is tested for connectivity, such as device management, device connectivity test, SNMP protocol connectivity test, etc. If there is no problem with the connectivity test, the device information and discovery formula are issued; secondly, Figure 6 As shown, the collector in the AP cluster sends an SNMP request and returns the OID (object identity) of the resource components that can be collected in the target network. The AP or distributed working node receives the resource information of the collected device, saves / updates the resource task table, and saves the return message to the resource task subtable; the master node scans the resource task table, and when count=total is reached, it sends the resource discovery information (Message Queue, MQ), the working node receives the message, creates a resource record to the resource table, creates a task to the task table, updates the threshold information and collection interval information in the task table, and sends it to the corresponding collector, and the collector generates a periodic task. Those skilled in the art can set the resource task table and resource task subtable according to actual management needs. The collector accesses the "collected device" through the SNMP protocol, and the "collected device" returns the resource collection that can be collected. The collector sends it to the work node, and the work node collects each type of resource collection, saves it in the task table, and saves the return message information in the item subtable. The master node scans the tasks that meet the conditions, sends them to the work node, and continues to do the subsequent business process.

[0129] from Figure 6 It can be seen that the above-mentioned network device resource collection method provided by the present disclosure removes the database for update operation, fundamentally eliminates the database waiting problem, and avoids database deadlock. At the same time, the master node in the multi-instance of the present disclosure polls the database, takes out the record of "status" reaching the next operation, and sends it to the work node to complete the business process, thereby ensuring data consistency and reducing data redundancy.

[0130] Figure 7 The schematic diagram shows the structure of a device for collecting network equipment resources according to an embodiment of the present disclosure.

[0131] like Figure 7As shown, the network device resource collection device 700 includes a multi-threaded collection module 710, a task table creation module 720, a single-threaded scanning module 730, a periodic resource collection module 740 and a task table dynamic update module 750.

[0132] The multi-threaded acquisition module 710 is used to perform multi-threaded acquisition of device resources in the target network and send the set of device resources that can be collected in the target network to the working node; in one embodiment, the multi-threaded acquisition module 710 can be used to execute the operation S210 described above, which will not be repeated here.

[0133] In the embodiment of the present disclosure, the multi-threaded acquisition module 710 can provide the target network manager with a corresponding operation entry for the target network manager to choose to agree or reject the automated decision result. That is, before the device resources are acquired, the target network manager can obtain the instruction to agree or reject the acquisition input through the corresponding operation entry. If the target network manager agrees to acquire, the device resources are acquired. If the target network manager refuses to acquire, the expert decision process is entered.

[0134] The task list creation module 720 is used to create a resource task list and use the resource task list to classify and save the set of collectible device resources received by the working node; in one embodiment, the task list creation module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0135] The single-thread scanning module 730 is used to use the main working node to perform a single-thread scan on the resource task table. When the scanning result meets the preset conditions, the resource discovery message queue is sent to the slave working node; in one embodiment, the single-thread scanning module 730 can be used to execute the operation S230 described above, which will not be repeated here.

[0136] The periodic resource collection module 740 is used to control the periodic resource collection of the target network from the working node based on the preset resource threshold and collection interval using the resource discovery message queue; in one embodiment, the periodic resource collection module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0137] The task table dynamic update module 750 is used to dynamically update the resource task table using the collectible device resource set obtained from the periodic scan of the working node. In one embodiment, the task table dynamic update module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0138] According to an embodiment of the present disclosure, any multiple modules of the multi-threaded acquisition module 710, the task table creation module 720, the single-threaded scanning module 730, the periodic resource acquisition module 740, and the task table dynamic update module 750 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the multi-threaded acquisition module 710, the task table creation module 720, the single-threaded scanning module 730, the periodic resource acquisition module 740, and the task table dynamic update module 750 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, at least one of the multi-threaded acquisition module 710, the task table creation module 720, the single-threaded scanning module 730, the periodic resource acquisition module 740, and the task table dynamic update module 750 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0139] According to an embodiment of the present disclosure, the multi-threaded acquisition module 710 includes a connectivity test submodule and a scanning and acquisition submodule.

[0140] The connectivity test submodule is used to perform connectivity test on the target network and send the device information to be collected and the predefined resource discovery formula to the distributed collector.

[0141] The scanning and collection submodule is used to scan the target network using a distributed collector based on the device information to be collected and the predefined resource discovery formula, and to perform multi-threaded collection of the collectible device resources in the target network to obtain a set of collectible device resources.

[0142] According to an embodiment of the present disclosure, the above-mentioned connectivity test submodule includes a network and protocol test unit.

[0143] The network and protocol test unit is used to perform connectivity tests on devices in the target network and to perform connectivity tests on the network protocols used in the scanning and acquisition process.

[0144] According to an embodiment of the present disclosure, the scanning and acquisition submodule includes an access unit, a scanning unit and an acquisition unit.

[0145] The access unit is used to send a request based on the SNMP protocol to the target network by using the distributed collector to access the target network.

[0146] The scanning unit is used to scan the target network using the distributed collector based on the SNMP protocol to determine the device resources that can be collected in the target network.

[0147] The collection unit is used to perform multi-threaded collection of collectible device resources in the target network using distributed collectors based on the SNMP protocol to obtain a collection of collectible device resources.

[0148] According to an embodiment of the present disclosure, the task table creation module 720 includes a classification submodule, a storage submodule and a subtable creation submodule.

[0149] The classification submodule is used to classify the device resources in the collectible device resource set based on the object identifier and using the working node.

[0150] The saving submodule is used to create a resource task table for each type of device resource based on the classification result, and save the identifier of each type of device resource in the resource task table.

[0151] The sub-table creation sub-module is used to create a resource task sub-table, and use the resource task sub-table to save the message information returned during the resource task table creation process.

[0152] According to an embodiment of the present disclosure, the above-mentioned single-thread scanning module 730 includes a determination submodule.

[0153] The discrimination submodule is used to use the master working node to perform a single-threaded scan on the number of currently received resources and the total number of resources in the resource task table. When the number of currently received resources and the total number of resources are equal, the resource discovery message queue is sent to the slave working node.

[0154] According to an embodiment of the present disclosure, the periodic resource collection module 740 includes: a screening and determination submodule, a resource acquisition submodule and a periodic collection submodule.

[0155] The screening and determination submodule is used to screen target device resources that meet a preset resource threshold from the set of collectible device resources, and determine the scanning period based on the collection interval.

[0156] The resource acquisition submodule is used to use the resource discovery message queue to control multiple distributed slave working nodes to obtain resource discovery messages in a distributed lock manner.

[0157] The periodic collection submodule is used to perform periodic resource collection on target device resources by controlling the collector from the working node based on the scanning cycle when the resource discovery message is obtained.

[0158] Figure 8 A block diagram of an electronic device suitable for implementing a method for collecting network device resources according to an embodiment of the present disclosure is schematically shown.

[0159] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0160] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0161] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0162] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0163] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0164] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the network device resource collection method provided by the embodiment of the present disclosure.

[0165] The above functions defined in the system / device of the embodiment of the present disclosure are executed when the computer program is executed by the processor 801. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0166] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0167] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0168] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0169] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0171] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for collecting network device resources, characterized in that: The method comprises: Perform multi-threaded collection of device resources in the target network, and send the set of device resources that can be collected in the target network to the working node; Creating a resource task table and using the resource task table to classify and save the set of collectible device resources received by the working node; Using the master working node to perform a single-threaded scan on the resource task table, and when the scan result meets the preset conditions, sending the resource discovery message queue to the slave working node; Based on a preset resource threshold and collection interval, using the resource discovery message queue to control the slave working node to perform periodic resource collection on the target network; The resource task table is dynamically updated using the collectible device resource set obtained from the periodic resource collection of the working nodes.

2. The method according to claim 1, characterized in that Multi-threaded acquisition of device resources in the target network includes: Performing a connectivity test on the target network, and sending the device information to be collected and the predefined resource discovery formula to the distributed collector; Based on the device information to be collected and the predefined resource discovery formula, the target network is scanned by the distributed collector, and the collectible device resources in the target network are collected by multi-threading to obtain the collectible device resource set.

3. The method according to claim 2, characterized in that The connectivity test of the target network includes: Perform connectivity tests on devices in the target network and on network protocols used during scanning and acquisition.

4. The method according to claim 2, characterized in that: Based on the device information to be collected and the predefined resource discovery formula, the target network is scanned by using the distributed collector, and the collectable device resources in the target network are collected by multi-threading, and the collectable device resource set obtained includes: Using the distributed collector to send a request based on the SNMP protocol to the target network to access the target network; Based on the SNMP protocol, the target network is scanned by the distributed collector to determine the device resources that can be collected in the target network; Based on the SNMP protocol, the distributed collector is used to perform multi-threaded collection of collectible device resources in the target network to obtain the collectible device resource set.

5. The method according to claim 1, characterized in that Creating a resource task table and using the resource task table to classify and save the set of collectible device resources received by the working node includes: Based on the object identifier, classify the device resources in the set of collectible device resources using the working node; Based on the classification result, creating the resource task table for each type of device resource, and storing the identifier of each type of device resource in the resource task table; A resource task subtable is created, and the resource task subtable is used to store the message information returned during the creation process of the resource task table.

6. The method according to claim 5, characterized in that The resource task table includes a task identifier, a device resource identifier, a number of currently received resources, and a total number of resources; The resource-task sub-table includes a task sub-table identifier, a task identifier of the resource-task table, and message information.

7. The method according to claim 1, characterized in that Using the master working node to perform a single-threaded scan on the resource task table, and when the scan result meets the preset conditions, sending the resource discovery message queue to the slave working node includes: The master working node is used to perform a single-threaded scan on the number of currently received resources and the total number of resources in the resource task table. When the number of currently received resources is equal to the total number of resources, the resource discovery message queue is sent to the slave working node.

8. The method according to claim 1, characterized in that Based on a preset resource threshold and collection interval, using the resource discovery message queue to control the slave working node to perform periodic resource collection on the target network includes: Filtering target device resources that meet the preset resource threshold from the set of collectible device resources, and determining a scanning period based on the collection interval; Using the resource discovery message queue to control multiple distributed slave working nodes to obtain resource discovery messages in a distributed lock manner; When the resource discovery message is obtained, based on a scanning cycle, the slave working node controls the collector to perform periodic resource collection on the target device resources.

9. A network equipment resource collection device, characterized in that: The device comprises: A multi-threaded acquisition module, used for performing multi-threaded acquisition of device resources in a target network, and sending a collection of device resources that can be acquired in the target network to a working node; A task table creation module, used to create a resource task table and use the resource task table to classify and save the set of collectible device resources received by the working node; A single-thread scanning module is used to use the master working node to perform a single-thread scan on the resource task table, and when the scanning result meets the preset conditions, the resource discovery message queue is sent to the slave working node; A periodic resource collection module, configured to control the slave working node to perform periodic resource collection on the target network using the resource discovery message queue based on a preset resource threshold and collection interval; The task table dynamic update module is used to dynamically update the resource task table using the set of collectible device resources obtained from the periodic resource collection of the working nodes.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images