A resource monitoring system, method, device, electronic device and storage medium

Abstract

The application discloses a resource monitoring system, method and device, electronic equipment and storage medium, and the system comprises a monitoring alarm server and at least one resource pool; the resource pool comprises at least one monitoring object and a resource monitoring device; the monitoring alarm server is connected with one end of the resource monitoring device, and index data is collected from the resource monitoring device; the other end of the resource monitoring device is connected with the monitoring object, and index data is collected from the monitoring object. The resource monitoring system disclosed by the application can quickly collect monitoring indexes in the resource pool of the existing monitoring architecture, effectively reduces the exposure risk of the resource pool, and improves the convenience of centralized monitoring and index collection. By being connected with the monitoring alarm server and the monitoring object respectively, the resource monitoring device can be seamlessly embedded into the existing monitoring system in the mode of an agent, and centralized monitoring in the internet multi-resource pool scene is realized.

Description

Technical Field

[0001] This invention relates to the field of information technology, specifically to a resource monitoring system, method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, with the rapid maturation of the Docker and Kubernetes ecosystems, Prometheus has gradually become mainstream in the monitoring field. Prometheus changes the traditional closed protocol approach of monitoring tools, adopting a more open protocol, thereby establishing a richer ecosystem.

[0003] Prometheus uses a client / server architecture for metric collection, meaning the server retrieves metric data from the exporter via HTTP calls. Figure 1 As shown, the advantage of this architecture lies in the openness of the protocol. Prometheus's data collection protocol is quite simple, just a text protocol carried by HTTP, which is simpler than SOAP and RESTful, and even does not require security. Simplicity means openness, making it easier for third parties to develop exporters. Secondly, it has network penetration capabilities. In the current network landscape, HTTP is probably the only protocol permitted by all security policies, and using HTTP can penetrate most networks.

[0004] However, if we place Prometheus in a more complex environment, its problems become apparent. The system establishes resource pools across different cloud service providers, with each pool containing several host devices. Each pool has only a limited number of egress IPs, and most devices are internal network devices that do not provide internet service. Under the Prometheus architecture, either all devices in the resource pool must have internet IPs, or service forwarding must be deployed on the egress server. The former would be significantly more expensive, while the latter lacks flexibility in service discovery. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the deficiency of insufficient resource pool output in the prior art, thereby providing a resource monitoring system, method, device, electronic device and storage medium.

[0006] According to a first aspect, the present invention discloses a resource monitoring system, comprising: a monitoring alarm server and at least one resource pool; the resource pool includes: at least one monitored object and a resource monitoring device; the monitoring alarm server is connected to one end of the resource monitoring device and collects indicator data from the resource monitoring device; the other end of the resource monitoring device is connected to the monitored object and collects indicator data from the monitored object.

[0007] According to a second aspect, the present invention discloses a resource monitoring method, comprising: collecting indicator data from a monitored object; and sending the indicator data to a monitoring alarm server according to a command sent by the monitoring alarm server.

[0008] Optionally, the step of collecting data from the monitored objects includes: obtaining a list of monitored objects; obtaining the indicator text of the monitored objects based on the list of monitored objects; and preprocessing the indicator text of the monitored objects to generate indicator items.

[0009] Optionally, obtaining the list of monitored objects includes: scanning all monitored objects in the resource pool; obtaining the list of monitored objects based on the scanning results; repeatedly scanning all monitored objects in the resource pool at preset time intervals; and updating the list of monitored objects based on the results of the repeated scanning.

[0010] Optionally, obtaining the indicator text of the monitored objects according to the list of monitored objects includes: sending an indicator retrieval command to each of the monitored objects according to the list of monitored objects; and receiving the indicator text returned by the monitored objects.

[0011] Optionally, the step of preprocessing the indicator text of the monitored object to generate indicator items includes: parsing and generating indicator data based on the indicator text; obtaining information about the monitored object; and generating indicator items based on the indicator data and the information about the monitored object.

[0012] Optionally, sending indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server includes: receiving a pull command sent by the monitoring and alarm server; the pull command sent by the monitoring and alarm server includes information about the object to be monitored; traversing the indicator items according to the information about the object to be monitored, finding the indicator item corresponding to the information about the object to be monitored; obtaining the required indicator data according to the indicator item corresponding to the information about the object to be monitored; and sending the required indicator data to the monitoring and alarm server.

[0013] According to a third aspect, the present invention discloses a resource monitoring device, comprising: a data acquisition module for acquiring data from a monitored object; and a data transmission module for sending indicator data to the monitoring alarm server according to a command sent by the monitoring alarm server.

[0014] According to a fourth aspect, the present invention discloses an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the steps of the resource monitoring method as described in the second aspect and any alternative embodiment of the second aspect.

[0015] According to a fifth aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the resource monitoring method as described in the second aspect and any alternative embodiment of the second aspect.

[0016] The technical solution of this invention has the following advantages:

[0017] The resource monitoring system disclosed in this invention, by setting up a resource monitoring device within the resource pool of an existing monitoring architecture, can quickly collect monitoring indicators within the resource pool, effectively reducing the risk of the resource pool being exposed to the outside world and improving the convenience of centralized monitoring and indicator collection. By connecting to both the monitoring alarm server and the monitored object respectively, the resource monitoring device can be seamlessly embedded into the existing system in a proxy mode. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is an architectural diagram illustrating a specific example from the background technology of this invention;

[0020] Figure 2 This is a principle block diagram of a specific example of a resource monitoring system in an embodiment of the present invention;

[0021] Figure 3 This is an architecture diagram of a specific example of a resource monitoring system in an embodiment of the present invention;

[0022] Figure 4 This is a flowchart illustrating a specific example of the resource monitoring method in this embodiment of the invention.

[0023] Figure 5 This is a flowchart illustrating another specific example of the resource monitoring method in this embodiment of the invention;

[0024] Figure 6This is a flowchart illustrating another specific example of the resource monitoring method in this embodiment of the invention;

[0025] Figure 7 This is a schematic diagram illustrating a specific example of a resource monitoring device in an embodiment of the present invention.

[0026] Figure 8 This is a specific example diagram of an electronic device in an embodiment of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0029] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can also refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] This invention discloses a resource monitoring system, such as... Figure 2 and Figure 3 As shown, it includes: a monitoring and alarm server 101 and at least one resource pool 102;

[0032] The resource pool 102 includes at least one monitored object 103 and a resource monitoring device 104.

[0033] The monitoring and alarm server 101 is connected to one end of the resource monitoring device 104 and collects indicator data from the resource monitoring device 104;

[0034] The other end of the resource monitoring device 104 is connected to the monitored object 103, and collects indicator data from the monitored object 103.

[0035] The resource monitoring system disclosed in this invention, by setting up resource monitoring devices within the resource pool of an existing monitoring architecture, can quickly collect monitoring indicators within the resource pool, effectively reducing the risk of the resource pool being exposed to the outside world and improving the convenience of centralized monitoring and indicator collection. By connecting to both the monitoring alarm server and the monitored object, the resource monitoring device can be seamlessly embedded into the existing monitoring system in a proxy mode, realizing centralized monitoring in multi-resource pool scenarios on the Internet.

[0036] This invention also discloses a resource monitoring method, such as... Figure 4 As shown, it includes the following steps:

[0037] Step S1: Collect indicator data from the monitored objects.

[0038] Specifically, the process of collecting data from monitored objects can first involve obtaining a list of monitored objects; then, based on the list of monitored objects, obtaining the indicator text of the monitored objects; and finally, preprocessing the indicator text of the monitored objects to generate indicator items.

[0039] Specifically, the process of obtaining the list of monitored objects can first scan all monitored objects in the resource pool; then, based on the scan results, obtain the list of monitored objects; next, repeat the scan of all monitored objects in the resource pool at preset time intervals; and finally, update the list of monitored objects based on the results of the repeated scans.

[0040] The process of scanning all monitored objects within the resource pool involves sending data packets to preset Internet Protocol (IP) addresses within the resource pool. The presence of a monitored object at each IP address is determined by whether a data packet is returned. The content of the returned data packet then identifies the monitored object information.

[0041] For example, the monitoring object information may include: an identifier, a business domain, a puller identifier, an IP address, a port number, an application identifier, and a service address. The identifier represents the unique identifier of the target, which can be encoded using IP address + port number; the business domain is used to classify the monitoring object, reducing the amount of data collected per instance; the puller identifier defines which puller processes the current target, and the puller determines the communication protocol and pull method; the IP address represents the Internet Protocol address of the monitoring object; the port number represents the protocol port used by the monitoring object; the application identifier represents the application running in the monitoring object; and the service address represents the address of the web service endpoint of the monitoring object.

[0042] The process of obtaining the list of monitored objects can be achieved by mapping each IP address to the information of the monitored objects.

[0043] For example, such as Figure 5 As shown, the process of obtaining a list of monitored objects can be implemented through a service discovery process. This process can be designed as a plug-in module, allowing for various implementations to be derived based on the application scenario. Taking one service discovery process as an example, the collection and scheduling server controls the process, initiating it periodically via a timer. First, a start command is used to notify the monitoring object acquirer to begin the current scan, facilitating the clearing of data from the previous scan. Then, the scanner's scanning function is invoked to start the scan. There may be multiple scanners; therefore, when a single scanner is processing its current scan, it first notifies the monitoring object acquirer via a start command. Each time a monitored object is discovered, a discovery command is used to notify the monitoring object acquirer so that the acquirer can save the discovered monitored objects. Finally, a stop command is used to notify the monitoring object acquirer that the scanner has completed its scan. Once all scanners have completed their scans, the collection and scheduling server notifies the monitoring object acquirer via a stop command, completing the process of obtaining the list of monitored objects.

[0044] The data collection and scheduling server is responsible for scheduling the metric collection activities. It dynamically discovers the list of monitored objects through scanners, then finds the corresponding puller based on the puller identifier attribute in the monitored object information, and uses corresponding commands to drive the puller to collect data from the monitored objects. The monitored object acquirer stores the list of collection objects and the service discovery results; the list of collection objects serves as the basis for the metric collection process. The scanner can be a scanner for collection objects, used to scan the target list under a preset configuration. The puller is used to perform metric collection and protocol conversion tasks, and can read the required data from the monitored object information and initiate metric collection activities according to its own implementation mode.

[0045] For example, the process of obtaining the list of monitored objects can also be implemented using existing technologies such as static file definition, hot file definition, and XScript script service discovery, and the present invention does not limit this.

[0046] The process of repeatedly scanning all monitored objects within the resource pool can be achieved by repeatedly scanning all monitored objects within the resource pool at preset time intervals. Specifically, this repeated scanning process allows for timely detection of changes in monitored objects within the resource pool, enabling real-time monitoring of the monitored objects' status. For example, the preset time interval can be pre-set according to the actual situation of the resource monitoring system; this invention does not limit this setting.

[0047] The process of updating the monitoring object list can be implemented by generating a new monitoring object list based on the results of repeatedly scanning all monitoring objects in the resource pool. Specifically, updating the monitoring object list can obtain the current status of monitoring objects in the resource pool, preventing the inability to retrieve indicator text due to an expired monitoring object list. For example, updating the monitoring object list can be implemented by overwriting the old monitoring object list with a new one, or by comparing the new and old monitoring object lists and only deleting expired items from the old list and adding new items. This invention does not limit the implementation to either method.

[0048] Specifically, the process of obtaining the indicator text of the monitored objects can first involve sending an indicator retrieval command to each of the monitored objects according to the list of monitored objects; and then receiving the indicator text returned by the monitored objects.

[0049] The process of sending an indicator retrieval command to each monitored object can be achieved by sending a preset data packet based on the IP address in the monitored object's information. Specifically, the preset data packet includes the retrieval command. The process of receiving the indicator text returned by the monitored object can be achieved by receiving the data packet returned by the monitored object and then extracting the indicator text from the data packet.

[0050] For example, such as Figure 6 As shown, the process of obtaining the indicator text of a monitored object can also be achieved through indicator data retrieval. This process is controlled by the data acquisition and scheduling server, which initiates the process at regular intervals using a timer. First, the list of target objects in the monitored object acquirer is obtained, and then each monitored object in the list is processed. The processing of each monitored object acquirer includes: first, obtaining the puller identifier to obtain the puller corresponding to the monitored object; then, calling the puller's execution function to perform the retrieval.

[0051] Specifically, the process of generating indicator items can first be based on the indicator text to parse and generate indicator data; then obtain the information of the monitored object; and finally generate indicator items based on the indicator data and the information of the monitored object.

[0052] Among them, parsing and generating indicator data can be achieved by converting indicator text into several items corresponding to several indicators and several data according to preset rules. Then, based on the information of the monitored object, the monitored object information, several indicators and several data are matched one by one to generate the corresponding number of indicator items.

[0053] For example, the process of parsing and generating indicator data can involve using a data extractor to parse the extracted data into data fragments, marking them according to preset rules based on the monitored object information, and then calling the storage module for storage. Here, a data fragment refers to all indicator data of a monitored object at a certain point in time; it represents the result of a single data collection by the data extractor at that point in time. The storage module stores the latest data fragments for all monitored objects.

[0054] Step S2: Send indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server.

[0055] Specifically, the process of sending indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server can be as follows: First, receive the pull command sent by the monitoring and alarm server; then, based on the information of the object to be monitored, traverse the indicator items to find the indicator items corresponding to the information of the object to be monitored; then, obtain the required indicator data based on the indicator items corresponding to the information of the object to be monitored; finally, send the required indicator data to the monitoring and alarm server.

[0056] Specifically, the monitoring and alarm server sends a retrieval command via an internet port. This retrieval command from the monitoring and alarm server includes information about the object to be monitored.

[0057] The resource monitoring method disclosed in this invention, by setting up a resource monitoring device within the resource pool of an existing monitoring architecture, can quickly collect monitoring indicators within the resource pool, effectively reducing the risk of the resource pool being exposed to the outside world and improving the convenience of centralized monitoring and indicator collection. By connecting to both the monitoring alarm server and the monitored object, the resource monitoring device can be seamlessly embedded into the existing monitoring system in a proxy mode, realizing centralized monitoring in multi-resource pool scenarios on the Internet.

[0058] This invention also discloses a resource monitoring device, such as... Figure 7 As shown, it includes:

[0059] The data acquisition module 201 is used to acquire data from the monitored object; for details, please refer to the relevant description of step S1 in the embodiment of the method of the present invention, which will not be repeated here.

[0060] The data acquisition module 201 includes: an indicator acquisition scheduling server PullServer, a monitoring object acquirer PullTargetCollection, a scanner Scanner, a puller Puller, and a storage module DataStore.

[0061] The data sending module 202 is used to send indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server; for details, please refer to the relevant description of step S2 in the embodiment of the method of the present invention, which will not be repeated here.

[0062] The resource monitoring device disclosed in this invention, by setting up a resource monitoring device within the resource pool of an existing monitoring architecture, can quickly collect monitoring indicators within the resource pool, effectively reducing the risk of the resource pool being exposed to the outside world and improving the convenience of centralized monitoring and indicator collection. By connecting to both the monitoring alarm server and the monitored object, the resource monitoring device can be seamlessly embedded into the existing monitoring system in a proxy mode, realizing centralized monitoring in multi-resource pool scenarios on the Internet.

[0063] This invention also provides an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 301 and a memory 302, wherein the processor 301 and the memory 302 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0064] Processor 301 can be a central processing unit (CPU). Processor 301 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0065] The memory 302, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the resource monitoring method in the embodiments of the present invention. The processor 301 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory 302, thereby implementing the resource monitoring method in the above method embodiments.

[0066] The memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 301, etc. Furthermore, the memory 302 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 302 may optionally include memory remotely located relative to the processor 301, and these remote memories may be connected to the processor 301 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] One or more modules are stored in memory 302, and when executed by processor 301, they perform actions such as... Figure 4 Resource monitoring method in the illustrated embodiment.

[0068] While exemplary embodiments and their advantages have been described in detail, those skilled in the art can make various changes, substitutions, and modifications to these embodiments without departing from the spirit of the invention and the scope of protection defined by the appended claims. Such modifications and variations all fall within the scope defined by the appended claims. For other examples, those skilled in the art should readily understand that the order of process steps can be changed while remaining within the scope of the invention.

[0069] Furthermore, the scope of this invention is not limited to the processes, mechanisms, manufacturing methods, material compositions, means, methods, and steps of the specific embodiments described in the specification. From the disclosure of this invention, those skilled in the art will readily understand that any existing or future processes, mechanisms, manufacturing methods, material compositions, means, methods, or steps that perform substantially the same function or obtain substantially the same results as the corresponding embodiments described in this invention can be applied according to this invention. Therefore, the appended claims are intended to include these processes, mechanisms, manufacturing methods, material compositions, means, methods, or steps within their scope of protection.

Claims

1. A resource monitoring method, characterized in that, Applied to a resource monitoring system, the resource monitoring system includes: a monitoring and alarm server and at least one resource pool; The resource pool includes at least one monitored object and a resource monitoring device; The monitoring and alarm server is connected to one end of the resource monitoring device and collects indicator data from the resource monitoring device; The other end of the resource monitoring device is connected to the monitored object, and collects indicator data from the monitored object; The resource monitoring method includes: Collecting indicator data from monitored objects; the process of collecting indicator data from monitored objects includes: obtaining a list of monitored objects; obtaining indicator text of the monitored objects based on the list of monitored objects; and preprocessing the indicator text of the monitored objects to generate indicator items. The list of monitored objects includes: Scan all monitored objects within the resource pool; Based on the scan results, obtain a list of monitored objects; Repeatedly scan all monitored objects in the resource pool at preset time intervals; Update the list of monitored objects based on the results of repeated scans; According to the command sent by the monitoring and alarm server, send indicator data to the monitoring and alarm server; the step of sending indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server includes: Receive a pull command from the monitoring and alarm server; the pull command from the monitoring and alarm server includes information about the object to be monitored; Based on the information of the object to be monitored, traverse the indicator items and find the indicator items that correspond to the information of the object to be monitored. Based on the indicator items corresponding to the information of the object to be monitored, obtain the required indicator data; Send the required indicator data to the monitoring and alarm server.

2. The resource monitoring method according to claim 1, characterized in that, The step of obtaining the indicator text of the monitored objects based on the list of monitored objects includes: According to the list of monitored objects, send an indicator retrieval command to each of the monitored objects; Receive the indicator text returned by the monitored object.

3. The resource monitoring method according to claim 1, characterized in that, The step of preprocessing the indicator text of the monitored object to generate indicator items includes: Based on the indicator text, parse and generate indicator data; Obtain information about the monitored object; Based on the indicator data and the information of the monitored object, indicator items are generated.

4. A resource monitoring device, characterized in that, Applied to a resource monitoring system, the resource monitoring system includes: a monitoring and alarm server and at least one resource pool; The resource pool includes at least one monitored object and a resource monitoring device; The monitoring and alarm server is connected to one end of the resource monitoring device and collects indicator data from the resource monitoring device; The other end of the resource monitoring device is connected to the monitored object, and collects indicator data from the monitored object; The resource monitoring device includes: The data acquisition module is used to collect indicator data from monitored objects; the collection of indicator data from monitored objects includes: obtaining a list of monitored objects; obtaining the indicator text of the monitored objects according to the list of monitored objects; and preprocessing the indicator text of the monitored objects to generate indicator items. The list of monitored objects includes: Scan all monitored objects within the resource pool; Based on the scan results, obtain a list of monitored objects; Repeatedly scan all monitored objects in the resource pool at preset time intervals; Update the list of monitored objects based on the results of repeated scans; The data sending module is used to send indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server; the step of sending indicator data to the monitoring and alarm server according to the command sent by the monitoring and alarm server includes: Receive a pull command from the monitoring and alarm server; the pull command from the monitoring and alarm server includes information about the object to be monitored; Based on the information of the object to be monitored, traverse the indicator items and find the indicator items that correspond to the information of the object to be monitored. Based on the indicator items corresponding to the information of the object to be monitored, obtain the required indicator data; Send the required indicator data to the monitoring and alarm server.

5. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the steps of the resource monitoring method as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the resource monitoring method as described in any one of claims 1-3.

Images