Resource monitoring method, device, electronic device and storage medium
By minimizing and dividing the acquisition tasks of the cloud management platform, creating task processes and adjusting the execution time of the acquisition group, the performance loss problem caused by timing task scheduling in large-scale cloud platform systems is solved, and efficient resource monitoring is achieved.
Patent Information
- Application Number
- CN202011077019.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-10
- Publication Date
- 2025-02-14
- Estimated Expiration
- 2040-10-10
AI Technical Summary
In large-scale cloud platform systems, when the monitoring resources are orders of magnitude, the scheduling of timing tasks will lead to performance losses and affect the efficiency of acquisition.
By minimizing the acquisition tasks of the cloud management platform, dividing them into multiple acquisition elements, and dividing them into acquisition groups, creating corresponding task processes for execution, and adjusting them according to the execution time to make each acquisition group's Execution time equalization.
On the premise of meeting the timeliness of the monitoring system acquisition, the performance overhead of the acquisition is reduced, the number of system tasks is reduced, and the stability of the monitoring system is improved.
Smart Images

Figure CN112269643B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and more specifically, to a resource monitoring method, device, electronic device, and computer-readable storage medium. Background Art
[0002] For cloud platform projects, various underlying virtualizations can be connected to manage various cloud resources. Cloud platforms generally have their own alarm systems. Based on the monitoring data of the underlying virtualized resources connected, cloud resource data can be monitored, and resource alarms and information can be pushed. For large-scale cloud platform systems, when the number of monitored resources is relatively large, some parallel computing load balancing methods are often required to alleviate the pressure on the monitoring system. In related technologies, scheduled tasks can be set for each resource to collect data, but the more system resources there are, the more scheduled tasks there are. When the number of tasks reaches a certain level, the scheduling of tasks is also a considerable performance loss. When the scale of resources is too large, the scheduling of tasks often affects the performance of collection.
[0003] Therefore, how to reduce the performance overhead of acquisition while meeting the acquisition timeliness of the monitoring system is a technical problem that technicians in this field need to solve.
[0004] Application Contents
[0005] The purpose of the present application is to provide a resource monitoring method, device, electronic device and computer-readable storage medium, which reduce the performance overhead of collection while meeting the collection timeliness of the monitoring system.
[0006] To achieve the above objectives, the present application provides a resource monitoring method, comprising:
[0007] Minimize the collection tasks of the cloud management platform to obtain multiple collection elements, and divide the collection elements into collection groups according to a preset division method to obtain multiple collection groups;
[0008] Creating a corresponding task process for each of the acquisition groups, and using the task process to execute all acquisition elements in the corresponding acquisition group;
[0009] The collection elements in the collection groups are adjusted according to the execution time of each collection group and the execution time of each collection element in each collection group to balance the execution time of each collection group, and the step of creating a corresponding task process for each collection group is re-entered.
[0010] The step of dividing the acquisition elements into acquisition groups according to a preset division method to obtain a plurality of acquisition groups includes:
[0011] The collection elements used to collect monitoring items of the same physical host are divided into a collection group to obtain multiple collection groups.
[0012] The startup frequency of each task process is consistent with the monitoring frequency of the cloud management platform.
[0013] The step of adjusting the acquisition elements in the acquisition group according to the execution time of each acquisition group and the execution time of each acquisition element in each acquisition group includes:
[0014] Determine the timeliness baseline;
[0015] If there is a first collection group whose execution time is greater than the aging baseline, split the first collection group so that the execution time of the collection groups after the splitting is less than the aging baseline;
[0016] If there is a second acquisition group whose execution time is less than a first preset proportion of the timeliness baseline, a plurality of the second acquisition groups are merged to obtain a target acquisition group; wherein the execution time of the target acquisition group is greater than the first preset proportion of the timeliness baseline and less than the timeliness baseline.
[0017] Wherein, the determination of the timeliness baseline includes:
[0018] A monitoring frequency of the cloud management platform is determined, and the timeliness baseline is determined as a second preset ratio of the monitoring frequency.
[0019] The splitting operation is performed on the first collection group so that the execution time of the collection groups after the splitting is less than the timeliness baseline, including:
[0020] Sorting all the acquisition elements of the first acquisition group from large to small according to the execution time to obtain a first sorting result, and taking the first acquisition element in the first sorting result as the target acquisition element;
[0021] Move the target acquisition element to a newly created acquisition group; wherein the execution time of the newly created acquisition group is greater than a first preset ratio of the aging baseline and less than the aging baseline;
[0022] Determine whether the current execution time of the first acquisition group is greater than the aging baseline;
[0023] If so, the next acquisition element of the target acquisition element in the first sorting result is used as the target acquisition element again, and the step of moving the target acquisition element to the newly created acquisition group is entered again.
[0024] The step of merging the plurality of second acquisition groups to obtain a target acquisition group includes:
[0025] Sort all the second acquisition groups according to their execution time from small to large to obtain a second sorting result, and use the first acquisition group in the second sorting result as the first target acquisition group and the second acquisition group as the second target acquisition group;
[0026] If the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline, merging the first target acquisition group and the second target acquisition group into a combined acquisition group;
[0027] Determining whether the execution time of the combined acquisition group is less than a first preset ratio of the timeliness baseline;
[0028] If so, the merged acquisition group is re-used as the first target acquisition group, the next acquisition group of the second target acquisition group in the second sorting result is re-used as the second acquisition group, and the step of merging the first target acquisition group and the second target acquisition group into a merged acquisition group is re-entered if the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline.
[0029] To achieve the above objectives, the present application provides a resource monitoring device, comprising:
[0030] A division module, used for performing minimization processing on the collection tasks of the cloud management platform to obtain a plurality of collection elements, and dividing the collection elements into collection groups according to a preset division method to obtain a plurality of collection groups;
[0031] An execution module, used for creating a corresponding task process for each of the acquisition groups, and using the task process to execute all acquisition elements in the corresponding acquisition group;
[0032] The adjustment module is used to adjust the collection elements in the collection group according to the execution time of each collection group and the execution time of each collection element in each collection group, so as to balance the execution time of each collection group and restart the workflow of the execution module.
[0033] To achieve the above objectives, the present application provides an electronic device, including:
[0034] Memory for storing computer programs;
[0035] A processor is used to implement the steps of the resource monitoring method as described above when executing the computer program.
[0036] To achieve the above objectives, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the resource monitoring method described above are implemented.
[0037] From the above scheme, it can be seen that a resource monitoring method provided by the present application includes: minimizing the collection tasks of the cloud management platform to obtain multiple collection elements, and dividing the collection elements into collection groups according to a preset division method to obtain multiple collection groups; creating a corresponding task process for each of the collection groups, and using the task process to execute all the collection elements in the corresponding collection group; adjusting the collection elements in the collection group according to the execution time of each of the collection groups and the execution time of each of the collection elements in each of the collection groups to balance the execution time of each collection group, and re-entering the step of creating a corresponding task process for each of the collection groups.
[0038] The resource monitoring method provided by the present application proposes a concept of a collection element and a collection group. The collection element is the smallest unit of monitoring data collection that can be executed, and the collection group is a collection of collection elements. A collection group contains at least one collection element, and a collection group corresponds to a scheduled task. When each scheduled task executes the collection, the collection program of each collection element in the collection group will be run one by one. Secondly, after each collection is completed, the collection group will be adjusted to minimize the number of collection groups within the allowed monitoring collection frequency set by the system, and maximize the number of collection elements in the collection group. This can reduce the number of scheduled tasks and thus reduce the system's overhead in task scheduling. At the same time, the execution time balance within each collection group is guaranteed as much as possible, so that the next collection can be executed as much as possible within a relatively reasonable range of task number and task execution volume. Under the premise of meeting the collection timeliness of the monitoring system, the system tasks are reduced, the performance overhead of the collection is reduced, and the stability of the monitoring system is improved. The present application also discloses a resource monitoring device, an electronic device, and a computer-readable storage medium, which can also achieve the above-mentioned technical effects.
[0039] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific implementation methods, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the drawings:
[0041] Figure 1is a flow chart of a resource monitoring method according to an exemplary embodiment;
[0042] Figure 2 is a flow chart showing a method for splitting a first acquisition group according to an exemplary embodiment;
[0043] Figure 3 is a flow chart of a method for merging a second acquisition group according to an exemplary embodiment;
[0044] Figure 4 is a structural diagram of a resource monitoring device according to an exemplary embodiment;
[0045] Figure 5 The figure is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION
[0046] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0047] The embodiment of the present application discloses a resource monitoring method, which reduces the performance overhead of collection while meeting the collection timeliness of the monitoring system.
[0048] See also Figure 1 , according to an exemplary embodiment, a flowchart of a resource monitoring method is shown, such as Figure 1 As shown, including:
[0049] S101: performing minimization processing on the collection task of the cloud management platform to obtain a plurality of collection elements, and dividing the plurality of collection elements into collection groups according to a preset division method to obtain a plurality of collection groups;
[0050] The purpose of this step is to minimize the collection tasks in the cloud management platform in order to form collection elements and divide the collection groups. The collection element is the smallest unit of monitoring data collection that can be executed. The collection group is a collection of collection elements. A collection group contains at least one collection element. It can be understood that the definition of a collection element is not just the collection of a resource, but the collection of a certain attribute of a resource. For example, the monitoring collection of a physical host can be divided into CPU collection, memory collection, network collection and other collection elements. The collection of a host's CPU is used as a collection element, and the collection of a host's memory is used as another collection element. Dividing the collection tasks into collection element granularity can minimize the difference in collection time between different collection tasks and facilitate the redivision of collection groups.
[0051] As a feasible implementation method, according to the actual situation of the cloud platform, the monitoring resources connected to the cloud platform are generally physical hosts and virtual machines corresponding to the underlying virtualization. They are grouped by physical hosts by default, and one collection group is added to one host. One collection group contains the collection elements of the host itself and the collection elements of all virtual machines on the host. In other words, each physical host in the cloud management platform corresponds to a collection group, and the collection of a physical host itself and the collection cloud of the virtual machines on the physical host are added to this collection group. That is, the plurality of collection elements are divided into a collection group according to a preset division method to obtain a plurality of collection groups, including: the collection elements used to collect monitoring items of the same physical host are divided into a collection group to obtain a plurality of collection groups. Of course, users can also customize the division method of the collection groups, for example, according to the type of collection task, collection item, etc., which is not specifically limited here.
[0052] S102: creating a corresponding task process for each of the acquisition groups, and using the task process to execute all acquisition elements in the corresponding acquisition group;
[0053] In this step, a scheduled task is added to each collection group, and one collection group corresponds to one system task, which reduces the number of system tasks. The monitoring frequency requirement is set in the monitoring system, and the monitoring system collects according to the scheduled tasks set by the collection group, that is, the startup frequency of each task process is consistent with the monitoring frequency of the cloud management platform.
[0054] In the specific implementation, after a collection cycle is completed, check whether all collection groups have been executed. If completed, proceed to the next step; if there are collection group tasks that are not completed, continue to wait for the next collection cycle to be completed until all collection group tasks are completed, to prevent the existence of unfinished collection elements when the collection group is divided twice, resulting in abnormal collection data.
[0055] S103: adjusting the collection elements in the collection groups according to the execution time of each collection group and the execution time of each collection element in each collection group to balance the execution time of each collection group, and re-entering the step of creating a corresponding task process for each collection group.
[0056] In the specific implementation, after each collection is completed, the collection group is adaptively divided according to the time baseline, mainly including the increase and decrease of the collection group and the re-division of the collection elements within the group. When the execution time of a collection group is longer than the system's time baseline, the collection group will be split to meet the collection baseline requirements. At the same time, the collection groups whose last collection was below the time performance threshold are merged to reduce the number of collection groups, reduce the scheduling overhead of scheduled tasks, reduce system pressure, and maintain system stability.
[0057] As a feasible implementation method, this step includes: determining an aging baseline; if there is a first acquisition group whose execution time is greater than the aging baseline, splitting the first acquisition group so that the execution time of the split acquisition groups is less than the aging baseline; if there is a second acquisition group whose execution time is less than a first preset proportion of the aging baseline, merging multiple second acquisition groups to obtain a target acquisition group; wherein the execution time of the target acquisition group is greater than the first preset proportion of the aging baseline and less than the aging baseline.
[0058] In a specific implementation, if the execution time of a collection group is longer than the aging baseline, the collection group needs to be split, and the collection elements of the collection group need to be divided. In addition, the collection groups whose execution time is less than the first preset proportion of the aging baseline are merged. The first preset proportion here can be 50%, and of course the user can also set it to other values, which are not specifically limited here. If the collection time is between half of the aging baseline and the aging baseline, it can be considered reasonable to assign this collection group without adjustment.
[0059] The timeliness baseline refers to the maximum duration allowed for the execution of a collection group, i.e., a thread task. Preferably, the determination of the timeliness baseline includes: determining the monitoring frequency of the cloud management platform, and determining the timeliness baseline as a second preset ratio of the monitoring frequency. In a specific implementation, the timeliness baseline is related to the monitoring frequency of the cloud management platform, and is the second preset ratio of the monitoring frequency. The second preset ratio is not specifically limited here, for example, it can be 80%, that is, if the monitoring frequency of the cloud management platform is set to 1 minute, it is allowed to return the underlying collection results at the latest once every 48 seconds.
[0060] The resource monitoring method provided in the embodiment of the present application proposes a concept of a collection element and a collection group. The collection element is the smallest unit of monitoring data collection that can be executed, and the collection group is a collection of collection elements. A collection group contains at least one collection element, and a collection group corresponds to a scheduled task. When each scheduled task executes the collection, the collection program of each collection element in the collection group will be run one by one. Secondly, after each collection is completed, the collection group will be adjusted to minimize the number of collection groups within the allowed monitoring collection frequency set by the system, and maximize the number of collection elements in the collection group. This can reduce the number of scheduled tasks and thus reduce the system's overhead in task scheduling. At the same time, the execution time balance within each collection group is guaranteed as much as possible, so that the next collection can be executed as much as possible within a relatively reasonable range of task number and task execution amount. On the premise of meeting the collection timeliness of the monitoring system, the system tasks are reduced, the performance overhead of the collection is reduced, and the stability of the monitoring system is improved.
[0061] The following describes a method for splitting the first collection group provided by the present application, specifically:
[0062] See also Figure 2 , according to an exemplary embodiment, a flowchart of a method for splitting a first acquisition group is shown, such as Figure 2 As shown, including:
[0063] S201: sorting all the collection elements of the first collection group from large to small according to the execution time to obtain a first sorting result, and taking the first collection element in the first sorting result as the target collection element;
[0064] S202: moving the target acquisition element to a newly created acquisition group; wherein the execution time of the newly created acquisition group is greater than a first preset ratio of the aging baseline and less than the aging baseline;
[0065] S203: Determine whether the current execution time of the first acquisition group is greater than the timeliness baseline; if so, proceed to S204;
[0066] S204: The next collection element of the target collection element in the first sorting result is used as the target collection element again, and the process goes back to S202.
[0067] The purpose of this embodiment is to split the first collection group whose execution time is greater than the aging baseline. In the specific implementation, the collection elements in the first collection group are sorted from large to small according to the execution time, and the collection element with the longest execution time is taken. If the execution time of this collection element is greater than the aging baseline, it is prompted that the aging baseline is too small, the collection element is already the smallest unit, and the execution time cannot be reduced any further. In this case, the aging baseline can be raised again, or it can be chosen to be not processed by default, or this division is skipped. After adding this collection element to the newly created collection group, check whether the execution time of the remaining collection elements in the original collection group meets the requirements. If it meets the requirements, it ends. If not, continue to divide the unsatisfied collection elements into other collection groups. If all other collection groups are not satisfied after being moved in, a new collection group is created and moved in until all collection groups meet the requirements. In addition, if the execution time of the newly created collection group is less than the first preset ratio of the aging baseline, it can be merged.
[0068] For example, collection group A contains 5 collection elements a, b, c, d, and e, and the execution time is 1, 2, 3, 4, and 5 respectively. The execution time of collection group A is 15, and the timeliness baseline is 6. The execution time of collection group A is greater than 6, so it needs to be split. First, sort all collection elements from large to small according to the execution time to get edcba. Secondly, take out collection element e and put it into the newly created collection group B. At this time, the execution time of collection group A is 10, and the execution time is still greater than 6. Continue to take out collection element d and put it into collection group B. At this time, collection group B includes collection elements e and collection element d. The execution time is 9, which is greater than 6, so collection group B does not accept collection element d. Continue to put collection element d into the newly created collection group C. At this time, collection group A includes collection elements a, b, and c, and the execution time is 6. The conditions are met and there is no need to split further. At this time, collection group B includes collection element e, and the execution time is 5, which is less than 6. Collection group C includes collection element d, and the execution time is 4, which is less than 6. Both conditions are met.
[0069] The following describes a method for merging a second collection group provided by the present application, specifically:
[0070] See also Figure 3 , according to an exemplary embodiment, a flowchart of a method for merging a second acquisition group is shown, such as Figure 3 As shown, including:
[0071] S301: sorting all the second acquisition groups from small to large according to the execution time to obtain a second sorting result, and taking the first acquisition group in the second sorting result as the first target acquisition group and the second acquisition group as the second target acquisition group;
[0072] S302: If the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline, merge the first target acquisition group and the second target acquisition group into a combined acquisition group;
[0073] S303: Determine whether the execution time of the combined collection group is less than a first preset ratio of the timeliness baseline; if so, proceed to S304;
[0074] S304: The merged acquisition group is used as the first target acquisition group again, the next acquisition group of the second target acquisition group in the second sorting result is used as the second acquisition group again, and S302 is repeated again.
[0075] In this embodiment, the second collection groups whose execution time is less than the first preset ratio of the time baseline are merged. If they are still less than the first preset ratio of the time baseline after merging, the merging will continue to reduce system tasks. For example, the first preset ratio is 50%, and the execution times of collection groups D, E, F, and G are 1, 1, 2, and 4 respectively. The baseline time is 8, and the first preset ratio of the time baseline is 4. Then, collection group D and collection group E are first merged into collection group H, and the execution time is 2, which is less than 4. Then, collection group H and collection group F are merged into a collection group with an execution time of 4. The execution time of collection group G is 4, and the execution time after merging is 8, which is not between 4 and 8, so it will not be merged. After the merger is completed, it becomes collection group H and collection group G, and the execution time of both is 4.
[0076] A resource monitoring device provided in an embodiment of the present application is introduced below. The resource monitoring device described below and the resource monitoring method described above can be referenced to each other.
[0077] See also Figure 4 , a structural diagram of a resource monitoring device according to an exemplary embodiment, such as Figure 4 As shown, including:
[0078] A division module 401 is used to perform minimization processing on the collection task of the cloud management platform to obtain multiple collection elements, and divide the collection elements into collection groups according to a preset division method to obtain multiple collection groups;
[0079] An execution module 402 is used to create a corresponding task process for each of the acquisition groups, and use the task process to execute all acquisition elements in the corresponding acquisition group;
[0080] The adjustment module 403 is used to adjust the collection elements in the collection group according to the execution time of each collection group and the execution time of each collection element in each collection group to balance the execution time of each collection group and restart the workflow of the execution module 402.
[0081] The resource monitoring device provided in the embodiment of the present application proposes a concept of a collection element and a collection group. The collection element is the smallest unit of monitoring data collection that can be executed, and the collection group is a collection of collection elements. A collection group contains at least one collection element, and a collection group corresponds to a scheduled task. When each scheduled task executes the collection, the collection program of each collection element in the collection group will be run one by one. Secondly, after each collection is completed, the collection group will be adjusted to minimize the number of collection groups within the allowed monitoring collection frequency set by the system, and maximize the number of collection elements in the collection group. This can reduce the number of scheduled tasks and thus reduce the system's overhead in task scheduling. At the same time, the execution time balance within each collection group is guaranteed as much as possible, so that the next collection can be executed as much as possible within a relatively reasonable range of task number and task execution amount. On the premise of meeting the collection timeliness of the monitoring system, the system tasks are reduced, the performance overhead of the collection is reduced, and the stability of the monitoring system is improved.
[0082] Based on the above embodiment, as a preferred implementation, the division module 401 includes:
[0083] A processing unit, used for minimizing the collection tasks of the cloud management platform to obtain multiple collection elements;
[0084] The division unit is used to divide the collection elements used for collecting monitoring items of the same physical host into a collection group to obtain multiple collection groups.
[0085] Based on the above embodiment, as a preferred implementation mode, the startup frequency of each task process is consistent with the monitoring frequency of the cloud management platform.
[0086] Based on the above embodiment, as a preferred implementation, the adjustment module 403 includes:
[0087] A determination unit is used to determine the aging baseline;
[0088] a splitting unit, configured to, if there is a first collection group whose execution time is greater than the aging baseline, split the first collection group so that the execution time of the collection groups after the splitting is less than the aging baseline;
[0089] A merging unit is used to merge multiple second acquisition groups to obtain a target acquisition group if there is a second acquisition group whose execution time is less than a first preset ratio of the aging baseline; wherein the execution time of the target acquisition group is greater than the first preset ratio of the aging baseline and less than the aging baseline.
[0090] Based on the above embodiment, as a preferred implementation mode, the determining unit is specifically a unit that determines the monitoring frequency of the cloud management platform and determines the timeliness baseline as a second preset ratio of the monitoring frequency.
[0091] Based on the above embodiment, as a preferred implementation, the splitting unit includes:
[0092] A first sorting subunit is used to sort all the collection elements of the first collection group from large to small according to the execution time to obtain a first sorting result, and use the first collection element in the first sorting result as the target collection element;
[0093] A moving subunit, used to move the target acquisition element to a newly created acquisition group; wherein the execution time of the newly created acquisition group is greater than a first preset ratio of the aging baseline and less than the aging baseline;
[0094] A first determination subunit is used to determine whether the current execution time of the first acquisition group is greater than the timeliness baseline; if so, start the workflow of the first determination subunit;
[0095] The first determining subunit is used to take the next collecting element of the target collecting element in the first sorting result as the target collecting element again, and restart the working process of the moving subunit.
[0096] Based on the above embodiment, as a preferred implementation, the merging unit includes:
[0097] A second sorting subunit is used to sort all the second collection groups from small to large according to the execution time to obtain a second sorting result, and use the first collection group in the second sorting result as the first target collection group and the second collection group as the second target collection group;
[0098] a merging subunit, configured to merge the first target acquisition group and the second target acquisition group into a merged acquisition group if the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline;
[0099] A first determining subunit is configured to determine whether the execution time of the combined acquisition group is less than a first preset ratio of the timeliness baseline; if so, start the workflow of the second determining subunit;
[0100] The second determining subunit is used to use the merged acquisition group as the first target acquisition group again, use the next acquisition group of the second target acquisition group in the second sorting result as the second acquisition group again, and restart the workflow of the merging subunit.
[0101] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0102] Based on the hardware implementation of the above program module, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, Figure 5 FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment. Figure 5 As shown, the electronic equipment includes:
[0103] Communication interface 1, capable of exchanging information with other devices such as network devices;
[0104] The processor 2 is connected to the communication interface 1 to implement information exchange with other devices and is used to execute the application access method provided by one or more of the above technical solutions when running a computer program. The computer program is stored in the memory 3.
[0105] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 4 .
[0106] The memory 3 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.
[0107] It can be understood that the memory 3 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 2 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.
[0108] The method disclosed in the above embodiment of the present application can be applied to the processor 2, or implemented by the processor 2. The processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 2 or the instruction in the form of software. The above processor 2 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 3, and the processor 2 reads the program in the memory 3 and completes the steps of the above method in combination with its hardware.
[0109] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present application are implemented, which will not be repeated here for the sake of brevity.
[0110] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 3 storing a computer program, and the computer program can be executed by a processor 2 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.
[0111] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.
[0112] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0113] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A resource monitoring method, characterized in that: include: Minimize the collection tasks of the cloud management platform to obtain multiple collection elements, and divide the collection elements into collection groups according to a preset division method to obtain multiple collection groups; Creating a corresponding task process for each of the acquisition groups, and using the task process to execute all acquisition elements in the corresponding acquisition group; Adjusting the collection elements in the collection groups according to the execution time of each collection group and the execution time of each collection element in each collection group to balance the execution time of each collection group, and re-entering the step of creating a corresponding task process for each collection group; The step of adjusting the acquisition elements in the acquisition group according to the execution time of each acquisition group and the execution time of each acquisition element in each acquisition group includes: Determine the timeliness baseline; If there is a first collection group whose execution time is greater than the aging baseline, split the first collection group so that the execution time of the collection groups after the splitting is less than the aging baseline; If there is a second acquisition group whose execution time is less than a first preset proportion of the timeliness baseline, a plurality of the second acquisition groups are merged to obtain a target acquisition group; wherein the execution time of the target acquisition group is greater than the first preset proportion of the timeliness baseline and less than the timeliness baseline.
2. The resource monitoring method according to claim 1, characterized in that: The step of dividing the acquisition elements into acquisition groups according to a preset division method to obtain a plurality of acquisition groups includes: The collection elements used to collect monitoring items of the same physical host are divided into a collection group to obtain multiple collection groups.
3. The resource monitoring method according to claim 1, characterized in that: The startup frequency of each task process is consistent with the monitoring frequency of the cloud management platform.
4. The resource monitoring method according to claim 1, characterized in that: Determining the timeliness baseline includes: A monitoring frequency of the cloud management platform is determined, and the timeliness baseline is determined as a second preset ratio of the monitoring frequency.
5. The resource monitoring method according to claim 1, characterized in that: The first collection group is split so that the execution time of the collection groups after the split is less than the timeliness baseline, including: Sorting all the acquisition elements of the first acquisition group from large to small according to the execution time to obtain a first sorting result, and taking the first acquisition element in the first sorting result as the target acquisition element; Move the target acquisition element to a newly created acquisition group; wherein the execution time of the newly created acquisition group is greater than a first preset ratio of the aging baseline and less than the aging baseline; Determine whether the current execution time of the first acquisition group is greater than the aging baseline; If so, the next acquisition element of the target acquisition element in the first sorting result is used as the target acquisition element again, and the step of moving the target acquisition element to the newly created acquisition group is entered again.
6. The resource monitoring method according to claim 1, characterized in that: The merging operation on the plurality of the second acquisition groups to obtain a target acquisition group includes: Sort all the second acquisition groups according to their execution time from small to large to obtain a second sorting result, and use the first acquisition group in the second sorting result as the first target acquisition group and the second acquisition group as the second target acquisition group; If the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline, merging the first target acquisition group and the second target acquisition group into a combined acquisition group; Determining whether the execution time of the combined acquisition group is less than a first preset ratio of the timeliness baseline; If so, the merged acquisition group is re-used as the first target acquisition group, the next acquisition group of the second target acquisition group in the second sorting result is re-used as the second acquisition group, and the step of merging the first target acquisition group and the second target acquisition group into a merged acquisition group is re-entered if the sum of the execution time of the first target acquisition group and the execution time of the second target acquisition group is less than the timeliness baseline.
7. A resource monitoring device, characterized in that: include: A division module, used for performing minimization processing on the collection tasks of the cloud management platform to obtain a plurality of collection elements, and dividing the collection elements into collection groups according to a preset division method to obtain a plurality of collection groups; An execution module, used for creating a corresponding task process for each of the acquisition groups, and using the task process to execute all acquisition elements in the corresponding acquisition group; An adjustment module, used for adjusting the collection elements in the collection groups according to the execution time of each collection group and the execution time of each collection element in each collection group, so as to balance the execution time of each collection group, and restart the workflow of the execution module; Wherein, the adjustment module includes: A determination unit is used to determine the aging baseline; a splitting unit, configured to, if there is a first collection group whose execution time is greater than the aging baseline, split the first collection group so that the execution time of the collection groups after the splitting is less than the aging baseline; A merging unit is used to merge multiple second acquisition groups to obtain a target acquisition group if there is a second acquisition group whose execution time is less than a first preset ratio of the aging baseline; wherein the execution time of the target acquisition group is greater than the first preset ratio of the aging baseline and less than the aging baseline.
8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the resource monitoring method as claimed in any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the resource monitoring method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Monitoring method, device and equipment and medium
CN109597736A
Resource monitoring method and device based on YARN, and terminal equipment
CN111290917A