Monitoring data cache flashing method and device and medium
By merging and cacheing monitoring data in the data storage service module of the ganglia monitoring framework, and swiping the timing database file under preset duration or forced cycle, the problem of excessive disk pressure in the ganglia framework under large-scale monitoring is solved, and the More efficient monitoring performance.
Patent Information
- Application Number
- CN202411375215.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2024-12-03
AI Technical Summary
In the large-scale monitoring scenario, the ganglia monitoring framework has high IOPS indicators of timing database files, resulting in excessive disk pressure, affecting system performance, and making it difficult to support larger-scale monitoring.
The monitoring data cache flushing method is adopted. By establishing a tree structure in the data storage service module, merging monitoring data and flushing to the timing database file under preset time or forced cycles, the number of times it is written directly to the disk and the IOPS indicator is reduced.
It effectively reduces the IOPS indicators of the ganglia framework, slows down the pressure on disk, supports larger-scale monitoring data, and improves monitoring performance and effectiveness.
Smart Images

Figure CN119065612A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a monitoring data cache flushing method, device and medium. Background Art
[0002] As a monitoring framework with a lightweight architecture and wide application, Ganglia is widely used in the monitoring industry. With its low CPU usage, low memory usage, and easy expansion, Ganglia is more popular in resource monitoring in the private cloud industry.
[0003] Ganglia is designed as one RRD file (a time series database file format) for one monitoring item. Therefore, when the number of monitoring items increases, the number of RRDs will also increase synchronously. Ganglia writes a sampling point at a fixed time, which means that when the sampling points of all monitoring items need to be updated every once in a while, a new sampling data needs to be written. The corresponding number of read and write operations per second (Input / Output Operations Per Second, IOPS) will reach the same scale, which will put greater pressure on the disk. The IOPS of the currently common mechanical hard disk (Hard Disk Drive, HDD) is only around 200. After adding a disk array (Redundant Arrays of Independent Disks, RAID) card, the IOPS will increase to 1000 or more, which is not enough to support large-scale monitoring sampling writing, which will cause the disk utilization (util) to soar for a long time, affecting the use of the entire system.
[0004] Therefore, technical personnel in this field now urgently need a monitoring data cache flushing method to reduce the overall IOPS index of the ganglia framework so that ganglia can support larger-scale monitoring. Summary of the invention
[0005] The purpose of the present invention is to provide a monitoring data cache flushing method, device and medium to reduce the overall IOPS index of the ganglia framework so that ganglia can support larger-scale monitoring.
[0006] In order to solve the above technical problems, the present invention provides a monitoring data cache flushing method, which is applied to a data storage service module of a monitoring framework, comprising:
[0007] Pull monitoring data from each sampling point in the cluster;
[0008] Merging the monitoring data according to the monitoring items to obtain a monitoring data array, wherein the monitoring data array is stored in a cache module;
[0009] According to the monitoring items, the monitoring data array in the cache module is flushed to the corresponding time series database file.
[0010] In a possible embodiment, the monitoring data is merged according to the monitoring items to obtain a monitoring data array, and the monitoring data array is stored in the cache module including:
[0011] Whenever the cache module receives a write request for the monitoring data, the monitoring data is written into a tree structure pre-established in the cache module;
[0012] Among them, the node key value of any node in the tree structure is the name of the monitoring item, and the node data is the monitoring item data structure corresponding to the monitoring item; the monitoring item data structure includes the monitoring data array and the data update time; the monitoring data array is used to write the monitoring data corresponding to the monitoring item; the data update time is the sampling time of the most recent monitoring data written.
[0013] In a possible embodiment, the monitoring item data structure further includes: the last refresh time;
[0014] After writing the monitoring data into the tree structure pre-established in the cache module, the method further includes:
[0015] Update the last refresh time according to the current system time;
[0016] Then, according to the monitoring item, flushing the monitoring data array in the cache module to the corresponding time series database file includes:
[0017] When the cache module receives a write request for the monitoring data, it determines the cached time length of the monitoring data according to the last refresh time corresponding to the monitoring data and the current system time;
[0018] If the cached time exceeds the preset maximum cache time, the monitoring data structure corresponding to the monitoring data is added to the end of the refresh queue as a refresh task.
[0019] In a possible embodiment, updating the last refresh time according to the current system time includes:
[0020] Generate a random number within a preset random range; wherein the lower limit of the random range is greater than or equal to 0, and the upper limit is less than the maximum cache duration;
[0021] The last refresh time is updated according to the difference between the current system time and the random number.
[0022] In a possible embodiment, according to the monitoring item, flushing the monitoring data array in the cache module to the corresponding time series database file further includes:
[0023] Traversing all nodes in the tree structure according to a preset forced refresh cycle; wherein the forced refresh cycle is greater than or equal to the maximum cache duration;
[0024] If the monitoring data structure corresponding to the cached time length exceeding the maximum cached time length exists in the tree structure, then this part of the monitoring data structure is determined as an abnormal data structure;
[0025] Adding all the abnormal data structures as refresh tasks to the end of the refresh queue;
[0026] After any of the abnormal data structures is flushed, the abnormal data structure is removed from the tree structure.
[0027] In a possible embodiment, pulling monitoring data of each sampling point in the cluster includes:
[0028] Pulling the sampling data corresponding to each of the sampling points at a time interval consistent with the storage interval of the time series database file;
[0029] After the sampled data is pulled, the validity and message format of the sampled data are verified;
[0030] The sampled data that has passed the verification is converted into the time series database file structure format, and the sampling time is filled in the time series database file structure to obtain the monitoring data; wherein, the sampling time is rounded according to the time series database file storage interval, and all the sampling times are rounded forward or rounded backward.
[0031] In a possible embodiment, when the monitoring data needs to be displayed on an interface, the method further includes:
[0032] Reading the monitoring data in the time series database file as a first intermediate result, and reading the monitoring data in the cache module as a second intermediate result;
[0033] Merging the first intermediate result and the second intermediate result, and removing the duplicate monitoring data therein, and removing the item with an earlier sampling time among the different monitoring data corresponding to the same sampling point and the same monitoring item, so as to obtain a final result;
[0034] The final result is returned.
[0035] In order to solve the above technical problems, the present invention also provides a monitoring data cache flushing device, which is applied to a data storage service module of a monitoring framework, comprising:
[0036] The pull-up module is used to pull the monitoring data of each sampling point in the cluster;
[0037] A cache module, used for merging the monitoring data according to monitoring items to obtain a monitoring data array, and storing the monitoring data array in the cache module;
[0038] The flush module is used to flush the monitoring data array in the cache module to the corresponding time series database file according to the monitoring item.
[0039] In order to solve the above technical problems, the present invention also provides a monitoring data cache flushing device, comprising:
[0040] Memory for storing computer programs;
[0041] A processor is used to implement the steps of the monitoring data cache flushing method as described above when executing the computer program.
[0042] In order to solve the above technical problem, the present invention also provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the monitoring data cache flushing method as described above are implemented.
[0043] The present invention provides a monitoring data cache flushing method, which transforms the data storage service module (gmetad) of ganglia so that after the monitoring data is pulled and sampled by the data acquisition service module (gmond) of ganglia, it is not directly written into the time series database (rrd) file; instead, it is first written into the cache module of the cluster, and the monitoring data is merged based on the monitoring items in the cache; then, the grouped monitoring data is flushed to the corresponding rrd file in units of monitoring items. It can be seen that this method realizes the group flushing of monitoring data in units of monitoring items under the ganglia framework.
[0044] According to input / output (IO) analysis, ganglia only writes 8 bytes of data for each sampling. However, for systems such as Linux (an operating system kernel), the page size is usually much larger than 8 bytes. That is, a single rrd file can write monitoring data of multiple sampling points at a time to be counted as one IO, which can reduce the pressure on the disk, that is, the IOPS indicator can be reduced.
[0045] In summary, a single rrd file is a database file designed to store a monitoring item in ganglia. This method uses cache modules such as memory as a transit, and merges the monitoring data according to the monitoring items in the cache module, so that the monitoring data of multiple sampling points of the same monitoring item pulled over a period of time are merged and flushed to the corresponding rrd file, realizing the merging and flushing of monitoring data, which greatly reduces the pressure on the disk. Therefore, under the same hardware conditions, the ganglia framework implemented based on this method can support larger-scale data monitoring, bringing better monitoring performance and monitoring effects.
[0046] The monitoring data cache flushing device and non-volatile storage medium provided by the present invention correspond to the above method and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0048] FIG1 is a flow chart of a monitoring data cache flushing method provided by an embodiment of the present invention;
[0049] FIG2 is a schematic diagram of an application framework of a monitoring data cache flushing method provided by an embodiment of the present invention;
[0050] FIG3 is a structural diagram of a monitoring data cache flushing device provided by an embodiment of the present invention;
[0051] FIG. 4 is a structural diagram of another monitoring data cache flushing device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] The core of the present invention is to provide a monitoring data cache flushing method, device and medium.
[0054] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0055] Ganglia is a monitoring framework with a lightweight architecture and wide application. It supports monitoring of thousands of nodes and hundreds of thousands of resources, and has great scalability in scale. Ganglia uses time series database (rrd) files as persistent storage. When the scale is large, the disadvantages of rrd will be reflected. Specifically, ganglia is designed so that one monitoring item corresponds to one rrd file. Therefore, when the number of monitoring items increases, the number of rrds will also increase synchronously. In addition, ganglia writes a sampling point at a fixed time. This will result in that when the sampling points of all monitoring items are updated at regular intervals, each time a new sampling data (monitoring data) needs to be written, the corresponding number of read and write operations per second (Input / Output Operations Per Second, IOPS) will reach the same scale, which will put too much pressure on the disk. The IOPS of the common mechanical hard disk drive (HDD) is only about 200. After adding a disk array (Redundant Arrays of Independent Disks, RAID) card, the IOPS will increase to 1000 or more, which is not enough to support large-scale monitoring sampling and writing. It will cause the disk utilization (util) to soar for a long time, affecting the use of the entire system, making the hard disk performance a bottleneck restricting the expansion of ganglia monitoring scale.
[0056] Based on this, the present invention provides a monitoring data cache flushing method, which is applied to the data storage service module of the monitoring framework. As shown in FIG1 , the method includes:
[0057] S10: Pull the monitoring data of each sampling point in the cluster.
[0058] S20: merging the monitoring data according to the monitoring items to obtain a monitoring data array, and storing the monitoring data array in a cache module.
[0059] S30: According to the monitoring items, the monitoring data array in the cache module is flushed to the corresponding time series database file.
[0060] First, the data storage service module to which this method is applied is explained. In the ganglia framework, it mainly includes three core parts: data collection service (gmond), data storage service (gmetad) and web interface (gweb). Among them, the data storage service module is also the functional module that implements the data storage service gmetad. Specifically, gmond is a daemon process that runs on each node that needs to be monitored, and is used to collect monitoring statistics, send and receive statistical information on the same multicast or unicast channel. Gmetad is also a daemon process. Gmetad regularly checks gmonds, pulls sampled monitoring data from gmonds, and stores the monitoring data in the rrd file. Gweb is a visualization tool in ganglia, which is a front-end that uses a browser to display the data stored by gmetad. In this method, the gmetad is mainly modified to solve the above-mentioned problems when the monitoring data is flushed to the rrd file.
[0061] As shown in FIG2 , in gmetad, step S10 of the present method first pulls the monitoring data of each sampling point in the cluster. This step, i.e., gmetad pulls the monitoring data sampled by gmond, is no different from the current use of ganglia, except that after the monitoring data is pulled, it is not directly written to the rrd file. Instead, as in step S20, the monitoring data is first merged according to the monitoring items to obtain a monitoring data array, and the monitoring data array is stored in the cache module. Afterwards, as in step S30, the monitoring data of multiple sampling points sampled by gmond within a period of time are merged and flushed to the corresponding rrd file in units of monitoring items and the cache module is used as a transfer, completing a merged flush of the monitoring data. Repeating the above process can realize the merged flushing of the monitoring data under the ganglia framework, so that each time the IO is called to flush and write the monitoring data to the rrd file, the monitoring data of multiple sampling points are written at one time, which can effectively solve the high IO pressure problem existing in the ganglia framework when it is large-scale. For example, assuming that the number of sampling points corresponding to a monitoring item is between 100 and 500, the monitoring data implemented by this method can be merged and flushed to the rrd file, which can expand the monitoring scale to 100 to 500 times the original under the same hardware conditions. Under the same monitoring scale, the disk IOPS, CPU usage, disk busyness and other indicators can be greatly reduced, making the cluster run more stable and secure.
[0062] As can be seen from the above, in this method, the process of directly writing the monitoring data obtained by sampling from gmond after pulling from gmetad is actually split into two data transmission processes. One is that the monitoring data obtained by sampling from gmond is first flushed to the cache module, and the monitoring data is finally stored in the cache module in the form of a monitoring data array merged according to the monitoring items. The second is to flush the monitoring data array in the cache module to the corresponding rrd file. In actual implementation, these two data transmission processes can be implemented by two processes respectively, as shown in Figure 2, and these two processes can be named data processing module and cache flushing module respectively. Combined with the cache module as the monitoring data transfer in the two data transmission processes, gmetad has three functional modules as shown in Figure 2 after the transformation of this method: data processing module, cache module and cache flushing module. That is, step S11 in this method mainly corresponds to the data processing module in Figure 2, step S20 mainly corresponds to the cache module in Figure 2, step S30 mainly corresponds to the cache flushing module in Figure 2, and Figure 2 is a possible functional module implementation schematic diagram of this method.
[0063] On the other hand, it should be noted that the monitoring data merging in this embodiment can occur before writing to the cache module, or it can occur in the cache module. That is, one possible implementation scheme is to use a process to merge the monitoring data after pulling the monitoring data returned by gmonds, and then store the merged monitoring data array in the cache module. Another possible implementation scheme is to use a thread to write the monitoring data into the cache module after pulling the monitoring data returned by gmonds, and complete the merging of the monitoring data in the cache module. This embodiment does not limit this, and it can be freely selected according to different actual needs.
[0064] However, this embodiment provides a possible implementation scheme for merging monitoring data in a cache module, and the above step S20 specifically includes:
[0065] S21: Whenever the cache module receives a request to write monitoring data, the monitoring data is written into a tree structure pre-established in the cache module.
[0066] Among them, the node key value of any node in the tree structure is the name of the monitoring item, and the node data is the monitoring item data structure of the corresponding monitoring item; the monitoring item data structure includes a monitoring data array and a data update time; the monitoring data array is used to write the monitoring data corresponding to the monitoring item; the data update time is the sampling time of the latest monitoring data written.
[0067] It should be noted that, based on the tree structure provided by the present embodiment, whenever there is monitoring data that needs to be written to the cache module, it is possible to query whether there is a corresponding monitoring item data structure in the monitoring item query tree structure corresponding to the monitoring data. If there is, the monitoring data is directly written into the monitoring data array in the monitoring item data structure, and stored as an element in the array. If not, the creation of the corresponding node is performed with the above-mentioned tree structure node rule, in order to prepare for the subsequent monitoring data write corresponding to the same monitoring item.
[0068] In addition, regarding the cache module in the above method, the high-performance storage media already in the cluster can be used, that is, the storage media usually used as cache in the cluster, such as memory, etc. As a daemon process deployed in the cluster, gmetad does not have any technical obstacles to calling the host's memory resources, so in this method, at least the memory resources in the cluster can be directly used as the cache module.
[0069] Based on the writing method provided by this embodiment, the monitoring data written into the cache module is naturally merged and stored according to the monitoring item classification, and the merged monitoring data is the monitoring data array in the monitoring item data structure. That is, this embodiment completes the merging of monitoring data according to monitoring items and the writing of monitoring data to the cache module in one step, which greatly improves efficiency and reduces the burden on the cluster.
[0070] It should also be noted that the data update time mentioned in this embodiment is used to characterize the most recent update time of the monitoring data currently stored in the cache module. Since the cache module is a newly added module in this method that serves as a transfer for monitoring data, in actual applications, what is of concern is not when the cache module last updated the monitoring data, but when the monitoring data was last sampled by gmonds (related to the timeliness of the monitoring data, etc.). Therefore, in this method, the sampling time of the monitoring data is used as the basis for the data update time, rather than the current system time when the monitoring data is written into the cache module.
[0071] In addition, from the description of the ganglia framework in the above embodiment, it can be seen that gmonds samples the sampling points to obtain data, and the rrd file stores not only the sampling data itself, but also various indicators of the sampling data. One of the very important indicators is the sampling time of the sampling data, which characterizes the timeliness of the sampling data. Therefore, in the above and subsequent descriptions of this embodiment, the monitoring data actually refers to a data structure, which includes the sampling data and the sampling time. The sampling time is generally spliced with the sampling data in the form of a timestamp.
[0072] Based on this, regarding how to pull monitoring data in the above step S10, in other words, it can also be the specific functional implementation of the data processing module in Figure 2. This embodiment provides a possible implementation scheme, and the above step S10 specifically includes:
[0073] S11: Pull the sampling data corresponding to each sampling point at a time interval consistent with the storage interval of the time series database file.
[0074] S12: After the sampled data is pulled, the validity and message format of the sampled data are verified.
[0075] S13: Convert the verified sampling data into a time series database file structure format, and fill the sampling time in the time series database file structure to obtain monitoring data.
[0076] The sampling time is rounded according to the storage interval of the time series database file, and all sampling times are rounded forward or backward.
[0077] For step S11, this embodiment pulls the sampling data of each sampling point, that is, the data actually sampled by gmonds deployed on each node. In step S11, the sampling data is pulled at a time interval consistent with the storage interval of the rrd file. The purpose of such setting is to balance the input and output loads of the cache module, thereby simplifying the complexity of the data processing process involved in the cache module, which is conducive to more efficient flushing of monitoring data to the rrd file.
[0078] Specifically, since the rrd interface will perform fitting calculations on the written data, when the cache module flushes the monitoring data to the rrd file, it is necessary to ensure that the flushing time fully meets the definition of the rrd storage time point. That is, the output side of the cache module flushes the monitoring data at the rrd file storage interval. This step implements the pulling of sampled data at the rrd file storage interval on the input side of the cache module, which can keep the input and output side rates of the cache module consistent and achieve load balancing.
[0079] It should also be noted that the rrd file storage interval mentioned in the above step S11 is one of the two inherent parameters of the rrd file. When the rrd file is first created, the rrd storage interval and the number of storage points need to be determined. These two parameters determine the size of the rrd file.
[0080] Step S12 is a data preprocessing process. In this embodiment, the validity of the sampled data and the message format are specifically verified. The validity check of the sampled data can specifically be to verify the data boundary, data format, etc. of the sampled data. If the pre-configured requirements are not met, it can be considered that the validity check has not passed, and this part of the data can be discarded. In addition, the message format check is not the same concept as the verification of the sampled data format in the above-mentioned sampled data validity check, but is a verification of the format of the pulled message to determine whether the pulled message is valid, and those that do not meet the message standard requirements can be discarded.
[0081] Step S13 is a specific sampling time filling process. After the sampling data is pulled and verified through the above steps, the corresponding sampling time can be filled in this part of the sampling data to obtain the final monitoring data. Specifically, the sampling time can be spliced at the head or tail of the sampling data as in the above example to obtain the final monitoring data. In particular, one of the major application scenarios targeted by the present invention is the application of ganglia in large-scale data monitoring scenarios. At this time, due to the large number of sampling data pulled at the same time and the limited processing capacity of the process, it is difficult to ensure that all sampling data completes the filling of the corresponding sampling time at the moment of pulling. Therefore, it is not advisable to directly use the current system time at the processing moment as the sampling data to fill the sampling time. This embodiment rounds the sampling time based on the current system time through the rrd file storage interval.
[0082] However, it is particularly important to note that, since it is necessary to distinguish the sampled data pulled at different times, the rounding must be performed in the same direction when the sampling time is rounded. For example, for a batch of sampled data pulled at 11:21:10, there is a sampled data whose processing time is 11:21:13. Assuming that it is rounded forward, the rounded sampling time is 11:21:10, then the sampling time of a batch of sampled data pulled at 11:21:10 (i.e., 11:21:10~11:21:20, including the sampling data pulled in the time range of the front end point but not including the back end point) is 11:21:10. The same is true for rounding backward, which will not be repeated in this embodiment. However, if different rounding directions are used for the same batch of sampling data, for example, the common "rounding" method, 11:21:13 is rounded to 11:21:10, and 11:21:18 is rounded to 11:21:20, the same batch of sampling data will be divided into two different sampling data pulling batches of 11:21:10 and 11:21:20 when filling the sampling time, which will cause confusion in subsequent data monitoring, so it is not advisable.
[0083] On the other hand, in the above embodiment, a possible implementation scheme has been provided for the specific implementation of step S20, and the monitoring data is stored through a tree structure pre-created in the cache module, so that the monitoring data stored in the cache module is naturally merged based on the monitoring items. And when writing to the rrd file later, it is also convenient to write to the rrd file according to the monitoring item merger, which is conducive to improving the efficiency of the monitoring data refresh.
[0084] Regarding how to cache monitoring data and complete the merging in the above step S20, in other words, it can also be the specific functional implementation of the cache module in Figure 2. This embodiment provides a possible implementation scheme, and the above monitoring item data structure also includes: the last refresh time.
[0085] After step S21: writing the monitoring data into the tree structure pre-established in the cache module, the method further includes:
[0086] S22: Update the last refresh time according to the current system time.
[0087] Then, step S30: according to the monitoring items, flushing the monitoring data array in the cache module to the corresponding time series database file specifically includes:
[0088] S31: When the cache module receives a write request for monitoring data, it determines the cached time of the monitoring data according to the last refresh time corresponding to the monitoring data and the current system time.
[0089] S32: If the cached time exceeds the preset maximum cached time, the monitoring data structure corresponding to the monitoring data is added to the end of the refresh queue as a refresh task.
[0090] In this embodiment, a condition is proposed for flushing monitoring data from the cache module to the rrd file, that is, the cached time exceeds the preset maximum cache time. It is easy to know that this condition can achieve the limitation of the maximum number of monitoring data cached in the cache module. When the monitoring scale remains unchanged, the monitoring data generated within a period of time always fluctuates within a certain range. Therefore, by stipulating the maximum cache time allowed for monitoring data in the cache module, the maximum number of monitoring data that can exist in the cache module at the same time can be indirectly limited. This adapts to the characteristics of limited cluster memory resources and will not cause excessive adverse effects on the normal operation of other functions of the cluster.
[0091] In addition, it can be known from this embodiment that the monitoring item data structure mentioned in the above embodiment can not only include the monitoring data array, data update time, and the last refresh time in this embodiment, but also can be expanded according to other needs. Exemplarily, the monitoring item data structure can also include: monitoring item file directory, monitoring data point quantity and other information, which is not limited in this embodiment.
[0092] It should also be noted that the scheme provided in this embodiment for merging and flushing monitoring data in a cache module to an rrd file is triggered by data update in the cache, so the triggering and execution of this flushing process mainly occurs in the cache module. In the above-mentioned possible embodiment, it is also explained that the merging and flushing of monitoring data in the cache module to an rrd file is mainly implemented by the cache flushing module in Figure 2. In view of this, this embodiment provides a corresponding solution, that is, steps S31 and S32 in this embodiment are mainly to submit the corresponding flushing task to the flushing queue after the cache module determines that the monitoring data needs to be merged and flushed to the rrd file, and the flushing queue is managed by the cache flushing module, that is, it meets the requirements of the above-mentioned embodiment. In addition, the implementation scheme provided in this embodiment for managing the merging and flushing of monitoring data to an rrd file using the flushing queue is also conducive to the independent management of the flushing of monitoring data by multiple subjects without confusion, and provides a prerequisite for the implementation of subsequent possible embodiments.
[0093] Exemplarily, this embodiment also provides a possible implementation scheme for implementing the merging and flushing of monitoring data into the rdd file by a cache flushing module. That is, another monitoring data merging and flushing scheme different from the above-mentioned embodiment. It is also a forced flushing scheme for the abnormal situation that the monitoring data corresponding to the monitoring items in the cache module has never been updated. The above-mentioned step S30 also includes:
[0094] S33: traverse all nodes in the tree structure according to a preset forced refresh cycle.
[0095] The forced refresh period is greater than or equal to the maximum cache duration.
[0096] S34: If there exists a corresponding monitoring data structure in the tree structure whose cached duration exceeds the maximum cached duration, this part of the monitoring data structure is determined as an abnormal data structure.
[0097] S35: Add all abnormal data structures as refresh tasks to the end of the refresh queue.
[0098] S36: After any abnormal data structure is refreshed, the abnormal data structure is removed from the tree structure.
[0099] It can be seen from the monitoring data flushing scheme provided by the above embodiment that the execution of steps S31 and S32 is triggered by the cache module receiving a write request for monitoring data, that is, only when any monitoring item has monitoring data refreshed, the monitoring data array corresponding to the monitoring item will execute the judgment of step S32, and it is possible to merge and flush. However, if there is an abnormal scenario where there is no monitoring data update for a certain monitoring item for a long time, the monitoring data flushing scheme provided by the above embodiment is no longer applicable, causing this part of the monitoring data to always remain in the cache module. On the one hand, the long-term retention causes this part of the monitoring data to lose its timeliness. On the other hand, the retention of monitoring data will also cause the space of the cache module to be unnecessary occupied, wasting memory resources.
[0100] Based on this, the present embodiment provides a forced flushing solution to solve this problem in a targeted manner. Specifically, the present embodiment can periodically traverse all monitoring data structures in the cache module tree structure through the process of the cache flushing module, and perform the same flushing condition judgment as in step S32 of the above embodiment. If all monitoring data structures do not meet the flushing conditions, it means that the data flushing solution of the above embodiment can currently still normally complete the purpose of merging monitoring data to flush to the rrd file, and the above-mentioned abnormal scenario does not occur. If the monitoring data structure corresponding to a monitoring item meets the flushing conditions, it means that the monitoring item has not had any monitoring data update for a long time, and the above-mentioned abnormal scenario has occurred, and the solution of the above embodiment is no longer applicable. At this time, the present solution can detect it through traversal and add it to the flushing queue to avoid the monitoring data from being stranded in the cache module for a long time.
[0101] Based on the forced down-sweep solution provided in the above embodiment, this embodiment further provides a possible implementation scheme. After step S36, the above method further includes:
[0102] S37: Return a monitoring data cache abnormality report to the web page.
[0103] The monitoring data cache abnormality report includes: the abnormal monitoring item name and the abnormal data structure. The above-mentioned web page is gweb.
[0104] This embodiment returns a monitoring data cache exception report containing the abnormal monitoring item name and abnormal data structure to the visual front-end of the ganglia framework, i.e., the web page gweb, to inform relevant personnel that some data in the cache has not been updated for a long time, so that relevant personnel can be informed of this abnormal situation in time and can take corresponding solutions, which is conducive to maintaining the stability of ganglia's data monitoring.
[0105] Furthermore, based on the above embodiment, this embodiment further provides a peak shaving and valley filling solution. Specifically, the above step S22: updating the last refresh time according to the current system time specifically includes:
[0106] S221: Generate a random number within a preset random range.
[0107] The lower limit of the random range is greater than or equal to 0, and the upper limit is less than the maximum cache duration.
[0108] S222: Update the last refresh time according to the difference between the current system time and the random number.
[0109] Based on the settings of this embodiment, based on the current system time now and the random number random (jitter), where (jitter) is a random range, it can be ensured that the last flush time of each monitoring data in the cache module is randomly distributed within (now – random (jitter)). It can be seen from the data flushing scheme provided by the above embodiment that the condition for flushing the monitoring data is that the cached duration of the monitoring data corresponding to the monitoring item exceeds the pre-configured maximum cache duration write_interval. Therefore, based on the generation scheme of the last flush time of this embodiment, it is ensured that the cache time of the monitoring data structure corresponding to each monitoring item in the cache module is as different as possible, and tends to be evenly distributed within the range of [write_interval–jitter, write_interval]. Therefore, when the monitoring data structure of each monitoring item is triggered to be merged and flushed to the rrd file, the monitoring data corresponding to different monitoring items is flushed to the rrd file in a relatively even manner within the range of [write_interval–jitter, write_interval], achieving the effect of peak shaving and valley filling, which can further reduce the pressure of the monitoring data flushing on the cluster hard disk and other hardware.
[0110] On the other hand, from the above description of the ganglia framework, it can be known that the monitoring data stored in gmetad can be displayed in the browser through gweb. In the original ganglia framework, since all monitoring data are stored in the rrd file, the monitoring data can be read from the rrd file when the interface is displayed. However, for this method, since this method transforms gmetad, some monitoring data is temporarily stored in the cache module, that is, the monitoring data in gmetad actually exists in two locations, the cache module and the rrd file. This also makes it impossible to simply read data from the rrd file for interface display. In view of this, the present embodiment provides an interface display scheme that is suitable for the monitoring data cache refresh method provided by the present invention. When the monitoring data needs to be displayed on the interface, the method also includes:
[0111] S41: reading the monitoring data in the time series database file as the first intermediate result, and reading the monitoring data in the cache module as the second intermediate result.
[0112] S42: Merge the first intermediate result and the second intermediate result, and remove duplicate monitoring data and an item with an earlier sampling time among different monitoring data corresponding to the same sampling point and the same monitoring item, to obtain a final result.
[0113] S43: Return the final result.
[0114] Under the implementation of the monitoring data cache flushing method provided in the above embodiment, the monitoring data in gmetad includes two parts stored in the cache module and the rrd file. Therefore, when displaying the interface, it is necessary to obtain the monitoring data stored in these two parts for display together.
[0115] Further, based on the implementation schemes of several possible cache modules provided in the above embodiments for merging and flushing monitoring data to rrd files, it can be known that when any monitoring item has monitoring data updated, the cache module will trigger the judgment of whether to flush the monitoring data array corresponding to the monitoring item to the rrd file, and when there is no monitoring data update in the mandatory period, the mandatory flushing mechanism will be triggered. Based on this, for the monitoring data corresponding to the monitoring items shared by the cache module and the rrd file, the cache module is always newer than the rrd file in terms of the "version" of the monitoring data. This is determined based on the cache module monitoring data writing and flushing scheme provided in the above embodiments. Whenever monitoring data is generated, the parallel steps S21 and S31, S32 will cause the cache module to flush the previously stored monitoring data (provided that the flushing conditions are met) to the rrd file, and the newly updated monitoring data will wait for subsequent merging. In addition, since the monitoring data in the cache module is merged into a monitoring data array according to the monitoring items, and the monitoring data array is used as the flushing object when flushing, the flushing is the update of the monitoring data. Therefore, when the monitoring data array triggered by an update is flushed to the rrd file, only part of the monitoring data in the monitoring data array corresponding to a monitoring item may be updated, while other part of the monitoring data is not updated. This part of the monitoring data is the duplicate data in the cache module and the rrd file.
[0116] In summary, after reading the monitoring data (i.e., the first intermediate result and the second intermediate result) in the cache module and the rrd file, this embodiment integrates the two read results and removes the duplicated parts. And for different monitoring data of the same monitoring item and the same sampling point, the cache module is used as the basis, so as to obtain the final monitoring data result, which can be used as the basis for interface display.
[0117] In the above embodiment, a monitoring data cache flushing method is described in detail, and the present invention also provides a corresponding embodiment of a monitoring data cache flushing device. It should be noted that the present invention describes the embodiment of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.
[0118] From the perspective of functional modules, as shown in FIG3 , this embodiment provides a monitoring data cache flushing device, including:
[0119] The pull-up module 11 is used to pull the monitoring data of each sampling point in the cluster;
[0120] A cache module 12, used for merging the monitoring data according to monitoring items to obtain a monitoring data array, and storing the monitoring data array in the cache module;
[0121] The flush module 13 is used to flush the monitoring data array in the cache module to the corresponding time series database file according to the monitoring item.
[0122] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0123] FIG4 is a structural diagram of a monitoring data cache flushing device provided by another embodiment of the present invention. As shown in FIG4 , a monitoring data cache flushing device includes: a memory 20 for storing computer programs;
[0124] The processor 21 is used to implement the steps of a monitoring data cache flushing method in the above embodiment when executing a computer program.
[0125] The monitoring data cache flushing device provided in this embodiment may include but is not limited to a mobile terminal, a personal computer, a workstation, etc.
[0126] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0127] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of a monitoring data cache flushing method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include, but is not limited to, a monitoring data cache flushing method, etc.
[0128] In some embodiments, a monitoring data cache flushing device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .
[0129] Those skilled in the art will appreciate that the structure shown in FIG. 4 does not constitute a limitation on a monitoring data cache flushing device, and may include more or fewer components than those shown in the figure.
[0130] A monitoring data cache flushing device provided by an embodiment of the present invention includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a monitoring data cache flushing method.
[0131] In addition to the embodiments of the monitoring data cache flushing method and device provided in the above embodiments, the present invention also provides an embodiment corresponding to a computer program product. A computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the monitoring data cache flushing method described in any of the above embodiments can be implemented.
[0132] Since the embodiments of the computer program product part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the computer program product part, which will not be repeated here.
[0133] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.
[0134] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes.
[0135] The above is a detailed introduction to a monitoring data cache flushing method, device and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.
[0136] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A monitoring data cache flushing method, characterized in that: The data storage service module applied to the monitoring framework includes: Pull monitoring data from each sampling point in the cluster; Merging the monitoring data according to the monitoring items to obtain a monitoring data array, wherein the monitoring data array is stored in a cache module; According to the monitoring items, the monitoring data array in the cache module is flushed to the corresponding time series database file.
2. The monitoring data cache flushing method according to claim 1, characterized in that: The monitoring data is merged according to the monitoring items to obtain a monitoring data array, and the monitoring data array is stored in the cache module, including: Whenever the cache module receives a write request for the monitoring data, the monitoring data is written into a tree structure pre-established in the cache module; Among them, the node key value of any node in the tree structure is the name of the monitoring item, and the node data is the monitoring item data structure corresponding to the monitoring item; the monitoring item data structure includes the monitoring data array and the data update time; the monitoring data array is used to write the monitoring data corresponding to the monitoring item; the data update time is the sampling time of the most recent monitoring data written.
3. The monitoring data cache flushing method according to claim 2, characterized in that: The monitoring item data structure also includes: the last refresh time; After writing the monitoring data into the tree structure pre-established in the cache module, the method further includes: Update the last refresh time according to the current system time; Then, according to the monitoring item, flushing the monitoring data array in the cache module to the corresponding time series database file includes: When the cache module receives a write request for the monitoring data, the cache module determines the cached time of the monitoring data according to the last refresh time corresponding to the monitoring data and the current system time; If the cached time exceeds the preset maximum cache time, the monitoring data structure corresponding to the monitoring data is added to the end of the refresh queue as a refresh task.
4. The monitoring data cache flushing method according to claim 3, characterized in that: The updating of the last refresh time according to the current system time comprises: Generate a random number within a preset random range; wherein the lower limit of the random range is greater than or equal to 0, and the upper limit is less than the maximum cache duration; The last refresh time is updated according to the difference between the current system time and the random number.
5. The monitoring data cache flushing method according to claim 3, characterized in that: According to the monitoring item, flushing the monitoring data array in the cache module to the corresponding time series database file also includes: Traversing all nodes in the tree structure according to a preset forced refresh cycle; wherein the forced refresh cycle is greater than or equal to the maximum cache duration; If the monitoring data structure corresponding to the cached time length exceeding the maximum cached time length exists in the tree structure, then this part of the monitoring data structure is determined as an abnormal data structure; Adding all the abnormal data structures as refresh tasks to the end of the refresh queue; After any of the abnormal data structures is flushed, the abnormal data structure is removed from the tree structure.
6. The monitoring data cache flushing method according to claim 2, characterized in that: The monitoring data of each sampling point in the pulling cluster includes: Pulling the sampling data corresponding to each of the sampling points at a time interval consistent with the storage interval of the time series database file; After the sampled data is pulled, the validity and message format of the sampled data are verified; The sampled data that has passed the verification is converted into the time series database file structure format, and the sampling time is filled in the time series database file structure to obtain the monitoring data; wherein, the sampling time is rounded according to the time series database file storage interval, and all the sampling times are rounded forward or rounded backward.
7. The monitoring data cache flushing method according to any one of claims 1 to 6, characterized in that: When the monitoring data needs to be displayed on an interface, the method further includes: Reading the monitoring data in the time series database file as a first intermediate result, and reading the monitoring data in the cache module as a second intermediate result; Merging the first intermediate result and the second intermediate result, and removing the duplicate monitoring data therein, and removing the item with an earlier sampling time among the different monitoring data corresponding to the same sampling point and the same monitoring item, so as to obtain a final result; The final result is returned.
8. A monitoring data cache flushing device, characterized in that: The data storage service module applied to the monitoring framework includes: The pull-up module is used to pull the monitoring data of each sampling point in the cluster; A cache module, used for merging the monitoring data according to monitoring items to obtain a monitoring data array, and storing the monitoring data array in the cache module; The flush module is used to flush the monitoring data array in the cache module to the corresponding time series database file according to the monitoring item.
9. A monitoring data cache flushing device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the monitoring data cache flushing method as described in any one of claims 1 to 7 when executing the computer program.
10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the monitoring data cache flushing method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Data transmission method and device, medium and product
CN120295958A
A data transmission method, device, medium and product
CN120295958B
