A data monitoring method and device for a distributed object storage system and a medium

Abstract

The application discloses a distributed object storage system data monitoring method and device and a medium thereof, relates to the technical field of distributed object storage, and aims to provide an implementation mode for data monitoring, and provides a distributed object storage system data monitoring method for solving the problem that the current data monitoring mode has a great influence on performance. According to the distributed object storage system data monitoring method, the request information is first stored in a cache, part of data in the request information is counted according to a preset monitoring requirement, data monitoring information is obtained, and the data monitoring information is saved in a storage pool, so that the data monitoring information can be called when needed. Therefore, the amount of data written into the storage system or the amount of data read from the storage system when a user queries the data monitoring information is less than the amount of data stored in the storage system in the current mode, and the difference is more obvious when the request information contains more requests.

Description

Technical Field

[0001] This application relates to the field of distributed object storage technology, and in particular to a data monitoring method, apparatus and medium for a distributed object storage system. Background Technology

[0002] With the development of distributed object storage technology, it is being increasingly widely applied in numerous fields, leading to a surge in user workloads and a corresponding increase in the demand for data monitoring. Due to the diverse and complex application scenarios of distributed object storage systems, users need to monitor storage performance over a specific period to determine the health status of the distributed storage system. If a cluster failure occurs or customers report abnormal read / write operations, troubleshooting can be conducted by analyzing monitoring data. Furthermore, by examining the increase in business volume per unit time in statistical metrics, the estimated time when the cluster capacity will reach its limit can be assessed, allowing for proactive expansion or other targeted measures to ensure the normal and stable operation of the distributed object storage system.

[0003] Data monitoring is a crucial part of distributed object storage. It involves monitoring all requests received within a given period as request information, with the total data contained within these requests constituting the monitored object. However, users don't need to know all the data of the monitored object. Instead, they obtain different data monitoring information by statistically analyzing different characteristics of the monitored object. Each type of data monitoring information is a high-level summary of different characteristics of the request information. Through these different data monitoring information, users can understand various characteristics of the request information to meet their monitoring needs. This data monitoring information includes: requests per second (TPS), bandwidth (data volume per second), latency (average latency of requests within the time period), total number of requests within the time period, space increment, object count increment, Hypertext Transfer Protocol (HTTP) return codes, sent traffic, and received traffic, etc. Regarding the total number of requests within the time period, this refers to the total number of requests contained in the request information of interest; simply counting how many requests are included in the request information is sufficient to obtain the total number of requests.

[0004] Current data monitoring typically involves receiving and storing all data from each request in a storage pool. When needed, the request information is retrieved, and statistics are compiled based on the required data monitoring criteria. The results are then displayed as data monitoring information on a visual interface for easy viewing by relevant personnel. However, this approach becomes increasingly demanding as the number of requests processed by the distributed object storage system increases. Large-scale distributed object storage systems can handle tens of thousands of requests per second. Recording detailed information for every single request would severely impact cluster performance, thereby affecting the usability of the distributed object storage system.

[0005] Therefore, those skilled in the art urgently need a data monitoring method for distributed object storage systems to address the problem that current data monitoring methods have a significant impact on performance. Summary of the Invention

[0006] The purpose of this application is to provide a data monitoring method, device, and medium for a distributed object storage system, thereby addressing the problem that current data monitoring methods have a significant impact on performance.

[0007] To address the aforementioned technical problems, this application provides a data monitoring method for a distributed object storage system, comprising:

[0008] Receive request information sent by the client and store the request information in the cache;

[0009] Based on the preset monitoring requirements, some data in the request information is statistically analyzed to obtain data monitoring information;

[0010] Data monitoring information is sent to the storage pool and saved.

[0011] Preferably, saving the data monitoring information to the storage pool includes:

[0012] Data monitoring information is saved to the omap of a pre-defined rados object in the storage pool.

[0013] Preferably, saving the data monitoring information to the omap of a preset rados object in the storage pool includes:

[0014] After each preset time interval, the acquired monitoring data will be inserted into omap.

[0015] Preferably, sending data monitoring information to the storage pool includes:

[0016] When the number of requests stored in the cache exceeds a preset threshold, the data monitoring information in the cache is sent to the storage pool.

[0017] Preferably, sending data monitoring information to the storage pool includes:

[0018] Every second preset time interval, the data monitoring information in the cache will be sent to the storage pool.

[0019] Preferably, it further includes:

[0020] Receive monitoring query commands sent by the client;

[0021] Retrieve the corresponding data monitoring information from the storage pool according to the monitoring query command;

[0022] Return data monitoring information as query results.

[0023] Preferably, after retrieving the corresponding data monitoring information from the storage pool according to the monitoring query instruction, the method further includes:

[0024] Display the data monitoring information on the management interface.

[0025] To address the aforementioned technical problems, this application also provides a data monitoring device for a distributed object storage system, comprising:

[0026] The receiving module is used to receive request information sent by the client and store the request information in the cache;

[0027] The statistics module is used to perform statistics on a portion of the data in the request information according to preset monitoring requirements in order to obtain data monitoring information.

[0028] The storage module is used to send data monitoring information to the storage pool and save it.

[0029] Preferably, it further includes:

[0030] The query module is used to receive monitoring query commands sent by the client; retrieve the corresponding data monitoring information from the storage pool according to the monitoring query commands; and return the data monitoring information as the query result.

[0031] The display module is used to display data monitoring information on the management interface.

[0032] To address the aforementioned technical problems, this application also provides a data monitoring device for a distributed object storage system, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor is used to implement the steps of the distributed object storage system data monitoring method described above when executing computer programs.

[0035] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data monitoring method for a distributed object storage system as described above.

[0036] This application provides a data monitoring method for a distributed object storage system. The method first stores request information in a cache, then statistically analyzes the required data from the request based on preset monitoring needs to obtain data monitoring information. Finally, this data monitoring information is saved to a storage pool for later retrieval. This data monitoring method avoids storing all request data in the distributed object storage system. Instead, it uses a cache as a buffer to temporarily store the request information and advances the step of statistically analyzing a portion of the request information to obtain data monitoring information, storing the obtained data monitoring information in the storage pool. Therefore, the amount of data written to the storage system and the amount of data retrieved from the storage system when a user queries data monitoring information are both less than the current method of storing all request information in the storage system. This difference becomes more pronounced as the request information contains more requests, and the impact on storage system performance is smaller.

[0037] The distributed object storage system data monitoring device and computer-readable storage medium provided in this application correspond to the above-mentioned method and have the same effect. Attached Figure Description

[0038] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A flowchart of a data monitoring method for a distributed object storage system provided by the present invention;

[0040] Figure 2 A flowchart of another data monitoring method for a distributed object storage system provided by the present invention;

[0041] Figure 3 A structural diagram of a data monitoring device for a distributed object storage system provided by the present invention;

[0042] Figure 4 This is a structural diagram of another distributed object storage system data monitoring device provided by the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0044] The core of this application is to provide a data monitoring method, device, and medium for a distributed object storage system.

[0045] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Distributed storage refers to a storage method in which a group of computer nodes communicate and interact through a network, coordinating to provide storage services. Object storage, on the other hand, is an object-based storage method, a general term used to describe methods for solving and processing discrete units, called objects. Each object is assigned a unique identifier, allowing a server or end user to retrieve the object without needing to know its physical address. This approach is helpful for automating and simplifying data storage in cloud computing environments.

[0047] Therefore, distributed object storage is widely used in application scenarios with high business volume. However, high business volume can bring problems, such as whether the storage system can support such a large number of business requests, and the increased likelihood of storage system failures. Therefore, it is currently necessary to monitor business data (i.e., requests) to understand the pressure on the storage system and take targeted measures.

[0048] However, there is also a problem. As mentioned above, distributed object storage systems are used when the business volume is large. At this time, the number of customer requests is large and the frequency is high. Large object storage systems can handle tens of thousands of requests per second. If you want to monitor business data, it is obviously unrealistic to record all the data of all requests. Ordinary storage media cannot accommodate such a huge amount of data, while storing it in a distributed storage system will put a lot of pressure on it and affect its performance, which runs counter to the original intention of data monitoring.

[0049] In summary, such as Figure 1 As shown, this application provides a data monitoring method for a distributed object storage system, including:

[0050] S11: Receive the request information sent by the client and store the request information in the cache.

[0051] It is easy to understand that data monitoring is a long-term and continuous process. Data monitoring information actually reflects some characteristics of all data within a certain period of time. Data monitoring means dividing all requested information into different time periods according to user needs, performing the required statistics on the data in each time period, and returning the statistical results as data monitoring information to inform the user.

[0052] As explained above, the data monitoring information includes: requests per second (TPS), bandwidth (data volume per second), latency (average latency of requests within the time period), total number of requests within the time period, space increment, object count increment, Hypertext Transfer Protocol (HTTP) return codes, sent traffic, and received traffic, among other information. TPS is calculated by dividing the total number of requests within the time period by the total duration of that time period (in seconds); bandwidth is calculated by dividing the total amount of data from the requests within the time period by the total duration of that time period (in seconds); other data monitoring information is similar, statistically analyzing a portion of the request data to provide users with data monitoring information that characterizes the features of all request data without returning all request data.

[0053] S12: Based on the preset monitoring requirements, perform statistics on some data in the request information to obtain data monitoring information.

[0054] It is easy to see that data monitoring information includes many different types, but not every type of data monitoring information is what users need. In practical applications, different users have their own unique monitoring needs, which are specifically reflected in their need to know one or more types of data monitoring information.

[0055] Therefore, to avoid collecting statistics on all types of data monitoring information, which would waste unnecessary operating resources and affect the performance of the distributed object storage system, it is necessary to pre-configure according to user needs so that only the data monitoring information that users care about is obtained during data monitoring.

[0056] S13: Send the data monitoring information to the storage pool and save it.

[0057] As can be seen from the above, each data monitoring information reflects a certain feature of all request information and is a statistical analysis of a portion of the request information data. It is mostly in the form of a single data value, and its data volume is negligible compared to the total number of request information. Therefore, even if it is stored in a distributed storage system, it will not affect the performance of the distributed storage system.

[0058] It is easy to understand that storing data monitoring information in the storage pool of the storage system is a preferred implementation method, the purpose of which is to achieve the storage of data monitoring information by reusing the existing storage system. At the same time, since the data monitoring data is relatively small compared to the data that the storage system can store, even if it is stored in the storage system, it will not bring a large load and has little impact on the performance of the storage system.

[0059] Of course, in practical applications, data monitoring data can also be stored using external storage media. This embodiment does not impose any restrictions on this, but this implementation requires additional storage media, which will bring additional costs.

[0060] Furthermore, to further illustrate the data monitoring method for a distributed object storage system provided in this application, the following example uses the total number of requests within a given time period as the data monitoring information to provide a practical method for data statistics and its subsequent flushing and saving to the storage pool, including:

[0061] 1) Define a global variable ops_num for statistical purposes. The initial value of the variable ops_num is 0.

[0062] 2) Receive request information. For each request received, increment the variable ops_num by 1.

[0063] 3) When the preset rules are met, the variable ops_num is flushed to the storage pool.

[0064] 4) Check if the variable ops_num exists in the storage pool. If it does not exist, store the variable ops_num directly in the storage pool. If it exists, add the value of the variable ops_num in this time to the value of the variable ops_num in the storage pool before storing it.

[0065] By following the above steps, data monitoring information can be obtained. When needed, users can use the management software to call the interface provided by the distributed object storage system to read information from the storage pool and obtain the variable ops_num, thereby knowing the total number of requests in a certain period of time.

[0066] Similarly, for data monitoring information such as requests per second (TPS), the same statistical method can be used. However, when the user retrieves the variable ops_num, the value of the variable ops_num needs to be divided by the total duration (in seconds). This division operation can be performed before the data is flushed to the storage pool or after the user retrieves the variable ops_num. This embodiment does not impose any restrictions on this.

[0067] However, it's important to note that since TPS is the average number of requests received over a period of time, if the statistical period is too long, data in the cache may be lost over time. Therefore, this part of the statistics is not suitable to be completed in the cache all at once. It is preferable to perform the division operation after the user retrieves the variable ops_num from the storage pool, and the cache only counts the number of received requests. In this way, the cache can count the data within this time period in batches and flush it to the storage pool without ensuring that no data is lost.

[0068] This application provides a data monitoring method for a distributed object storage system. It uses a cache as a temporary storage location for request information and performs statistical analysis on a portion of the data in the request information based on preset monitoring requirements to obtain data monitoring information. This allows the data monitoring information to be directly stored in a storage pool, eliminating the need to store all request information in the pool. This significantly reduces the amount of data stored, thus greatly reducing the load on the storage system and minimizing its performance impact. Furthermore, this method of directly storing data monitoring information in the storage system allows users to directly retrieve the relevant data monitoring information from the storage pool during queries. Compared to the current method of storing all request information in the storage system and then statistically analyzing the request information upon receiving a user's query to obtain and return the data monitoring information, this method requires fewer steps from sending a query to receiving the data monitoring information. Consequently, the system's response time is shorter, allowing users to obtain data monitoring information faster, resulting in higher query efficiency and a better user experience.

[0069] To further illustrate the data monitoring method for a distributed object storage system provided in this application, the following explanation details the process of saving data monitoring information to a storage pool:

[0070] The data storage layer of a distributed storage system, also known as the Rados layer, is a crucial part that performs the core functions of the storage system. Correspondingly, the basic unit for storing data in the data storage layer is the Rados object, which differs from objects in object storage. A Rados object includes a map structure called omap, and omap has a map structure.<k,v> Structure. Where k represents a certain moment, usually the insertion time; v represents the statistical value within a certain time period (this time value is adjustable) starting from moment k.

[0071] Generally, there are three possible storage methods for storing data in a storage pool:

[0072] One is the data portion stored in the storage pool (i.e., the data part). However, to ensure security, the data portion usually undergoes data locking, so storing data monitoring information in this part is more complex.

[0073] The second part is the extended attribute part (i.e., the XATTR part) stored in the storage pool. However, the number of statistical information entries that can be stored in the extended attribute part is limited, and it cannot be stored when there are a large number of data monitoring information entries.

[0074] Thirdly, the data is stored in the omap of a specific Rados object in the storage pool. `k` records the current insertion time, and `v` represents the current statistical time period (unlike the time periods mentioned above, this time period is a preset time value; every time this preset time elapses, omap inserts a record).<k,v> Statistical values ​​(statistical information).

[0075] The third storage method stores a large amount of data and is itself a map structure, so there is no need for data locking. It is simple and quick to implement and is the optimal solution.

[0076] Furthermore, this embodiment also provides a preferred implementation, wherein the settable time value is 1 minute, that is:

[0077] After each preset time interval, the acquired monitoring data will be inserted into omap.

[0078] The first preset duration is 1 minute. The advantage of setting the first preset duration to 1 minute is that it is simpler for omap to set up compared to the smaller time unit of seconds. For larger time units, using 1 minute as the unit can receive the data monitoring information from the cache more promptly, avoiding data loss due to excessive data idling time.

[0079] This embodiment achieves simple storage of data monitoring information by storing it on the omap of a Rados object in the storage pool. It can also meet the storage needs of a large amount of data monitoring information, making it a better solution for storing data monitoring information in a distributed object storage system, thereby better meeting the monitoring needs of users.

[0080] As can be seen from the above embodiments, the process of flushing data monitoring information from the cache to the storage pool can be performed according to preset rules. Therefore, this embodiment provides a preferred solution with preset rules, and the above-mentioned sending of data monitoring information to the storage pool specifically involves:

[0081] When the number of requests stored in the cache exceeds a preset threshold, the data monitoring information in the cache is sent to the storage pool.

[0082] The specific value of the aforementioned preset threshold should be related to the size of the OMAP. Setting the preset threshold too high will overload the storage system and affect its performance, while setting it too low will waste storage resources. Based on the above reasons and in combination with the actual situation, a preferred preset threshold is 10,000 records. That is, when the number of requests stored in the cache exceeds 10,000, the data monitoring information in the cache is sent to the storage pool for storage.

[0083] In addition, this embodiment also provides another preferred method for preset rules, wherein the above-mentioned sending of data monitoring information to the storage pool is specifically as follows:

[0084] Every second preset time interval, the data monitoring information in the cache will be sent to the storage pool.

[0085] It is readily known that a cache is a device used to temporarily store data. Data loss is likely to occur if data is left idle for too long; therefore, the second preset duration should not be too long. Furthermore, if the second preset duration is too short, data interaction between the cache and the storage system will be too frequent, consuming significant operating resources and impacting storage system performance. Based on these reasons, this embodiment provides a preferred setting for the second preset duration: 30 seconds.

[0086] This embodiment provides two preferred schemes for preset rules, enabling those skilled in the art to select an appropriate method to implement the above-described distributed object storage system data monitoring method according to actual conditions. Furthermore, this embodiment also provides further optimized methods for each preferred scheme of preset rules, achieving technical effects such as reducing the impact on storage system performance and minimizing waste of operating resources, further ensuring that the distributed object storage system data monitoring method provided in this application is applicable to actual situations.

[0087] Furthermore, as can be clearly seen from the above embodiments, statistically analyzing request information in the cache to obtain data monitoring information before storing it in the storage pool can reduce the load on the storage system. When a client needs to query monitoring data, it can then retrieve it from the storage pool. Therefore, to further illustrate the data monitoring method for a distributed object storage system provided in this application, this embodiment also provides a specific implementation scheme for a client to query monitoring data, such as... Figure 2 As shown, the above method also includes:

[0088] S21: Receive monitoring query commands sent by the client.

[0089] The monitoring query command should include at least the following: the type of data monitoring information being queried (i.e., specifically TPS, bandwidth, latency, total number of requests in the time period, space increment, object increment, HTTP return code, one or more of the sent and received traffic), and the corresponding time period.

[0090] S22: Retrieve the corresponding data monitoring information from the storage pool according to the monitoring query command.

[0091] Based on the type of data monitoring information queried, find the corresponding data stored in omap. Since omap is a map...<k,v> The structure is such that k represents a certain moment, so all data monitoring information within the corresponding time period can be found based on the value of k.

[0092] S23: Return the data monitoring information as the query result.

[0093] The data monitoring information obtained in step S22 is returned as the result. When there are multiple data monitoring information obtained in step S22, they can also be calculated and integrated into one data monitoring information before being returned. This embodiment does not impose any restrictions on this.

[0094] In addition to the steps shown above, this embodiment also provides another preferred implementation scheme. After retrieving the corresponding data monitoring information from the storage pool according to the monitoring query instruction, the method further includes:

[0095] S24: Display the data monitoring information on the management interface.

[0096] It is important to note that Figure 2 The step S24 shown is only one possible implementation after step S23. In fact, there is no restriction on the order of steps S23 and S24.

[0097] The management interface may include display devices such as a screen or touch screen. Data monitoring information is also displayed on the aforementioned display devices. The data monitoring information may be displayed in the form of pictures, reports, text, etc. This embodiment does not limit this and can select a suitable implementation method according to actual needs.

[0098] This embodiment provides an implementation scheme for users to query data monitoring information based on the above embodiments. It is simple to implement and, since it only queries and calls data monitoring information, it is simpler and faster than the current method of calling all requested information data and then performing statistics to obtain data monitoring information. It also has a lower load on the storage system and reduces the impact on the performance of the distributed object storage system.

[0099] Finally, the above descriptions include various terms related to time periods, such as: a request message for a period of time, after a first preset duration, after a second preset duration, etc. Furthermore, the durations of different time periods are not necessarily equal.

[0100] To clearly distinguish the different time periods mentioned above and avoid confusion, the following examples will illustrate this:

[0101] Based on the preferred solution provided by the above embodiments, the first preset duration is preferably 1 minute and the second preset duration is preferably 30 seconds; and the preset rule is preferably to send the data monitoring information in the cache to the storage pool every second preset duration; in a possible application scenario, the user needs to monitor the request information from 11:00 to 12:00 (that is, the total duration of this time period is 1 hour).

[0102] At this time, the cache continuously receives all request information from 11:00 to 12:00. Starting from 11:00, every 30 seconds, the cache flushes the statistical data monitoring information to the storage pool. In other words, the cache statistically analyzes the request information and obtains data monitoring information in 30-second increments.

[0103] The storage pool inserts data monitoring information into the omap of a certain Rados object every minute. That is, inserting statistics into the omap once is to combine two consecutive data monitoring messages sent by the cache into one data monitoring message for storage (the quantitative relationship between the data monitoring information of the cache and the storage pool in this example is determined by the first preset duration and the second preset duration).

[0104] Starting at 11:00 AM, the buffer flushes statistical results (i.e., data monitoring information) to the storage pool every 30 seconds. Upon receiving this statistical information, the storage pool stores it every minute until 12:00 PM. After that, users can send query requests to retrieve the data monitoring information requested between 11:00 AM and 12:00 PM.

[0105] As can be seen from the above, omap is a map<k,v> The structure is defined by k, where k represents a certain moment. Therefore, based on the value of k, all data monitoring information within the time period from 11:00 to 12:00 can be found. These data monitoring information are then recalculated to be integrated into a single data monitoring information and returned to the management interface as the query result. This completes the data monitoring and query process.

[0106] It's easy to understand that, given the known type of data monitoring information, integrating multiple hourly data monitoring records into a single long-term data monitoring record is straightforward. For example, regarding the total number of requests from 11:00 AM to 12:00 PM, the cache flushes the data to record the total number of requests every 30 seconds, while the storage pool stores the number of requests every minute. Therefore, obtaining the total number of requests from 11:00 AM to 12:00 PM involves summing the values ​​of the 60 data monitoring records extracted from the storage for that time period to arrive at the final data monitoring record representing the total number of requests from 11:00 AM to 12:00 PM. Other data monitoring information, such as bandwidth, latency, and space increment, follows the same principle, and will not be elaborated upon in this embodiment.

[0107] The above embodiments have described in detail a data monitoring method for a distributed object storage system. This application also provides an embodiment of a data monitoring device for a distributed object storage system. It should be noted that this application describes the device embodiment from two perspectives: one based on functional modules and the other based on hardware.

[0108] From the perspective of functional modules, such as Figure 3 As shown, this embodiment provides a data monitoring device for a distributed object storage system, including:

[0109] The receiving module 31 is used to receive request information sent by the client and store the request information in the cache;

[0110] The statistics module 32 is used to perform statistics on a portion of the data in the request information according to preset monitoring requirements in order to obtain data monitoring information.

[0111] Storage module 33 is used to send data monitoring information to the storage pool and save it.

[0112] Preferably, it further includes:

[0113] The query module is used to receive monitoring query commands sent by the client; retrieve the corresponding data monitoring information from the storage pool according to the monitoring query commands; and return the data monitoring information as the query result.

[0114] The display module is used to display data monitoring information on the management interface.

[0115] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0116] This embodiment provides a distributed object storage system data monitoring device that uses a cache as a temporary storage location for request information. A statistics module performs statistical analysis on a portion of the data in the request information based on preset monitoring requirements to obtain data monitoring information. Furthermore, the data monitoring information can be stored in a storage pool via a storage module, eliminating the need to store all request information. This significantly reduces the amount of data required for storage, decreasing the load on the storage system and minimizing the impact on its performance. Additionally, the distributed object storage system data monitoring device provided in this embodiment allows users to directly retrieve the corresponding data monitoring information from the storage pool during queries. Compared to the current method of storing all request information in the storage system and then statistically analyzing the request information upon receiving a user's query to obtain and return the data monitoring information, this method requires fewer steps from sending a query to receiving the data monitoring information. Consequently, the system's response time is shorter, allowing users to obtain data monitoring information more quickly, resulting in higher query efficiency and a better user experience.

[0117] Figure 4 A structural diagram of a data monitoring device for a distributed object storage system provided in another embodiment of this application is shown below. Figure 4 As shown, a data monitoring device for a distributed object storage system includes: a memory 40 for storing computer programs;

[0118] The processor 41 is used to execute computer programs to implement the steps of a data monitoring method for a distributed object storage system as described in the above embodiments.

[0119] The distributed object storage system data monitoring device provided in this embodiment can include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0120] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 41 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0121] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, can implement the relevant steps of a distributed object storage system data monitoring method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, a distributed object storage system data monitoring method.

[0122] In some embodiments, a distributed object storage system data monitoring device may further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0123] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on a data monitoring device for a distributed object storage system and may include more or fewer components than shown.

[0124] This application provides a distributed object storage system data monitoring device, including a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a distributed object storage system data monitoring method.

[0125] This embodiment provides a distributed object storage system data monitoring device. By executing a computer program stored in the storage via an executor, it can statistically analyze a portion of the requested data to obtain data monitoring information based on preset monitoring requirements. Furthermore, the data monitoring information is stored in a storage pool, eliminating the need to store all the requested information, thus reducing the amount of data required and minimizing the impact on storage system performance. Moreover, it allows users to directly retrieve the corresponding data monitoring information from the storage pool when querying, reducing the steps required from sending a query request to receiving the returned data monitoring information. Consequently, the system's response time is shorter, allowing users to obtain data monitoring information more quickly, resulting in higher query efficiency and a better user experience.

[0126] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0127] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] This embodiment provides a computer-readable storage medium. When the computer program stored therein is executed, it can statistically analyze a portion of the data in the requested information according to preset monitoring requirements to obtain data monitoring information. This data monitoring information is then stored in a storage pool, eliminating the need to store the entire request information, thus reducing the amount of data required and minimizing the impact on storage system performance. Furthermore, it allows users to directly retrieve the corresponding data monitoring information from the storage pool when querying, reducing the steps required from sending a query request to receiving the returned data monitoring information. Consequently, the system's response time is shorter, allowing users to obtain data monitoring information more quickly, resulting in higher query efficiency and a better user experience.

[0129] The foregoing provides a detailed description of a data monitoring method, apparatus, and medium for a distributed object storage system provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0130] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data monitoring method for a distributed object storage system, characterized in that, include: Receive request information sent by the client and store the request information in the cache; Based on preset monitoring requirements, some data in the request information are statistically analyzed to obtain data monitoring information; The data monitoring information is sent to the storage pool and saved. Saving the data monitoring information to the storage pool includes: The data monitoring information is saved to omap, a predefined rados object in the storage pool; wherein, omap has a map structure.<k,v> The structure involves inserting the acquired data monitoring information into the omap every first preset time interval, where k represents the insertion time and v represents the statistical value within a certain time period starting from time k. Sending the data monitoring information to the storage pool includes: When the number of requests stored in the cache exceeds a preset threshold, the data monitoring information in the cache is sent to the storage pool. Every second preset time interval, the data monitoring information in the cache is sent to the storage pool; wherein, the second preset time interval is half of the first preset time interval; This method also includes: Receive a monitoring query instruction sent by the client; wherein the monitoring query instruction includes: the type of data monitoring information to be queried, and the corresponding time period; According to the type of data monitoring information, the corresponding data stored in omap is found from the storage pool, and the data monitoring information within the corresponding time period is found according to the k value in the omap structure; The data monitoring information will be returned as the query result.

2. The data monitoring method for a distributed object storage system according to claim 1, characterized in that, After retrieving the corresponding data monitoring information from the storage pool according to the monitoring query instruction, the method further includes: The data monitoring information is displayed on the management interface.

3. A data monitoring device for a distributed object storage system, characterized in that, include: The receiving module is used to receive request information sent by the client and store the request information in a cache; The statistics module is used to perform statistics on a portion of the data in the request information according to preset monitoring requirements, so as to obtain data monitoring information; A storage module is used to send the data monitoring information to a storage pool and save it; wherein saving the data monitoring information to the storage pool includes: saving the data monitoring information to an omap object of a preset rados object in the storage pool; wherein the omap has a map structure.<k,v> The structure involves inserting the acquired data monitoring information into the omap after a first preset time interval, where k represents the insertion time and v represents the statistical value within a certain period starting from time k. Sending the data monitoring information to the storage pool includes: sending the data monitoring information in the cache to the storage pool when the number of requests stored in the cache exceeds a preset threshold; and sending the data monitoring information in the cache to the storage pool after a second preset time interval; wherein the second preset time interval is half of the first preset time interval. The query module is used to receive monitoring query instructions sent by the client; wherein, the monitoring query instructions include: the type of data monitoring information to be queried, and the corresponding time period; according to the type of data monitoring information, the module finds the corresponding data stored in the omap in the storage pool, and finds the data monitoring information within the corresponding time period according to the k value in the omap structure; and returns the data monitoring information as the query result.

4. A data monitoring device for a distributed object storage system, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data monitoring method for a distributed object storage system as described in claim 1 or 2 when executing the computer program.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data monitoring method for a distributed object storage system as described in claim 1 or 2.

Images