Object storage bucket operation log storage method and device, equipment and storage medium

Abstract

The present invention discloses a method, apparatus, device and computer-readable storage medium for storing operation logs of object storage buckets, and relates to the field of object storage technology. The method comprises: obtaining bucket operation information corresponding to an operation request of a target storage bucket; wherein the bucket operation information comprises a request time of the operation request and a bucket operation log; using the request time as an index, storing the bucket operation information corresponding to each target storage bucket, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information according to the log time information in the log query request of the current target storage bucket; the present invention can use the request time of the operation request as an index to store the bucket operation information of the storage bucket that starts the bucket log recording function in the object storage system, thereby realizing the organization and collection of the operation information of the storage bucket, facilitating the user to monitor and query the operation information of the storage bucket, and improving the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of object storage, in particular to an object storage bucket operation log storage method, device and equipment and a computer readable storage medium. BACKGROUND

[0002] With the development of Internet technology, the amount of unstructured data in the Internet is increasing, which leads to the emergence of distributed object storage. This emerging storage method is suitable for the storage of unstructured data such as pictures and videos. However, as the application of object storage in massive storage continues to deepen, users pay more and more attention to the monitoring of big data, especially the operations such as data addition, modification and deletion. However, there is no user-oriented bucket operation record mechanism for object storage at present, and the background running log in the storage system is very chaotic for users, which is difficult to collect and effective information.

[0003] Therefore, how to arrange and collect the operation information of the object storage bucket, facilitate the monitoring and query of the operation information of the object storage bucket by the user, and improve the user experience, is a problem that needs to be solved urgently at present. SUMMARY

[0004] The purpose of the present application is to provide an object storage bucket operation log storage method, device and equipment and a computer readable storage medium, to arrange and collect the operation information of the object storage bucket, facilitate the monitoring and query of the operation information of the object storage bucket by the user, and improve the user experience.

[0005] To solve the above technical problems, the present application provides an object storage bucket operation log storage method, comprising:

[0006] Obtaining bucket operation information corresponding to an operation request of a target storage bucket; wherein the target storage bucket is a storage bucket in an object storage system that starts a bucket log recording function, the bucket operation information includes a request time of the operation request and a bucket operation log, and the bucket operation log includes request results and / or calling function information;

[0007] Taking the request time as an index, storing the bucket operation information corresponding to each target storage bucket, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information according to the log time information in the log query request of the current target storage bucket; wherein the current target storage bucket is any target storage bucket.

[0008] In some embodiments, the obtaining of the bucket operation information corresponding to the operation request of the target storage bucket comprises:

[0009] The object storage server receives a current operation request sent by a service client device; wherein the current operation request is a request for operating a storage bucket in the object storage system;

[0010] If the storage bucket corresponding to the current operation request is the target storage bucket, during processing of the current operation request, the bucket operation information corresponding to the current operation request is collected and cached.

[0011] In some embodiments, after storing the bucket operation information corresponding to each of the target storage buckets using the request time as the index, the method further comprises:

[0012] The object storage server receives a current log query request for a specified storage bucket sent by an operation and maintenance client device;

[0013] It is judged whether the specified storage bucket is the target storage bucket;

[0014] If yes, according to the log query time period in the current log query request, the bucket operation information corresponding to the current log query request is found from the stored bucket operation information of the specified storage bucket, and the found bucket operation information corresponding to the current log query request is sent to the operation and maintenance client device; wherein the bucket operation information corresponding to the current log query request is the bucket operation information corresponding to the request time in the log query time period, and the log time information is the log query time period.

[0015] In some embodiments, the storing of the bucket operation information corresponding to each of the target storage buckets using the request time as the index comprises:

[0016] Using a background processing thread, the bucket operation information corresponding to each of the target storage buckets cached in a preset time is stored using the request time in the bucket operation information as the index.

[0017] In some embodiments, the storing of the bucket operation information corresponding to each of the target storage buckets using the request time as the index comprises:

[0018] The bucket operation information of each of the target storage buckets is stored using a data pool, and the index corresponding to the request time of each of the target storage buckets is stored using a metadata pool, so that according to the log time information in the log query request of the current target storage bucket, the target index corresponding to the log time information in the metadata pool is found, and the bucket operation information of the current target storage bucket corresponding to the target index is found from the data pool.

[0019] In some embodiments, the data pool is used to store the bucket operation information of each of the target storage buckets, and the metadata pool is used to store the index corresponding to the request time of each of the target storage buckets, including:

[0020] The cached map structure data corresponding to the current target storage bucket is obtained;The map structure data corresponding to the current target storage bucket includes the request time and the bucket operation log corresponding to each of the bucket operation information of the current target storage bucket.

[0021] According to the map structure data, a first object corresponding to the current target storage bucket is stored in the data pool;The object name of the first object includes the minimum request time in the map structure data corresponding to the current target storage bucket.

[0022] According to the object name of the first object, a corresponding key-value pair is added to the key-value map of a second object corresponding to the current target storage bucket in the metadata pool.

[0023] In some embodiments, the key of the key-value pair is the object name of the first object, and the value of the key-value pair is empty.

[0024] The application also provides an object storage bucket operation log storage device, including:

[0025] A log acquisition module is configured to acquire bucket operation information corresponding to an operation request of a target storage bucket;The target storage bucket is a storage bucket in an object storage system that starts a bucket log recording function, and the bucket operation information includes a request time of the operation request and a bucket operation log, and the bucket operation log includes a request result and / or calling function information.

[0026] A log storage module is configured to store the bucket operation information corresponding to each of the target storage buckets by taking the request time as an index, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information according to the log time information in a log query request of the current target storage bucket;The current target storage bucket is any of the target storage buckets.

[0027] The application also provides an object storage bucket operation log storage device, including:

[0028] A memory is configured to store a computer program.

[0029] A processor is configured to execute the computer program to implement the steps of the object storage bucket operation log storage method described above.

[0030] In addition, the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the object storage bucket operation log storage method.

[0031] The object storage bucket operation log storage method provided by the application comprises the following steps: obtaining bucket operation information corresponding to an operation request of a target storage bucket; the target storage bucket is a storage bucket in an object storage system that starts a bucket log recording function, the bucket operation information comprises a request time of the operation request and a bucket operation log, and the bucket operation log comprises request results and / or calling function information; taking the request time as an index, storing the bucket operation information corresponding to each target storage bucket respectively; and according to log time information in a log query request of a current target storage bucket, finding the bucket operation information of the current target storage bucket corresponding to the stored log time information; the current target storage bucket is any target storage bucket.

[0032] It can be seen that the application can take the request time of an operation request as an index to store the bucket operation information of the storage bucket in the object storage system that starts the bucket log recording function, realizes the arrangement and collection of the operation information of the storage bucket, enables the corresponding bucket operation information to be queried through the request time subsequently, facilitates the monitoring and querying of the operation information of the object storage bucket by a user, and improves the user experience. In addition, the application further provides an object storage bucket operation log storage device, equipment and computer readable storage medium, which also have the beneficial effects described above. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute the embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0034] Figure 1 The flow chart of the object storage bucket operation log storage method provided by the embodiment of the application;

[0035] Figure 2 The flow chart of the object storage bucket operation log storage method provided by the embodiment of the application;

[0036] Figure 3 The flow chart of the object storage bucket operation log storage method provided by the embodiment of the application;

[0037] Figure 4A flowchart of a log query process of another object storage bucket operation log storage method provided by the embodiment of the present application is shown in the figure.

[0038] Figure 5 A structure block diagram of an object storage bucket operation log storage device provided by the embodiment of the present application is shown in the figure.

[0039] Figure 6 A structure diagram of an object storage bucket operation log storage device provided by the embodiment of the present application is shown in the figure.

[0040] Figure 7 A specific structure diagram of an object storage bucket operation log storage device provided by the embodiment of the present application is shown in the figure.

[0041] Figure 8 A structure diagram of a computer readable storage medium provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0042] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0043] Please refer to Figure 1 , Figure 1 A flowchart of an object storage bucket operation log storage method provided by the embodiment of the present application is shown in the figure. The method can include:

[0044] Step 101: obtaining bucket operation information corresponding to an operation request of a target storage bucket; wherein the target storage bucket is a storage bucket in an object storage system that starts a bucket log recording function, the bucket operation information includes a request time of the operation request and a bucket operation log, and the bucket operation log includes a request result and / or calling function information.

[0045] It can be understood that the object storage system in the embodiment can be a storage system using object storage (RadosGateway), such as a distributed object storage system. The target storage bucket in the embodiment can be a storage bucket that starts the bucket log recording function in the object storage system, that is, a storage bucket that needs to store bucket operation information. The bucket operation information in the embodiment can include the request time (such as the receiving time) of the operation request, and the operation log made to the storage bucket in the processing process of the operation request, such as the request result (such as successful deletion or deletion failure) of the operation request and / or the information of the function called in the processing process of the operation request (that is, the calling function information), and can also include the request type of the operation request and other information.

[0046] Correspondingly, for the specific manner in which the processor in the embodiment obtains the bucket operation information corresponding to the operation request of the target storage bucket, the designer can set it according to the practical scene and user demand, such as the processor of the object storage (RGW, RadosGateway) server can collect the bucket operation information corresponding to the operation request in the process of processing the operation request after receiving the operation request of the target storage bucket. Correspondingly, the processor also buffers the bucket operation information corresponding to the operation request, so as to facilitate the batch aggregation and storage of the bucket operation information through the buffer, that is, the processor in this step can collect and buffer the bucket operation information corresponding to the operation request of the target storage bucket, such as the RGW server can collect and buffer the bucket operation information corresponding to the operation request of the target storage bucket after receiving the operation request of the target storage bucket issued by the business client (that is, the business client device), as shown in the following table. Figure 2 Correspondingly, for the specific manner in which the processor in the embodiment obtains the bucket operation information corresponding to the operation request of the target storage bucket, the designer can set it according to the practical scene and user demand, such as the processor of the object storage (RGW, RadosGateway) server can collect the bucket operation information corresponding to the operation request in the process of processing the operation request after receiving the operation request of the target storage bucket. Correspondingly, the processor also buffers the bucket operation information corresponding to the operation request, so as to facilitate the batch aggregation and storage of the bucket operation information through the buffer, that is, the processor in this step can collect and buffer the bucket operation information corresponding to the operation request of the target storage bucket, such as the RGW server can collect and buffer the bucket operation information corresponding to the operation request of the target storage bucket after receiving the operation request of the target storage bucket issued by the business client (that is, the business client device), as shown in the following table.

[0047] For example, the processor of the object storage server in this step can receive a current operation request sent by the service client device; wherein the current operation request is a request for operating a storage bucket in the object storage system; if the storage bucket corresponding to the current operation request is the target storage bucket, then in the process of processing the current operation request, the bucket operation information corresponding to the current operation request is collected and cached. For example, after receiving the current operation request sent by the service client device, the processor can determine whether the storage bucket corresponding to the current operation request is the target storage bucket; if yes, the bucket operation information corresponding to the current operation request is collected and cached; if no, the current process can be ended or the current operation request can be processed normally without collecting the bucket operation information corresponding to the current operation request. The processor can also collect the bucket operation information corresponding to the current operation request after receiving the current operation request sent by the service client device, and determine whether the storage bucket corresponding to the current operation request is the target storage bucket; if yes, the bucket operation information corresponding to the current operation request is cached; if no, the current process can be ended or the current operation request can be processed normally without caching the bucket operation information corresponding to the current operation request.

[0048] Correspondingly, for the specific data structure of the bucket operation information in this embodiment, the designer can set it according to the practical scene and user demand, for example, the bucket operation information can be map structure data, for example, each storage bucket that starts the bucket log recording function (i.e. the target storage bucket) can use a map structure cache to save the bucket operation information of the storage bucket, and the bucket operation information corresponding to each operation request of the target storage bucket can be saved in the map structure cache as map structure data (i.e. map structure data); when the bucket operation information is map structure data, each map structure data can include request time and bucket operation log, and the map structure data corresponding to the current target storage bucket can include the request time and the bucket operation log corresponding to each bucket operation information of the current target storage bucket; for example, the map structure data can be the data of the map structure of the key-value type, the key in each map structure data can be the request time, and the value in each map structure data can be the operation log, i.e. the operation log of the target storage bucket at the request time (key); that is, a map structure data can be <time, value>, and within a preset time (such as 1s), 100 cache requests of the current target storage bucket are received, and the map structure data recorded in the cache of the current target storage bucket can be map1={<time1, value1>, <time2, value2>, …, <time100, value100>}, and the current target storage bucket is any target storage bucket.

[0049] Step 102: Use the request time as an index to store the bucket operation information corresponding to each target storage bucket, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information based on the log time information in the log query request of the current target storage bucket; wherein, the current target storage bucket is any target storage bucket.

[0050] It can be understood that in this embodiment, the processor can use the request time in the bucket operation information as an index to store the bucket operation information corresponding to each target storage bucket, so that the request time of any target storage bucket can be used to query the stored bucket operation information corresponding to the target storage bucket, thereby realizing the query of the bucket operation log in the bucket operation information.

[0051] Correspondingly, the method provided in this embodiment may also include a query process for bucket operation information, such as the processor can find the bucket operation information of the current target bucket corresponding to the stored log time information based on the log time information in the log query request of the current target bucket; for example, the processor of the object storage server can receive the current log query request for the specified bucket sent by the operation and maintenance client device; the current log query request can be any log query request; determine whether the specified bucket is the target bucket; if so, find the bucket operation information corresponding to the current log query request of the stored specified bucket based on the log query time period in the current log query request, and send the found bucket operation information or bucket operation log corresponding to the current log query request to the operation and maintenance client device; wherein, the bucket operation information corresponding to the current log query request is the bucket operation information corresponding to the request time within the log query time period, and the request time information of the bucket operation log to be queried in the log query request (i.e., log time information) can be a time period specified by the user (i.e., log query time period).

[0052] It should be noted that in order to reduce the impact of the collection and storage of bucket operation information on the request processing performance of the object storage system, the processor in this embodiment can obtain and cache the bucket operation information corresponding to the operation request of the target storage bucket in step 101. In this step, the background processing thread is used to store the bucket operation information corresponding to each target storage bucket using the request time as an index. For example, in this step, the processor can use the background processing thread to store the bucket operation information corresponding to each target storage bucket cached within the preset time at a preset time interval using the request time in the bucket operation information corresponding to each target storage bucket cached within the preset time as an index; Figure 2As shown, the RGW server can use a background processing thread to read the bucket operation information corresponding to each target bucket in the cache at a preset time interval. For example, if the preset time interval is 1 s, i.e., the working interval of the background processing thread is 1 s, the processor can use the background processing thread to intercept the cache of all target buckets every second, and store the bucket operation information of each target bucket cached in 1 s, using the request time in the bucket operation information of the target bucket as the index.

[0053] Correspondingly, the specific way in which the processor stores the bucket operation information corresponding to each target bucket using the request time as the index in this step, i.e., the specific storage method of the bucket operation information and the index, can be set by the designer according to the practical scenario and user demand. For example, the processor can directly use the data pool of the object storage system to store the bucket operation information corresponding to each target bucket. The processor can also use the data pool to store the bucket operation information of each target bucket, and use the metadata pool to store the index corresponding to the request time of each target bucket; so that the processor can subsequently find the index corresponding to the log time information in the log query request of the current target bucket (i.e., the target index) in the metadata pool, and find the bucket operation information of the current target bucket corresponding to the target index in the data pool.

[0054] In the object storage system, the objects (rados objects) in the storage pool are different from the objects in the storage buckets in the object storage system. The objects in the storage buckets refer to a file uploaded by a client as a whole and saved in the storage bucket as an object, while the rados objects refer to a piece of data saved in the storage pool after being encoded as binary data. The storage pool is divided into a metadata pool and a data pool, and the rados objects in the two pools are different. The rados objects in the data pool are only a piece of data without an omap structure; the rados objects in the metadata pool have an omap structure; the omap (key-value map) refers to a K-V type map structure on the rados objects in the storage pool, which can be used to save K-V type data. Generally, the omap only exists on the rados objects in the metadata pool.

[0055] Correspondingly, the processor can store the bucket operation information of each target storage bucket in the data pool and store the index corresponding to the request time of each target storage bucket in the metadata pool in the following manners, which can be set by the designer. For example, when the bucket operation information corresponding to the operation request of the target storage bucket is temporarily stored in the cache in the form of map structure data, the processor can obtain the map structure data corresponding to the current target storage bucket in the cache; the map structure data corresponding to the current target storage bucket includes the request time and the bucket operation log corresponding to each bucket operation information of the current target storage bucket; according to the map structure data, a first object (rados object) corresponding to the current target storage bucket is stored in the data pool; the object name of the first object includes the minimum request time or the maximum request time in the map structure data corresponding to the current target storage bucket, and the first object can be the binary encoding result of the map structure data; according to the object name of the first object, a corresponding key-value pair (K-V) is added to the key-value map (omap) of the second object corresponding to the current target storage bucket in the metadata pool. That is, the processor can store the rados object (first object) corresponding to each target storage bucket in the storage pool according to the entire bucket operation information (map structure data) corresponding to each target storage bucket in the cache, and the object name of the rados object can include the minimum request time in the entire bucket operation information corresponding to the target storage bucket; and a corresponding key-value pair corresponding to the object name of the first object is added to the omap of the rados object (second object) corresponding to each target storage bucket in the metadata pool, wherein the key of the key-value pair can be the object name of the first object, and the value of the key-value pair is empty.

[0056] For example, the processor can use a background processing thread to obtain the map structure data corresponding to each target storage bucket in the cache at a preset time interval; according to the map structure data corresponding to each target storage bucket, the first object corresponding to each target storage bucket is stored in the data pool; and according to the object name of the first object corresponding to each target storage bucket newly stored in the data pool, a corresponding key-value pair is added to the key-value map of the second object corresponding to each target storage bucket in the metadata pool. For example, Figure 2As shown, the RGW server uses a background processing thread to read the bucket operation information (i.e., map structure data) corresponding to each target storage bucket in the cache at a preset time interval and save the bucket operation information, so that the rados object (i.e., the first object) named time (i.e., the minimum request time) in the data pool can be saved to save the bucket operation information in the preset time, which can be recorded as the bucket log object (i.e., the first object); correspondingly, the object name of the bucket log object (i.e., the first object) in the data pool is recorded as an index on the omp of the rados object (i.e., the second object) corresponding to each storage bucket in the metadata pool. For example, when the map structure data corresponding to the current target storage bucket in the cache within 1S is map1={<time1,value1>,<time2,value2>,…,<time100,value100>}, the background processing thread can read map1, encode map1 into a binary rados object (i.e., the first object), and save the rados object in the data pool, and the object name of the rados object can include the minimum time "time1"; and each storage bucket (i.e., the target storage bucket) that starts the bucket log recording function in the metadata pool has a rados object (i.e., the second object), and the object name of the first object saved in the data pool is recorded on the omap of the second object corresponding to the current target storage bucket in the metadata pool, for example, key="time1" and value is empty are recorded on the omap of the second object.

[0057] In the embodiment, the operation request request time is used as an index to store the bucket operation information of the storage bucket starting the bucket log recording function in the object storage system, the operation information of the storage bucket is sorted and collected, the corresponding bucket operation information can be queried through the request time in the future, the user can conveniently monitor and query the operation information of the object storage bucket, and the user experience is improved.

[0058] Based on the above embodiment, the embodiment of the present application also provides another object storage bucket operation log storage method to realize the bucket operation log query function of the storage bucket, and details are referred to Figure 3 , Figure 3 The flowchart of another object storage bucket operation log storage method provided by the embodiment of the present application. The method can include:

[0059] Step 201: The object storage server receives the current log query request of the specified storage bucket sent by the operation and maintenance client device.

[0060] In the embodiment, the object storage server can be a server in an object storage system, and the operation and maintenance client device can be an electronic device running an operation and maintenance client. The specified storage bucket can be any storage bucket in the object storage system, and the current log query request of the specified storage bucket can be a request for querying the bucket operation log of the specified storage bucket.

[0061] Correspondingly, the embodiment is not limited to the specific content of the current log query request. For example, the current log query request can include information of a request time of the bucket operation log to be queried (i.e., log time information). For example, the log time information in the embodiment can be a log query time period, i.e., the bucket operation log of the specified storage bucket in a time period is required to be queried. The current log query request can also include identification information of the specified storage bucket, such as a name or a number of the specified storage bucket.

[0062] Step 202: determining whether the specified storage bucket is a target storage bucket; if yes, proceeding to step 203.

[0063] It can be understood that, in the embodiment, the object storage server can determine whether the bucket operation information corresponding to the specified storage bucket is stored by determining whether the specified storage bucket corresponding to the current log query request is a storage bucket starting the bucket log recording function (i.e., a target storage bucket), so as to find the stored bucket operation information corresponding to the current log query request of the specified storage bucket through step 203 when it is determined that the bucket operation information corresponding to the specified storage bucket is stored.

[0064] Correspondingly, for the case that the object storage server determines that the specified storage bucket is not the target storage bucket, the designer can set it by himself / herself. For example, the process can be directly ended as shown in FIG. 8, or the object storage server can return the starting recommendation information of the bucket log recording function to the operation and maintenance client device to prompt the user to start the bucket log recording function of the specified storage bucket. Figure 4

[0065] Step 203: finding the bucket operation information corresponding to the current log query request of the specified storage bucket according to the log query time period in the current log query request, and sending the found bucket operation information corresponding to the current log query request of the specified storage bucket to the operation and maintenance client device; wherein the bucket operation information corresponding to the current log query request in the current log query request is the bucket operation information corresponding to the request time in the log query time period, and the log time information is the log query time period.

[0066] ​It should be noted that in this step, the object storage server can query the bucket operation information corresponding to the request time of the specified storage bucket within the log query time period according to the log query time period in the current log query request when determining the specified storage bucket as the target storage bucket, and return the bucket operation information or the bucket operation log in the bucket operation information to the operation and maintenance client device, thereby realizing the query function of the bucket operation log of the storage bucket.

[0067] Correspondingly, for the specific way in which the object storage server finds the bucket operation information corresponding to the current log query request of the specified storage bucket according to the log query time period in the current log query request in this step, the designer can set it according to the practical scene and user demand, such as when the object storage server stores the bucket operation information and the corresponding index of the target storage bucket by using the data pool and the metadata pool, the object storage server can query the index (i.e., the target index) of the specified storage bucket in the log query time period in the metadata pool; according to the target index, the bucket operation information of the specified storage bucket in the data pool is queried as the bucket operation information corresponding to the current log query request. For example, the object storage server can query the target key-value pair in the second object (rados object) corresponding to the specified storage bucket in the metadata pool; wherein the request time in the key of the target key-value pair is within the log query time period and the value is empty; the first object of the specified storage bucket corresponding to the target key-value pair in the data pool is queried; the first object of the specified storage bucket is decoded to obtain the bucket operation information corresponding to the current log query request.

[0068] For example, after the user starts the bucket log recording function of the storage bucket bucket1, the RGW server receives the operation request of the service client device to bucket1, collects the bucket operation information related to bucket1 in the operation request processing flow, and then temporarily stores it in the cache map corresponding to bucket1, the key is the reception time of this operation request recorded as time1, and the value is the operation log related to bucket1 collected in this operation request recorded as log1. The contents corresponding to nine operation requests stored in the cache map can be map={<time1,log1>,…,<time9,log9>}, that is, the contents corresponding to nine operation requests can be sorted in ascending order according to the reception time; then the operation request processing is completed, and the bucket operation information is temporarily stored in the cache, waiting for the processing of the background processing thread.

[0069] Correspondingly, after the background processing thread is started, the cache map content of the bucket 1 can be intercepted every 1s, and then the cache map content is encoded and stored in the data pool as a rados object (i.e., a first object) corresponding to the bucket 1, and the object name is the key of the first element in the intercepted map, that is, time 1. At the same time, a kv (i.e., a key-value pair) is added to the omap of the rados object (i.e., a second object) of the bucket log index corresponding to the bucket 1 in the metadata pool, and the key is time 1, that is, the object name of the first object in the data pool, and the value is empty. Thus, the background processing thread completes the storage of the bucket operation information of the bucket 1 once.

[0070] Correspondingly, as shown in Figure 4 , when the user wants to query the bucket operation log of the bucket 1, the log query request of the bucket 1 can be initiated through the operation and maintenance client device, and the time period to be queried (i.e., the log query time period) is specified. Then, the object storage server (RGW server) can go to the rados object of the bucket log index corresponding to the bucket 1 in the metadata pool, filter the key of the omap with the log query time period, find the target key-value pair, obtain the object name of the rados object in the data pool that needs to be accessed, and then read the data of the rados object of the bucket 1 corresponding to the object name in the data pool, decode the corresponding bucket operation information, and return to the operation and maintenance client device.

[0071] In the embodiment, the application can realize the query function of the bucket operation log by finding the bucket operation information corresponding to the current log query request of the specified storage bucket according to the log query time period in the current log query request and sending the found bucket operation information corresponding to the current log query request to the operation and maintenance client device, which facilitates the user to query and call the bucket operation information and improves the user experience.

[0072] Corresponding to the above method embodiment, the application also provides an operation log storage device of an object storage bucket. The operation log storage device described below can be referred to the operation log storage method of the object storage bucket described above.

[0073] Please refer to Figure 5 , Figure 5 The structure block diagram of the operation log storage device of the object storage bucket provided by the application embodiment. The device can include:

[0074] The log obtaining module 10 is configured to obtain bucket operation information corresponding to an operation request of a target storage bucket; the target storage bucket is a storage bucket in an object storage system that starts a bucket log recording function, and the bucket operation information includes a request time of the operation request and a bucket operation log, and the bucket operation log includes a request result and / or calling function information;

[0075] The log storage module 20 is configured to store the bucket operation information corresponding to each target storage bucket respectively by taking the request time as an index, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information according to log time information in a log query request of the current target storage bucket; the current target storage bucket is any target storage bucket.

[0076] In some embodiments, when the apparatus is applied to an object storage server, the log obtaining module 10 can include:

[0077] The operation receiving sub-module is configured to receive a current operation request sent by a business client device; the current operation request is a request for operating a storage bucket in an object storage system;

[0078] The cache sub-module is configured to, if the storage bucket corresponding to the current operation request is a target storage bucket, collect and cache the bucket operation information corresponding to the current operation request in the process of processing the current operation request.

[0079] In some embodiments, the apparatus can further include:

[0080] The query receiving sub-module is configured to receive a current log query request of a specified storage bucket sent by an operation and maintenance client device;

[0081] The judgment sub-module is configured to judge whether the specified storage bucket is a target storage bucket;

[0082] The judgment sub-module is configured to, if the specified storage bucket is a target storage bucket, find the bucket operation information corresponding to the current log query request of the specified storage bucket stored according to a log query time period in the current log query request, and send the found bucket operation information corresponding to the current log query request to the operation and maintenance client device; the bucket operation information corresponding to the current log query request of the current log query request is the bucket operation information corresponding to the request time in the log query time period, and the log time information is the log query time period.

[0083] In some embodiments, the log storage module 20 can be specifically configured to store the bucket operation information corresponding to each target storage bucket respectively cached in a preset time by taking the request time in the bucket operation information corresponding to each target storage bucket cached in the preset time as an index.

[0084] In some embodiments, the log storage module 20 can be specifically configured to store the bucket operation information of each target storage bucket by using a data pool, and store the index corresponding to the request time of each target storage bucket by using a metadata pool, so as to find the target index corresponding to the log time information in the metadata pool according to the log time information in the log query request of the current target storage bucket, and find the bucket operation information of the current target storage bucket corresponding to the target index from the data pool.

[0085] In some embodiments, the log storage module 20 can include:

[0086] The cache obtaining sub-module is configured to obtain the cached map structure data corresponding to the current target storage bucket; wherein the map structure data corresponding to the current target storage bucket includes the request time and the bucket operation log corresponding to each bucket operation information of the current target storage bucket.

[0087] The data pool storage sub-module is configured to store the first object corresponding to the current target storage bucket in the data pool according to the map structure data; wherein the object name of the first object includes the smallest request time in the map structure data corresponding to the current target storage bucket.

[0088] The metadata pool storage sub-module is configured to add the corresponding key-value pair to the key-value map of the second object corresponding to the current target storage bucket in the metadata pool according to the object name of the first object.

[0089] In some embodiments, the key of the key-value pair is the object name of the first object, and the value of the key-value pair is empty.

[0090] In this embodiment, the log storage module 20 stores the bucket operation information of the storage bucket starting the bucket log recording function in the object storage system by taking the request time of the operation request as an index, realizes the arrangement and collection of the operation information of the storage bucket, so that the corresponding bucket operation information can be queried by the request time subsequently, facilitates the monitoring and querying of the operation information of the object storage bucket by the user, and improves the user experience.

[0091] Corresponding to the above method embodiments, the present embodiment also provides an object storage bucket operation log storage device. The object storage bucket operation log storage device described below can be mutually corresponding to the object storage bucket operation log storage method described above.

[0092] Please refer to Figure 6 , Figure 6 The object storage bucket operation log storage device provided by the present embodiment can include:

[0093] The memory D1 is configured to store a computer program.

[0094] Processor D2, configured to implement the steps of the object storage bucket operation log storage method provided by the above method embodiments when executing a computer program.

[0095] Specifically, refer to Figure 7 , Figure 7 The specific structure diagram of the object storage bucket operation log storage device provided by the embodiment of the present application, the operation log storage device 310 can have great differences due to different configurations or performances, and can include one or more central processing units (CPUs) 322 (for example, one or more processors) and a memory 332, one or more storage application programs 342 or data 344 storage media 330 (for example, one or more mass storage devices). Among them, the memory 332 and the storage medium 330 can be temporary storage or persistent storage. The program stored in the storage medium 330 can include one or more modules (not shown in the figure), each module can include a series of instruction operations in the data processing device. Further, the central processing unit 322 can be configured to communicate with the storage medium 330 and execute a series of instruction operations in the storage medium 330 on the operation log storage device 310.

[0096] The operation log storage device 310 can also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341. For example, Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0097] Specifically, the object storage bucket operation log storage device provided by the embodiment can be a server, such as the server of the object storage system, that is, the above-mentioned object storage server.

[0098] The steps in the object storage bucket operation log storage method described above can be implemented by the structure of the object storage bucket operation log storage device.

[0099] Corresponding to the above method embodiments, the embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium described below can be mutually corresponding with the object storage bucket operation log storage method described above.

[0100] Please refer to Figure 8 , Figure 8A structural diagram of a computer readable storage medium provided by an embodiment of the present application is shown. The computer readable storage medium 40 stores a computer program 41, and the computer program 41 is executed by a processor to implement the steps of the object storage bucket operation log storage method provided by the above method embodiment.

[0101] The computer readable storage medium 40 can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or various storage media that can store program codes.

[0102] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the apparatus, device and computer readable storage medium disclosed by the embodiments, since they correspond to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0103] The above describes in detail the object storage bucket operation log storage method, apparatus, device and computer readable storage medium provided by the present application. The principle and implementation manner of the present application are described by applying specific examples. The above embodiment is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principle of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method for storing operation logs in an object storage bucket, characterized in that: include: Obtaining bucket operation information corresponding to the operation request for the target bucket; wherein the target bucket is a bucket in the object storage system with bucket logging enabled, the bucket operation information including the request time of the operation request and a bucket operation log, the bucket operation log including the request result and / or call function information; Using the request time as an index, the bucket operation information corresponding to each of the target storage buckets is stored, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information according to the log time information in the log query request of the current target storage bucket; wherein the current target storage bucket is any of the target storage buckets; The storing of bucket operation information corresponding to each target storage bucket using the request time as an index includes: The data pool is used to store the bucket operation information of each target storage bucket, and the metadata pool is used to store the index corresponding to the request time of each target storage bucket, so as to find the target index corresponding to the log time information in the metadata pool according to the log time information in the log query request of the current target storage bucket, and find the bucket operation information of the current target storage bucket corresponding to the target index from the data pool; The step of using a data pool to store the bucket operation information of each target bucket and using a metadata pool to store the index corresponding to the request time of each target bucket includes: Obtaining cached map structure data corresponding to the current target bucket; wherein the map structure data corresponding to the current target bucket includes the request time and bucket operation log corresponding to each bucket operation information of the current target bucket; According to the map structure data, a first object corresponding to the current target storage bucket is stored in the data pool; wherein the object name of the first object includes the minimum request time in the map structure data corresponding to the current target storage bucket; According to the object name of the first object, a corresponding key-value pair is added to the key-value map of the second object corresponding to the current target storage bucket stored in the metadata pool; the key of the key-value pair is the object name of the first object, and the value of the key-value pair is empty.

2. The method for storing operation logs of an object storage bucket according to claim 1, wherein: The step of obtaining bucket operation information corresponding to the operation request for the target bucket includes: The object storage server receives a current operation request sent by a business client device; wherein the current operation request is a request to operate a storage bucket in the object storage system; If the storage bucket corresponding to the current operation request is the target storage bucket, then in the process of processing the current operation request, the bucket operation information corresponding to the current operation request is collected and cached.

3. The method for storing operation logs of an object storage bucket according to claim 1, wherein: After storing the bucket operation information corresponding to each target storage bucket using the request time as an index, the method further includes: The object storage server receives a current log query request for a specified bucket from the operation and maintenance client device; Determine whether the specified storage bucket is the target storage bucket; If so, based on the log query time period in the current log query request, the bucket operation information corresponding to the current log query request stored in the specified storage bucket is searched, and the bucket operation information corresponding to the current log query request found is sent to the operation and maintenance client device; wherein, the bucket operation information corresponding to the current log query request is the bucket operation information corresponding to the request time within the log query time period, and the log time information is the log query time period.

4. The method for storing operation logs of an object storage bucket according to claim 1, wherein: The storing of bucket operation information corresponding to each target storage bucket using the request time as an index includes: Using a background processing thread, the bucket operation information corresponding to each target storage bucket cached within the preset time is stored at preset time intervals using the request time in the bucket operation information corresponding to each target storage bucket cached within the preset time as an index.

5. An operation log storage device for an object storage bucket, characterized in that: include: A log acquisition module is configured to acquire bucket operation information corresponding to an operation request for a target bucket; wherein the target bucket is a bucket in the object storage system with bucket logging enabled, and the bucket operation information includes the request time of the operation request and a bucket operation log, wherein the bucket operation log includes the request result and / or call function information; a log storage module, configured to use the request time as an index to store bucket operation information corresponding to each of the target storage buckets, so as to find the bucket operation information of the current target storage bucket corresponding to the stored log time information based on the log time information in the log query request of the current target storage bucket; wherein the current target storage bucket is any of the target storage buckets; The log storage module is specifically configured to use a data pool to store the bucket operation information of each target storage bucket, and use a metadata pool to store the index corresponding to the request time of each target storage bucket, so as to find the target index corresponding to the log time information in the metadata pool according to the log time information in the log query request of the current target storage bucket, and find the bucket operation information of the current target storage bucket corresponding to the target index from the data pool; including: The cache acquisition submodule is used to obtain the map structure data corresponding to the current target storage bucket in the cache; wherein the map structure data corresponding to the current target storage bucket includes the request time corresponding to each bucket operation information of the current target storage bucket and the bucket operation log; A data pool storage submodule, configured to store a first object corresponding to the current target storage bucket in the data pool according to the map structure data; wherein the object name of the first object includes the minimum request time in the map structure data corresponding to the current target storage bucket; The metadata pool storage submodule is used to add a corresponding key-value pair to the key-value map of the second object corresponding to the current target storage bucket stored in the metadata pool according to the object name of the first object; the key of the key-value pair is the object name of the first object, and the value of the key-value pair is empty.

6. An operation log storage device for an object storage bucket, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for storing operation logs in an object storage bucket according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for storing operation logs in an object storage bucket according to any one of claims 1 to 4.

Images