Log recording methods, devices, electronic devices and computer-readable storage media

Abstract

This application discloses a logging method, comprising: upon receiving an access request for a target storage bucket, determining a log storage bucket corresponding to the target storage bucket based on the access request; generating an access log based on the access request and its processing result; storing the access log in a preset storage space and determining the storage location of the access log in the preset storage space; generating a log index for the access log based on the storage location; and storing the log index in the log storage bucket. By applying the technical solution provided in this application, logging of storage buckets within an object storage system can be implemented. This application also discloses a logging device, an electronic device, and a computer-readable storage medium, which also possess the aforementioned technical effects.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a logging method, a logging device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Currently, in object storage applications, customers require audit logs for written data to facilitate operation tracking and troubleshooting in case of anomalies. From a data security perspective, storage systems also need to support audit logs for bucket and object-related operations to meet the specific needs of certain customer scenarios. However, current object storage systems only support recording system logs and object service logs for troubleshooting, but do not record audit logs for bucket configuration and object management operations.

[0003] Therefore, how to implement logging of storage buckets within an object storage system is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a logging method that can realize logging of buckets within an object storage system; another purpose of this application is to provide a logging device, electronic device, and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0005] Firstly, this application provides a logging method, including:

[0006] When an access request for a target storage bucket is received, the log storage bucket corresponding to the target storage bucket is determined based on the access request;

[0007] An access log is generated based on the access request and the processing result of the access request;

[0008] The access log is stored in a preset storage space, and the storage location of the access log in the preset storage space is determined.

[0009] Generate a log index for the access log based on the storage location;

[0010] The log index is stored in the log storage bucket.

[0011] Optionally, storing the access log to a preset storage space includes:

[0012] Store the access logs in a preset cache;

[0013] Determine whether the access logs in the preset cache meet the preset flush conditions;

[0014] If so, all access logs in the preset cache will be flushed to the preset storage space.

[0015] Optionally, determining whether the access logs in the preset cache meet the preset flush conditions includes:

[0016] Determine whether the number of access logs in the preset cache has reached a preset number, or whether the cache duration of the preset cache has reached a preset duration;

[0017] If the number of access logs reaches the preset number, or the cache duration reaches the preset duration, then it is determined that the access logs in the preset cache meet the preset flushing conditions.

[0018] Optionally, the preset storage space is a data pool, and the step of flushing all access logs in the preset cache to the preset storage space includes:

[0019] All access logs in the preset cache are flushed to the data pool in the form of striped logs.

[0020] Optionally, after flushing all access logs in the preset cache to the data pool in the form of striped logs, the process further includes:

[0021] Generate the stripe identifier for the striped log;

[0022] Accordingly, generating the log index of the access log based on the storage location includes:

[0023] The object structure diagram in the log pool is inserted into the corresponding storage location and the stripe identifier to generate the log index of the access log.

[0024] Optionally, generating access logs based on the access request and the processing result of the access request includes:

[0025] Determine the request type and resource access information based on the access request;

[0026] The processing result information and timestamp information are determined based on the processing result of the access request;

[0027] The access log is generated based on the request type, the resource access information, the processing result information, and the timestamp information.

[0028] Optionally, determining the log storage bucket corresponding to the target storage bucket based on the access request includes:

[0029] The target storage bucket is determined based on the access request;

[0030] The log storage bucket corresponding to the target storage bucket is determined by querying the preset storage bucket mapping relationship; the preset storage bucket mapping relationship is the mapping relationship between accessible storage buckets and log storage buckets.

[0031] Optionally, when an access request for a target storage bucket is received, determining the log storage bucket corresponding to the target storage bucket based on the access request includes:

[0032] When an access request for the target storage bucket is received, determine whether the bucket log function is enabled;

[0033] If so, the log storage bucket corresponding to the target storage bucket is determined based on the access request.

[0034] Optionally, the logging method further includes:

[0035] When a bucket log query request is received, the target query bucket and the target log bucket corresponding to the target query bucket are determined based on the bucket log query request.

[0036] Based on the bucket log query request, query the access log of the target query bucket in the log storage bucket.

[0037] Optionally, the logging method further includes:

[0038] Perform preset operations on the access logs; the preset operations include download operations and / or deletion operations.

[0039] Secondly, this application also discloses a log recording device, comprising:

[0040] The determination module is used to determine the log storage bucket corresponding to the target storage bucket based on the access request when an access request for the target storage bucket is received.

[0041] The first generation module is used to generate access logs based on the access request and the processing result of the access request;

[0042] The first storage module is used to store the access log to a preset storage space and determine the storage location of the access log in the preset storage space;

[0043] The second generation module is used to generate a log index for the access log based on the storage location;

[0044] The second storage module is used to store the log index into the log storage bucket.

[0045] Thirdly, this application also discloses an electronic device, comprising:

[0046] Memory, used to store computer programs;

[0047] A processor for executing the computer program to implement any of the logging methods described above.

[0048] Fourthly, this application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the logging methods described above.

[0049] This application provides a logging method, comprising: when receiving an access request for a target storage bucket, determining a log storage bucket corresponding to the target storage bucket based on the access request; generating an access log based on the access request and the processing result of the access request; storing the access log in a preset storage space and determining the storage location of the access log in the preset storage space; generating a log index of the access log based on the storage location; and storing the log index in the log storage bucket.

[0050] By applying the technical solution provided in this application, for a target storage bucket that needs to be logged, a corresponding log storage bucket is pre-set. When an access request for the target storage bucket is received, the corresponding log storage bucket can be determined based on the access request, and an access log for this access request can be generated based on the processing result. Furthermore, a corresponding log index is generated based on the storage location of the access log in the preset storage space and saved to the log storage bucket. Thus, log data storage for the storage bucket is realized based on the preset storage space, and log index storage for the storage bucket is realized based on the log storage bucket, so that bucket log data can be queried based on the log index later. It can be seen that this technical solution realizes the logging of storage buckets in the object storage system, effectively meeting user needs.

[0051] In one embodiment, a cache-based temporary log implementation is adopted. After the access logs are collected, they are first stored in a preset cache. Then, when the access logs in the preset cache meet the preset flushing conditions, a batch of access logs are flushed uniformly. This can effectively avoid log flushing conflicts and thus avoid the problem of log data loss.

[0052] In one embodiment, an object storage approach is used to store the log index. The log identifier and storage location of the striped log are generated into a data pair and inserted into the object structure graph in the log pool to generate the log index. This achieves log index storage based on object storage, which makes it easier to perform log query processing later.

[0053] In one embodiment, before logging the target storage bucket, it is first determined whether the bucket logging function of the current system is enabled, and then it is determined whether bucket logging needs to be performed based on the enabled status. It can be seen that the object storage system in this embodiment supports the custom settings of the bucket logging function, which makes it convenient for users to determine whether to enable the bucket logging service according to their actual needs, effectively meeting the different needs of users.

[0054] The logging device, electronic device, and computer-readable storage medium provided in this application also have the above-mentioned technical effects, and will not be described in detail here. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the prior art and the embodiments of this application, the accompanying drawings used in the description of the prior art and the embodiments of this application will be briefly introduced below. Of course, the accompanying drawings described below with respect to the embodiments of this application are only a part of the embodiments in this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and such other drawings also fall within the protection scope of this application.

[0056] Figure 1 A flowchart illustrating a logging method provided in this application;

[0057] Figure 2 A flowchart illustrating another logging method provided in this application;

[0058] Figure 3 A schematic diagram of a log recording device provided in this application;

[0059] Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0060] The core of this application is to provide a logging method that can realize logging of buckets within an object storage system; another core aspect of this application is to provide a logging device, electronic device, and computer-readable storage medium, all of which have the aforementioned beneficial effects.

[0061] To provide a clearer and more complete description of the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0062] This application provides a log recording method.

[0063] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a logging method provided in this application, which may include the following steps S101 to S105.

[0064] S101: When an access request for the target storage bucket is received, the log storage bucket corresponding to the target storage bucket is determined based on the access request.

[0065] This step aims to determine the log bucket. For an object storage system, when it receives an access request for a target bucket, it can determine the corresponding log bucket based on the request. The target bucket is the bucket that needs to be logged; that is, all access requests processed by it need to be logged for subsequent operation tracking and troubleshooting. Access requests can be various data processing requests initiated by users through the client (such as op requests, preflight requests), including but not limited to upload requests, modification requests, and deletion requests. The log bucket corresponds to the target bucket, and its function is to record relevant log information about access requests to the target bucket. The relationship between target buckets and log buckets can be one-to-many, meaning one log bucket can be used to log multiple target buckets.

[0066] S102: Generate access logs based on access requests and the processing results of access requests.

[0067] This step aims to generate access logs, specifically, to generate access logs corresponding to access requests. Specifically, to ensure the completeness of the recorded access log information, access logs can be generated by combining the access request and its processing result. The specific content of the access logs can be set by technical personnel according to the actual situation; for example, it may include the request type, reception time, processing duration, and processing result of the access request, etc., which is not limited in this application.

[0068] S103: Store the access log to the preset storage space and determine the storage location of the access log in the preset storage space.

[0069] This step aims to store the access log. Specifically, after generating the access log corresponding to the access request, the access log can be stored in a preset storage space, and its storage location within that preset storage space can be determined. The preset storage space is a pre-created storage space used to record bucket log data (i.e., access logs about the target bucket), which can specifically be a data pool. Correspondingly, the storage location of the access log in the preset storage space is its storage location within the data pool (e.g., offset address).

[0070] S104: Generate a log index for the access log based on the storage location.

[0071] This step aims to generate a log index for the access logs. Specifically, after determining the storage location of the access logs in the preset storage space, a log index can be generated using that location. This log index is used to implement subsequent log query functions; therefore, it must contain the storage location of the access logs in the preset storage space. Of course, the log index can also contain other types of data information for subsequent log queries; its specific content can be set according to the actual situation.

[0072] S105: Store the log index in the log storage bucket.

[0073] This step aims to store the log index, specifically the log index of accessed logs, in the log bucket. Thus, log data storage for the bucket is achieved based on the preset storage space, and the log index for the bucket is stored accordingly. Therefore, subsequent query functionality for bucket log data can be implemented based on the log index.

[0074] As can be seen, the logging method provided in this application pre-sets a corresponding log storage bucket for the target storage bucket that needs to be logged. When an access request for the target storage bucket is received, the corresponding log storage bucket can be determined based on the access request, and an access log for this access request can be generated based on the processing result. Furthermore, a corresponding log index is generated based on the storage location of the access log in the preset storage space and saved to the log storage bucket. Thus, log data storage of the storage bucket is realized based on the preset storage space, and log index storage of the storage bucket is realized based on the log storage bucket, so that the bucket log data can be queried based on the log index later. It can be seen that this technical solution realizes the logging of storage buckets in the object storage system and effectively meets the user's needs.

[0075] Based on the above embodiments:

[0076] In one embodiment of this application, storing the access log to a preset storage space may include:

[0077] Store access logs in a preset cache;

[0078] Determine whether the access logs in the preset cache meet the preset flush conditions;

[0079] If so, all access logs in the preset cache will be flushed to the preset storage space.

[0080] This application provides an implementation method for storing access logs to a preset storage space, namely, an implementation method based on cached temporary log storage. Specifically, after the access logs are collected, they can be stored in a preset cache first. Then, it is determined whether the access logs in the preset cache meet the preset flushing conditions. The preset flushing conditions refer to the pre-set execution conditions that allow the access logs in the preset cache to be flushed to the preset storage space. The specific content can be customized by technicians according to actual needs. Furthermore, when the access logs in the preset cache meet the preset flushing conditions, all access logs in the preset cache can be flushed to the preset storage space in a unified manner, thereby realizing access log storage.

[0081] As can be seen, the embodiments of this application adopt a method based on cached temporary logs. After the access logs are collected, they are first stored in a preset cache. Then, when the access logs in the preset cache meet the preset flushing conditions, the batch access logs are flushed uniformly. This can effectively avoid log flushing conflicts and thus avoid the problem of log data loss.

[0082] In one embodiment of this application, determining whether the access logs in the preset cache meet the preset flushing conditions may include:

[0083] Determine whether the number of access logs in the preset cache has reached the preset number, or whether the cache duration of the preset cache has reached the preset duration;

[0084] If the number of access logs reaches the preset number, or the cache duration reaches the preset duration, then the access logs in the preset cache are determined to meet the preset flushing conditions.

[0085] This application provides a specific type of preset flushing condition, including: whether the number of access logs in the preset cache reaches a preset quantity, and whether the caching duration of the preset cache reaches a preset duration. Therefore, when determining whether the access logs in the preset cache meet the preset flushing condition, specifically, it can be determined whether the number of access logs stored in the preset cache (access log quantity) reaches a preset quantity, or whether the caching duration of the access logs in the preset cache (caching duration) reaches a preset duration. If either condition is met, it can be determined that the access logs in the preset cache have met the preset flushing condition, and the flushing of access log data can continue. Furthermore, the values ​​of the preset quantity and preset duration do not affect the implementation of this technical solution; they can be set by those skilled in the art according to actual conditions, and this application does not limit them in this regard.

[0086] In one embodiment of this application, the preset storage space can be a data pool. The above-mentioned flushing of all access logs in the preset cache to the preset storage space may include: flushing all access logs in the preset cache to the data pool in the form of striped logs.

[0087] This application provides a specific type of preset storage space, namely a data pool. Based on this, when the access logs in the preset cache are flushed down, they can be flushed down to the data pool in the form of striped logs. Each flushing operation corresponds to a striped log.

[0088] In one embodiment of this application, after flushing all access logs in the preset cache to the data pool in the form of striped logs, it may further include: generating stripe identifiers for the striped logs;

[0089] Accordingly, the log index for generating access logs based on storage location may include: inserting the storage location and stripe identifier into the object structure graph in the log pool to generate the log index for access logs.

[0090] This application provides a method for generating a log index. As described above, when flushing access logs in a preset cache, they can be flushed to the data pool in the form of striped logs. Each flush operation corresponds to all access logs as a striped log. Based on this, a stripe identifier, such as a stripe ID (Identity document), can be generated for the striped log. Thus, the log index can be generated based on the storage location of the access logs in the data pool and the stripe identifier. Specifically, the storage location and stripe identifier can be inserted into the object structure graph (omap) in the log pool to generate the log index.

[0091] As can be seen, in this embodiment, the log index is stored using object storage. The log identifier and storage location of the striped log are generated into a data pair and inserted into the object structure graph in the log pool to generate the log index. This realizes log index storage based on object storage, which makes it easier to perform log query processing later.

[0092] In one embodiment of this application, generating access logs based on access requests and the processing results of those access requests may include:

[0093] Determine the request type and resource access information based on the access request;

[0094] The processing result information and timestamp information are determined based on the processing result of the access request;

[0095] An access log is generated based on the request type, resource access information, processing result information, and timestamp information.

[0096] This application provides a method for generating access logs, the content of which may include request type, resource access information, processing result information, and timestamp information. The request type and resource access information can be determined based on the access request, while the processing result information and timestamp information can be determined based on the processing result of the access request. The request type may include upload requests, deletion requests, modification requests, etc.; resource access information refers to the resource information specified in the access request (including but not limited to resource type, resource storage location, etc.); the processing result information is the final processing result of the access request; and the timestamp information may include the time node when the access request was received, the time node when the access request was completed, and the processing duration of the access request.

[0097] In one embodiment of this application, determining the log storage bucket corresponding to the target storage bucket based on the access request may include:

[0098] Determine the target storage bucket based on the access request;

[0099] The target bucket is determined by querying the preset bucket mapping relationship to find the corresponding log bucket; the preset bucket mapping relationship is the mapping relationship between accessible buckets and log buckets.

[0100] This application provides an implementation method for determining the log bucket corresponding to a target bucket. Specifically, a mapping relationship between accessible buckets and log buckets can be pre-created, i.e., the aforementioned preset mapping relationship. For example, multiple accessible buckets correspond to one log bucket, or one accessible bucket corresponds to one log bucket. The target bucket is any one of all accessible buckets. Thus, the log bucket corresponding to the target bucket can be determined by querying the preset mapping relationship.

[0101] In one embodiment of this application, the process of determining the log storage bucket corresponding to the target storage bucket based on the access request when an access request for the target storage bucket is received may include:

[0102] When an access request for the target storage bucket is received, determine whether the bucket log function is enabled;

[0103] If so, determine the log storage bucket corresponding to the target storage bucket based on the access request.

[0104] In this embodiment, the object storage system supports custom settings for bucket logging, allowing users to enable or disable the bucket logging function according to their actual needs. Specifically, when an access request for a target storage bucket is received, before determining the corresponding log storage bucket based on the access request, it can first determine whether the bucket logging function is currently enabled. If the bucket logging function is enabled, it means that bucket logging services are currently required, and therefore, the step of determining the corresponding log storage bucket based on the access request can continue. Conversely, if the bucket logging function is not enabled, it means that bucket logging services are currently not required, and the access request can continue, but logging is necessary. Furthermore, by default, the bucket logging function can be predefined as disabled.

[0105] Clearly, the object storage system in this embodiment supports custom settings for the bucket log function, allowing users to determine whether to enable the bucket log recording service based on their actual needs, effectively meeting different user requirements.

[0106] In one embodiment of this application, the logging method may further include:

[0107] When a bucket log query request is received, the target query bucket and the target log bucket corresponding to the target query bucket are determined based on the bucket log query request.

[0108] Based on the bucket log query request, query the access logs of the target bucket in the log storage bucket.

[0109] The logging method provided in this application embodiment can further implement bucket log query functionality. Specifically, when a user needs to perform a bucket log query, they can initiate a log query request through a client. For an object storage system, the target query bucket and its corresponding target log bucket can be determined based on the bucket log query request. The target query bucket is the storage bucket for which the user needs to perform the log query, and the target log bucket is the log storage bucket of the storage bucket for which the log query is performed, recording the access logs of the storage bucket for which the log query is performed. Therefore, the access logs of the target query bucket can be retrieved from the log storage bucket based on the bucket log query request, thus realizing bucket log query.

[0110] In one embodiment of this application, the logging method may further include: performing a preset operation on the access log; the preset operation includes a download operation and / or a deletion operation.

[0111] The log recording method provided in this application embodiment can further implement bucket log management operations, namely the aforementioned preset operations. The preset operations may include, but are not limited to, download operations and deletion operations. The download operation can be used to download the access logs of the target query bucket from the log storage bucket, and the deletion operation can delete the access logs of the target query bucket from the log storage bucket.

[0112] Based on the above embodiments, this application provides another logging method.

[0113] Please refer to Figure 2 , Figure 2 A flowchart illustrating another logging method provided in this application is shown below, and its implementation process is as follows:

[0114] 1. By default, the bucket logging feature in Radosgw (an object storage system) is disabled. You can enable the bucket logging feature through custom settings.

[0115] 2. With bucket logging enabled, when an op request (preflight request) arrives at Radosgw, a management thread (such as the bl thread) is started to record its access log and save it to the thread cache. The op request then completes. The recorded access log may include information such as the request type, the resource specified in the request, and the request processing time and date. Each op request adds a log entry to the thread cache.

[0116] 3. When there is continuous business pressure, the access logs in the thread cache keep increasing. When an operator inserts data into the thread cache, it can determine whether to actively flush the content in the thread cache to the data pool using the following two conditions:

[0117] (1) When the number of access log entries in the thread cache exceeds the maximum number of entries, a refresh is triggered;

[0118] (2) When the time interval between the current time and the last time refreshed exceeds the maximum interval time, refresh is triggered.

[0119] When the thread cache meets any of the above flush conditions, the management thread can read the thread cache and flush all its contents to the data pool, saving it as a log object stripe. The log object stripe is named oid = ${tenant_name}:{bucket_name}:${date}:${hostname}, where tenant_name is the tenant name of the source bucket (target storage bucket), bucket_name is the source bucket name, date is the timestamp in "year month day hour minute second.millisecond", and hostname is the host node name.

[0120] 4. After the flush is complete, a record corresponding to the oid can be inserted into the rados object omap (object structure graph) of the log pool to generate a log index. Here, the rados object name rados_oid = ${tenant_name}:{bucket_name}:${date}, where date is "year, month, day, hour, minute, second.nanosecond", and both seconds and nanoseconds are 0. This rados object can be called the log object header. The omap of this log object header records the log object stripe names and their storage locations in the data pool for the access logs within a preset duration (e.g., one minute) corresponding to date, used to index the log object stripes within the preset duration.

[0121] 5. In the log pool, after the log object header is inserted for the first time, an object will be generated in the destination log bucket (log storage bucket). The object name object_name is rados_oid. At this time, the object in the destination log bucket can be downloaded and deleted, and log queries can also be performed through the RestAPI interface (an application programming interface).

[0122] This completes the logging and querying functionality for buckets within the object storage system.

[0123] Furthermore, it should be noted that, in order to implement the above technical solution, the following constraints need to be pre-set for the object storage system:

[0124] 1. When the bucket log function is enabled, the destination bucket for log storage must be specified. The two buckets (the storage bucket for logging and the storage bucket for recording bucket logs) cannot be the same bucket and must belong to the same user.

[0125] 2. The bucket logging function and the bucket as a destination are mutually exclusive. That is, if a bucket is already a destination bucket for log storage, the bucket logging function cannot be enabled. If the bucket logging function is already enabled, the bucket cannot be designated as a destination bucket for log storage.

[0126] 3. If a bucket has already been designated as the destination bucket, it cannot be used as a normal storage bucket. Therefore, a new empty bucket needs to be created for the destination bucket. Objects in this bucket can only be downloaded and deleted, and are incompatible with other attributes.

[0127] 4. RestAPI queries have a delay. You can set a preset time delay after the log upload is complete before querying the logs to ensure the integrity of the retrieved data.

[0128] As can be seen, the logging method provided in this application pre-sets a corresponding log storage bucket for the target storage bucket that needs to be logged. When an access request for the target storage bucket is received, the corresponding log storage bucket can be determined based on the access request, and an access log for this access request can be generated based on the processing result. Furthermore, a corresponding log index is generated based on the storage location of the access log in the preset storage space and saved to the log storage bucket. Thus, log data storage of the storage bucket is realized based on the preset storage space, and log index storage of the storage bucket is realized based on the log storage bucket, so that the bucket log data can be queried based on the log index later. It can be seen that this technical solution realizes the logging of storage buckets in the object storage system and effectively meets the user's needs.

[0129] This application provides a log recording device.

[0130] Please refer to Figure 3 , Figure 3 This application provides a schematic diagram of the structure of a log recording device, which may include:

[0131] Module 1 is used to determine the log storage bucket corresponding to the target storage bucket based on the access request when an access request for the target storage bucket is received.

[0132] The first generation module 2 is used to generate access logs based on access requests and the processing results of access requests;

[0133] The first storage module 3 is used to store the access logs to a preset storage space and determine the storage location of the access logs in the preset storage space;

[0134] The second generation module 4 is used to generate a log index for the access log based on the storage location;

[0135] The second storage module 5 is used to store the log index to the log storage bucket.

[0136] As can be seen, the log recording device provided in this application pre-sets a corresponding log storage bucket for the target storage bucket that needs to be logged. When an access request for the target storage bucket is received, the device can determine the corresponding log storage bucket based on the access request, and generate an access log for this access request based on the processing result. Furthermore, a corresponding log index is generated based on the storage location of the access log in the preset storage space and saved to the log storage bucket. Thus, log data storage of the storage bucket is realized based on the preset storage space, and log index storage of the storage bucket is realized based on the log storage bucket, so that the bucket log data can be queried based on the log index later. Therefore, this technical solution realizes the log recording of storage buckets in the object storage system, effectively meeting user needs.

[0137] In one embodiment of this application, the first storage module 3 may include:

[0138] Storage unit, used to store access logs to a preset cache;

[0139] The judgment unit is used to determine whether the access logs in the preset cache meet the preset flush conditions;

[0140] The flush unit is used to flush all access logs in the preset cache to the preset storage space if the access logs in the preset cache meet the preset flush conditions.

[0141] In one embodiment of this application, the aforementioned determination unit may be specifically used to determine whether the number of access logs in the preset cache reaches a preset number, or whether the cache duration of the preset cache reaches a preset duration; if the number of access logs reaches a preset number, or the cache duration reaches a preset duration, then it is determined that the access logs in the preset cache meet the preset flushing conditions.

[0142] In one embodiment of this application, the preset storage space is a data pool, and the aforementioned flushing unit can be specifically used to flush all access logs in the preset cache to the data pool in the form of striped logs.

[0143] In one embodiment of this application, the log recording device may further include an identifier generation module, used to generate a stripe identifier for the stripe log after all access logs in the preset cache are flushed to the data pool in the form of stripe logs;

[0144] Accordingly, the second generation module 4 can be specifically used to insert the storage location and stripe identifier into the object structure diagram in the log pool to generate the log index of the access log.

[0145] In one embodiment of this application, the first generation module 2 described above can be specifically used to determine the request type and resource access information based on the access request; determine the processing result information and timestamp information based on the processing result of the access request; and generate an access log based on the request type, resource access information, processing result information, and timestamp information.

[0146] In one embodiment of this application, the determination module 1 can be specifically used to determine the target storage bucket based on the access request; and to query and determine the log storage bucket corresponding to the target storage bucket based on a preset storage bucket mapping relationship; the preset storage bucket mapping relationship is the mapping relationship between accessible storage buckets and log storage buckets.

[0147] In one embodiment of this application, the determination module 1 can be specifically used to determine whether the bucket log function is enabled when an access request for the target storage bucket is received; if so, the log storage bucket corresponding to the target storage bucket is determined according to the access request.

[0148] In one embodiment of this application, the log recording device may further include a query module for performing preset operations on access logs; the preset operations include download operations and / or deletion operations. Upon receiving a bucket log query request, the system determines the target query bucket and the corresponding target log bucket based on the bucket log query request; and queries the access logs of the target query bucket in the log storage bucket according to the bucket log query request.

[0149] In one embodiment of this application, the logging device may further include an operation module for performing preset operations on the access log; the preset operations include download operations and / or deletion operations.

[0150] For a description of the apparatus provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0151] This application provides an electronic device.

[0152] Please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an electronic device, which may include:

[0153] Memory, used to store computer programs;

[0154] A processor is used to execute computer programs and implement the steps of any of the logging methods described above.

[0155] like Figure 4 The diagram shows the structural composition of an electronic device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0156] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0157] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the logging method.

[0158] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0159] When an access request for the target storage bucket is received, the corresponding log storage bucket is determined based on the access request;

[0160] An access log is generated based on the access request and the processing result of the access request.

[0161] Store the access logs to a preset storage space and determine the storage location of the access logs within the preset storage space;

[0162] A log index is generated based on the storage location of the access logs;

[0163] Store the log index in the log storage bucket.

[0164] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0165] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0166] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0167] Of course, it should be noted that, Figure 4 The structure shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than Figure 4 More or fewer components as shown, or combinations of certain components.

[0168] This application provides a computer-readable storage medium.

[0169] The computer-readable storage medium provided in this application embodiment stores a computer program, which, when executed by a processor, can implement the steps of any of the logging methods described above.

[0170] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] For a description of the computer-readable storage medium provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0175] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method of logging, characterized by, The method comprises the following steps: When receiving an access request for a target storage bucket, determining a log storage bucket corresponding to the target storage bucket according to the access request; the target storage bucket is a storage bucket that needs to be logged; each storage bucket has a corresponding relationship with each log storage bucket, and the corresponding storage bucket and log storage bucket belong to the same user; Generating an access log according to the access request and the processing result of the access request; Storing the access log in a preset storage space and determining the storage location of the access log in the preset storage space; Generating a log index of the access log according to the storage location; Storing the log index in the log storage bucket; When receiving a bucket log query request, determining a target query bucket and a target log bucket corresponding to the target query bucket according to the bucket log query request; the target query bucket is a storage bucket that needs to be logged; the target log bucket is a log storage bucket of the storage bucket that needs to be logged, and the target log bucket records the access log of the storage bucket that needs to be logged; When the log uploading completion time of the target log bucket reaches a preset time length, querying the access log of the target query bucket in the log storage bucket according to the bucket log query request by using a RestAPI interface; The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

2. The logging method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

3. The logging method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

4. The logging method of claim 1, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises According to a preset storage bucket mapping relationship query, the target storage bucket corresponding log storage bucket is determined.

5. The logging method of claim 1, wherein, When the access request about the target storage bucket is received, the target storage bucket corresponding log storage bucket is determined according to the access request, including: When the access request about the target storage bucket is received, it is judged whether the bucket log function is started. If yes, the target storage bucket corresponding log storage bucket is determined according to the access request.

6. The logging method of claim 1, wherein, Also including: The preset operation is performed on the access log; the preset operation includes download operation and / or deletion operation.

7. A logging device characterized by Including: The determination module is used for determining the target storage bucket corresponding log storage bucket according to the access request when the access request about the target storage bucket is received; the target storage bucket is the storage bucket needing log recording; each storage bucket and each log storage bucket have corresponding relationship, and the corresponding storage bucket and log storage bucket belong to the same user; The first generation module is used for generating access log according to the access request and the processing result of the access request; The first storage module is used for storing the access log to the preset storage space and determining the storage position of the access log in the preset storage space; The second generation module is used for generating the log index of the access log according to the storage position; The second storage module is used for storing the log index to the log storage bucket; The query module is used for determining the target query bucket and the target log bucket corresponding to the target query bucket according to the bucket log query request when the bucket log query request is received; the target query bucket is the storage bucket needing log query; the target log bucket is the log storage bucket of the storage bucket needing log query, and the target log bucket records the access log of the storage bucket needing log query; when the log uploading completion time of the target log bucket reaches the preset time length, the access log of the target query bucket is queried in the log storage bucket according to the bucket log query request by using RestAPI interface; The first storage module is specifically used for storing the access log to the preset cache; judging whether the access log in the preset cache meets the preset flushing condition; if yes, all access logs in the preset cache are flushed to the preset storage space; The preset storage space is a data pool, and the first storage module is specifically used for flushing all access logs in the preset cache to the data pool in the form of strip log, and generating the strip identification of the strip log; Correspondingly, the second generation module is specifically used for inserting the storage position and the strip identification into the object structure diagram in the log pool to generate the log index of the access log.

8. An electronic device, comprising: Including: Memory, used for storing computer program; Processor, used for executing the computer program to realize the steps of the log recording method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the log recording method in any one of claims 1 to 6.

Images