Object data processing methods, systems, and application products

Abstract

This application discloses an object data processing method, system, and program product. The method, applied to an object data management system, includes: a forwarding node receiving a data operation request from a client, the data operation request including a first public IP address; the forwarding node performing load balancing on multiple proxy nodes configured with the first public IP address, determining a first proxy node to handle the data operation request, and forwarding the data operation request to the first proxy node; the first proxy node receiving the data operation request and performing corresponding operations based on local cache information; the local cache information includes: object data information, object data access counts, object data creation time, and a checksum. The method provided by this application can reduce the complexity and cost of operation and maintenance management, and enable the first proxy node to quickly respond to data operation requests, thereby improving the query and download efficiency of object data.

Description

Technical Field

[0001] This disclosure generally relates to the field of data processing technology, and specifically to an object data processing method, system, and program product. Background Technology

[0002] Minio, as an open-source, high-performance object storage software, is widely used in the field of data storage due to its relatively simple deployment and operation, as well as its relatively complete storage services.

[0003] However, when the software is used to store and manage objects in buckets that reach hundreds of thousands or even millions of AWS S3 (i.e., object storage service provided by Amazon Web Services), slow query speeds are likely to occur (for example, when performing the enumeration operation ls, there is a high probability that it will get stuck and unable to return), and queries can easily affect write operations; and storing small files in the minio software can also lead to severe disk fragmentation used by minio, thus affecting query speed.

[0004] Secondly, although minio can support all nodes in the cluster to provide services to the outside world at the same time, when providing services to the outside world, they generally use a single domain name or a single IP (that is, there is only one entry point exposed to the outside world), which makes it impossible to fully utilize the load capacity of all nodes; and in multi-tenant scenarios, if tenants are isolated and managed by allocating different ports to each tenant, it will lead to a significant increase in the complexity of operation and maintenance and the cost of use. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an object data processing method, system and program product. This method reduces the complexity and cost of operation and maintenance management by providing a port to the outside through the forwarding node, and uses the forwarding node to perform load balancing processing on multiple proxy nodes, so as to improve system performance and reliability while optimizing resource utilization. At the same time, it enables the first proxy node to respond quickly to data operation requests, thereby improving the query efficiency and download efficiency of object data.

[0006] In a first aspect, the present invention provides an object data processing method applied to an object data management system, the object data management system including forwarding nodes, multiple proxy nodes, and multiple miniio storage nodes, the method comprising:

[0007] The forwarding node receives data operation requests sent by the client, and the data operation requests include the first public IP address;

[0008] The forwarding node performs load balancing on multiple proxy nodes configured with the first public IP address, determines the first proxy node to handle the data operation request, and forwards the data operation request to the first proxy node.

[0009] The first proxy node receives data operation requests and performs corresponding operations based on local cache information. Local cache information includes: object data information, object data access count, object data creation time, and verification value.

[0010] Secondly, an object data management system is provided, characterized in that the object data management system includes forwarding nodes, multiple proxy nodes, and multiple miniIO storage nodes, wherein at least one proxy node is configured with the same public virtual address, and one of the proxy nodes configured with the same public virtual address is configured with a private virtual address.

[0011] The forwarding node is configured to receive data operation requests sent by clients, the data operation requests including the first public IP address;

[0012] The forwarding node is configured to: perform load balancing on multiple proxy nodes configured with the first public IP address, determine the proxy node to handle data operation requests, and forward the data operation requests to the proxy node;

[0013] The proxy node is configured to receive data operation requests and perform corresponding operations based on local cache information. The local cache information includes: object data information, object data access count, object data creation time, and verification value.

[0014] Thirdly, a computer program product is provided, which contains instructions that, when the instructions are executed, are performed any of the methods described in the first aspect above.

[0015] Fourthly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method of any one of the first aspects above.

[0016] Fifthly, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method of any one of the first aspects above.

[0017] Compared to existing technologies that allocate different ports to different tenants, the object data processing method, system, and program products provided in this application, on the one hand, provide a port to the outside through a forwarding node to receive and process data operation requests sent by clients, thereby reducing the complexity and cost of operation and maintenance management; on the other hand, utilize the forwarding node to perform load balancing on multiple proxy nodes, thereby optimizing resource utilization and improving system performance and reliability while determining the proxy node capable of handling data operation requests; in addition, by caching data locally on each proxy node, the first proxy node can quickly respond to data operation requests, thereby improving the query and download efficiency of object data. Attached Figure Description

[0018] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0019] Figure 1 This is a schematic diagram of the architecture of the object data management system 10 provided in this application embodiment;

[0020] Figure 2 This is a flowchart illustrating the object data processing method provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating the response data upload request provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the process for updating or creating local cache information provided in the embodiments of this application;

[0023] Figure 5 This is a schematic diagram of the synchronization data provided in the embodiments of this application;

[0024] Figure 6 This is a flowchart illustrating the response data download request provided in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the cache statistics process provided in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the data cleaning process provided in the embodiments of this application;

[0027] Figure 9 This is a flowchart illustrating the response to a data deletion request provided in an embodiment of this application;

[0028] Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0029] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments. Furthermore, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The terms "first" and "second," etc., in the specification and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects.

[0031] First, the terminology used in this application will be explained as follows:

[0032] (1) MinIO: is a high-performance object storage released under the GNU Affero General Public License v3.0. It is an API compatible with Amazon S3 cloud storage services and can build high-performance infrastructure for machine learning, analytics and application data workloads.

[0033] (2) Single instance: An instance of an object is generated only once during the entire application lifecycle, and all requests (method calls) are handled by this instance; in this embodiment, it can be understood as a process;

[0034] (3) Multiple instances: Composed of multiple single instances; in this embodiment, it can be understood as multiple processes;

[0035] (4) Multi-tenant technology: also known as multi-tenancy technology (SaaS), is a software architecture technology.

[0036] (5) Nginx: is a high-performance web server, reverse proxy server, load balancer and HTTP caching software;

[0037] (6) Port: A logical concept in a computer network used to identify a specific network service or application;

[0038] (7) Bucket: It is a core concept in object storage, used to store and manage objects, similar to a folder in a file system;

[0039] (8) Object: refers to the basic data unit stored in the object storage system, usually in the form of a file; unlike the traditional file system, objects in object storage have greater flexibility and scalability.

[0040] Currently, in multi-tenant scenarios, existing technologies include providing different port access to tenants by starting multiple processes on different ports (i.e., tenant A uses port 1, and tenant B uses port 2); and creating random prefixes for different tenants and appending them to the bucket to distinguish different tenants by a single process (i.e., tenant A's data is stored in the bucket with prefix a_, and tenant B's data is stored in the bucket with prefix b_). In addition, compression processing can be implemented in the front end to speed up data query processing.

[0041] However, in the above solutions, starting multiple processes for different tenants can easily lead to a significant increase in the complexity and cost of maintaining the external management port; using a single instance to create random prefixes for different tenants can easily lead to increased process complexity; compression processing can easily lead to low efficiency; and when the front-end nginx encounters an anomaly, it cannot provide services, thus causing a single point of failure.

[0042] In response, this application provides an object data processing method that can improve query performance when there are too many small files by providing a single external port; enable data synchronization and traffic balancing in the nginx cluster to achieve high availability of the nginx cluster; and prevent errors in cached data caused by rapid changes in the cache when small files are pre-cached, thus ensuring the accuracy of the cached data.

[0043] In one possible implementation, the object data processing method provided in this application is applicable to an object data processing system. Figure 1 This is a schematic diagram of the architecture of the object data management system 10 provided in this application embodiment, as shown below. Figure 1 As shown, the object data management system 10 includes a forwarding node 11, multiple proxy nodes 12, and multiple miniio storage nodes 13.

[0044] For example, forwarding node 11 corresponds to a switch for receiving data operation requests from client 14; at least one proxy node 12 constitutes an nginx cluster and is configured with the same public virtual address VIP. One of the proxy nodes 12 configured with the same public virtual address VIP is configured with a private virtual address VIP. That is, the proxy node 12 configured with the private virtual address VIP can be understood as the master node among at least one proxy node 12.

[0045] In the specific implementation, the forwarding node 11 receives the data operation request sent by the client 14, performs load balancing on the multiple proxy nodes 12 configured with the first public network address, determines the first proxy node 121 that can handle the data operation request, and forwards the data operation request to the first proxy node 121; correspondingly, after receiving the data operation request, the first proxy node 121 that handles the data operation request can perform the corresponding operation based on the local cache information.

[0046] It should be noted that the master node is contained in the first proxy node 121. That is, the first proxy node 121 that can handle data operation requests can be either the master node or the backup node. The local cache information includes the object data information, the number of times the object data is accessed, the creation time of the object data, and the checksum.

[0047] Specifically, load balancing of multiple agent nodes 12 can be achieved through OSPF (Open Shortest Path First, a dynamic routing protocol) to distribute traffic evenly among the agent nodes 12.

[0048] In the object data management system 10 provided in this application, the Nginx cluster and the Minio cluster run on the same machine by running a Minio cluster (i.e., the backend storage layer) on the backend and setting up an Nginx cluster (i.e., the frontend proxy layer) on the frontend. The traffic is evenly distributed to the Nginx cluster through the OSPF protocol. The Nginx cluster stores file metadata simply and forwards business traffic evenly to the Minio cluster, enabling the Minio cluster to obtain business traffic evenly and make full use of it, thereby speeding up the query speed of client data operation requests. In addition, for multi-tenant scenarios, it is no longer necessary to provide different ports to different tenants. Business processing is carried out through a unified port, thereby reducing the complexity of management and maintenance.

[0049] Figure 2 This is a flowchart illustrating the object data processing method provided in an embodiment of this application. Figure 2 As shown, the method may include the following steps:

[0050] Step S201: Forwarding node 11 receives a data operation request sent by client 14. The data operation request includes the first public network address.

[0051] For example, client 14 is configured with an API endpoint for the object storage service (i.e., a business domain name exposed externally) to access the external access endpoint of the object storage service via API. Correspondingly, forwarding node 11 receives data operation requests sent by client 14 through API access.

[0052] For example, the first public IP address included in the data operation request corresponds to the public virtual IP address (VIP) configured by each of the above-mentioned proxy nodes 12. For instance, multiple proxy nodes 12 are configured with a first public IP address corresponding to the data operation request.

[0053] Specifically, data operation requests include at least data upload requests, data download requests, and data deletion requests, but this application does not impose specific limitations on these.

[0054] In step S202, the forwarding node 11 performs load balancing on the multiple proxy nodes 12 configured with the first public network address, determines the first proxy node 121 to handle the data operation request, and forwards the data operation request to the first proxy node 121.

[0055] For example, after receiving the data operation request sent by the client 14, the forwarding node 11 can perform load balancing on multiple proxy nodes 12 to load the data operation request to the first proxy node 121 (i.e., one of the nginx nodes in the above-mentioned nginx cluster) based on the OSPF protocol, thereby realizing the forwarding of the data operation request to the first proxy node 121.

[0056] In step S203, the first proxy node 121 receives a data operation request and performs the corresponding operation based on the local cache information; the local cache information includes: object data information, object data access count, object data creation time, and verification value.

[0057] The object data information includes at least the object data path ( / tenant name / <bucket name> / <object name>). Based on the example content of local cache information, which can also be called a cache statistics table, the information included in the local cache information can be represented as table entries. It should be noted that local cache information only exists on the primary node; it does not exist on the backup node.

[0058] In one possible implementation, after receiving a data operation request, the first proxy node 121 can perform corresponding operations based on the local cache information according to the type of the data operation request.

[0059] For example, when the data operation request is a data upload request, the first proxy node 121 first determines whether it is the master node, and performs the corresponding operation based on the master node's determination result. The data upload request includes the first object data to be uploaded.

[0060] Specifically, when the first proxy node 121 is the master node, it performs the data upload operation; otherwise, it forwards the data upload request to the master node to instruct the master node to perform the data upload operation.

[0061] In one example, the data upload operation includes: clearing the cache space corresponding to the first object data and caching the first object data locally, updating the local cache information corresponding to the first object data in the local cache information maintained by the master node, and sending the first object data to the first storage node among multiple miniio storage nodes 13, instructing the first storage node to store the first object data.

[0062] In another example, the data upload operation includes: caching the first object data locally, creating local cache information corresponding to the first object data in the local cache information maintained by the master node, and sending the first object data to the first storage node among multiple miniio storage nodes 13, instructing the first storage node to store the first object data.

[0063] For example, the first proxy node 121 can determine whether it is a master node by checking whether it has a private network virtual address (VIP). Correspondingly, if the first proxy node 121 has a private network virtual address (VIP), then the first proxy node 121 is determined to be the master node.

[0064] For example, when the data operation request is a data download request, the first proxy node 121 performs the corresponding operation based on the local cache information maintained by the first proxy node 121. The data download request is used to request the download of the second object data. It should be noted that the local cache information maintained by the first proxy node 121 here does not mean that the local cache information exists within the first proxy node 121, but rather that the first proxy node 121 uses the local cache information when performing the corresponding operation.

[0065] Specifically, the first proxy node 121 determines whether the second object data is cached locally.

[0066] Correspondingly, if the second object data is not cached locally, the second object data is requested from the first storage node among the multiple miniio storage nodes 13.

[0067] Correspondingly, if the local cache contains the second object data, the information of the second object data is sent to the master node, and after the master node instructs the first agent node 121 to delete the second object data in the local cache, the second object data is requested from the first storage node among the multiple miniio storage nodes 13.

[0068] Optionally, if the local cache contains the second object data, the information of the second object data is sent to the master node, and the second object data is retrieved from the local cache after the master node instructs the first agent node 121 to maintain or update the local cache.

[0069] For example, when the data operation request is a data deletion request, the first proxy node 121 first determines whether it is the master node, and performs the corresponding operation based on the master node's judgment result. The data deletion request includes the third object data to be deleted.

[0070] Correspondingly, when the first agent node 121 is the master node, it performs the data deletion operation; otherwise, it forwards the data deletion request to the master node to instruct the master node to perform the data deletion operation.

[0071] Specifically, the data deletion operation includes deleting the cached data corresponding to the third object data, resetting the local cache information corresponding to the third object data, and sending a data deletion instruction to the second storage node among the multiple miniio storage nodes 13 to instruct the second storage node to delete the third object data.

[0072] In another embodiment of this application, a specific implementation method is provided when the data operation request is a data upload request.

[0073] In one possible implementation, Figure 3 This is a schematic diagram of the response data upload request provided in an embodiment of this application, such as... Figure 3 As shown, taking the master node corresponding to the master nginx as an example, the specific steps include:

[0074] Step S301: Forwarding node 11 sends the data upload request to the first proxy node 121.

[0075] In step S302, the first proxy node 121 determines whether it is the primary nginx. If not, proceed to step S303; otherwise, proceed to step S304.

[0076] Step S303: Forward the first object data corresponding to the data upload request to the main nginx.

[0077] The first object data can be the uploaded object data packet.

[0078] In step S304, the main nginx checks whether the cache space has cached the corresponding data of the first object. If it does, step S305 is executed; otherwise, step S306 is executed.

[0079] Step S305: The main nginx clears the cache space corresponding to the first object data.

[0080] For example, the main nginx clears the cache space corresponding to the first object data, and at the same time clears the corresponding table entry information in the local cache information.

[0081] In step S306, the main nginx determines the size of the first object data. If the data size is less than or equal to the first preset threshold, then step S307 is executed; otherwise, step S308 is executed.

[0082] For example, the first preset threshold can correspond to the maximum value of the pre-cached files of proxy node 12.

[0083] Step S307: The main nginx caches the first object data locally.

[0084] For example, the master nginx writes the first object data to the nginx cache and sets the cache time to the system-set threshold (i.e., the pre-cache time). At the same time, it calls the updated pre-cache interface to update or create the local cache information corresponding to the first object data in the local cache information maintained by the master node.

[0085] Step S308: Update or create the local cache information corresponding to the first object data in the local cache information maintained by the master node.

[0086] For example, the master nginx updates or creates an in-memory metadata information table to record metadata information corresponding to the first object data; at the same time, it updates or creates a local metadata information file.

[0087] The metadata information entries should at least correspond to the tenant name, bucket name, object name, and object size (in bytes). The format of the local metadata information file should at least correspond to the path object path (example: / tenant name / <bucket name> / <object name>) and object size (in bytes).

[0088] It should be noted that the update or creation operation in this step can correspond to the judgment result of step S304. If the corresponding data of the first object data is cached (for example, the table entry of the first object data corresponds to the local cache information), then the update operation is performed; otherwise, the creation operation is performed.

[0089] In step S309, the main nginx sends the first object data to the first storage node among the multiple miniio storage nodes 13, instructing the first storage node to store the first object data.

[0090] For example, the main nginx can use a consistent hashing algorithm to load the first object data onto the first storage node among multiple miniio storage nodes 13 based on the requested URL, in order to complete the response to the data upload request.

[0091] In another embodiment of this application, a method for updating or creating local cache information corresponding to the first object data is also provided.

[0092] In one possible implementation, corresponding to step S306 above, when the data size of the first object data is less than the first preset threshold, the access count of the first object data is set to the second preset threshold, and the verification value is calculated based on the first object data. At the same time, the cache creation time of the first object data is set to the current time, thereby creating or updating the local cache information of the first object data based on the access count of the first object data, the verification value of the first object data, the cache creation time of the first object data, and the information of the first object data.

[0093] The second preset threshold can be the minimum number of consecutive accesses.

[0094] For example, Figure 4 This is a schematic diagram of the process for updating or creating local cache information provided in the embodiments of this application, such as... Figure 4 As shown, the specific steps include:

[0095] Step S401: Obtain the incoming object information and object data.

[0096] For example, corresponding to step S307 above, after the main nginx caches the first object data locally, it can obtain the passed-in object information and object data by calling the update pre-cached interface.

[0097] Step S402: Query the local cache information and determine whether the table entry exists. If the table entry does not exist, proceed to step S403; otherwise, proceed to step S404.

[0098] For example, local cache information can be queried based on the information in the first object's data.

[0099] Step S403: Create table entries.

[0100] Step S404: Set the number of accesses to the first object data to the minimum number of consecutive accesses.

[0101] The minimum number of consecutive accesses corresponds to the minimum number of consecutive access requests preset by the system.

[0102] Step S405: Set the cache creation time of the first object data to the current time.

[0103] The current time is the current second-level timestamp.

[0104] Step S406: Set the check value md5 to the check value md5 calculated based on the first object data.

[0105] It should be noted that the reason for using the MD5 checksum during the pre-caching process is that the size of the first object data is smaller than the first preset threshold (i.e., the pre-cached file is small), and the calculation speed is faster. The MD5 setting can prevent small files from updating data quickly and frequently during the pre-caching period. The MD5 checksum is not used for non-pre-cached files because non-pre-cached files need to be accessed a certain number of times before they are cached. Hot data files that are accessed a certain number of times normally do not usually update quickly and frequently, so there is no need for MD5 to prevent rapid data updates. In addition, the cache will be cleared before non-hot data files.

[0106] In another embodiment of this application, a specific implementation method for synchronizing data by proxy node 12 is also provided.

[0107] To prevent unintended modifications to local cache information by the pre-cached file (i.e., the corresponding first object data) in the above embodiments, this application uses an MD5 verification mechanism for protection.

[0108] In one possible implementation, the first proxy node 121 requests multiple local cache information from the master node. The multiple local cache information correspond to multiple different object data. The first proxy node 121 is either the master node or the backup node (i.e., both the master and backup nodes perform data synchronization operations).

[0109] For example, for each piece of local cache information, if the MD5 checksum recorded in the first proxy node 121 corresponding to the local cache information is not 0, then it is determined whether the first proxy node 121 has a local cache file corresponding to the local cache information.

[0110] Correspondingly, if a local cache file exists, a new checksum md5 is calculated based on the current actual cache of the first proxy node 121. If the new checksum md5 is different from the checksum md5 recorded in the local cache information, the current actual cache of the first proxy node 121 is deleted. The current actual cache is the object data corresponding to the object data information in the local cache information.

[0111] For example, if the local cache information corresponds to a checksum MD5 value recorded in the first proxy node 121 that is 0, then the hot data recorded in the local cache information is requested from the minio storage node 13, and the hot data is cached locally on the first proxy node 121.

[0112] For example, Figure 5 This is a schematic diagram of the synchronization data provided in the embodiments of this application, such as... Figure 5 As shown, taking the example of starting a synchronization task to the main nginx instance once per second, the specific steps include:

[0113] Step S501: Request local cache information and metadata information from the main nginx.

[0114] For example, the first proxy node 121 obtains metadata information and local cache information (i.e., statistics cache table) from the main nginx.

[0115] Step S502: Update the metadata information to the local machine.

[0116] Step S503: Traverse the cache statistics table.

[0117] Step S504: Obtain the table entries corresponding to each cache statistics table.

[0118] Step S505: Determine whether the md5 checksum recorded in the cache statistics table is 0. If it is 0, proceed to step S506; otherwise, proceed to step S5071.

[0119] Step S506: Determine whether the number of accesses is less than the minimum number of consecutive accesses. If so, repeat step S504 to obtain the next table entry; otherwise, proceed to step S5072.

[0120] Step S5071: Determine if a corresponding local cache file exists locally. If it exists, proceed to step S5081; otherwise, proceed to step S5083.

[0121] Step S5072: Determine if there is a local cache. If there is, repeat step S504 to obtain the next table entry; otherwise, proceed to step S5083.

[0122] Step S5081: Calculate a new checksum md5 based on the current actual cache, and determine whether the new checksum md5 is the same as the checksum md5 recorded in the local cache information. If they are the same, repeat step S504 to obtain the next table entry; otherwise, execute step S5082.

[0123] For example, if the new checksum MD5 is the same as the checksum MD5 recorded in the local cache information, it indicates that the file has not changed; otherwise, it indicates that the file has changed.

[0124] Step S5082: Delete the cache.

[0125] Step S5083: Request the hot data recorded in the local cache information from the miniio storage node 13 to cache it locally.

[0126] Step S509: Set the cache time to the hot data cache time.

[0127] For example, after step S509 is completed, step S504 is repeated to obtain the next entry.

[0128] It should be noted that the synchronization process is only used to handle metadata updates, pre-cache updates, and cache synchronization, while deletion and caching and updating of non-pre-cache data are handled during download.

[0129] In another embodiment of this application, a specific implementation method is provided when the data operation request is a data download request.

[0130] In one possible implementation, when the first proxy node 121 determines that the second object data is not cached locally, it sets the locally maintained reporting flag to the first value.

[0131] Optionally, after deleting the second object data in the local cache according to the instruction of the master node, the first agent node 121 sets the locally maintained reporting identifier to the second value.

[0132] In another possible implementation, after the first proxy node 121 requests the second object data from the first storage node among the multiple miniio storage nodes 13, it reports the information of the second object data to the master node if the reported identifier is a first value, and responds to the master node's instructions.

[0133] For example, Figure 6 This is a flowchart illustrating the response data download request provided in an embodiment of this application, as shown below. Figure 6 As shown, the specific steps include:

[0134] Step S601: Forwarding node 11 sends the data download request to the first proxy node 121.

[0135] In step S602, the first proxy node 121 determines whether the second object data is cached locally. If the second object data is cached, then step S6031 is executed; otherwise, step S6032 is executed.

[0136] Step S6031: Report access information to the master node.

[0137] Step S6032: Set the reporting flag to 0.

[0138] For example, the first value mentioned above corresponds to 0.

[0139] Step S6041: Obtain the response indication from the master node.

[0140] For example, if the master node's response is a delete instruction, then step S6051 is executed; if the master node's response is a keep-cache instruction, then step S6052 is executed; if the master node's response is an update instruction, then step S6053 is executed.

[0141] Step S6051: Delete the cache.

[0142] Step S6052: Response data and end.

[0143] Step S6053: Update the cache time to the hot data cache time.

[0144] Step S606: Set the reporting flag to 1.

[0145] For example, the second value mentioned above corresponds to 1.

[0146] Step S607: Request the second object data from the first storage node among the multiple miniio storage nodes 13.

[0147] Step S608: Determine whether the second object data has been downloaded. If it exists, proceed to step S6091; otherwise, proceed to step S6092.

[0148] Step S6091, response data.

[0149] Step S6092, no response found, end.

[0150] Step S610: Determine if the reporting flag is 1. If it is 1, end the execution process; if it is 0, proceed to step S611.

[0151] Step S611: Report the information of the second object data to the master node.

[0152] For example, the information for the second object data could be the object data path ( / tenant name / <bucket name> / <object name>).

[0153] Step S612: Obtain the response indication from the master node.

[0154] For example, if the master node's response indicates a delete instruction, the execution process ends; if the master node's response indicates a cache retention instruction, step S613 is executed; if the master node's response indicates an update instruction, step S613 is executed.

[0155] Step S613: Cache.

[0156] Step S614: Set the cache time to the hot data cache time.

[0157] In this embodiment of the application, by judging the retention time in the absence of caching, the accurate cache time can be determined when the cache expires or when the load is placed on a machine that has not been accessed and does not have a synchronous cache.

[0158] In another embodiment of this application, a specific implementation method for cache statistics is also provided.

[0159] In one possible implementation, corresponding to step S611 above, after the first proxy node 121 reports the information of the second object data to the master node, the master node can query the local cache information based on the information of the second object data to obtain the access count of the second object data recorded in the local cache information.

[0160] Correspondingly, if the number of accesses is less than the second preset threshold, a deletion instruction is returned to the first proxy node 121 to instruct the first proxy node 121 to delete the second data object cache in the cache; if the number of accesses is greater than or equal to the second preset threshold, the number of accesses is moduloed based on the second preset threshold.

[0161] Specifically, if the remainder result is 0, the cached data creation time of the second object data is updated to the current time, and an update instruction is sent to the first agent node 121 to instruct the first agent node 121 to update the cache time to the current time; if the remainder result is not 0, a cache retention instruction is sent to the first agent node 121.

[0162] In another possible implementation, when the master node determines that no table entry corresponding to the second object data is recorded in the local cache information, it creates a table entry for the second object data and sets the access count of the second object data to 1. This table entry is used to record the local cache information corresponding to the second object data.

[0163] Specifically, the table entries for creating second object data on the master node include object information ( / tenant name / <bucket name> / <object name>), access count, creation time, and MD5 checksum.

[0164] Optionally, if the table entry corresponding to the second object data has already been recorded in the local cache information, the access count of the second object data will be increased by 1.

[0165] For example, Figure 7 This is a schematic diagram of the cache statistics process provided in an embodiment of this application, such as... Figure 7 As shown, taking the cache statistics table corresponding to local cache information as an example, the specific steps include:

[0166] Step S701: The master node provides a reporting interface.

[0167] Step S702: Receive the reporting request from the first agent node 121.

[0168] Step S703: Obtain information about the second object data.

[0169] For example, the information for the second object data could be the object data path ( / tenant name / <bucket name> / <object name>).

[0170] Step S704: Query whether the table entry of the cache statistics table exists based on the information of the second object data. If it does not exist, proceed to step S7051; otherwise, proceed to step S7052.

[0171] Step S7051: Create table entries.

[0172] Step S7052: Increment the access count of the second object data by 1.

[0173] Step S706: Set the access count of the second object data to 1.

[0174] Step S707: Set the creation time to the current second-level timestamp.

[0175] Step S708: Determine whether the number of accesses to the second object data is less than the second preset threshold. If it is less, proceed to step S7091; otherwise, proceed to step S7092.

[0176] For example, the second preset threshold is greater than the number of nginx nodes, such as the minimum number of consecutive accesses.

[0177] Step S7091: Return a deletion instruction to the first agent node 121, instructing the first agent node 121 to delete the second data object cache in the cache.

[0178] Step S7092: Take the remainder of the number of visits based on the second preset threshold.

[0179] For example, the remainder can be obtained by dividing the number of visits by the second preset threshold.

[0180] Step S710: Determine if the remainder is 0. If it is, proceed to step S7111; otherwise, proceed to step S7112.

[0181] Step S7111: Update the cached data creation time to the current second-level timestamp, return the updated cache time, and send an update command to the first agent node 121.

[0182] Step S7112: Send a cache retention instruction to the first agent node 121.

[0183] It should be noted that, regardless of whether the first proxy node 121 is the primary or backup node, it performs caching operations according to the reporting interface provided in step S701.

[0184] In another embodiment of this application, a specific implementation method for the timed task when the first proxy node 121 acts as the master node is also provided.

[0185] In one possible implementation, any local cache information can be obtained to determine the expiration time of the local cache information based on the creation time and preset duration recorded in the local cache information, and the local cache information can be deleted when the system real-time time exceeds the expiration time.

[0186] For example, Figure 8 This is a schematic diagram of the data cleaning process provided in the embodiments of this application, such as... Figure 8 As shown, taking the cache statistics table corresponding to local cache information as an example, the master node executes a scheduled task once per second, which specifically includes the following steps:

[0187] Step S801: Traverse the cache statistics table.

[0188] Step S802: Obtain any table entry.

[0189] Step S803: Add the creation time to the system preset duration to obtain the expiration time.

[0190] For example, the system's preset duration is the statistical time, in seconds.

[0191] Step S804: Obtain the system real-time time.

[0192] Step S805: Determine whether the system real-time time is greater than or equal to the expiration time. If yes, proceed to step S806; otherwise, repeat step S801 to proceed to the next process.

[0193] Step S806: Delete the table entry.

[0194] It should be noted that the timed task provided in this application embodiment is used to control the hot statistics time, that is, it only processes table entries and does not process cache cleanup. Cache cleanup corresponds to automatic expiration and download cleanup operations.

[0195] In another embodiment of this application, a specific implementation of a data deletion request is also provided.

[0196] In one possible implementation, Figure 9 This is a flowchart illustrating the response to a data deletion request provided in an embodiment of this application, as shown below. Figure 9 As shown, taking the nginx corresponding to proxy node 12 as an example, the specific steps include:

[0197] Step S901: Forwarding node 11 sends the data deletion request to the first proxy node 121.

[0198] In step S902, the first proxy node 121 determines whether it is the primary nginx. If not, proceed to step S903; otherwise, proceed to step S904.

[0199] Step S903: Forward the data deletion request to the main nginx.

[0200] Step S904: The main nginx deletes the cache of the third object data corresponding to the data deletion request, clears the table entries in the third object data statistics cache table, and deletes the third object data from the local metadata information.

[0201] Step S905: Send a data deletion command to the second storage node among the multiple miniio storage nodes 13, instructing the second storage node to delete the third object data.

[0202] In another embodiment of this application, a specific implementation of querying the object list is also provided.

[0203] For example, forwarding node 11 receives an object list request sent by client 14, performs load balancing on multiple proxy nodes 12 configured with a first public IP address (VIP) to determine a second proxy node 122 that can handle the object list request, and forwards the object list request to the second proxy node 122. Correspondingly, the second proxy node 122 obtains the corresponding object list through metadata information stored in a local file and returns the object list to client 14 through forwarding node 11.

[0204] Specifically, the above list of objects can be a list of files stored in the minio storage node 13, such as a list of files under the / test directory of a certain tenant.

[0205] Based on this, the second proxy node 122 can create a scheduled update task to update the object list periodically. It should be noted that the request to query the object list is responded to by different ports on the miniio storage node 13 (i.e., switching to the miniio storage node 13 involves accessing different ports).

[0206] In another embodiment of this application, other configurations for the proxy node 12 are also provided.

[0207] In one possible implementation, each proxy node 12 is also configured with a public IP address and a private IP address.

[0208] For example, the object data management system 10 can achieve high availability by using the public IP address configured for each proxy node 12 and the same public virtual address (VIP) for each proxy node 12.

[0209] Specifically, when traffic enters the nginx cluster, the load balancing of each nginx can be achieved based on a public IP address and the same VIP configured for each nginx instance, as well as the OSPF protocol configured between the server and the Layer 3 switch.

[0210] For example, the main nginx can be identified by using the private network IP configured on proxy node 12 and the private network virtual address (VIP) configured on one of the proxy nodes 12.

[0211] Specifically, each nginx node is configured with a private IP address and a private VIP is planned. When using keepalived to configure the nginx cluster, one nginx node can be selected as the master node to start (this master node has a private VIP, while the other nodes do not). When the master node fails, a new master node will be selected from the remaining nodes to migrate the private VIP to the new master node.

[0212] Correspondingly, when any nginx node in the nginx cluster starts a scheduled task, it can continuously check whether its own network card has a private network VIP in order to determine whether it is the master node.

[0213] In one possible implementation, the business domain name rule can be defined as follows: if the tenant name is test and the main domain name used by the object storage is object.com, then the domain name provided to users is test.object.com.

[0214] In another possible implementation, the system threshold setting rule is specifically: pre-caching time < statistical time < hot caching time.

[0215] For example, by using hot data files to reduce repeated data retrieval and cache update time, pre-cached but non-hot data files can be quickly expired and evicted.

[0216] In another embodiment of this application, a pre-cached instance is also provided, which corresponds to a data upload request.

[0217] Taking tenant 'test', request domain 'test.object.com', bucket 'bucket', object 'object', file size '1KB', maximum pre-cached file size '4KB', and pre-caching time '10 seconds' as an example, in an nginx cluster of three nodes A, B, and C, node A is the master node:

[0218] When tenant 'test' initiates an upload of object 'object' to the bucket via test.object.com, the switch load is shifted to node A. Node A, determining itself as the master node, deletes the cache entry with ' / test / bucket / object' as the key and also deletes the entry for ' / test / bucket / object' in the statistics cache table. Next, it compares the file size with the maximum cache file size. If the file size is less than the maximum pre-cached file size, it stores the object in the cache with ' / test / bucket / object' as the key and sets the cache time to the pre-cached time. Then, it calls the update pre-cached interface to write an entry to the in-memory metadata information table: tenant name 'test', bucket name 'bucket', object name 'object', object size '1024', and records this information as ' / test / bucket / object 1024' in the local metadata information file. Finally, it calculates a consistent hash via the request URL http: / / test.object.com / bucket / object and uploads it to a storage node in the miniio cluster, ending the response.

[0219] It should be noted that this application pre-caches files based on file size because uploaded files have a probability of being accessed within a system-set threshold range. Small files consume fewer cache resources, and pre-caching can speed up probabilistic access. Furthermore, the client's perception of transmission speed is more noticeable for small files compared to large files; therefore, probabilistic pre-caching is implemented. Based on this, user experience can be improved while conserving nginx cache resources.

[0220] In another embodiment of this application, a non-pre-cached instance is also provided, which corresponds to a data download request.

[0221] Taking tenant 'test', request domain 'test.object.com', bucket 'bucket', object 'object', statistics time 5 seconds, minimum number of consecutive requests 10, 9 downloads within 5 seconds, and hot file caching time 30 seconds as an example, in an nginx cluster of three nodes A, B, and C, node A is the master node:

[0222] The non-main process includes: when tenant test initiates the 10th request to download the object in the bucket via test.object.com within 5 seconds, the switch load is shifted to node B. At this time, node B has no cache, so it sets the reported flag to 0; then it uses the request URL http: / / test.object.com / bucket / object to calculate a consistent hash and downloads it to a storage node in the miniio cluster to respond to the tenant.

[0223] Since the reported identifier is already 0, node B will report the / test / kubect / object information to node A. At this time, node A will respond by updating the cache time. After obtaining the information, node B will cache the object information with / test / kubect / object as the key and set the cache time to the hot file cache time.

[0224] The main process includes: when node A receives an object access request reported by node B, it retrieves / test / kubect / object, searches for an entry in the statistics cache table using / test / kubect / object, and increments the access count by 1 when the entry is found. If the access count is the same as the minimum consecutive request count, the access count is divided by the minimum consecutive request count to obtain the remainder. If the remainder is equal to 0, the creation time of the entry is updated to the current timestamp, and the updated cache time is returned.

[0225] The object data processing method provided in this application can achieve the cleaning and updating of object change cache through the combination of algorithm and synchronization design, achieve high availability of cluster through OSPF, keepalived and public-private VIP design, speed up access to small files by using pre-caching technology, speed up access to hot files by using intelligent algorithm, and at the same time adopt a decentralized design so that any server can quickly switch to become the master node when the master node is abnormal.

[0226] The following is for reference. Figure 10 , Figure 10 A schematic diagram of a computer device suitable for implementing embodiments of this application is shown, such as... Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage section 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the system's operating instructions. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0227] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0228] Specifically, according to embodiments of this application, the flowchart above refers to... Figures 2-9 The described process can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program contains program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined in the system of this application.

[0229] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0230] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operational instructions of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two connected blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operational instructions, or using a combination of dedicated hardware and computer instructions.

[0231] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor may be described as including a semantic extraction unit, a weight allocation unit, and a determination unit. The names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0232] On the other hand, this application also provides a computer-readable storage medium, which may be included in the computer device described in the above embodiments, or may exist independently and not assembled into the computer device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application. For example, it may execute... Figures 2-9 The steps of the method shown are as follows.

[0233] This application provides a computer program product including instructions that, when executed, cause the method described in this application to be performed. For example, it can execute... Figures 2-9 The steps of the method shown are as follows.

[0234] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An object data processing method characterized by, The method is applied to an object data management system, the object data management system comprising a forwarding node, a plurality of agent nodes and a plurality of minio storage nodes, and the method comprises the following steps: The forwarding node receives a data operation request sent by a client, wherein the data operation request comprises a first public network address; The forwarding node performs load balancing processing on a plurality of agent nodes configured with the first public network address, determines a first agent node for processing the data operation request, and forwards the data operation request to the first agent node; The first agent node receives the data operation request and performs a corresponding operation based on local cache information; the local cache information comprises information of object data, access times of object data, creation time of object data and a check value; If the data operation request is a data upload request, the data upload request comprises first object data to be uploaded, The first agent node performs a corresponding operation based on local cache information, comprising: The first agent node determines whether it is a master node; if yes, it performs a data upload operation, otherwise, it forwards the data upload request to a master node and instructs the master node to perform the data upload operation; The data upload operation comprises clearing a cache space corresponding to the first object data, locally caching the first object data, updating local cache information corresponding to the first object data in local cache information maintained by the master node, or locally caching the first object data and creating local cache information corresponding to the first object data in local cache information maintained by the master node; and sending the first object data to a first storage node in the plurality of minio storage nodes and instructing the first storage node to store the first object data; The updating or creating of the local cache information corresponding to the first object data comprises: obtaining information of the first object data, setting access times of the first object data as a second preset threshold value in a case where a size of the first object data is smaller than a first preset threshold value, calculating the check value according to the first object data, and setting a cache creation time of the first object data as a current time; creating or updating the local cache information of the first object data based on the access times of the first object data, the check value of the first object data, the cache creation time of the first object data and the information of the first object data.

2. The method of claim 1, wherein, The method further comprises: The first agent node requests a plurality of local cache information from the master node, wherein the plurality of local cache information correspond to a plurality of different object data; For each local cache information, if the local cache information corresponds to a check value recorded in the first proxy node is not 0, determine whether the first proxy node exists with the local cache information corresponding to the local cache file; If it exists, a new check value is calculated based on the first proxy node current actual cache; If the new check value is different from the check value recorded in the local cache information, delete the current actual cache; The current actual cache is the object data corresponding to the object data information in the local cache information; If the local cache information corresponds to a check value recorded in the first proxy node is 0, request the hot data recorded in the local cache information to the minio storage node, and cache the hot data in the first proxy node.

3. The method of claim 1, wherein, The first proxy node judges whether itself is a master node, comprising: The first proxy node judges whether itself is configured with a private network virtual address, if the first proxy node is configured with a private network virtual address, it is determined that the first proxy node is a master node.

4. The method according to claim 1 or 3, characterized in that, If the data operation request is a data download request, the data download request is used to request to download the second object data, Then the first proxy node executes corresponding operation based on local cache information, comprising: The first proxy node judges whether the local cache has the second object data, if the local cache does not have the second object data, the first storage node in the plurality of minio storage nodes is requested for the second object data; If the local cache has the second object data, the information of the second object data is sent to the master node, and after the master node instructs the first proxy node to delete the local cache of the second object data, the first storage node in the plurality of minio storage nodes is requested for the second object data; Or, the information of the second object data is sent to the master node, and after the master node instructs the first proxy node to maintain the local cache or update the local cache, the second object data is obtained from the local cache.

5. The method of claim 4, wherein, The method further comprises: The first proxy node determines that the local cache does not have the second object data, sets the reporting identifier maintained by the local cache to a first value, or after the first proxy node deletes the local cache of the second object data according to the instruction of the master node, sets the reporting identifier maintained by the local cache to a second value; After the first proxy node requests the second object data to the first storage node in the plurality of minio storage nodes, if the reporting identifier is the first value, the information of the second object data is reported to the master node, and the indication of the master node is responded.

6. The method of claim 4, wherein, The method further comprises: After the master node receives the information of the second object data, the local cache information is queried according to the information of the second object data, and the access times of the second object data recorded in the local cache information are obtained. if the access frequency is less than a second preset threshold, a deletion instruction is returned to the first proxy node, instructing the first proxy node to delete the second data object cache in the cache; if the access frequency is greater than or equal to the second preset threshold, the access frequency is taken modulo based on the second preset threshold; if the modulo result is 0, the cache data creation time of the second object data is updated to the current time, and an update instruction is sent to the first proxy node, instructing the first proxy node to update the cache time to the current time; if the modulo result is not 0, a cache retention instruction is sent to the first proxy node.

7. The method of claim 6, wherein, The method further comprises: if the local cache information does not record the table item corresponding to the second object data, the table item of the second object data is created, and the access frequency of the second object data is set to 1, the table item being used to record the local cache information corresponding to the second object data; if the local cache information has recorded the table item corresponding to the second object data, the access frequency of the second object data is increased by 1.

8. The method of claim 1, wherein, The method further comprises: acquiring one of the local cache information, and determining the expiration time of the local cache information according to the creation time and the preset time length recorded in the local cache information; acquiring the system real-time time, and if the system real-time time exceeds the expiration time, deleting the local cache information.

9. The method of claim 1, wherein, if the data operation request is a data deletion request, the data deletion request including third object data to be deleted, then the first proxy node performs corresponding operations based on the local cache information, including: the first proxy node judges whether it is the master node; if yes, a data deletion operation is performed, otherwise, the data deletion request is forwarded to the master node, instructing the master node to perform the data deletion operation; the data deletion operation includes: deleting the cache data corresponding to the third object data, resetting the local cache information corresponding to the third object data, and sending a data deletion instruction to a second storage node in the plurality of minio storage nodes, instructing the second storage node to delete the third object data.

10. The method of claim 1, wherein, The method further comprises: The forwarding node receives an object list request sent by a client, the forwarding node performs load balancing processing on a plurality of proxy nodes configured with the first public network address, determines a second proxy node processing the object list request, and forwards the object list request to the second proxy node; The second proxy node acquires the corresponding object list through the metadata information stored in the local file, and returns the object list to the client through the forwarding node.

11. An object data management system, characterized by The object data management system comprises a forwarding node, a plurality of proxy nodes and a plurality of minio storage nodes, at least one of the proxy nodes being configured with the same public network virtual address, one of the proxy nodes configured with the same public network virtual address being configured with a private network virtual address, The forwarding node is configured to receive a data operation request sent by a client, the data operation request including a first public network address; The forwarding node is configured to: perform load balancing on a plurality of proxy nodes configured with the first public network address, determine a first proxy node for processing the data operation request, and forward the data operation request to the first proxy node; The first proxy node is configured to: receive the data operation request, and perform a corresponding operation based on local cache information; The local cache information includes: information of object data, access times of object data, creation time of object data, and a check value; If the data operation request is a data upload request, the data upload request includes first object data to be uploaded, The first proxy node performs a corresponding operation based on local cache information, including: The first proxy node determines whether it is a master node; if yes, it performs a data upload operation, otherwise, it forwards the data upload request to a master node and instructs the master node to perform the data upload operation; The data upload operation includes: clearing cache space corresponding to the first object data, locally caching the first object data, updating local cache information corresponding to the first object data in local cache information maintained by the master node, or locally caching the first object data, and creating local cache information corresponding to the first object data in local cache information maintained by the master node; and sending the first object data to a first storage node in the plurality of minio storage nodes, instructing the first storage node to store the first object data; The updating or creating of the local cache information corresponding to the first object data includes: obtaining information of the first object data, setting access times of the first object data to a second preset threshold value in a case where a size of the first object data is less than a first preset threshold value, calculating the check value according to the first object data, and setting cache creation time of the first object data to a current time; creating or updating the local cache information of the first object data based on the access times of the first object data, the check value of the first object data, the cache creation time of the first object data, and the information of the first object data.

12. A computer program product, characterised in that, The computer program product includes instructions that, when executed, cause the method of any one of claims 1-10 to be implemented.

Images