Data prefetching method, computing node and storage system

Abstract

The embodiment of the invention provides a data prefetching method, a computing node and a storage system. The method comprises the steps that the computing node obtains access information of a first application to the storage node within a preset time period; the computing node determines information of prefetched data based on the access information; the computing node determines a cache node for prefetching the prefetched data according to the information of the prefetched data, and generates a prefetching request for prefetching the prefetched data; and the computing node sends the prefetching request to the cache node. And the cache node responds to the pre-fetching request to execute a pre-fetching operation on the pre-fetching data. According to the scheme provided by the embodiment of the invention, the prefetching accuracy is improved.

Description

[0001] This application is a divisional application of the Chinese application with application date of September 23, 2021 and application number 202111117681.6, claiming the priority of the Chinese patent application filed with the State Intellectual Property Office of China on September 23, 2021, with application number 202111117681.6 and invention name “A data prefetching method, computing node and storage system”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data pre-fetching method, a computing node and a computer system. Background Art

[0003] The storage system usually includes a plurality of computing nodes and a plurality of storage nodes connected to each other. The computing node writes the generated data into the storage node and reads the data from the storage node. In order to shorten the data access path from the computing node to the storage node, the memory of the storage system is usually used to store the data written or read by the computing node, or the data is preloaded from the main memory of the storage node into the memory in the storage system. With the rapid growth of the amount of data, global cache technology for storage systems has emerged. By using global cache technology, the cache resources (such as memory) in the storage system can be uniformly named to form a cache pool. Each computing node can cache data to any address in the cache pool. The cache pool is composed of cache resources in multiple storage nodes, for example, or can be composed of cache resources in multiple cache nodes included in the storage system. Taking the cache node as an example, data pre-fetching recommendation is usually performed on each cache node side, and the pre-fetching accuracy of this scheme is poor. Alternatively, a central node is set in the cache node for data pre-fetching recommendation, which results in a long pre-fetching delay and increases the network communication cost. Summary of the invention

[0004] The embodiments of the present application aim to provide a data pre-fetching method, a computing node and a storage system, which improve the pre-fetching accuracy and reduce the network communication cost by making pre-fetched data recommendations on the computing node side.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application provides a data prefetching method, which includes: a computing node obtains access information of a first application to a storage node within a preset time period; the computing node determines information of prefetched data based on the access information; the computing node determines a cache node for prefetching the prefetched data according to the information of the prefetched data, and generates a prefetch request for prefetching the prefetched data; the computing node sends the prefetch request to the cache node; the cache node performs a prefetch operation on the prefetched data in response to the prefetch request.

[0006] By having the computing node determine the information of pre-fetched data according to the local access information, the accuracy of pre-fetching is improved and the network communication cost is reduced.

[0007] In a possible implementation manner of the first aspect, the computing node determining the information of pre-fetching data based on the access information includes: the computing node determining the information of pre-fetching data based on the access information according to a pre-fetching recommendation model.

[0008] By determining the information of pre-fetched data by the pre-fetch recommendation model, the accuracy and efficiency of the pre-fetched data recommendation are improved.

[0009] In a possible implementation manner of the first aspect, the pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

[0010] In a possible implementation of the first aspect, the access information includes access information of a first user, and the computing node determines information for pre-fetching data based on the access information according to a pre-fetching recommendation model, including: the pre-fetching recommendation model determines an access pattern of the first user based on the access information of the first user, and determines data to be pre-fetched according to the access pattern.

[0011] In a possible implementation of the first aspect, the prefetch request is a prefetch request for a data block, file data, or object data, and the method further includes, after the cache node receives the prefetch request for the prefetch data from the computing node, converting the prefetch request into a format and semantics uniformly set for the data block, file data, and object data.

[0012] By converting prefetch requests into a unified format and semantics, cache nodes only need to set up one prefetch interface, avoiding the cost and operational complexity of maintaining multiple protocols. In addition, global cache pools corresponding to different applications and different data types can be set up, improving the utilization of cache resources.

[0013] In a possible implementation of the first aspect, the information of the pre-fetched data includes a first identifier of the pre-fetched data, and converting the pre-fetch request into a format and semantics uniformly set for data blocks, file data, and object data includes converting the first identifier in the pre-fetch request into a second identifier that conforms to a preset format.

[0014] In a possible implementation manner of the first aspect, converting the first identifier in the pre-fetch request into a second identifier that complies with a preset format includes converting the first identifier into the second identifier through a hash algorithm.

[0015] In a possible implementation of the first aspect, the cache node includes a write cache and a read cache, and the cache node performs a prefetch operation on the prefetch data in response to the prefetch request, including: the cache node determines whether the prefetch data is stored in the write cache based on the second identifier, and when it is determined that the prefetch data is stored in the write cache, stores the prefetch data in the read cache corresponding to the second identifier.

[0016] In a possible implementation of the first aspect, the cache node performs a prefetch operation on the prefetch data in response to the prefetch request, and further includes: when the cache node determines that the prefetch data is not stored in the write cache, determining whether the prefetch data is stored in the read cache based on the second identifier; when it is determined that the prefetch data is not stored in the read cache, generating a data read request based on the second identifier, and sending the data read request to the storage node; the storage node reads the prefetch data according to the data read request, and returns the prefetch data to the cache node; the cache node stores the prefetch data in the read cache corresponding to the second identifier.

[0017] A second aspect of the present application provides a storage system, including a computing node, a cache node and a storage node, wherein the computing node is used to: obtain access information of a first application to a storage node within a preset time period; determine information of pre-fetched data based on the access information; determine a cache node for pre-fetching the pre-fetched data based on the information of the pre-fetched data, and generate a pre-fetch request for pre-fetching the pre-fetched data; send the pre-fetch request to the cache node; the cache node is used to perform a pre-fetch operation on the pre-fetched data in response to the pre-fetch request.

[0018] In a possible implementation manner of the second aspect, the computing node is used to determine the information for pre-fetching data based on the access information specifically includes: the computing node is used to determine the information for pre-fetching data based on the access information according to a pre-fetching recommendation model.

[0019] In a possible implementation manner of the second aspect, the pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

[0020] In a possible implementation of the second aspect, the access information includes access information of a first user, and the computing node is used to determine information for pre-fetching data based on the access information according to a pre-fetching recommendation model, including: the computing node is used to determine an access pattern of the first user based on the access information of the first user through the pre-fetching recommendation model, and determine the data to be pre-fetched according to the access pattern.

[0021] In a possible implementation of the second aspect, the prefetch request is a prefetch request for a data block, file data, or object data, and the cache node is further used to: after receiving the prefetch request for the prefetch data from the computing node, convert the prefetch request into a format and semantics uniformly set for the data block, file data, and object data.

[0022] In a possible implementation of the second aspect, the information of the pre-fetched data includes a first identifier of the pre-fetched data, and the cache node is used to convert the pre-fetch request into a format and semantics uniformly set for data blocks, file data and object data, including that the cache node is used to convert the first identifier in the pre-fetch request into a second identifier that conforms to a preset format.

[0023] In a possible implementation manner of the second aspect, the cache node is used to convert the first identifier in the pre-fetch request into a second identifier that complies with a preset format, including the cache node being used to convert the first identifier into the second identifier through a hash algorithm.

[0024] In a possible implementation of the second aspect, the cache node includes a write cache and a read cache, and the cache node is used to perform a prefetch operation on the prefetch data in response to the prefetch request, including: the cache node is used to determine whether the prefetch data is stored in the write cache based on the second identifier, and in the case of determining that the prefetch data is stored in the write cache, the prefetch data is stored in the read cache corresponding to the second identifier.

[0025] In a possible implementation of the second aspect, the cache node is used to perform a prefetch operation on the prefetch data in response to the prefetch request, and further includes: the cache node is used to determine whether the prefetch data is stored in the read cache based on the second identifier when it is determined that the prefetch data is not stored in the write cache, and when it is determined that the prefetch data is not stored in the read cache, generate a data read request based on the second identifier, and send the data read request to the storage node; the storage node is also used to read the prefetch data according to the data read request, and return the prefetch data to the cache node; the cache node is also used to store the prefetch data in the read cache corresponding to the second identifier.

[0026] A third aspect of the present application provides a data prefetching method, which is executed by a computing node and includes: obtaining access information of a first application to a storage node within a preset time period; determining information of prefetched data based on the access information; determining a cache node for prefetching the prefetched data based on the information of the prefetched data, and generating a prefetch request for prefetching the prefetched data; and sending the prefetch request to the cache node.

[0027] In a possible implementation manner of the third aspect, determining the information of pre-fetched data based on the access information includes: determining the information of pre-fetched data based on the access information according to a pre-fetch recommendation model.

[0028] In a possible implementation manner of the third aspect, the pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

[0029] In a possible implementation of the third aspect, the access information includes access information of a first user, and the information for determining pre-fetched data based on the access information according to the pre-fetch recommendation model includes: the pre-fetch recommendation model determines an access pattern of the first user based on the access information of the first user, and determines the data to be pre-fetched according to the access pattern.

[0030] A fourth aspect of the present application provides a computing node, including a processor and a memory, wherein the memory stores executable computer program instructions, and the processor executes the executable computer program instructions to implement the method described in the third aspect and its possible implementation methods.

[0031] The fifth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed in a computer or a processor, the computer or the processor implements the method described in the third aspect and its possible implementation methods.

[0032] In a sixth aspect, the present application provides a computer program product, including computer program instructions. When the computer program instructions are executed in a computer or a processor, the computer or the processor implements the method described in the third aspect and its possible implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The embodiments of the present application may be described more clearly by referring to the accompanying drawings:

[0034] Figure 1 An architectural diagram of a computer system provided for an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the structure of the computing node and the cache node provided in the embodiment of the present application;

[0036] Figure 3 A schematic diagram of a method for data routing for a client adaptation layer;

[0037] Figure 4 A flow chart of a method for writing data in a storage system provided in an embodiment of the present application;

[0038] Figure 5 A flow chart of a method for reading data in a storage system provided in an embodiment of the present application;

[0039] Figure 6 A flow chart of a method for pre-fetching data in a storage system provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a user access mode provided in an embodiment of the present application;

[0041] Figure 8 A structural diagram of a computing node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0043] Figure 1 The computer system architecture diagram provided in the embodiment of the present application. The computer system is, for example, a storage system, including a computing cluster 100, a cache cluster 200, and a storage cluster 300. The computing cluster 100 includes a plurality of computing nodes. Figure 1 The computing node 10a and the computing node 10b are schematically shown in FIG. The computing node can access data from the storage node through an application program or application (APP), so it is also called an "application server". The computing node can be a physical machine or a virtual machine. Physical machines include but are not limited to desktop computers, servers, laptops, and mobile devices.

[0044] The cache cluster 200 may be an independent physical cluster, or it may share the same cluster with the storage cluster 300 (i.e., be deployed in the same cluster). In the case where the cache cluster 200 and the storage cluster 300 belong to the same cluster, the resources in the cluster (such as storage resources, computing resources, etc.) are pre-divided into resources for performing cache operations and resources for performing storage operations. The cache cluster 200 includes a plurality of cache nodes, of which cache nodes 20a, 20b, and 20c are schematically shown in the figure. The cache nodes are interconnected via a network. The storage cluster 300 includes a plurality of storage nodes, of which storage nodes 30a, 30b, and 30c are schematically shown in the figure. Among them, the cache nodes and the storage nodes may be physical machines or virtual machines.

[0045] Taking cache node 20a as an example, cache node 20a includes processor 201, memory 202 and hard disk 203. Processor 201 is a central processing unit (CPU), which is used to process operation requests from computing nodes or operation requests from other cache nodes, and is also used to process requests generated within the cache node.

[0046] Memory 202 refers to an internal memory that directly exchanges data with processor 201. It can read and write data at any time and at a relatively fast speed, and serves as a temporary data storage for an operating system or other running programs. Memory 202 includes at least two types of memories, for example, memory can be either random access memory (RAM) or read only memory (ROM). For example, RAM can include dynamic random access memory (DRAM) or storage class memory (SCM) and other memories. DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memories. It is a non-volatile memory. SCM can provide faster read and write speeds than hard disks, but its computing speed is slower than DRAM, and its cost is also cheaper than DRAM. However, DRAM and SCM are only exemplary in this embodiment, and the memory may also include other random access memories, such as static random access memory (SRAM), etc. In addition, the volatile memory in the memory 202 may be configured to have a power-saving function, so that when the system is powered off, the data stored in the memory 202 will not be lost. The memory with the power-saving function is called a non-volatile memory.

[0047] The hard disk 203 is used to provide non-volatile storage resources, and its access latency is usually higher than that of the memory, and its cost is lower than that of the memory. The hard disk 203 can be, for example, a solid state disk (SSD) or a mechanical storage hard disk (HDD).

[0048] In order to adapt to the current massive computing and storage needs, a global cache pool can be provided by aggregating the storage resources (such as memory or hard disk) of multiple cache nodes in the cache cluster 200, so that the applications in each computing node can use the cache resources in any cache node. Among them, a storage medium (such as RAM, SCM, SSD) with a lower access latency than the hard disk in the storage node is usually selected as the storage resource in the cache node, thereby providing a faster data access speed than the storage node. In this way, when the cache demand of the computing cluster 100 increases, more cache nodes can be added to the cache cluster 200, thereby horizontally expanding the capacity of the global cache pool. The global cache pool provides a unified address space (or namespace) for each computing node, and the computing node can route data to the cache node for caching the data, avoiding data redundancy and consistency problems caused by repeated caching of data. In addition, in the global cache pool, high availability of data can be achieved through multiple copies, replication, multi-active and other technologies.

[0049] A computing node (e.g., computing node 10a) may send a data access request (read request or write request) to a storage cluster to a cache node (e.g., cache node 20a) for caching the data. The cache node 20a may include a write cache and a read cache, for example. If the data access request is a write request, the cache node 20a may write the data into the write cache, return a write success to the computing node 10a, and then write from the write cache to the storage node in the background, thereby improving the feedback speed of the write request. If the data access request is a read request, the cache node 20a may first determine whether the data is hit in the write cache. If it is determined that the data is not in the write cache, it may determine whether the data is stored in the read cache. If the data is stored in the read cache, the cache node 20a may directly read the data from the read cache and return the data to the computing node 10a, thereby not having to go to the storage cluster to read the data, shortening the data reading path, and improving the feedback speed of the read request.

[0050] In another structure, independent cache nodes do not need to be deployed, but storage resources in storage nodes, such as memory, PCM, etc., are used to form a cache pool to provide it to applications in computing nodes.

[0051] Generally, in order to improve data access efficiency, the data to be accessed will be pre-fetched into the cache pool in advance, which improves the hit rate of the data in the cache and thus improves the data access efficiency. Data pre-fetching generally includes two processes: one is the recommendation of pre-fetched data, and the other is to read the recommended pre-fetched data from the storage node to the cache pool in advance.

[0052] In the related art, data is generally pre-fetched on the cache cluster side, and there are generally two implementation methods for recommending pre-fetched data. In one implementation, a recommendation for pre-fetched data is made at each cache node, so that the cache node can pre-fetch the pre-fetched data according to the recommendation. However, since data access requests generated by the same application in the computing node are usually distributed to multiple cache nodes, each cache node can only obtain the data access history of part of the data accessed by a certain application within the recent preset time period, and cannot obtain the data access history of the full amount of data accessed by the application, wherein the data access history includes the identifiers of multiple data requested to be accessed by the application and the corresponding access times. The cache node only recommends pre-fetched data based on the access history of part of the data of the application processed by this node, and the recommendation accuracy is low, and the pre-fetching bandwidth and cache resources are wasted.

[0053] In another embodiment, a specific cache node is set as a central node for data prefetching in the cache cluster. The central node collects the data access history of each application of each computing node from other cache nodes, so that the prefetched data can be recommended based on the complete data access history of a single application. Other cache nodes can send prefetch recommendation requests to the central node, thereby receiving the prefetch recommendation results from the central node and prefetching data based on the prefetch recommendation results. However, this data prefetching method adds additional network communication, increases communication costs, and may result in untimely prefetching.

[0054] In the embodiment of the present application, each computing node makes a pre-fetched data recommendation. The computing node can make a pre-fetched data recommendation based on the data access history of a single application in the computing node in a recent preset period through a pre-fetched recommendation model, so that the recommendation accuracy is high, no additional network communication is caused, and the delay of the pre-fetched recommendation is low.

[0055] Figure 2 A schematic diagram of the structure of the computing nodes and cache nodes provided in the embodiment of the present application. Figure 2 FIG. 1 shows a computing node 10a, a cache node 20a, and a storage node 30a as an example. Figure 2 As shown, one or more applications may be installed in the computing node 10a, and the multiple applications are as follows: Figure 2For example, the applications shown in the figure include databases, virtual machines (VMs), big data, high performance computing (HPC), artificial intelligence (AI), etc. These applications may use different data services provided by the storage cluster 300. For example, the storage cluster 300 is a Ceph cluster, which is a distributed file system. Librados service components are deployed in computing nodes to provide block storage services, object storage services, file storage services, and other services to various applications in the computing nodes.

[0056] The client adaptation layer 11 can be deployed in the computing node 10a, and the client adaptation layer 11 can be embedded in the Librados service component deployed in the computing node 10a in the form of a function library. Thus, the client adaptation layer 11 can intercept the data access requests to the storage cluster generated by each application, determine the identifier of the cache node corresponding to the target data of the data access request, generate an operation request sent to the cache node based on the data access request and the identifier of the corresponding cache node, and send the operation request to the corresponding cache node 20a (i.e., the cache server). The operation request includes, for example, information such as the operation type, the destination node, and the original data access request. The cache node 20a performs corresponding operations according to the operation request and returns a response message to the client adaptation layer 11. After receiving the response message from the server, the client adaptation layer 11 parses the message and returns the parsing result to the application in the computing node 10a.

[0057] A data analysis service (DAS) module 12 (hereinafter referred to as DAS12) is also deployed in the computing node 10a. DAS12 is used to register a message service with the client adaptation layer 11, so that the user's read and write access requests at multiple times can be pulled from the client adaptation layer 11. DAS12 includes a pre-fetch recommendation model, which mines the user's access pattern based on the user's read and write access requests at multiple times, and makes pre-fetch data recommendations based on this, and pushes the recommendation results to the client adaptation layer 11. The client adaptation layer 11 generates a data pre-fetch request based on the recommendation result, and sends it to the corresponding cache node, so that the cache node performs data pre-fetching.

[0058] A server adaptation layer 21 is deployed in the cache node 20a. The server adaptation layer 21 is used to receive an operation request from the client adaptation layer 11 through the network. The operation request includes information such as the operation type and the user's original data access request. As mentioned above, since different applications use different data services, the user's original data access request may have different formats and semantics. To this end, the server adaptation layer 21 is also used to perform a unified protocol translation and conversion on the original data access request, thereby converting the original data access request into a data access request with a unified format and semantics. Afterwards, the server adaptation layer 21 can call the operation interface to process the request according to the operation type. The operation interface includes, for example, a write interface, a read interface, a pre-fetch interface, etc.

[0059] like Figure 2 As shown in FIG. 1 , cache node 20a includes a write cache 22, an L1 read cache (i.e., a first-level read cache) 23, and an L2 read cache (i.e., a second-level read cache) 24. The write cache 22 includes, for example, Figure 1 The RAM storage space in the memory 202 and the SSD storage space in the hard disk 203, wherein the RAM storage space is used to accelerate query and disk flushing, and the SSD storage space is used to protect the data written to the RAM (i.e., dirty data). For example, the written data can be stored in the form of multiple copies in the SSD storage space, thereby ensuring high reliability of dirty data and high availability in fault scenarios.

[0060] The L1 read cache mainly uses small-capacity, high-performance storage media, such as DRAM, SCM, etc. in the memory 202. The L1 read cache serves as a unified entry for read operations, shielding the existence of the secondary read cache upward to avoid introducing management and interaction complexity. The L2 read cache mainly uses a large-capacity storage medium to receive data eliminated by the first-level read cache. The L2 read cache can use a non-volatile storage medium (such as SCM or SSD). The large-capacity L2 read cache can avoid the L1 read cache from being pre-fetched in time due to scenarios such as limited space, or the performance fluctuation and degradation caused by a large amount of hot data being eliminated to the L2 read cache. The global cache of the present invention supports the expansion of three or more levels of cache.

[0061] Figure 2 The aggregation module 25 is used to aggregate small data volumes stored in the write cache 22 into large data volumes, and then the storage agent module 26 writes the large data volumes into the storage cluster 300 .

[0062] The cache node 20a also includes a cluster management module 27. The cache cluster 200 generates a partition view of the cache cluster through the cluster management module in each cache node. Specifically, a master node for cluster management can be set in the cache cluster 200, and each newly launched cache node in the cache cluster registers with the master node through the cluster management module, so that the master node can obtain information about cache resources in each cache node. The master node can map the cache resources in each cache node to each partition (part it ion) according to a preset algorithm, and generate a partition view, which includes a mapping relationship between each cache node and each partition. Among them, in the scenario of multi-copy storage, a partition can be mapped to multiple cache nodes. After generating the partition view, the master node can send the partition view to other cache nodes.

[0063] The cache resources include, for example, cache resources such as write cache, L1 read cache, and L2 read cache in each cache node. The client adaptation layer 11 of the computing node 10a can, for example, obtain a partition view of the cache cluster 200 from the cluster management module 27 of any cache node (for example, cache node 20a). After intercepting a data access request from an application, the client adaptation layer 11 can determine the cache node that processes the data to be accessed from the partition view according to preset rules. For example, the client adaptation layer can hash the key (Key) of the data to be accessed to obtain a digest, and then take the modulus of the partition number with the digest to determine the partition number corresponding to the data, and then determine the cache node corresponding to the data access request based on at least one cache node corresponding to the partition number in the partition view.

[0064] Figure 3 Schematic diagram of the method for data routing for the client adaptation layer. Figure 3 As shown, after determining that the data to be accessed corresponds to the partition pt0, the client adaptation layer 11 Figure 3 The partition view in the partition can determine the partition pt0 and Figure 3 , so the data access request can be routed to cache node 20a, cache node 20b, cache node 20c respectively. Figure 3 One partition in the table corresponds to three cache nodes, indicating that three copies of the data are stored to increase reliability.

[0065] The following will combine Figure 2 Describe the method flow of writing data, reading data and pre-fetching data provided by the embodiments of this specification.

[0066] Figure 4 A flow chart of a method for writing data in a storage system provided in an embodiment of the present application. Figure 4The method shown may be executed by a computing node, a cache node, and a storage node in a storage system. The following description will be made by taking a computing node 10a, a cache node 20a, and a storage node 30a as examples.

[0067] like Figure 4 As shown, first, in step S401, the computing node 10a generates a cache node write request based on the data write request of the application.

[0068] As mentioned above Figure 2 As described, one or more applications, such as a database application, may be installed in the computing node 10a, and the computing node 10a is also installed with a client adaptation layer 11 and a DAS 12. When the database application wishes to write data to the storage cluster, a data write request is generated. Among them, the database application may select a data storage service provided by the storage cluster as needed, such as a block storage service, an object storage service, or a file storage service. In the case of a block storage service, the data write request includes, for example, the logical address of the data and the data to be written, and the logical address of the data includes, for example, information such as a logical unit number (logical unit number, LUN), a logical block address (logical block address, LBA), and a data length, and the logical address of the data is equivalent to the Key of the data. In the case of an object storage service, the data write request includes, for example, the object name of the data and the data to be written, and the object name of the data is the Key of the data. In the case of a file storage service, the data write request includes, for example, the file name of the file data and the directory path where the file is located, and the file name and directory path are equivalent to the Key of the data. That is to say, when different applications use different data services, the formats of the keys (such as the form of the key, the length of bytes, etc.) in the data access requests (including data write requests, data read requests, etc.) generated by them vary greatly. At the same time, the attributes of each field (such as field length, field semantics, etc.) in the data access requests generated by different applications when using different data services may also be different. In addition, the attributes of the fields in the data access requests generated by different applications may also be different.

[0069] refer to Figure 2 After the database application generates the above data write request and sends it to the Librados component, the client adaptation layer 11 can intercept the data write request from the Librados component and generate a cache node write request for sending to the cache cluster 200 based on the data write request. Specifically, the client adaptation layer 11 first determines that the data to be written corresponds to the partition pt0, for example, Figure 3In the routing process shown, the client adaptation layer 11 can determine that the data to be written should be routed to cache nodes 20a, 20b and 20c, so that the client adaptation layer 11 can generate three cache node write requests sent to cache nodes 20a, 20b and 20c respectively. The following description takes cache node 20a as an example, and the operations of cache node 20b and cache node 20c can refer to the operations of cache node 20a.

[0070] The generated cache node write request sent to the cache node 20a includes, for example, information such as the node identification of the cache node 20a, the operation type (write request type), and the initial data access request.

[0071] In step S402, after generating a cache node write request, the computing node 10a sends the cache node write request to a corresponding cache node, such as the cache node 20a.

[0072] Specifically, after generating the cache node write request, the client adaptation layer 11 may send the cache node write request to the server adaptation layer 21 in the cache node 20a.

[0073] In step S403, the cache node converts the cache node write request into a unified format and semantics.

[0074] As described above, since data access requests corresponding to different applications and / or different data storage services have different formats and / or semantics, in the embodiment of the present application, the cache node write request is converted into a unified format and semantics through the server adaptation layer 21. The unified format and semantics correspond to, for example, a data storage service in the storage cluster, such as an object storage service, so that only one data storage service can be provided in the storage cluster.

[0075] Specifically, the conversion operation may include converting the Key (e.g., Key1) of the data to be written in the cache node write request into a preset length. For example, assuming that the preset length is 20 bytes, when the data storage service used by the application is a block storage service, the length of the data Key is usually less than or equal to 20 bytes. When Key1 is less than 20 bytes, the server adaptation layer 21 may adaptively add bytes to the Key1 in a preset manner, thereby increasing the length of the Key1 to 20 bytes of Key2. When the data storage service corresponding to the application is an object storage service, the length of the object name may not be fixed, and the server adaptation layer 21 may map Key1 to 20 bytes of Key2 through, for example, a hash algorithm. The cache node 20a may maintain a corresponding relationship table between the initial Key and the mapped Key through a data table. After mapping Key1 to Key2 through a hash algorithm, the server adaptation layer 21 may determine whether there is a hash collision based on the data table. If there is a hash collision, the server adaptation layer 21 may remap Key1 to a different 20-byte Key through a preset algorithm and record the hash collision for query.

[0076] By converting the Keys of the data to be written in the write requests of different cache nodes into the same preset length as described above, the complexity of data management in the cache cluster is reduced and storage space is saved.

[0077] The above conversion operation also includes converting the semantics of the cache node write request into preset semantics. Specifically, the server adaptation layer 21 converts the cache node write request according to the preset attributes such as the length and semantics of multiple fields.

[0078] After the above conversion process, the cache node can use a unified interface to process data service requests corresponding to different applications and different data storage services. Therefore, a unified global cache pool can be created for different applications, thereby improving the utilization rate of cache resources.

[0079] In step S404, the cache node executes the write request and writes the data to the cache node.

[0080] As described above, when executing a write request, the cache node calls the write interface to write the data in the write request into the write cache 22 in the cache node, and the write cache 22 includes, for example, RAM storage space and SSD storage space. After converting the cache node write request into a unified format and semantics, the server-side adaptation layer 21 can call the write interface set in the cache node 20a. The computer code included in the write interface is used to perform a series of operations such as data caching and writing data to the storage cluster after execution. After starting to execute the write interface, according to the write interface, the data requested to be written in the cache node write request is written into the write cache 22 corresponding to the converted Key (for example, Key3). This data is also the data requested to be written in the aforementioned data write request.

[0081] Specifically, the data is written in the SSD space in the write cache 22 in the form of, for example, three copies, so as to protect the written data. At the same time, the data is stored in the RAM space in the write cache to speed up the query and disk flushing of the data (i.e., storage to the storage cluster).

[0082] In step S405, after writing the data, the cache node returns write request completion information to the computing node.

[0083] After completing writing to the write cache, the cache node 20a can immediately return write request completion information to the computing node 10a, without having to return the write request completion information after writing the data to the storage cluster, thereby shortening the feedback time and improving system efficiency.

[0084] After writing the data as described above, the cache node 20a can determine whether the write cache meets the disk flushing condition. The disk flushing condition includes, for example, any of the following conditions: the data stored in the write cache reaches a preset water level; the current time is a preset disk flushing time (for example, the idle time of the cache node); a disk flushing instruction is received from a business person. When it is determined that the disk flushing condition is met, the cache node 20a performs data deduplication, data merging, and other processing on some of the data stored in the write cache RAM with an earlier storage time, so as to store it in the storage cluster.

[0085] Optionally, in step S406, the cache node 20a aggregates multiple data in the write cache received from the write cache.

[0086] Specifically, the aggregation module 25 in the cache node 20a aggregates multiple data in the write cache received from the write cache.

[0087] Assume that the multiple data to be flushed in the write cache are multiple small objects, and the data volume of the small objects is small, for example, 8Kb in size. The multiple small objects include multiple new data that rewrite the old data, and these old data may be distributed in different storage addresses in different storage clusters. Therefore, if these small objects are directly written into the storage cluster separately, it is necessary to address the aforementioned different storage addresses separately, which will generate a large amount of random data writes in the storage cluster. Each random data write in the storage cluster requires re-seeking and rotating the disk in the HDD, which leads to a decrease in the flushing speed. In addition, the data storage speed of the storage medium in the storage cluster is usually slower than the cache medium in the cache cluster. In this way, in a scenario with high concurrency, the speed of flushing the data in the write cache cannot keep up with the speed of writing data in the write cache, and the capacity of the write cache of the cache node 20a is easily filled, resulting in the application data having to be directly written to the backend storage, so that the write cache will not be able to provide acceleration services.

[0088] To address this problem, in the embodiment of the present application, the aggregation module 25 aggregates multiple data in the write cache, and writes the aggregated larger data to the storage cluster, thereby improving the speed of writing to the storage cluster.

[0089] Specifically, the aggregation module 25 can aggregate, for example, 1,000 small objects in the write cache into an 8Mb large object for sequential writing to the storage cluster. This can convert multiple HDD random write operations into one sequential write operation, which only requires one seek + rotation delay time instead of 1,000 times, thereby improving the data writing speed of the storage cluster.

[0090] After aggregating multiple small objects into a large object, the aggregation module 25 generates a unique key for the large object and Figure 2 The information of the large object is recorded in the metadata in the large object, including the keys of multiple small objects contained in the large object, and the offset address (offset) and data length (length) of each small object stored in the large object. The aggregation module 25 can store the metadata in the memory, and at the same time store the metadata in the form of multiple copies in a non-volatile medium (such as an SSD), and synchronously update the metadata in the SSD after each update of the metadata in the memory.

[0091] After aggregating multiple small objects into a large object, the aggregation module 25 may provide the large object to the storage agent module 26 for writing into the storage cluster.

[0092] In step S407 , the cache node 20 a generates a data write request.

[0093] Specifically, after obtaining the above 8Mb large object, the storage agent module 26 in the cache node 20a determines the storage node (e.g., storage node 30a) corresponding to the data according to the preset data allocation rule, and generates a write request for the large object accordingly. Specifically, the write request includes, for example, the identifier of the storage node 30a, the key of the large object, and the large object.

[0094] In the case where small objects are not aggregated, the cache node 20a may provide each small object to the storage agent module 26 respectively, and the storage agent module 26 may similarly generate a data write request for each small object respectively.

[0095] In step S408, the cache node 20a sends the generated data write request to the storage node 30a.

[0096] Specifically, the storage agent module 26 sends the generated data write request to the storage node 30a.

[0097] In step S409, after receiving the data write request, the storage node 30a writes the corresponding data.

[0098] Specifically, after receiving the write request, the storage node 30a calls the write interface to write the data. The storage agent module 26 generates a data write request in a unified format, which has the semantics and format of the object storage service, for example. Therefore, the storage node 30a only needs to set a write interface corresponding to the object storage service. It can be understood that the storage agent module 26 is not limited to generating write requests with the semantics and format of the object storage service, but can generate write requests with the semantics and format of other data storage services.

[0099] After completing the writing of the data, the storage node 30a may return a write success message to the cache node 20a. After receiving the write success message, the cache node 20a may update the old versions of each small object stored in the L1 read cache 23 and / or L2 read cache 24 with the latest versions of each small object written, so that the data stored in the read cache is the latest version. At the same time, the cache node 20a may delete the data stored in the write cache that has been flushed to disk.

[0100] After the aggregation module 25 aggregates the small objects as described above, when most of the small objects in the large objects that have been flushed to disk become invalid data due to being deleted or modified, it will cause the large objects stored in the storage cluster to occupy a lot of invalid space. To this end, the aggregation module 25 can recycle large objects with more invalid space during idle time. Specifically, the aggregation module 25 can request the storage cluster to read the small objects that are still valid in the large object through the storage agent module 26, and send a request to delete the large object to the storage cluster after the reading is completed to complete the recycling of the large object. The small objects that are still valid in the recycled large object can be aggregated into a new large object again and then rewritten to the storage cluster. Specifically, multiple large objects can be recycled in the order of invalid space in each large object from most to least. After recycling the large object, the aggregation module 25 modifies the metadata accordingly.

[0101] Figure 5 A flow chart of a method for reading data in a storage system provided in an embodiment of the present application. Figure 5 The method shown may be executed by a computing node, a cache node, and a storage node in a storage system. The following description will be made by taking a computing node 10a, a cache node 20a, and a storage node 30a as examples.

[0102] like Figure 5 As shown, first, in step S501, the computing node 10a generates a cache node read request based on the data read request of the application.

[0103] Referring to the description of step S401 above, when the database application in the computing node 10a wishes to read data (e.g., the object named Key1) from the storage cluster, a data read request is generated, and the data read request includes the name of the object to be read, "Key1". Similarly, the data read request has a format and semantics corresponding to the database application and the data storage service used by the application.

[0104] After computing node 10a generates the above data read request, client adaptation layer 11 intercepts the data read request and generates a cache node read request based on the data read request to be sent to cache cluster 200. Specifically, client adaptation layer 11 can similarly determine that the data to be read should be routed to cache nodes 20a, 20b, and 20c, so that client adaptation layer 11 can generate three cache node read requests sent to cache nodes 20a, 20b, and 20c respectively. The following description takes cache node 20a as an example.

[0105] The generated cache node read request sent to the cache node 20a includes, for example, information such as the node identifier of the cache node 20a, the operation type (read request type), and the initial data read request.

[0106] In step S502 , the computing node 10 a sends a cache node read request to the cache node 20 a .

[0107] Specifically, after generating the cache node read request, the client adaptation layer 11 may send the cache node read request to the server adaptation layer 21 in the cache node 20a.

[0108] In step S503, the cache node 20a converts the cache node read request into a unified format and semantics.

[0109] This step can refer to the description of step S403 above, which will not be repeated here. After the conversion, the cache node read request is converted to read the object Key2.

[0110] In step S504 , the cache node 20 a determines whether the local cache stores data to be read.

[0111] Specifically, the cache node 20a calls the read interface to determine whether the local cache stores data to be read.

[0112] When the cache node 20a determines that the data to be read is stored in the local cache, it may read the data from the local cache and execute step S508 to return the read data to the computing node 10a.

[0113] Specifically, after executing the read interface, the cache node 20a first determines whether the value of the object Key2 is stored in the RAM storage space of the write cache 22. If it is determined that it is, the value can be read and returned to the computing node 10a. In the case where the value of the object Key2 is not stored in the write cache 22, the cache node 20a can determine whether the value of the object Key2 is stored in the L1 read cache 23. If it is determined that it is, the value can be read and returned to the computing node 10a. In the case where the value of the object Key2 is not stored in the L1 read cache 23, the cache node 20a can determine whether the value of the object Key2 is stored in the L2 read cache 24. If it is determined that it is, the value can be read and returned to the computing node 10a.

[0114] In step S505 , when the cache node 20 a determines that the data to be read is not stored in the local cache, it generates a data read request and sends the data read request to the storage node 30 a .

[0115] In one implementation, cache node 20a may generate a data read request for reading object Key2.

[0116] In another embodiment, for the above-mentioned scenario of aggregating small objects into large objects, the cache node 20a first reads the metadata, determines that the object Key2 corresponds to the large object Key3, and determines the offset address and length of reading Key2 in the object Key3, and then generates a data read request, which includes the name of the object to be read "Key3", and the offset address and length of the data to be read in the object Key3.

[0117] In step S506, the storage node 30a reads the data.

[0118] After receiving the data read request from the cache node 20a, the storage node 30a reads the data corresponding to the offset address and length in the object Key3, thereby reading the object Key2.

[0119] In step S507 , the storage node 30 a returns the read data to the cache node 20 a .

[0120] In step S508 , the cache node 20 a returns the read data to the computing node 10 a .

[0121] Specifically, the cache node 20a converts key2 into key1 through the server adaptation layer 21, and returns the value of key2 received from the storage node 30a as the value of key1 to the computing node 10a, so that the computing node 10a returns the value of key1 to the application.

[0122] Figure 6 A flow chart of a method for pre-fetching data in a storage system provided in an embodiment of the present application. Figure 6 The method shown may be executed by a computing node, a cache node, and a storage node in a storage system. The following description will be made by taking a computing node 10a, a cache node 20a, and a storage node 30a as examples.

[0123] like Figure 6 As shown, first, in step S601, the computing node 10a obtains the data access history of the application in the most recent preset period of time.

[0124] Specifically, the DAS 12 in the computing node 10a obtains the data access history of the application within a recent preset period of time.

[0125] As described above, DAS 12 can pull the read and write access requests of the application from the client adaptation layer 11, thereby obtaining the data access history of the application user in the recent preset period. The data access history includes, for example, the data identifier read or written by the user in the recent preset period, and the time information of the reading or writing.

[0126] In step S602 , the computing node 10 a recommends data to be pre-fetched based on the data access history of each application.

[0127] Specifically, in an embodiment of the present application, the DAS 12 in the computing node recommends the data to be pre-fetched through a pre-fetch recommendation model. The pre-fetch recommendation model can use a variety of algorithms. For example, the pre-fetch recommendation model may include a clustering model for performing multi-dimensional feature clustering on the data in the user's data access history, thereby making data pre-fetch recommendations based on the clustering results. The pre-fetch recommendation model may also include a time series prediction model for predicting the data that the user will access at the next moment, thereby making data pre-fetch recommendations based on the prediction results. The pre-fetch recommendation model may also include algorithms such as frequent pattern mining and hot data identification.

[0128] The pre-fetch recommendation model can determine the user access pattern based on a variety of algorithms, and the pattern includes, for example, a streaming pattern, a hotspot pattern, an association pattern, a working set association pattern, and the like. Figure 7 A schematic diagram showing various user access modes. Figure 7 In each coordinate axis, the horizontal axis represents time, for example, and the vertical axis represents data identification (ie, the key of the data).

[0129] like Figure 7 As shown, in the streaming mode, the data accessed by the user is linearly related to time, so the pre-fetch recommendation model can predict the data that the user will access at the next moment as the recommended pre-fetch data based on the relationship. Specifically, the pre-fetch recommendation model outputs the identifier of the recommended pre-fetch data. In the hotspot mode, the hotspot data at different moments can be predicted, so the hotspot data at the next moment can be predicted as the recommended pre-fetch data based on the mode. In the association mode (read-read association or write-read association), the data read by the user in the next period is associated with the data read or written in the previous period. Therefore, the data that the user will read at the next moment can be predicted as the recommended pre-fetch data based on the mode. In the work machine association mode, the user's access to a data table (such as Table 2) is associated with the user's access to another data table (such as Table 1). Therefore, the data that the user will access at the next moment can be predicted as the recommended pre-fetch data based on the mode.

[0130] Among them, DAS12 supports stateless deployment, and pattern mining can be re-performed after the computing node 10a fails or the process is restarted. Alternatively, DAS12 can also write the access pattern mined by the pre-fetched recommendation model to a persistent medium, and read the user access pattern from the persistent medium after events such as failure, restart, and upgrade, thereby achieving rapid preheating.

[0131] After predicting the identifier of the recommended pre-fetched data (eg, Key1 ), the pre-fetch recommendation model provides the identifier of the recommended pre-fetched data to the client adaptation layer 11 .

[0132] The pre-fetch recommendation model is only one implementation method of the embodiment of the present application, and other methods for recommending pre-fetch data are also included in the scope disclosed in the embodiment of the present application.

[0133] In step S603, the computing node 10a generates a data prefetch request according to the recommended prefetch data.

[0134] Similar to generating a cache node read request, the client adaptation layer 11 of the computing node 10a determines the corresponding cache node (e.g., cache node Key2) according to the identifier Key1 of the recommended pre-fetched data, thereby generating a data pre-fetch request. The data pre-fetch request includes the operation request type (pre-fetch type), the identifier of the cache node 20a, and the identifier of the data to be pre-fetched (Key2).

[0135] In step S604, the computing node sends a data pre-fetch request to the cache node 20a.

[0136] Specifically, the client adaptation layer 11 sends the data pre-fetch request to the server adaptation layer 21 in the cache node 20a.

[0137] In step S605 , the cache node 20 a converts the data pre-fetch request into a unified format and semantics.

[0138] This step can refer to the description of step S403 above, which will not be repeated here. After the conversion, the data pre-fetch request is used to pre-fetch the value of the object Key2.

[0139] In step S606 , the cache node 20 a determines whether the write cache stores data to be pre-fetched.

[0140] Specifically, the cache node 20a first calls the prefetch interface, and after executing the prefetch interface, the cache node 20a first determines whether the value of the object Key2 is stored in the RAM storage space of the write cache 22. When the cache node 20a determines that the data to be prefetched is stored in the write cache 22, the cache node 20a can read the data from the write cache, and execute step S611 to store the data in the L1 read cache or the L2 read cache, and end the prefetch operation.

[0141] In step S607 , when the cache node 20 a determines that the write cache does not store data to be prefetched, it may determine whether the read cache stores data to be prefetched.

[0142] When the cache node 20a determines that the data to be pre-fetched is stored in any read cache, the pre-fetch operation may be terminated. Alternatively, the cache node 20a may read the data from the read cache and execute step S612 to return the data to the computing node 10a.

[0143] Specifically, if the value of object Key2 is not stored in the write cache 22, the cache node 20a may determine whether the value of object Key2 is stored in the L1 read cache 23, and if it is determined that the value is stored, the prefetch operation may be terminated. If the value of object Key2 is not stored in the L1 read cache 23, the cache node 20a may determine whether the value of object Key2 is stored in the L2 read cache 24, and if it is determined that the value is stored, the prefetch operation may be terminated.

[0144] In step S608, when the cache node 20a determines that the data to be pre-fetched is not stored in the read cache, it generates a data read request and sends the data read request to the storage node 30a. In step S609, the storage node 30a reads the data according to the data read request. In step S610, the storage node 30a returns the read data to the cache node 20a. Steps S608 to S610 can refer to the description of steps S505 to S507 above, which will not be repeated here.

[0145] In step S611, the cache node 20a stores the data returned by the storage node 30a into the read cache.

[0146] The cache node 20a can store the returned object Key2 in the L1 read cache or the L2 read cache and end the pre-fetch operation. When the computing node 10a sends a read request for the object Key1, the cache node 20a determines that Key1 corresponds to Key2 by converting the read request, so that the value of the variable Key2 can be read from the read cache, and the value of Key2 is returned to the computing node 10a as the value of Key1 without having to read from the storage cluster, thereby shortening the user access delay.

[0147] Optionally, after storing the value of the object Key in the read cache, the cache node 20a may execute step S612 to return the pre-fetched data to the computing node.

[0148] Figure 8 A structural diagram of a computing node provided in an embodiment of the present application, wherein the computing node is used to execute Figure 4 , Figure 5 or Figure 6 The method shown, the computing node comprises:

[0149] An acquisition unit 81 is used to acquire access information of the first application to the storage node within a preset period of time;

[0150] A determining unit 82, configured to determine information of pre-fetched data based on the access information;

[0151] A generating unit 83, configured to determine a cache node for prefetching the prefetched data according to information of the prefetched data, and generate a prefetch request for prefetching the prefetched data;

[0152] The sending unit 84 is configured to send the pre-fetch request to the cache node.

[0153] In an implementation manner, the determining unit 82 is specifically configured to determine information of pre-fetched data based on the access information according to a pre-fetch recommendation model.

[0154] In one implementation, the access information includes access information of a first user, and the determination unit 82 is specifically configured to determine, by the pre-fetch recommendation model, an access pattern of the first user based on the access information of the first user, and determine data to be pre-fetched according to the access pattern.

[0155] It should be understood that the descriptions such as “first” and “second” in this article are only used to distinguish similar concepts for the sake of simplicity of description and do not have any other limiting effect.

[0156] Those skilled in the art can clearly understand that the descriptions of the various embodiments provided in the present application can refer to each other. For the convenience and conciseness of the description, for example, the functions of the various devices and equipment provided in the embodiments of the present application and the execution steps can refer to the relevant description of the method embodiments of the present application, and the various method embodiments and the various device embodiments can also refer to each other.

[0157] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes all or part of the steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: read-only memory (ROM), random-access memory (RAM), disk or optical disk and other media that can store program codes.

[0158] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid-state drive (Solid State Dis sk, SSD), etc.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways without exceeding the scope of the present application. For example, the embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0160] In addition, the described devices and methods and schematic diagrams of different embodiments may be combined or integrated with other systems, modules, technologies or methods without exceeding the scope of the present application. In addition, the coupling or direct coupling or communication connection between each other shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electronic, mechanical or other forms.

[0161] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data pre-fetching method, characterized in that: The method comprises: The computing node determines information of pre-fetching data based on access information of the first application to the storage node within a preset period of time; The computing node determines a cache node to pre-fetch the pre-fetched data according to information of the pre-fetched data; The computing node generates and sends a prefetch request to the cache node, wherein the prefetch request is used to request prefetching of the prefetch data; The cache node performs a prefetch operation on the prefetch data in response to the prefetch request.

2. The method according to claim 1, characterized in that: The information of pre-fetching data determined by the computing node based on the access information of the first application to the storage node within a preset period includes: The computing node determines information of pre-fetching data based on the access information according to a pre-fetching recommendation model.

3. The method according to claim 2, characterized in that The pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

4. The method according to claim 2 or 3, characterized in that: The access information includes access information of a first user, and the computing node determines information for pre-fetching data based on the access information according to a pre-fetching recommendation model, including: the pre-fetching recommendation model determines an access pattern of the first user based on the access information of the first user, and determines data to be pre-fetched according to the access pattern.

5. The method according to any one of claims 1 to 4, characterized in that: The prefetch request is a prefetch request for a data block, file data or object data. The method also includes, after the cache node receives the prefetch request for the prefetch data from the computing node, converting the prefetch request into a format and semantics uniformly set for the data block, file data and object data.

6. The method according to claim 5, characterized in that The information of the pre-fetched data includes a first identifier of the pre-fetched data, and converting the pre-fetch request into a format and semantics uniformly set for data blocks, file data and object data includes converting the first identifier in the pre-fetch request into a second identifier that conforms to a preset format.

7. The method according to claim 6, characterized in that The converting the first identifier in the pre-fetch request into the second identifier conforming to a preset format includes converting the first identifier into the second identifier by using a hash algorithm.

8. The method according to claim 6 or 7, characterized in that: The cache node includes a write cache and a read cache, and the cache node performs a prefetch operation on the prefetch data in response to the prefetch request, including: The cache node determines whether the pre-fetched data is stored in the write cache based on the second identifier, and if it is determined that the pre-fetched data is stored in the write cache, stores the pre-fetched data in the read cache correspondingly to the second identifier.

9. The method according to claim 8, characterized in that The cache node performing a prefetch operation on the prefetch data in response to the prefetch request further includes: When the cache node determines that the pre-fetched data is not stored in the write cache, the cache node determines whether the pre-fetched data is stored in the read cache based on the second identifier; when it is determined that the pre-fetched data is not stored in the read cache, the cache node generates a data read request based on the second identifier, and sends the data read request to the storage node; The storage node reads the pre-fetched data according to the data read request, and returns the pre-fetched data to the cache node; The cache node stores the pre-fetched data and the second identifier in the read cache in correspondence.

10. A storage system, characterized in that: Including computing nodes, cache nodes and storage nodes, The computing node is used to: determine information of pre-fetched data based on access information of the first application to the storage node within a preset period of time; Determining a cache node for pre-fetching the pre-fetched data according to information of the pre-fetched data; Generate and send a prefetch request to the cache node, where the prefetch request is used to request prefetching of the prefetch data; The cache node is used to perform a prefetch operation on the prefetch data in response to the prefetch request.

11. The storage system according to claim 10, characterized in that: The information used by the computing node to determine the pre-fetched data based on the access information of the first application to the storage node within a preset period of time specifically includes: The computing node is used to determine information of pre-fetched data based on the access information according to the pre-fetch recommendation model.

12. The storage system according to claim 11, characterized in that: The pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

13. The storage system according to claim 11 or 12, characterized in that: The access information includes access information of a first user, and the computing node is used to determine information for pre-fetching data based on the access information according to a pre-fetching recommendation model, including: the computing node is used to determine an access pattern of the first user based on the access information of the first user through the pre-fetching recommendation model, and determine data to be pre-fetched according to the access pattern.

14. The storage system according to any one of claims 10 to 13, characterized in that: The prefetch request is a prefetch request for a data block, file data or object data, and the cache node is further used to: after receiving the prefetch request for the prefetch data from the computing node, convert the prefetch request into a format and semantics uniformly set for the data block, file data and object data.

15. The storage system according to claim 14, characterized in that: The information of the pre-fetched data includes a first identifier of the pre-fetched data, and the cache node is used to convert the pre-fetch request into a format and semantics uniformly set for data blocks, file data and object data, including: the cache node is used to convert the first identifier in the pre-fetch request into a second identifier that conforms to a preset format.

16. The storage system according to claim 15, characterized in that: The cache node is used to convert the first identifier in the pre-fetch request into a second identifier that complies with a preset format, including: the cache node is used to convert the first identifier into the second identifier by using a hash algorithm.

17. The storage system according to claim 15 or 16, characterized in that: The cache node includes a write cache and a read cache, and the cache node is used to perform a prefetch operation on the prefetch data in response to the prefetch request, including: The cache node is used to determine whether the pre-fetched data is stored in the write cache based on the second identifier, and if it is determined that the pre-fetched data is stored in the write cache, store the pre-fetched data in the read cache correspondingly to the second identifier.

18. The storage system according to claim 17, characterized in that: The cache node is used to perform a prefetch operation on the prefetch data in response to the prefetch request, and further includes: The cache node is used to determine whether the pre-fetched data is stored in the read cache based on the second identifier when it is determined that the pre-fetched data is not stored in the write cache, and generate a data read request based on the second identifier when it is determined that the pre-fetched data is not stored in the read cache, and send the data read request to the storage node; The storage node is further configured to read the pre-fetched data according to the data read request and return the pre-fetched data to the cache node; The cache node is further configured to store the pre-fetched data and the second identifier in the read cache in correspondence thereto.

19. A data pre-fetching method, characterized in that: The method is performed by a computing node and includes: Acquiring access information of the first application to the storage node within a preset period of time; determining information for pre-fetching data based on the access information; Determine a cache node for pre-fetching the pre-fetched data according to information of the pre-fetched data, and generate a pre-fetch request for pre-fetching the pre-fetched data; The pre-fetch request is sent to the cache node.

20. The method according to claim 19, characterized in that The information for determining the pre-fetched data based on the access information includes: The information of pre-fetching data is determined based on the access information according to the pre-fetching recommendation model.

21. The method according to claim 20, characterized in that The pre-fetch recommendation model is based on at least one of the following algorithms: a clustering algorithm, a time series prediction algorithm, a frequent pattern mining algorithm, and a hot data identification algorithm.

22. The method according to claim 20 or 21, characterized in that The access information includes access information of a first user, and the information for determining pre-fetched data based on the access information according to the pre-fetch recommendation model includes: the pre-fetch recommendation model determines an access pattern of the first user based on the access information of the first user, and determines data to be pre-fetched according to the access pattern.

23. A computing node, characterized in that: It comprises a processor and a memory, wherein the memory stores executable computer program instructions, and the processor executes the executable computer program instructions to implement the method according to any one of claims 19 to 22.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed in a computer or a processor, the computer or the processor executes the method according to any one of claims 19 to 22.

Images