A data access method, a data server, and a data storage system

Abstract

The present invention provides a data access method, a data server and a data storage system. When receiving a data request sent by a client, the data server selects a target cache server from a primary cache server and a secondary cache server, and sends a cache location query request including the data number of the data accessed by the data access request to the target cache server. If there is data cache location information corresponding to the data number in the target cache server, the data server receives the data cache location information returned by the target cache server. The data server reads data from the cache location specified by the data cache location information through a solid-state disk server and returns the read data to the client. By modifying the configuration of the data server and the data access process, multiple cache servers can provide cache location query services simultaneously, solving the problem of uneven load among multiple cache servers.

Description

Technical Field

[0001] The present invention relates to the technical field of computer storage, and in particular, to a data access method, a data server, and a data storage system. Background Art

[0002] Although existing highly available storage systems allow multiple cache servers to be configured in the system, only one server provides external services at the same time, and other servers are standby machines. When the load exceeds the processing capacity of a single server, the standby servers cannot provide external services either. It is only possible to improve the hardware performance of the single server providing external services, and it is not possible to achieve horizontal expansion by adding standby servers, resulting in the problem of unbalanced loads among multiple cache servers. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data access method, a data server, and a data storage system, achieving the purpose of balancing the loads of multiple cache servers.

[0004] To achieve the above object, on the one hand, the present invention provides a data access method, including:

[0005] The data server receives a data access request sent by a client;

[0006] The data server selects a target cache server from a primary cache server and a secondary cache server, and sends a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request;

[0007] If there is data cache location information corresponding to the data number in the target cache server, the data server receives the data cache location information returned by the target cache server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number;

[0008] The data server reads data from the cache location specified by the data cache location information through a solid-state disk server based on the data cache location information;

[0009] The data server returns the read data to the client.

[0010] Optionally, the data cache location information includes a solid-state disk number and at least one disk block number;

[0011] The data server reads data from the cache location specified by the data cache location information through a solid-state disk server based on the data cache location information, including:

[0012] The data server sends the data read request to the solid-state disk server based on the data cache location information; wherein, the data read request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number, the data block number, and the data cache location information;

[0013] If the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data read request, the solid-state disk server reads data from the disk block of the solid-state disk specified by the data cache location information in the mapping relationship;

[0014] The data server receives the data read by the solid-state disk server.

[0015] Optionally, the method further includes:

[0016] If the solid-state disk server fails to read data, the data server receives the read error message returned by the solid-state disk server;

[0017] The data server sends a storage location query request to the index server;

[0018] The data server receives the data storage location information corresponding to the data number sent by the index server;

[0019] The data server reads data from the storage location specified by the data storage location information through the mechanical disk server based on the data storage location information;

[0020] The data server returns the read data to the client.

[0021] Optionally, the method further includes:

[0022] The main cache server determines the mapping relationship of the data to be stored, and the mapping relationship of the data to be stored includes the correspondence between the data number and data block number of the data to be stored and the data cache location information of the data to be stored;

[0023] The main cache server generates a cache update instruction based on the mapping relationship of the data to be stored, and sends the cache update instruction to the solid-state disk server;

[0024] The solid-state disk server updates the mapping relationship of the data stored in the solid-state disk server according to the cache update instruction and updates the stored data of the solid-state disk in the solid-state disk server;

[0025] The slave cache server synchronizes based on the mapping relationship of the data stored in the solid-state disk server.

[0026] Optionally, the method further includes:

[0027] If the target cache server does not have the data cache location information corresponding to the data number, the data server receives the query failure information returned by the target cache server;

[0028] The data server sends a storage location query request to the index server;

[0029] The data server receives the data storage location information corresponding to the data number sent by the index server;

[0030] Based on the data storage location information, the data server reads data from the storage location specified by the data storage location information through the mechanical disk server;

[0031] The data server returns the read data to the client.

[0032] On the other hand, the present invention provides a data server, including: a receiving unit, a sending unit, and a reading unit;

[0033] The receiving unit is configured to receive a data access request sent by a client;

[0034] The sending unit is configured to select a target cache server from a primary cache server and a secondary cache server, and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request;

[0035] The receiving unit is further configured to, if the target cache server has the data cache location information corresponding to the data number, receive the data cache location information returned by the target cache server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number;

[0036] The reading unit is configured to, based on the data cache location information, read data from the cache location specified by the data cache location information through the solid state disk server, and send the read data to the client through the sending unit.

[0037] Optionally, the reading unit is specifically configured to send the data reading request to the solid-state disk server based on the data cache location information; if the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data reading request, receive the data read by the solid-state disk server from the disk block of the solid-state disk specified by the data cache location information in the mapping relationship; wherein, the data cache location information includes a solid-state disk number and at least one disk block number, the data reading request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number, the data block number, and the data cache location information.

[0038] Optionally, the receiving unit is further configured to receive a read error message returned by the solid-state disk server if the solid-state disk server fails to read the data; and / or, is further configured to receive a query failure message returned by the target cache server if the target cache server does not have the data cache location information corresponding to the data number.

[0039] The sending unit is further configured to send a storage location query request to the index server.

[0040] The receiving unit is further configured to receive the data storage location information corresponding to the data number sent by the index server.

[0041] The reading unit is further configured to read data from the storage location specified by the data storage location information through the mechanical disk server based on the data storage location information, and send the read data to the client through the sending unit.

[0042] Another aspect of the present invention provides a data storage system, including: a data server, a main cache server, at least one slave cache server, and a solid-state disk server.

[0043] The data server is configured to receive a data access request sent by a client, select a target cache server from the main cache server and at least one slave cache server, and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request.

[0044] The target cache server among the main cache server and the at least one slave cache server is configured to return the data cache location information to the data server if the target cache server has the data cache location information corresponding to the data number; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number.

[0045] The data server is further configured to send the data cache location information to the solid-state disk server;

[0046] The solid-state disk server is configured to read data from the cache location specified by the data cache location information, and send the read data to the client through the data server.

[0047] Optionally, the data server is specifically configured to send a data read request to the solid-state disk server, where the data read request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number and the data block number and the data cache location information;

[0048] The solid-state disk server is further configured to detect the mapping relationship recorded in the solid-state disk server and the mapping relationship in the data read request; if the mapping relationships are inconsistent, send a read error message to the data server;

[0049] The main cache server is further configured to determine the mapping relationship of the data to be stored, where the mapping relationship of the data to be stored includes the correspondence between the data number and the data block number of the data to be stored and the data cache location information of the data to be stored; generate a cache update instruction based on the mapping relationship of the data to be stored, and send the cache update instruction to the solid-state disk server;

[0050] The solid-state disk server is further configured to update the mapping relationship of the data stored in the solid-state disk server and update the stored data of the solid-state disk in the solid-state disk server according to the cache update instruction;

[0051] The slave cache server is further configured to perform synchronization based on the mapping relationship of the data stored in the solid-state disk server.

[0052] As described in the above solution, when a data request sent by a client is received, the data server selects a target cache server from the primary cache server and the secondary cache servers, and sends a cache location query request to the target cache server. Among them, the cache location query request includes the data number of the data accessed by the data access request. If there is data cache location information corresponding to the data number in the target cache server, the data server receives the data cache location information returned by the target cache server. Based on the data cache location information, the data server reads the data from the cache location specified by the data cache location information through the solid-state disk server and returns the read data to the client. Different from the prior art where only one cache server provides external services at the same time, by modifying the configuration of the data server and the data access process, the data server can configure the parameters (such as IP addresses) of multiple cache servers (such as a primary cache server and at least one secondary cache server), and multiple cache servers can simultaneously provide cache location query services to the data server to solve the problem of uneven load of multiple cache servers. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0054] Figure 1 It is a flowchart of a data access method according to an embodiment of the present invention;

[0055] Figure 2 It is a flowchart of a data access method according to another embodiment of the present invention;

[0056] Figure 3 It is a flowchart of a data access method according to another embodiment of the present invention;

[0057] Figure 4 It is a signaling diagram of the data access process according to the above embodiment of the present invention (in the case where the cache location is not found);

[0058] Figure 5 It is a signaling diagram of the data access process according to the above embodiment of the present invention (in the case where the cache location is found and the mapping relationship is consistent);

[0059] Figure 6 It is a signaling diagram of the data access process according to the above embodiment of the present invention (in the case where the cache location is found and the mapping relationship is inconsistent);

[0060] Figure 7Schematic diagram of the structure of a data server according to another embodiment of the present invention;

[0061] Figure 8 Schematic diagram of the structure of a data storage system according to another embodiment of the present invention. Detailed implementation manners

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] In this application, the term "including", "comprising" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including a..." does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0064] The prior art generally realizes the high availability of cache servers based on the open-source Keepalived. The two cache servers in the cache system each have an actual IP address. Assume that their IP addresses are address A and address B respectively. Keepalived sets a common, external virtual IP address. At the same time, the virtual IP address will only point to one of address A or address B. Only the cache server pointed to by the virtual IP address can provide the service of querying the cache to the data server. When the query cache load exceeds the processing capacity of a single cache server, the standby cache server cannot share a part of the query cache service either, resulting in the problem of uneven load among multiple cache servers.

[0065] On the other hand, when the cache server connected to the data server fails, Keepalived switches the virtual IP address pointing to the IP address of another cache server. The switching process of the virtual IP address pointing generally takes 3 to 5 seconds. During this period, the query service of the cache server is interrupted, and the data server will consider the query failed and send all data access requests to the mechanical disk server, resulting in waste of solid-state disk resources and at the same time reducing the data access efficiency. Because accessing data through the solid-state disk server is faster than accessing data through the mechanical disk server. These 3 to 5 seconds of interrupted cache service are unacceptable for a data server running at high load.

[0066] Aiming at the problems of uneven load among multiple cache servers and untimely switching of cache servers existing in the existing high-availability solutions. The solution of the present invention modifies the configuration of the data server and the data access process, enabling the data server to configure the parameters of multiple cache servers, and multiple cache servers can simultaneously provide cache location query services for it, thus solving the problem of uneven load among multiple cache servers. If a network anomaly occurs or the cache server currently providing services fails, resulting in the data server being unable to connect to the cache server, the data server will send a query request to another cache server according to the polling order, solving the problem of untimely switching of cache servers.

[0067] Please refer to FIG. 1, which shows a flowchart of a data access method according to an embodiment of the present invention, including:

[0068] S101, the data server receives a data access request sent by the client.

[0069] In this step, the data server can configure the parameters of multiple cache servers, such as IP addresses. The multiple cache servers include a primary cache server and at least one secondary cache server. In terms of the function of querying data cache location information, the functions of all cache servers are equivalent. Therefore, in the cache-related configuration items of the data server, the differences between the primary cache server and the secondary cache servers are not reflected. For example, the configuration item of the data server can adopt the following configuration description:

[0070] cache-server = IP address of primary cache server A, IP address of secondary cache server B, IP address of secondary cache server C;

[0071] Or, cache-server = IP address of secondary cache server B, IP address of primary cache server A, IP address of secondary cache server C;

[0072] The sequence of the IP addresses of the primary cache server and the secondary cache servers is not limited, and the IP addresses of different cache servers are separated by commas.

[0073] The data access request in this step includes the data identifier of the data accessed by the client, such as the data name, and the data name is used to indicate what data the client wants to access. For example, the data name can be the file name of the file to be accessed.

[0074] S102, the data server selects a target cache server from the primary cache server and the secondary cache servers, and sends a cache location query request to the target cache server.

[0075] In this step, the data server can randomly select a target cache server from the primary cache server and the secondary cache server; or sequentially select a target cache server from the primary cache server and the secondary cache server in a fixed order; or use other polling methods that can achieve balanced access to the primary cache server and the secondary cache server to determine the target cache server.

[0076] For example, if the primary cache server and the secondary cache server are numbered as cache servers No. 1 to No. 3 respectively, the data server can use the sequential polling method to select cache servers No. 1 to No. 3 as the target cache server: the data server selects cache server No. 1 as the target cache server for the first time and sends a cache location query request to cache server No. 1; the data server selects cache server No. 2 as the target cache server for the second time and sends a cache location query request to cache server No. 2; the data server selects cache server No. 3 as the target cache server for the third time and sends a cache location query request to cache server No. 3; the data server selects cache server No. 1 as the target cache server for the fourth time and sends a cache location query request to cache server No. 1...

[0077] If a network anomaly occurs or the current target cache server fails, resulting in a connection failure between the data server and the target cache server, the data server will select the next cache server as the target cache server according to the polling order and send a cache location query request to it. This speed of selecting a cache server is faster than sending a data access request to the mechanical disk server, thus solving the problem of untimely switching of the cache server.

[0078] Before sending the cache location query request, determine the data number (fid), data offset position (offset), and data length (length) according to the data identifier in the data access request. In this step, the cache location query request includes the data number, data offset position, and data length of the data accessed by the data access request.

[0079] S103. Determine whether there is data cache location information corresponding to the data number in the target cache server. If so, execute steps S104 - S105; if not, execute steps S106 - S109.

[0080] Among them, the data cache location information is used to indicate the storage location of the data block on the solid - state disk, including the solid - state disk number (did) and at least one disk block number (disk_block_id). A solid - state disk server includes multiple slots and can insert multiple solid - state disks. A solid - state disk includes multiple disk blocks, and a solid - state disk can be regarded as a device that stores a byte stream continuously. The data block size of the target cache server is the same as the disk block size of the solid - state disk, so as to store a data block in one disk block of the solid - state disk.

[0081] In practical applications, the data to be accessed by the data server can be regarded as a byte stream, and the size of a data block can be preset, for example, it can be 1M, 2M, 4M or other sizes. For the data to be accessed with the same data length, the larger a data block is, the smaller the total number of data blocks for accessing the data is. The number of data blocks and the data block numbers corresponding to different data are different. Therefore, in this step, the target cache server obtains the data block number (file_black_id) of the accessed data based on the data number, data offset position, and data length in the data access request.

[0082] There is a corresponding relationship between the data block number and the data cache location information, and the corresponding data cache location information is found through the data block number. It should be noted here that the corresponding relationship between the data block number and the data cache location information can be recorded in the target cache server in the form of a mapping relationship. For example, the mapping relationship includes the corresponding relationship between (fid, file_blcok_id) and the data cache location information.

[0083] S104. The data server receives the data cache location information returned by the target cache server.

[0084] S105. The data server, based on the data cache location information, reads data from the cache location specified by the data cache location information through the solid-state disk server and returns the read data to the client.

[0085] In this step, the data server sends a data read request to the solid-state disk server based on the data cache location information. The solid-state disk server reads the data at the corresponding location according to the data cache location information in the data read request. For example, it reads data from the disk blocks of the corresponding solid-state disk based on the solid-state disk number (did) and at least one disk block number (disk_block_id) in the data cache location information, and sends the read data back to the data server. The data server receives the data read by the solid-state disk server and returns the read data to the client.

[0086] S106. The data server receives the query failure information returned by the target cache server.

[0087] S107. The data server sends a storage location query request to the index server.

[0088] The storage location query request includes the data number, data offset position, and data length of the data accessed by the data access request. The index server obtains the index data block number (file_block_id2) of the accessed data according to the data number, data offset position, and data length, where the size of the index data block and the data block in the cache server are not necessarily the same.

[0089] S108, the data server receives the data storage location information corresponding to the data number sent by the index server.

[0090] The index server records the storage locations of all data blocks on the mechanical disk (i.e., the data storage location information). According to the index data block number, the data storage location information corresponding to the index data block number is obtained. The data storage location information includes the mechanical disk number (did2), the disk number of the mechanical disk (disk_block_id2), and the location of the mechanical disk service implementation example (ip, port). The index data block and the mechanical disk disk block have the same size. The mechanical disk can also be regarded as a device that stores byte streams continuously.

[0091] S109, the data server reads the data based on the data storage location information through the mechanical disk server and returns the data to the client.

[0092] In this step, the data server sends a data read request to the mechanical disk server. The mechanical disk server reads the data at the corresponding location according to the data storage location information in the data read request and sends it to the data server. The data server receives the data read by the mechanical disk server and returns the read data to the client.

[0093] The solution of this embodiment solves the problem of uneven load among multiple cache servers by modifying the configuration of the data server and the data access process, enabling the data server to configure the IP addresses of multiple cache servers, and multiple cache servers can simultaneously provide cache location query services for it. If the network is abnormal or the currently serving cache server fails, resulting in the data server being unable to connect to the cache server, the data server will switch to another cache server as the target cache server and send a query request to the target cache server, which is faster than the existing speed of switching the server and sending the data access request to the mechanical disk server, thus solving the problem of untimely switching of cache servers.

[0094] In the actual application process, it is found that in the storage system, cache data is often updated according to the usage of the data, and the mapping relationships of multiple cache servers need to be synchronized. The existing synchronization method synchronizes by converting the data to be cached into a snapshot. However, the three steps of converting the data to be cached into a snapshot, transmitting the snapshot between cache servers, and reconstructing the data to be cached from the snapshot are all very time-consuming.

[0095] To solve the problem of synchronization time consumption, please refer to Figure 2 , which shows the flowchart of a data access method according to another embodiment of the present invention. Compared with Figure 1 , steps S110 - S113 are added, and the process of caching in the master-slave cache servers includes:

[0096] S110, the primary cache server determines the mapping relationship of the data to be stored.

[0097] Among them, the mapping relationship of the data to be stored includes the corresponding relationship between the data number and data block number of the data to be stored and the data cache location information of the data to be cached, that is, the corresponding relationship between (fid, file_block_id) of the data to be stored and (did, disk_block_id) of the data to be stored. The data to be stored can be determined according to the statistically hot data blocks. The hot data blocks are, but not limited to, the data blocks that are frequently accessed. For example, the primary cache server can regularly count the access requests of the data server, and regard the data blocks whose access times exceed the preset number as hot data blocks, then the data in the hot data blocks is used as the data to be stored.

[0098] S111, the primary cache server generates a cache update instruction based on the mapping relationship of the data to be stored, and sends the cache update instruction to the solid-state disk server.

[0099] Among them, the cache update instruction includes instructions such as adding cache, deleting cache or modifying cache.

[0100] S112, the solid-state disk server updates the mapping relationship of the data stored in the solid-state disk server and updates the stored data in the solid-state disk of the solid-state disk server according to the cache update instruction.

[0101] Specifically, the solid-state disk server may add new mapping relationships, delete the stored mapping relationships or modify the stored mapping relationships according to the cache update instruction. At the same time, the solid-state disk in the solid-state disk server adds, deletes or modifies the stored data based on the change of the mapping relationship.

[0102] S113, the secondary cache server synchronizes based on the mapping relationship of the data stored in the solid-state disk server.

[0103] In this step, the secondary cache server and the primary cache server are independent of each other. Because the solid-state disk is a natural snapshot, the secondary cache server directly synchronizing the mapping relationship from the solid-state disk server will be more efficient and time-saving than the existing synchronization methods. For example, the secondary cache server can regularly synchronize the mapping relationship of the data to be stored from the solid-state disk server.

[0104] It should be noted that the caching process of the primary and secondary cache servers in steps S110 - S113 is not necessarily executed after step S109. Figure 2 This only shows one of the implementation manners, and the execution timing of steps S110 - S113 is not specifically limited here.

[0105] The solution of this embodiment separates the master and slave cache servers, and the slave cache server synchronizes based on the mapping relationship of the data stored in the solid-state disk server, solving the problem of time-consuming synchronization between the master and slave cache servers and improving the cache efficiency.

[0106] Based on the above method where the master cache server instructs the solid-state disk server to update the mapping relationship, and the slave cache server synchronizes to update the mapping relationship, there is a problem of weak consistency. This is because when the master cache server sends a cache update instruction to the solid-state disk server, the mapping relationship of the entire cache system has actually been updated. However, since the slave cache server and the master cache server are independent and do not communicate in real time. This means that within a short period of time before the slave cache server synchronizes the mapping relationship from the solid-state disk server, the cache mapping relationship on the slave cache server is the unupdated mapping relationship, which is different from the updated mapping relationship, that is, the mapping relationships on the master and slave cache servers are inconsistent.

[0107] To solve the problem of weak consistency between the master and slave cache servers, the present invention provides another embodiment of a data access method. Please refer to FIG. 3, which shows a flowchart of the method, including:

[0108] S301, the data server receives a data access request sent by the client.

[0109] S302, the data server selects a target cache server from the master cache server and the slave cache server, and sends a cache location query request to the target cache server.

[0110] S303, determine whether there is data cache location information corresponding to the data number in the target cache server. If it exists, execute steps S304 - S307. If it does not exist, execute steps S308 - S311.

[0111] The above steps S301 - S303 are similar to steps S101 - S103. For specific descriptions, please refer to the descriptions of steps S101 - S103 in the above embodiment, and will not be elaborated here.

[0112] S304, the data server receives the data cache location information returned by the target cache server.

[0113] Among them, the data cache location information can be returned by the target cache server in the form of a mapping relationship, where the mapping relationship is the mapping relationship of the data accessed by the data access request, including the correspondence between the data number and the data block number and the data cache location information, that is, the correspondence between (fid, file_blcok_id) and (did, disk_block_id). Based on the data cache location information in the mapping relationship, the solid-state disk server can know the storage location of the data to be read. For the explanations of terms such as data cache location information and solid-state disk server, please refer to step S104 and will not be elaborated here.

[0114] S305. The data server sends a data read request to the solid-state disk server.

[0115] Since the mapping relationship enables the solid-state server to know the storage location of the data to be read, the data read request can be sent to the solid-state disk server based on the mapping relationship when sending the data read request. Among them, the data read request includes the mapping relationship returned by the target cache server.

[0116] S306. The data server reads the data from the cache location specified by the data cache location information through the solid-state disk server and returns the read data to the client.

[0117] In this step, the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data read request. The target cache server may be the main cache server or a slave cache server that has been synchronized in a timely manner, and the returned mapping relationship is the latest mapping relationship. Since the mapping relationship on the solid-state disk server is consistent with the mapping relationship in the main cache server, the data cache location information in the mapping relationship is accurate, and the data in the cache location is also the data that the client wants to access. For the description of the process of the solid-state disk server reading data, please refer to step S105 and will not be elaborated here.

[0118] S307. If the solid-state disk server fails to read the data, the data server receives the read error message returned by the solid-state disk server.

[0119] In this step, the reason why the solid-state disk server fails to read the data may be that the mapping relationship recorded in the solid-state disk server is inconsistent with the mapping relationship in the data read request, indicating that the target cache server may be a slave cache server that has not yet synchronized the latest mapping relationship. Therefore, the data cache location information in the returned mapping relationship is inaccurate, and the data in the cache location is not the data that the client wants to access. So the solid-state disk server returns a read error message. The reason why the solid-state disk server fails to read the data may also be that the solid-state disk server and / or the solid-state disk have anomalies, resulting in a read failure.

[0120] After the data server receives the message indicating a read error, it executes steps S309 - S311.

[0121] S308, the data server receives the query failure information returned by the target cache server.

[0122] S309, the data server sends a storage location query request to the index server.

[0123] S310, the data server receives the data storage location information corresponding to the data number sent by the index server.

[0124] S311, the data server reads the data based on the data storage location information through the mechanical disk server and returns the data to the client.

[0125] The above steps S308 - S311 are similar to steps S106 - S109. For the specific description of the steps, please refer to the description of steps S106 - S109 in the above embodiments, and will not be elaborated here.

[0126] The solution of this embodiment ensures the accuracy of the queried data and solves the problem of weak consistency of the master - slave cache servers by modifying the query process, that is, by carrying the mapping relationship queried by the target cache server in the data read request sent by the data server to the solid - state disk server, and having the solid - state disk server check the mapping relationship.

[0127] The following describes the data access process of the above - mentioned embodiment in combination with the signaling diagram. The process is as follows:

[0128] Please refer to Figure 4 , which shows the signaling diagram when the data to be accessed by the client is not cached in the target cache server.

[0129] The client sends a data access request to the data server; among them, the data access request carries the data name to be accessed by the client.

[0130] The data server receives the data access request, determines the data number fid, data offset position offset, and data length length according to the data name; the data server sends a cache location query request to the target cache server, and the request carries (fid, offset, length).

[0131] The target cache server obtains (fid, file_block_id) according to (fid, offset, length) in the cache location query request, and does not find the data cache location information (did, disk_block_id) corresponding to (fid, file_block_id), and returns a query failure message to the data server.

[0132] After the data server receives the query failure message, it sends a storage location query request to the index server, and the request carries (fid, offset, length).

[0133] Based on (fid, offset, length) in the storage location query request, the index server obtains (fid, file_block_id2); and based on (fid, file_block_id2), it finds the corresponding data storage location information (did2, disk_block_id2, ip_port), and returns the data storage location information to the data server.

[0134] The data server sends a data read request to the mechanical disk server, and the data read request carries (did2, disk_block_id2, ip_port).

[0135] The mechanical disk server reads the data at the corresponding location according to (did2, disk_block_id2, ip_port) and returns it to the data server.

[0136] The data server receives the read data and returns it to the client.

[0137] Please refer to Figure 5 , which shows a signaling diagram when the data to be accessed by the client is cached in the target cache server and the mapping relationship recorded by the solid-state disk server is consistent with the mapping relationship in the data read request.

[0138] The client sends a data access request to the data server; among them, the data access request carries the data name to be accessed by the client.

[0139] The data server receives the data access request, determines the data number fid, data offset position offset, and data length length according to the data name; the data server sends a cache location query request to the target cache server, and the request carries (fid, offset, length).

[0140] Based on (fid, offset, length) in the cache location query request, the target cache server obtains (fid, file_block_id), obtains the corresponding data cache location information (did, disk_block_id) of (fid, file_block_id), and sends the data cache location information (did, disk_block_id) to the data server in the form of a mapping relationship. Among them, the mapping relationship is the corresponding relationship between (fid, file_block_id) and (did, disk_block_id).

[0141] The data server sends a data read request to the solid-state disk server, where the data read request carries a mapping relationship, (fid, file_block_id) → (did, disk_block_id).

[0142] The solid-state disk server checks that the mapping relationship in the data read request is consistent with the recorded mapping relationship, and reads the data at the corresponding location according to (did, disk_block_id) in the mapping relationship and returns it to the data server.

[0143] The data server receives the read data and returns it to the client.

[0144] Please refer to Figure 6 , which shows a signaling diagram when the data to be accessed by the client is cached in the target cache server, but the mapping relationship recorded by the solid-state disk server is inconsistent with the mapping relationship in the data read request.

[0145] The client sends a data access request to the data server; among them, the data access request carries the data name to be accessed by the client.

[0146] The data server receives the data access request, determines the data number fid, data offset position offset, and data length according to the data name; the data server sends a cache location query request to the target cache server, and the request carries (fid, offset, length).

[0147] The target cache server obtains (fid, file_block_id) according to (fid, offset, length) in the cache location query request, obtains the corresponding data cache location information (did, disk_block_id) of (fid, file_block_id), and returns the mapping relationship to the data server. Among them, the mapping relationship is the corresponding relationship between (fid, file_block_id) and (did, disk_block_id).

[0148] The data server sends a data read request to the solid-state disk server, where the data read request carries a mapping relationship, (fid, file_block_id) → (did, disk_block_id).

[0149] The solid-state disk server checks that the mapping relationship in the data read request is inconsistent with the recorded mapping relationship, and returns a read failure message to the data server.

[0150] After the data server receives a read failure message, it sends a storage location query request to the index server, and the request carries (fid, offset, length).

[0151] Based on (fid, offset, length) in the storage location query request, the index server obtains (fid, file_block_id2); and based on (fid, file_block_id2), it finds the corresponding data storage location information (did2, disk_block_id2, ip_port), and returns the data storage location information to the data server.

[0152] The data server sends a data read request to the mechanical disk server, and the data read request carries (did2, disk_block_id2, ip_port).

[0153] The mechanical disk server reads the data at the corresponding location according to (did2, disk_block_id2, ip_port) and returns it to the data server.

[0154] The data server receives the read data and returns it to the client.

[0155] Please refer to Figure 7 , which shows a schematic structural diagram of a data server according to another embodiment of the present invention. The data server in this embodiment can configure the parameters of multiple cache servers, such as IP addresses. The multiple cache servers include one main cache server and at least one slave cache server. In the function of querying data cache location information, the functions of all cache servers are equivalent. Therefore, in the cache-related configuration items of the data server, the differences between the main cache server and the slave cache server are not reflected. The data server includes: a receiving unit 701, a sending unit 702, and a reading unit 703. The functions of each unit are as follows:

[0156] The receiving unit 701 is used to receive a data access request sent by the client. The data access request includes the data identifier of the data accessed by the client, such as the data name, and the data name is used to indicate what data the client wants to access. For example, the data name can be the file name of the file to be accessed.

[0157] The sending unit 702 is used to select a target cache server from the main cache server and the slave cache servers, and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request. For the description of the selection method of the target cache server, please refer to step S102 of the above embodiment, which will not be elaborated here.

[0158] The receiving unit 701 is further configured to, if there is data cache location information corresponding to the data number in the target cache server, receive the data cache location information returned by the target cache server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number.

[0159] In one implementation, the data cache location information received by the receiving unit 701 is returned by the target cache server in the form of a mapping relationship, where the mapping relationship includes the correspondence between the data number, the data block number, and the data cache location information. For the conceptual explanations of the data block number and the mapping relationship, please refer to steps S103 and S304 of the above embodiments. For the description of the working process of different implementations of the receiving unit 701, please refer to steps S104 and S304 of the above embodiments.

[0160] The reading unit 703 is configured to read data from the cache location specified by the data cache location information through the solid-state disk server based on the data cache location information.

[0161] One implementation of the reading unit 703 is specifically configured to, based on the data cache location information, send a data reading request to the solid-state disk server; and receive the data read by the solid-state disk server from the disk block of the solid-state disk specified by the data cache location information. Among them, the data reading request includes the data cache location information, and the data cache location information is used to indicate the storage location of the data block in the solid-state disk, including the solid-state disk number and at least one disk block number.

[0162] Another implementation of the reading unit 703 is specifically configured to, based on the data cache location information, send a data reading request to the solid-state disk server, and the data reading request includes a mapping relationship. The reading unit 703 sends the data cache location information to the solid-state disk server in the form of a mapping relationship; if the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data reading request, the reading unit 703 receives the data read by the solid-state disk server from the disk block of the solid-state disk specified by the data cache location information in the mapping relationship.

[0163] The receiving unit 701 is further configured to, if the solid-state disk server fails to read the data, receive the read error message returned by the solid-state disk server; and / or, is further configured to, if there is no data cache location information corresponding to the data number in the target cache server, receive the query failure information returned by the target cache server. For the description of this function, please refer to steps S307 and S308 of the above embodiments.

[0164] The sending unit 702 is further configured to send a storage location query request to the index server.

[0165] The receiving unit 701 is further configured to receive the data storage location information corresponding to the data number sent by the index server.

[0166] The reading unit 703 is further configured to read data from the storage location specified by the data storage location information through the mechanical disk server based on the data storage location information.

[0167] For the functional descriptions of the sending unit 701, the fourth receiving unit 702, and the second reading unit 703 to read data through the mechanical disk server, please refer to steps S106 - S109 of the above embodiments, which will not be elaborated here.

[0168] The sending unit 701 is further configured to send the read data to the client. The read data can be cached data read through the solid - state disk server or stored data read through the mechanical disk server.

[0169] In this embodiment, the data server modifies the configuration and data access process, enabling the data server to configure the IP addresses of multiple cache servers. Multiple cache servers can simultaneously provide cache location query services for it, solving the problem of uneven load among multiple cache servers. If a network anomaly occurs or the cache server currently providing services fails, resulting in a connection failure between the data server and the cache server, the data server will switch to another cache server as the target cache server and send a query request to the target cache server. This is faster than the existing method of switching servers and sending data access requests to the mechanical disk server, thus solving the problem of untimely switching of cache servers.

[0170] Please refer to Figure 8 , which shows a schematic structural diagram of a data storage system according to another embodiment of the present invention, including a data server, a main cache server, at least one slave cache server, and a solid - state disk server.

[0171] The data server is configured to receive a data access request sent by the client, select a target cache server from the main cache server and at least one slave cache server, and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request.

[0172] The target cache server among the main cache server and at least one slave cache server is configured to, if there is data cache location information corresponding to the data number in the target cache server, return the data cache location information to the data server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number. For the functional description of this, please refer to steps S103, S104, and S106 of the above embodiments, which will not be elaborated here.

[0173] The data server is also used to send data cache location information to the solid-state disk server. For example, it is specifically used to send a data reading request to the solid-state disk server, where the data reading request includes data cache location information; or the data reading request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number and the data block number and the data cache location information.

[0174] The solid-state disk server is used to read data from the cache location specified by the data cache location information, and send the read data to the client through the data server. For the description of this function, please refer to step S112 of the above embodiment, which will not be elaborated here.

[0175] When the data reading request sent by the data server includes a mapping relationship, the solid-state disk server is also used to detect the mapping relationship recorded in the solid-state disk server and the mapping relationship in the data reading request; if the mapping relationships are consistent, read data from the cache location specified by the data cache location information, and send the read data to the client through the data server; if the mapping relationships are inconsistent, send a read error message to the data server. For the description of this function, please refer to steps S306 and S307 of the above embodiment, which will not be elaborated here.

[0176] In addition, the main cache server is also used to determine the mapping relationship of the data to be stored. The mapping relationship of the data to be stored includes the correspondence between the data number and the data block number of the data to be stored and the data cache location information of the data to be stored. For the description of this function, please refer to step S110 of the above embodiment, which will not be elaborated here. The main cache server is also used to generate a cache update instruction based on the mapping relationship of the data to be stored, and send the cache update instruction to the solid-state disk server. For the description of this function, please refer to step S111 of the above embodiment, which will not be elaborated here.

[0177] The solid-state disk server is also used to update the mapping relationship of the data stored in the solid-state disk server and update the stored data of the solid-state disk in the solid-state disk server according to the cache update instruction. For the description of this function, please refer to step S112 of the above embodiment, which will not be elaborated here.

[0178] The slave cache server is also used to perform synchronization based on the mapping relationship of the data stored in the solid-state disk server. For the description of this function, please refer to step S113 of the above embodiment, which will not be elaborated here.

[0179] The solution of this embodiment separates the master and slave cache servers. The slave cache server synchronizes based on the mapping relationship of the data stored in the solid-state disk server, solving the problem of time-consuming synchronization between the master and slave cache servers and improving the cache efficiency. At the same time, the solution of this embodiment modifies the query process, that is, the mapping relationship queried by the target cache server is carried in the data read request sent by the data server to the solid-state disk server, and the solid-state disk server checks the mapping relationship, ensuring the accuracy of the queried data and solving the problem of weak consistency between the master and slave cache servers.

[0180] In addition, the data storage system further includes an index server and a mechanical disk server.

[0181] The index server is used to receive the storage location query request sent by the data server and return the data storage location information according to the storage location query request. The storage location query request can be sent when the solid-state disk server fails to read the data, such as when the data server receives a read error message. The mechanical disk server is used to receive the data read request sent by the data server, read the data according to the data read request, and send it to the data server, so as to continue to provide data to the data server through the index server and the mechanical disk server when the solid-state disk server fails to read the data.

[0182] For the working processes of the index server and the mechanical disk server, please refer to steps S108 and S109 of the above embodiment, which will not be elaborated here.

[0183] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0184] Those skilled in the art may further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of function in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0185] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data access method, characterized in that, it includes: The data server receives a data access request sent by the client; The data server selects a target cache server from the primary cache server and the secondary cache server, and sends a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request; If there is data cache location information corresponding to the data number in the target cache server, the data server receives the data cache location information returned by the target cache server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number; The data server reads data from the cache location specified by the data cache location information through the solid-state disk server based on the data cache location information; If the solid-state disk server fails to read the data, the data server receives a read error message returned by the solid-state disk server; the data server sends a storage location query request to the index server; the data server receives the data storage location information corresponding to the data number sent by the index server; the data server reads data from the storage location specified by the data storage location information through the mechanical disk server based on the data storage location information; If the target cache server does not have data cache location information corresponding to the data number, the data server receives a query failure message returned by the target cache server; the data server sends a storage location query request to the index server; the data server receives the data storage location information corresponding to the data number sent by the index server; the data server reads data from the storage location specified by the data storage location information through the mechanical disk server based on the data storage location information; The data server returns the read data to the client.

2. The method according to claim 1, characterized in that, the data cache location information includes a solid-state disk number and at least one disk block number; The data server reads data from the cache location specified by the data cache location information through the solid-state disk server based on the data cache location information, including: The data server sends the data read request to the solid-state disk server based on the data cache location information; wherein, the data read request includes a mapping relationship, and the mapping relationship includes the corresponding relationship between the data number and the data block number and the data cache location information; If the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data read request, the solid-state disk server reads data from the disk block of the solid-state disk specified by the data cache location information in the mapping relationship; The data server receives the data read by the solid-state disk server.

3. The method according to any one of claims 1-2, characterized in that, it further includes: The primary cache server determines the mapping relationship of the data to be stored, where the mapping relationship of the data to be stored includes the correspondence between the data number and data block number of the data to be stored and the data cache location information of the data to be stored; Based on the mapping relationship of the data to be stored, the primary cache server generates a cache update instruction and sends the cache update instruction to the solid-state disk server; According to the cache update instruction, the solid-state disk server updates the mapping relationship of the data stored in the solid-state disk server and updates the stored data of the solid-state disk in the solid-state disk server; The secondary cache server synchronizes based on the mapping relationship of the data stored in the solid-state disk server.

4. A data server, characterized in that, it includes: a receiving unit, a sending unit, and a reading unit; The receiving unit is configured to receive a data access request sent by a client; The sending unit is configured to select a target cache server from the primary cache server and the secondary cache server and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request; The receiving unit is further configured to, if there is data cache location information corresponding to the data number in the target cache server, receive the data cache location information returned by the target cache server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number; The reading unit is configured to, based on the data cache location information, read data from the cache location specified by the data cache location information through the solid-state disk server and send the read data to the client through the sending unit; The receiving unit is further configured to, if the solid-state disk server fails to read the data, receive a read error message returned by the solid-state disk server; and / or, is further configured to, if the target cache server does not have data cache location information corresponding to the data number, receive a query failure message returned by the target cache server; The sending unit is further configured to send a storage location query request to the index server; The receiving unit is further configured to receive the data storage location information corresponding to the data number sent by the index server; The reading unit is further configured to, based on the data storage location information, read data from the storage location specified by the data storage location information through the mechanical disk server and send the read data to the client through the sending unit.

5. The data server according to claim 4, characterized in that, The reading unit is specifically configured to send the data reading request to the solid-state disk server based on the data cache location information; if the mapping relationship recorded in the solid-state disk server is consistent with the mapping relationship in the data reading request, receive the data read by the solid-state disk server from the disk block of the solid-state disk specified by the data cache location information in the mapping relationship; wherein, the data cache location information includes a solid-state disk number and at least one disk block number, the data reading request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number, the data block number, and the data cache location information.

6. A data storage system characterized in that it includes: a data server, a main cache server, at least one slave cache server, a solid-state disk server, an index server, and a mechanical disk server; The data server is configured to receive a data access request sent by a client, select a target cache server from the main cache server and at least one slave cache server, and send a cache location query request to the target cache server; wherein, the cache location query request includes the data number of the data accessed by the data access request. The target cache server among the main cache server and the at least one slave cache server is configured to, if there is data cache location information corresponding to the data number in the target cache server, return the data cache location information to the data server; wherein, the data cache location information is obtained by the target cache server based on the data block number corresponding to the data number. The data server is further configured to send the data cache location information to the solid-state disk server. The solid-state disk server is configured to read data from the cache location specified by the data cache location information, and send the read data to the client through the data server. The data server is further configured to, if the solid-state disk server fails to read the data, receive a read error message returned by the solid-state disk server; and / or, is further configured to, if the target cache server does not have data cache location information corresponding to the data number, receive a query failure message returned by the target cache server. The data server is further configured to send a storage location query request to the index server. The index server is configured to return data storage location information corresponding to the data number according to the storage location query request. The data server is further configured to, based on the data storage location information, read data from the storage location specified by the data storage location information through the mechanical disk server, and return the read data to the client.

7. The data storage system according to claim 6 characterized in that The data server is specifically configured to send a data reading request to the solid-state disk server, the data reading request includes a mapping relationship, and the mapping relationship includes the correspondence between the data number, the data block number, and the data cache location information. The solid-state drive server is further configured to detect the mapping relationship recorded in the solid-state drive server and the mapping relationship in the data read request; if the mapping relationships are inconsistent, send a read error message to the data server; The main cache server is further configured to determine the mapping relationship of the data to be stored, where the mapping relationship of the data to be stored includes the correspondence between the data number and data block number of the data to be stored and the data cache location information of the data to be stored; generate a cache update instruction based on the mapping relationship of the data to be stored, and send the cache update instruction to the solid-state drive server; The solid-state drive server is further configured to update the mapping relationship of the data stored in the solid-state drive server and update the stored data of the solid-state drive in the solid-state drive server according to the cache update instruction; The secondary cache server is further configured to perform synchronization based on the mapping relationship of the data stored in the solid-state drive server.

Images