A data reading method, device, storage medium and electronic device

Abstract

The application relates to a data reading method and device, a storage medium and an electronic device. The method comprises the following steps: receiving a data reading request, wherein the data reading request is used for requesting reading target data from a distributed data storage system, the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers; reading data from a target server corresponding to the target data among the plurality of distributed data storage servers; determining the target data according to the data read from the target server; and sending the target data in response to the data reading request. The application solves the technical problem that a large amount of network resources is wasted in the process of reading data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, and in particular to a data reading method and device, a storage medium, and an electronic device. BACKGROUND

[0002] In a distributed storage system, when a user initiates an EC (error correction code) read request for data reading, the original data blocks stored in each distributed data storage server are read back concurrently, the complete data is assembled, and data consistency verification is performed. After verification, the data is returned to the user according to the requirements. When the user initiates a Range read request to read only part of the data, the complete data stored in each distributed data storage server is also read and verified. This processing method magnifies the consumption of the network and the disk IO (input / output), which is a waste of overall system resources, and the time delay of reading is also increased accordingly.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The present application provides a data reading method and device, a storage medium, and an electronic device to at least solve the technical problem of wasting a large amount of network resources in the process of reading data in the related art.

[0005] According to an aspect of an embodiment of the present application, a data reading method is provided, comprising:

[0006] receiving a data reading request, wherein the data reading request is used to request reading target data from a distributed data storage system, and the target data is part of the data stored in the distributed data storage system by a plurality of distributed data storage servers;

[0007] reading data from a target server corresponding to the target data in the plurality of distributed data storage servers;

[0008] determining the target data according to the data read from the target server;

[0009] sending the target data in response to the data reading request.

[0010] Optionally, reading data from the target server corresponding to the target data in the plurality of distributed data storage servers comprises:

[0011] determining the target server from the plurality of distributed data storage servers according to data offset information and data length information carried in the data read request and an amount of data stored by each of the distributed data storage servers, wherein the data offset information is used to indicate an offset of the target data in data stored by the plurality of distributed data storage servers, and the data length information is used to indicate a length of the target data;

[0012] sending a first instruction to each of the target servers, wherein the first instruction is used to instruct reading all data stored in each of the servers, and the data read request comprises the first instruction;

[0013] receiving a first data block returned by each of the target servers in response to the first instruction, wherein the first data block is all data stored in each of the servers.

[0014] Optionally, determining the target data according to the data read from the target servers comprises:

[0015] determining a corresponding data position of the target data in each first data block according to the data offset information and the data length information carried in the data read request and the amount of data stored by each of the distributed data storage servers;

[0016] extracting data from the corresponding data position of each of the first data blocks and splicing the extracted data to obtain the target data.

[0017] Optionally, reading data from the target servers corresponding to the target data from the plurality of distributed data storage servers comprises:

[0018] determining the target server from the plurality of distributed data storage servers according to data offset information and data length information carried in the data read request and an amount of data stored by each of the distributed data storage servers, and a storage position of the target data in each of the servers included in the target server;

[0019] sending a second instruction to each of the target servers, wherein the second instruction is used to instruct reading data stored in the storage position in each of the servers, and the data read request comprises the second instruction;

[0020] receiving a second data block returned by each of the target servers in response to the second instruction, wherein the second data block is data stored in the storage position in each of the servers.

[0021] Optionally, determining the target data according to the data read from the target server comprises:

[0022] acquiring signature data returned by each of the servers, wherein the signature data is data obtained by each of the servers signing the second data block;

[0023] verifying the second data block using the signature data;

[0024] in a case where the second data block passes the verification, splicing the second data block returned by each of the servers as the target data.

[0025] Optionally, determining the target data according to the data read from the target server comprises:

[0026] performing data consistency verification on the data read from the target server, wherein the data read from the target server passes the data consistency verification performed by the target server;

[0027] in a case where the data read from the target server passes the data consistency verification, determining the target data according to the data read from the target server.

[0028] Optionally, after reading data from the target server corresponding to the target data among the plurality of distributed data storage servers, the method further comprises:

[0029] in a case where reading data from the target server abnormally, reading all data stored by the distributed data storage system from the plurality of distributed data storage servers;

[0030] extracting the target data from all data stored by the distributed data storage system;

[0031] sending the target data in response to the data read request.

[0032] According to another aspect of the embodiments of the present application, a data reading device is further provided, comprising:

[0033] a receiving module configured to receive a data read request, wherein the data read request is used to request reading target data from a distributed data storage system, and the target data is part of data stored by the distributed data storage system through a plurality of distributed data storage servers;

[0034] a first reading module configured to read data from a target server corresponding to the target data among the plurality of distributed data storage servers;

[0035] determining module, configured to determine the target data according to the data read from the target server;

[0036] a first sending module, configured to send the target data in response to the data read request.

[0037] According to another aspect of the embodiments of the present application, a storage medium is also provided, which includes a stored program, and the program performs the above method when running.

[0038] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor performs the above method through the computer program.

[0039] In the embodiments of the present application, a data read request is received, wherein the data read request is used to request reading target data from a distributed data storage system, and the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers; data is read from a target server corresponding to the target data among the plurality of distributed data storage servers; the target data is determined according to the data read from the target server; and the target data is sent in response to the data read request. When the received data read request is used to request reading target data from a distributed data storage system, and the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers, data is read from a target server among the plurality of distributed data storage servers, which is the distributed data storage server corresponding to the target data, and the target data is determined according to the data read from the target server and then sent, thereby avoiding the operation of reading and verifying all data from all distributed data storage servers, reducing a large amount of network interaction, also reducing a large amount of disk IO interaction, and reducing the overall data volume and process of interaction, and the reading time is also reduced, thereby realizing the technical effect of reducing the network resource waste caused by the process of reading data, and further solving the technical problem of a large amount of network resource waste caused by the process of reading data. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is a schematic diagram of a hardware environment of a data reading method according to an embodiment of the present application;

[0043] Figure 2 is a flow chart of an optional data reading method according to an embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a data storage structure according to an optional embodiment of the present application;

[0045] Figure 4 is a schematic diagram of an optional data reading device according to an embodiment of the present application;

[0046] Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the personnel in the technical field better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

[0048] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] According to an aspect of an embodiment of the present application, a method embodiment of data reading is provided.

[0050] Optionally, in the present embodiment, the above-mentioned data reading method can be applied to a hardware environment composed of a terminal 101 and a server 103 as shown in Figure 1 Figure 1 ​As shown, the server 103 is connected with the terminal 101 through a network, which can be used to provide services (such as game services, application services, etc.) for the terminal or the client installed on the terminal, and a database can be set on the server or independently of the server, which is used to provide data storage services for the server 103. The above network includes but is not limited to a wide area network, a metropolitan area network or a local area network, and the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, etc. The data reading method of the embodiment of the application can be executed by the server 103, or can be executed by the terminal 101, or can be executed by the server 103 and the terminal 101 together. Wherein, the terminal 101 executing the data reading method of the embodiment of the application can also be executed by the client installed thereon.

[0051] Figure 2 is a flow chart of an optional data reading method according to the embodiment of the application, as shown in the figure, the method can include the following steps: Figure 2

[0052] Step S202, receiving a data reading request, wherein the data reading request is used to request reading target data from a distributed data storage system, and the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers;

[0053] Step S204, reading data from a target server corresponding to the target data among the plurality of distributed data storage servers;

[0054] Step S206, determining the target data according to the data read from the target server;

[0055] Step S208, sending the target data in response to the data reading request.

[0056] Through the above steps S202 to S208, when the received data reading request is used to request reading target data from a distributed data storage system, and the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers, the data is read from the target server among the plurality of distributed data storage servers, which is the distributed data storage server corresponding to the target data, and then the target data is determined according to the data read from the target server and sent, so as to avoid the operation of reading and verifying all data from all distributed data storage servers, reduce a large amount of network interaction, also reduce a large amount of disk IO interaction, and the overall interactive data amount and process are reduced, and the reading time is also reduced, so as to realize the technical effect of reducing the network resource waste caused by the data reading process, and further solve the technical problem that the data reading process causes a large amount of network resource waste. ​

[0057] Optionally, in the embodiment, the data reading method can be applied to, but is not limited to, a server in a distributed data storage system for processing user requests. The servers included in the distributed data storage system can be divided into, but are not limited to, distributed data storage servers for storing data and servers for processing user requests according to functions. The servers for processing user requests can be one or several servers in the distributed data storage servers for storing data. Or the servers for processing user requests can also be deployed separately from the distributed data storage servers for storing data.

[0058] Optionally, in the embodiment, the above-mentioned distributed data storage system can be, but is not limited to, a distributed data storage system using a redundancy technology of erasure coding technology. The erasure coding technology is to encode the original data blocks through an erasure coding algorithm to obtain the check blocks, and store the data blocks and the check blocks together to achieve fault tolerance. The basic idea is to obtain N check blocks through calculation from M original data blocks, and for the M+N blocks, any N block elements lost can be recovered to the original data through the corresponding algorithm. The process of generating check blocks is called encoding (ENCODE), and the process of recovering lost data blocks is called decoding (DECODE). The method based on erasure coding has the advantages of low redundancy and high disk utilization compared with the multiple copy method. When the client needs to read all the data, it will initiate an EC read request to the distributed data storage system, the distributed data storage system reads the M original data blocks concurrently, assembles the complete data and performs data consistency check, and returns to the client according to the requirements in the EC read request after the check.

[0059] In the technical solution provided in step S202, the data reading request is used to request reading target data from the distributed data storage system, and the target data is part of the data stored by the distributed data storage system through the multiple distributed data storage servers. The data reading request can include, but is not limited to, a Range read request.

[0060] Optionally, in the embodiment, the data reading request can be, but is not limited to, sent by the user using the client to the server in the distributed data storage system for processing user requests. After receiving the data reading request, the server in the distributed data storage system for processing user requests processes the data reading request and returns data to the client.

[0061] In the technical solution provided in step S204, the target server is a server corresponding to the target data in the multiple distributed data storage servers, or in other words, the target server is a server storing the target data in the multiple distributed data storage servers.

[0062] Optionally, in the embodiment, the data stored in the distributed data storage system is distributed in a plurality of distributed data storage servers, each of which stores a part of the total data. For example, if the total data is 4M and the distributed data storage system has 8 distributed data storage servers, the 4M data can be divided into 8 parts, which can be (0, 0.5M], (0.5M, 1M], (1M, 1.5M]…(3.5M, 4M], each of which is stored in a distributed data storage server.

[0063] Optionally, in the embodiment, the target data can be stored in one or more of the plurality of distributed data storage servers. For example, if the target data range is [0.6M, 0.9M], the target server is a distributed data storage server storing data range (0.5M, 1M], and if the target data range is [0.6M, 2M], the target server is a distributed data storage server storing data range (0.5M, 1M], (1M, 1.5M] and (1.5M, 2M].

[0064] Optionally, in the embodiment, the data read from the target server can be all the data stored in the target server, or the target data can be directly read from the target server.

[0065] As an optional embodiment, reading data from the target server corresponding to the target data in the plurality of distributed data storage servers comprises:

[0066] S11, determining the target server from the plurality of distributed data storage servers according to the data offset information and the data length information carried in the data read request and the amount of data stored in each distributed data storage server, wherein the data offset information is used to indicate the offset of the target data in the data stored in the plurality of distributed data storage servers, and the data length information is used to indicate the length of the target data;

[0067] S12, sending a first instruction to each server in the target server, wherein the first instruction is used to instruct to read all the data stored in each server, and the data read request comprises the first instruction;

[0068] S13, receiving the first data block returned by each server in the target server in response to the first instruction, wherein the first data block is all the data stored in each server.

[0069] Optionally, in this embodiment, the data read request can indicate the target data to be read by carrying data offset information and data length information, the data offset information is used to indicate the offset of the target data in the data stored in the plurality of distributed data storage servers, and the data length information is used to indicate the length of the target data. For example, if the data offset information carried in the data read request is 0.6M and the data length information is 0.3M, it can be determined that the range of the target data is [0.6M, 0.9M]. If the data offset information carried in the data read request is 0.6M and the data length information is 1.4M, it can be determined that the range of the target data is [0.6M, 2M].

[0070] Optionally, in this embodiment, according to the data offset information and the data length information carried in the data read request and the amount of data stored in each distributed data storage server, the target server can be located from the plurality of distributed data storage servers. For example, according to the data offset information and the data length information carried in the data read request, the range of the target data can be determined, and then according to the range of the target data and the amount of data stored in each distributed data storage server, it can be determined that the target data is stored in which server or servers, so as to locate the target server.

[0071] For example, if the data offset information carried in the data read request is 0.6M and the data length information is 0.3M, it can be determined that the range of the target data is [0.6M, 0.9M], and then combined with the data range of each distributed data storage server (0, 0.5M], (0.5M, 1M], (1M, 1.5M]… (3.5M, 4M], it can be determined that the target server is the server with the data range of (0.5M, 1M].

[0072] Optionally, in this embodiment, the target server can also be determined according to the number of each distributed data storage server. The distributed data storage servers are numbered in the order of data storage, and are numbered as part 0 to part n servers. For example, the data ranges of 8 distributed data storage servers are (0, 0.5M], (0.5M, 1M], (1M, 1.5M]… (3.5M, 4M], and the numbers of the 8 distributed data storage servers are part 0 to part 7 respectively. Assuming that the range read by the user is 0.7M to 1.6M, the starting position of the target data can be located at part 1 by ceil((0.7M+1) / 0.5M), and the end position of the target data can be located at part 3 by ceil((1.6M+1) / 0.5M). (Wherein the ceil function means rounding up).

[0073] Optionally, in the embodiment, all the data stored in the target server can be read, and the data can be returned according to the requirement. The target server can include one or more servers. The first instruction sent to each server of the target server can be used to instruct reading all the data stored in each server. Each server of the target server returns all the data stored in the server as the first data block in response to the first instruction.

[0074] For example, the range of the target data is [0.6M, 0.9M], the first instruction is sent to the server with the data range of (0.5M, 1M] to instruct reading all the data stored in the server, and then the data with the data range of (0.5M, 1M] returned by the server with the data range of (0.5M, 1M] is received.

[0075] Optionally, in the embodiment, each server can perform data consistency check on the first data block before the server returns the first data block in response to the first instruction.

[0076] As an optional embodiment, reading the data from the target server corresponding to the target data in the plurality of distributed data storage servers includes:

[0077] S21, determining the target server and the storage location of the target data in each server included in the target server from the plurality of distributed data storage servers according to the data offset information and the data length information carried in the data reading request and the data amount stored in each distributed data storage server;

[0078] S22, sending a second instruction to each server of the target server, wherein the second instruction is used to instruct reading the data stored in the storage location in each server, and the data reading request includes the second instruction;

[0079] S23, receiving the second data block returned by each server of the target server in response to the second instruction, wherein the second data block is the data stored in the storage location in each server.

[0080] Optionally, in the embodiment, the storage location of the target data in each server included in the target server can be directly determined while determining the target server from the plurality of distributed data storage servers according to the data offset information and the data length information carried in the data reading request and the data amount stored in each distributed data storage server.

[0081] For example, if the data offset information carried in the data reading request is 0.6M and the data length information is 1.2M, it can be determined that the range of the target data is [0.6M, 1.8M], and in combination with the data range of each distributed data storage server (0, 0.5M], (0.5M, 1M], (1M, 1.5M]… (3.5M, 4M], it can be determined that the target server is the server with the data range of (0.5M, 1M], (1M, 1.5M] and (1.5M, 2M]. And it can be determined that the storage location of the target data in the server with the data range of (0.5M, 1M] is [0.6M, 1M], in the server with the data range of (1M, 1.5M] is (1M, 1.5M], and in the server with the data range of (1.5M, 2M] is (1.5M, 1.8M].

[0082] Optionally, in this embodiment, the corresponding part of the target data can be read directly from the target server, and the data can be returned according to the requirement. The target server can include one or more servers. The second instruction sent to each server in the target server can be used to instruct to read the data stored in the above storage location in each server. Each server in the target server returns the data stored in its storage location as a second data block in response to the second instruction.

[0083] For example, the range of the target data is [0.6M, 0.9M], the second instruction is sent to the server with the data range of (0.5M, 1M] to instruct to read the data with the storage location of [0.6M, 0.9M], and then the [0.6M, 0.9M] data returned by the server with the data range of (0.5M, 1M] is received.

[0084] Optionally, in this embodiment, before each server returns the second data block in response to the second instruction, each server can perform data consistency verification on the second data block, and return the second data block after the verification is passed.

[0085] As an optional embodiment, after reading the data from the target server corresponding to the target data in the plurality of distributed data storage servers, the method further comprises:

[0086] S31, in the case that reading data from the target server abnormally, reading all data stored by the distributed data storage system from the plurality of distributed data storage servers;

[0087] S32, extracting the target data from all data stored by the distributed data storage system;

[0088] S33, in response to the data read request, sending the target data.

[0089] Optionally, in this embodiment, if the above-mentioned manner of reading part of data is abnormal, all data stored in the distributed data storage system is read from the plurality of distributed data storage servers, and the target data is extracted therefrom and returned, thereby ensuring the success rate of the data read operation. For example, if the above-mentioned Range read process is abnormal, the complete data is decompressed by EC degradation read, and the corresponding content is read and returned to the client.

[0090] In the technical solution provided in step S206, the target data is determined according to different manners of reading data from the target server, for example, if the data read from the target server is all the data on the target server, the target data is extracted therefrom. If the data read from the target server is directly the target data, the data read from the target server is directly determined as the target data.

[0091] As an optional embodiment, determining the target data according to the data read from the target server comprises:

[0092] S41, determining the data position of the target data in each first data block according to the data offset information and the data length information carried in the data read request and the amount of data stored by each distributed data storage server;

[0093] S42, extracting data from the data position corresponding to each first data block and splicing the extracted data to obtain the target data.

[0094] Optionally, in this embodiment, if all the data stored by each server in the target server is read, that is, the first data block, the data position of the target data in each first data block can be determined according to the data offset information and the data length information carried in the data read request and the amount of data stored by each distributed data storage server, data is extracted from the data position corresponding to each first data block, and the extracted data is spliced, thereby obtaining the target data.

[0095] For example, if the data offset information carried in the data reading request is 0.6M and the data length information is 1.2M, it can be determined that the range of the target data is [0.6M, 1.8M]. In combination with the data (0.5M, 1M], (1M, 1.5M] and (1.5M, 2M] read from the target server, it can be determined that the data position of the target data in the data (0.5M, 1M] is [0.6M, 1M], the data position of the target data in the data (1M, 1.5M] is (1M, 1.5M], and the data position of the target data in the data (1.5M, 2M] is (1.5M, 1.8M]. The target data [0.6M, 1.8M] is obtained by splicing the data [0.6M, 1M], the data (1M, 1.5M] and the data (1.5M, 1.8M].

[0096] As an optional embodiment, determining the target data according to the data read from the target server comprises:

[0097] S51, obtaining the signature data returned by each server, wherein the signature data is data obtained by signing the second data block by each server;

[0098] S52, verifying the second data block using the signature data;

[0099] S53, in the case where the second data block passes the verification, splicing the second data block returned by each server as the target data.

[0100] Optionally, in the embodiment, if each server reads a part of the target data, i.e., each second data block, from the target server, the signature data provided by each server can be received while reading the part of the target data, the signature data being data obtained by signing the second data block by each server, the corresponding second data block is verified using the corresponding signature data, and the target data is spliced and returned after passing the verification.

[0101] As an optional embodiment, determining the target data according to the data read from the target server comprises:

[0102] S61, performing data consistency verification on the data read from the target server, wherein the data read from the target server passes the data consistency verification performed by the target server;

[0103] S62, in the case where the data read from the target server passes the data consistency verification, determining the target data according to the data read from the target server.

[0104] Optionally, in this embodiment, before determining the target data according to the data read from the target server, data consistency check can also be performed on the data read from the target server, and only if the data read from the target server passes the data consistency check, the target data is determined according to the data read from the target server.

[0105] Optionally, in this embodiment, if the data read from the target server does not pass the data consistency check, error reporting, retry, data recovery and the like can be used for processing.

[0106] Optionally, in this embodiment, the data read from the target server passes the data consistency check performed by the target server, that is, before the target server returns the data, the target server performs data consistency check on the data to be returned, and if the data to be returned passes the data consistency check, the return operation is performed. If the data to be returned fails to pass the data consistency check, error reporting, retry, data recovery and the like can be used for processing.

[0107] In the technical solution provided in step S208, the target data can be sent according to the data receiving end indicated by the data read request, for example, the target data can be sent to the client or server initiating the data read request, or other clients or servers indicated by the data read request.

[0108] The application also provides an optional embodiment, which provides a data reading method. When a user initiates a Range read request, according to the data offset information and the data length information carried in the request, it is determined which data blocks need to be read, and only the servers where the data blocks are located are initiated to read data. When the data is returned, data check is performed on each data block, and after the check is passed, the complete data is spliced and returned to the client.

[0109] In this way, only the data of interest to the client is initiated to read network and disk IO requests, without amplifying the resource consumption of the system, and at the same time, due to the reduction of data block reading, the reading delay is also reduced accordingly.

[0110] In this optional embodiment, a data reading process is provided, which can include but is not limited to the following steps:

[0111] Step 1: The client initiates a data read request, and the server judges whether the request is a Range read request after receiving the request.

[0112] Step 2.1: If it is not a Range read request, the reading process of the EC read request is normally executed.

[0113] Step2.2, if it is a Range read request, the data block position that needs to be read is calculated. For example: Figure 3 is a schematic diagram of a data storage structure according to an optional embodiment of the application. The complete data is divided into 8 parts, which are respectively stored in the distributed data storage servers of Part 0 to Part 7. This time, the reading involves a part of Part 1, all of Part 2, and a part of Part 3. Therefore, only Part 1, Part 2 and Part 3 need to be read, and each distributed data storage server returns the corresponding data that passes the verification. After receiving the complete data, the corresponding range is returned according to the needs of the client.

[0114] Step3, if the processing of the Range read request is abnormal, the complete data is decompressed through EC degradation reading, and the corresponding content is read and returned to the client.

[0115] Through the above process, for the storage scheme using the erasure code technology, the above Range read mode optimizes the reading of the partial Part, instead of reading all the data blocks, which reduces a large amount of network interaction, reduces a large amount of disk IO interaction, and reduces the overall data volume and process of interaction, and the reading time is also reduced.

[0116] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited to the action sequence described, because according to the application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the application.

[0117] From the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making an electronic device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the application.

[0118] According to another aspect of the embodiments of the application, a data reading device for implementing the above-mentioned data reading method is also provided.Figure 4 is a schematic diagram of an optional data reading device according to an embodiment of the present application, as shown in the figure, the device can include: Figure 4

[0119] a receiving module 42, configured to receive a data reading request, wherein the data reading request is used to request reading target data from a distributed data storage system, the target data is part of data stored by the distributed data storage system through a plurality of distributed data storage servers;

[0120] a first reading module 44, configured to read data from a target server corresponding to the target data among the plurality of distributed data storage servers;

[0121] a determining module 46, configured to determine the target data according to the data read from the target server;

[0122] a first sending module 48, configured to send the target data in response to the data reading request.

[0123] It should be noted that the receiving module 42 in this embodiment can be used to execute step S202 in the embodiments of the present application, the first reading module 44 in this embodiment can be used to execute step S204 in the embodiments of the present application, the determining module 46 in this embodiment can be used to execute step S206 in the embodiments of the present application, and the first sending module 48 in this embodiment can be used to execute step S208 in the embodiments of the present application.

[0124] It should be noted that the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the above disclosed contents. It should be noted that the above modules as part of the device can run in the hardware environment as shown in Figure 1 , can be realized by software, or can be realized by hardware.

[0125] ​When the received data reading request is used to request reading of target data from the distributed data storage system, and the target data is part of the data stored by the distributed data storage system through the plurality of distributed data storage servers, the data is read from a target server in the plurality of distributed data storage servers, which is the distributed data storage server corresponding to the target data, and the target data is determined and sent according to the data read on the target server, thereby avoiding the operation of reading and checking all data from all distributed data storage servers, reducing a large amount of network interaction, a large amount of disk IO interaction, and the amount of data and the process of overall interaction, and the reading time is also reduced, thereby realizing the technical effect of reducing the network resource waste caused by the process of reading data, and further solving the technical problem that the process of reading data causes a large amount of network resource waste.

[0126] As an optional embodiment, the first reading module comprises:

[0127] A first determining unit is configured to determine the target server from the plurality of distributed data storage servers according to data offset information and data length information carried in the data reading request and the amount of data stored by each distributed data storage server, wherein the data offset information is used to indicate the offset of the target data in the data stored by the plurality of distributed data storage servers, and the data length information is used to indicate the length of the target data.

[0128] A first sending unit is configured to send a first instruction to each server in the target server, wherein the first instruction is used to instruct reading of all data stored in each server, and the data reading request comprises the first instruction.

[0129] A first receiving unit is configured to receive a first data block returned by each server in the target server in response to the first instruction, wherein the first data block is all data stored in each server.

[0130] As an optional embodiment, the determining module comprises:

[0131] A second determining unit is configured to determine the data position of the target data in each first data block according to the data offset information and the data length information carried in the data reading request and the amount of data stored by each distributed data storage server.

[0132] A processing unit is configured to extract data from the data position corresponding to each first data block and splice the extracted data to obtain the target data.

[0133] As an optional embodiment, the first reading module comprises:

[0134] a third determining unit, configured to determine the target server from the plurality of distributed data storage servers according to data offset information and data length information carried in the data reading request and an amount of data stored by each distributed data storage server, and determine a storage location of the target data in each server included in the target server;

[0135] a second sending unit, configured to send a second instruction to each server in the target server, where the second instruction is used to instruct to read data stored at the storage location in the each server, and the data reading request comprises the second instruction;

[0136] a second receiving unit, configured to receive a second data block returned by the each server in the target server in response to the second instruction, where the second data block is the data stored at the storage location in the each server.

[0137] As an optional embodiment, the determining module comprises:

[0138] an obtaining unit, configured to obtain signature data returned by the each server, where the signature data is data obtained by the each server by signing the second data block;

[0139] a first checking unit, configured to check the second data block using the signature data;

[0140] a splicing unit, configured to splice the second data block returned by the each server into the target data in a case where the second data block passes the checking.

[0141] As an optional embodiment, the determining module comprises:

[0142] a second checking unit, configured to perform data consistency checking on data read from the target server, where the data read from the target server passes data consistency checking performed by the target server;

[0143] a fourth determining unit, configured to determine the target data according to the data read from the target server in a case where the data read from the target server passes the data consistency checking.

[0144] As an optional embodiment, the apparatus further comprises:

[0145] The second reading module is configured to read all data stored in the distributed data storage system from the plurality of distributed data storage servers in the case that reading data from the target server abnormally occurs after reading data from the target server corresponding to the target data.

[0146] The extraction module is configured to extract the target data from all data stored in the distributed data storage system.

[0147] The second sending module is configured to send the target data in response to the data reading request.

[0148] It should be noted that the above modules and examples and application scenarios implemented by the corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules as part of the device can run in the hardware environment as shown in Figure 1 , which can be implemented by software, or by hardware, wherein the hardware environment includes a network environment.

[0149] According to another aspect of the embodiments of the present application, an electronic device for implementing the above-mentioned data reading method is further provided.

[0150] Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application, as shown in Figure 5 , the electronic device can include one or more (only one is shown in the figure) processors 501, a memory 503, and a transmission device 505, as shown in Figure 5 , the electronic device can further include an input / output device 507.

[0151] The memory 503 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data reading method and device in the embodiments of the present application. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 503, that is, implements the above-mentioned data reading method. The memory 503 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 503 can further include a memory remotely arranged with respect to the processor 501, which can be connected to the electronic device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0152] The transmission device 505 described above is used to receive or send data via a network, and can also be used for data transmission between the processor and the memory. Specific examples of the network described above can include wired networks and wireless networks. In an example, the transmission device 505 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable to communicate with the Internet or a local area network. In an example, the transmission device 505 is a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

[0153] Specifically, the memory 503 is configured to store an application program.

[0154] The processor 501 can call the application program stored in the memory 503 through the transmission device 505 to perform the following steps:

[0155] receiving a data reading request, wherein the data reading request is used to request to read target data from a distributed data storage system, and the target data is part of data stored in the distributed data storage system by a plurality of distributed data storage servers;

[0156] reading data from a target server corresponding to the target data among the plurality of distributed data storage servers;

[0157] determining the target data according to the data read from the target server;

[0158] sending the target data in response to the data reading request.

[0159] By adopting the embodiments of the present application, a data reading scheme is provided. When a received data reading request is used to request to read target data from a distributed data storage system, and the target data is part of data stored in the distributed data storage system by a plurality of distributed data storage servers, data is read from a target server among the plurality of distributed data storage servers, the target server being a distributed data storage server corresponding to the target data, and then the target data is determined according to the data read from the target server and is sent, thereby avoiding the operation of reading and verifying all data from all distributed data storage servers, reducing a large amount of network interaction, reducing a large amount of disk IO interaction, and reducing the overall data amount and process of interaction, and the reading time is also reduced, thereby realizing the technical effect of reducing network resource waste caused by the data reading process, and further solving the technical problem of a large amount of network resource waste caused by the data reading process.

[0160] Optionally, the specific examples in the embodiments can refer to the examples described in the above embodiments, and the embodiments will not be described here again.

[0161] Those skilled in the art can understand that, Figure 5 The structure shown is only schematic, and the electronic device can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a mobile Internet device (MID), a PAD, or other electronic devices. Figure 5 This does not limit the structure of the above electronic device. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, etc.) than Figure 5 or have a different configuration than Figure 5 shown.

[0162] Those skilled in the art can understand that all or part of the steps in the above embodiments can be completed by instructing the hardware related to the electronic device by a program, which can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0163] The embodiments of the present application also provide a storage medium. Optionally, in the embodiments, the above storage medium can be used to execute the program code of the data reading method.

[0164] Optionally, in the embodiments, the above storage medium can be located on at least one of the network devices in the network shown in the above embodiments.

[0165] Optionally, in the embodiments, the storage medium is configured to store program code for executing the following steps:

[0166] receiving a data reading request, wherein the data reading request is used to request to read target data from a distributed data storage system, and the target data is part of the data stored by the distributed data storage system through a plurality of distributed data storage servers;

[0167] reading data from a target server corresponding to the target data among the plurality of distributed data storage servers;

[0168] determining the target data according to the data read from the target server;

[0169] sending the target data in response to the data reading request.

[0170] Optionally, specific examples in the present embodiment can refer to the examples described in the above embodiments, which will not be repeated here.

[0171] Optionally, in the present embodiment, the storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0172] The serial numbers of the embodiments of the present application described above are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0173] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0174] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0175] In the several embodiments provided by the present application, it should be understood that the disclosed client can be implemented in other ways. Of course, the above device embodiment is only illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0176] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0177] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each of the units can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0178] The above is only the preferred embodiment of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for reading data, characterized in that, include: Receive a data read request, wherein the data read request is used to request to read target data from the distributed data storage system, and the target data is a portion of the data stored by the distributed data storage system through multiple distributed data storage servers; Reading data from the target server corresponding to the target data from the plurality of distributed data storage servers includes: determining the target server from the plurality of distributed data storage servers based on the data offset information and data length information carried in the data read request, and the amount of data stored in each distributed data storage server, wherein the data offset information is used to indicate the offset of the target data in the data stored in the plurality of distributed data storage servers, and the data length information is used to indicate the length of the target data; sending a first instruction to each of the target servers, wherein the first instruction is used to instruct the reading of all data stored in each server, and the data read request includes the first instruction; receiving a first data block returned by each of the target servers in response to the first instruction, wherein the first data block is all data stored in each server; determining the target server from the plurality of distributed data storage servers based on the data offset information and data length information carried in the data read request, and the amount of data stored in each distributed data storage server includes: determining the range of the target data based on the data offset information and data length information carried in the data read request, and determining the target server storing the target data based on the range of the target data and the amount of data stored in each distributed data storage server; Determining the target data based on data read from the target server includes: determining the data position of the target data in each first data block based on the data offset information and data length information carried in the data read request, as well as the amount of data stored in each distributed data storage server; extracting data from the data position corresponding to each first data block and concatenating the extracted data to obtain the target data; In response to the data read request, send the target data.

2. The method according to claim 1, characterized in that, Reading data from the target server corresponding to the target data from the plurality of distributed data storage servers includes: Based on the data offset information and data length information carried in the data read request, as well as the amount of data stored in each distributed data storage server, the target server is determined from the plurality of distributed data storage servers, and the storage location of the target data in each server included in the target server is determined. Send a second instruction to each of the target servers, wherein the second instruction is used to instruct the reading of data stored at the storage location in each of the servers, and the data reading request includes the second instruction; Receive a second data block returned by each of the target servers in response to the second instruction, wherein the second data block is data stored at the storage location in each server.

3. The method according to claim 2, characterized in that, The target data is determined based on data read from the target server, including: Obtain the signature data returned by each server, wherein the signature data is the data obtained by each server signing the second data block; The signature data is used to verify the second data block; If the second data block passes verification, the second data blocks returned by each server are concatenated to form the target data.

4. The method according to claim 1, characterized in that, The target data is determined based on data read from the target server, including: Data consistency verification is performed on the data read from the target server, wherein the data read from the target server passes the data consistency verification performed by the target server; If the data read from the target server passes the data consistency check, the target data is determined based on the data read from the target server.

5. The method according to claim 1, characterized in that, After reading data from the target server corresponding to the target data among the plurality of distributed data storage servers, the method further includes: In the event of an anomaly when reading data from the target server, all data stored in the distributed data storage system shall be read from the plurality of distributed data storage servers; Extract the target data from all the data stored in the distributed data storage system; In response to the data read request, send the target data.

6. A data reading device, characterized in that, include: A receiving module is used to receive a data read request, wherein the data read request is used to request to read target data from the distributed data storage system, and the target data is a portion of the data stored by the distributed data storage system through multiple distributed data storage servers; The first reading module is configured to read data from the target server corresponding to the target data among the plurality of distributed data storage servers, including: determining the target server from the plurality of distributed data storage servers based on data offset information and data length information carried in the data reading request, and the amount of data stored in each distributed data storage server, wherein the data offset information is used to indicate the offset of the target data in the data stored in the plurality of distributed data storage servers, and the data length information is used to indicate the length of the target data; sending a first instruction to each of the target servers, wherein the first instruction is used to instruct the reading of all data stored in each server, and the data reading request includes the first instruction; receiving a first data block returned by each of the target servers in response to the first instruction, wherein the first data block is all data stored in each server; the step of determining the target server from the plurality of distributed data storage servers based on the data offset information and data length information carried in the data reading request, and the amount of data stored in each distributed data storage server includes: determining the range of the target data based on the data offset information and data length information carried in the data reading request, and determining the target server storing the target data based on the range of the target data and the amount of data stored in each distributed data storage server; The determination module is used to determine the target data based on the data read from the target server, including: determining the data position of the target data in each first data block based on the data offset information and data length information carried in the data read request, and the amount of data stored in each distributed data storage server; extracting data from the data position corresponding to each first data block and concatenating the extracted data to obtain the target data; The first sending module is used to send the target data in response to the data reading request.

7. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 5 when it is run.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the method described in any one of claims 1 to 5 through the computer program.

Images