Data storage method, system, data node and computer readable storage medium

Abstract

The present disclosure relates to a data storage method, system, data node and computer-readable storage medium, and relates to the field of computer technology. The method of the present disclosure includes: a data node receives a data packet to be written sent by a client; the data node allocates a disk array for the data packet to be written, and the disk array corresponds to multiple disks; the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disks corresponding to the disk array.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data storage method, system, data node, and computer-readable storage medium. Background Art

[0002] Distributed storage systems, such as HDFS (Hadoop Distributed File System), store data in a distributed manner on multiple data nodes. Currently, a common storage method for data nodes is to format multiple disks as local file systems and mount them to multiple local directories. The data nodes use these local directories as data directories, and the data nodes select a data directory to store data blocks as a file.

[0003] When the client reads and writes data blocks, it transmits them serially in packets. The data nodes read and write the files corresponding to the data blocks sequentially, which is to say, sequential reading and writing of disks. Summary of the invention

[0004] The inventors have found that in the current data reading and writing method, without considering the operating system cache, the rate at which the client writes data blocks cannot exceed the performance of sequential reading and writing of a single disk, and the writing rate is low.

[0005] A technical problem to be solved by the present disclosure is: how to improve the efficiency of writing data.

[0006] According to some embodiments of the present disclosure, a data storage method is provided, including: a data node receives a data packet to be written sent by a client; the data node allocates a disk array for the data packet to be written, and the disk array corresponds to multiple disks; the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disks corresponding to the disk array.

[0007] In some embodiments, the disk array is divided into multiple stripes, each stripe includes multiple storage slices, and different storage slices in a stripe belong to different disks.

[0008] In some embodiments, the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, including: the data node allocates storage space in the disk array for the data packet to be written according to the size of the data packet to be written; the data node divides the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space.

[0009] In some embodiments, the capacity of the storage slice is less than or equal to the amount of data that can be read and written by a disk at a single time.

[0010] In some embodiments, writing each data slice in parallel to different disk partitions in the disk array includes: the data node sends the data slice to the disk array according to the data directory of each disk array, so as to write each data slice in parallel to different disks corresponding to the disk array; wherein the data directory of the disk array is generated after the disk array is formatted as a local file system and mounted to a local directory.

[0011] In some embodiments, the method further includes: the data node receives a data read request sent by a client for a data packet to be read; the data node searches the disk array storing the data block to be read according to the data directory, and searches the disk storing at least one data slice corresponding to the data packet to be read according to the file system corresponding to the disk array; the data node reads each data slice corresponding to the data packet to be read from each storage disk in parallel.

[0012] In some embodiments, the method also includes: the client receives block group information sent by the management node, the information in the block group includes address information of multiple data nodes; the client divides the data to be written into data units, and calculates corresponding verification units based on a set of preset number of data units; the client sends the data of the data unit and the verification unit to each data node in the form of a data packet to be written according to the address information of each data node.

[0013] According to other embodiments of the present disclosure, a data node is provided, including: a data packet receiving module, used to receive a data packet to be written sent by a client; a disk array determining module, used to allocate a disk array for the data packet to be written, the disk array corresponding to multiple disks; a data writing module, used to divide the data packet to be written into multiple data slices according to the size of the data packet to be written, and write each data slice in parallel to different disks corresponding to the disk array.

[0014] In some embodiments, the disk array is divided into multiple stripes, each stripe includes multiple storage slices, and different storage slices in a stripe belong to different disks.

[0015] In some embodiments, the data writing module is used to allocate storage space in the disk array for the data packet to be written according to the size of the data packet to be written; and divide the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space.

[0016] In some embodiments, the capacity of the storage slice is less than or equal to the amount of data that can be read and written by a disk at a single time.

[0017] In some embodiments, the data writing module is used to send data slices to the disk array according to the data directory of each disk array, so as to write each data slice in parallel to different disks corresponding to the disk array; wherein the data directory of the disk array is generated after the disk array is formatted as a local file system and mounted to a local directory.

[0018] In some embodiments, the data node also includes: a read request receiving module, which is used to receive a data read request sent by a client for a data packet to be read; a data block search module, which is used to search the disk array storing the data block to be read according to the data directory, and search the disk storing at least one data slice corresponding to the data packet to be read according to the file system corresponding to the disk array; a data block reading module, which is used to read each data slice corresponding to the data packet to be read in parallel from each storage disk.

[0019] According to some further embodiments of the present disclosure, a data node is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the data storage method of any of the aforementioned embodiments based on instructions stored in the memory.

[0020] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, the data storage method of any of the aforementioned embodiments is implemented.

[0021] According to some other embodiments of the present disclosure, a data storage system is provided, comprising: a data node of any of the aforementioned embodiments; and a client, for receiving block group information sent by a management node, the information in the block group including address information of multiple data nodes; dividing the data to be written into data units, and calculating corresponding verification units based on a set of preset number of data units; and sending the data of the data unit and the verification unit to each data node in the form of a data packet to be written according to the address information of each data node.

[0022] In the present disclosure, disk partitions of multiple disks form a disk array, and a data node divides a data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disk partitions in the disk array. In the case where the data packet to be written is large, the method of the present disclosure writes data in parallel by multiple disks, thereby improving the efficiency of data writing.

[0023] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A schematic flowchart showing a data storage method according to some embodiments of the present disclosure is provided.

[0026] Figure 2 A schematic diagram showing a partitioned disk array according to some embodiments of the present disclosure.

[0027] Figure 3 A schematic diagram showing a striped layout of some embodiments of the present disclosure.

[0028] Figure 4 A schematic flowchart showing a data storage method according to some other embodiments of the present disclosure is provided.

[0029] Figure 5 A schematic diagram showing the data storage format of some embodiments of the present disclosure.

[0030] Figure 6 A schematic diagram showing the structure of data nodes according to some embodiments of the present disclosure.

[0031] Figure 7 A schematic diagram showing the structure of data nodes according to other embodiments of the present disclosure.

[0032] Figure 8 A schematic diagram showing the structure of data nodes according to some further embodiments of the present disclosure.

[0033] Fig. 9 A schematic diagram showing the structure of a data storage system according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0035] The present disclosure proposes a data storage method that can improve the reading and writing efficiency of data. Figure 1 Give a description.

[0036] Figure 1 Flowcharts of some embodiments of the data storage method disclosed herein. Figure 1 As shown, the method of this embodiment includes: steps S102 to S106.

[0037] In step S102, the client sends a data packet to be written to the data node, and correspondingly, the data node receives the data packet to be written sent by the client.

[0038] The client divides the data to be written into packets and sends them to the data nodes. The process of dividing the packets and finding the corresponding data nodes will be described later.

[0039] In some embodiments, the size of the data packet to be written sent by the client can be preset according to the number of disks corresponding to the disk array and the maximum amount of data that can be read and written at a single time on each disk. For example, the product of the number of disks corresponding to the disk array and the maximum amount of data that can be read and written at a single time on each disk is used as the upper limit of the capacity of the data packet to be written, and the data stream in the data packet to be written needs to be less than or equal to the upper limit of the capacity. For example, the disk array corresponds to 4 disks, and the maximum amount of data that can be read and written at a single time on each disk is 256KiB, then the upper limit of the capacity of the data packet to be written is 1MiB. The client can divide the data packet according to the upper limit of the capacity, or, in the case where the size of the data packet to be written sent by the client exceeds the preset upper limit of the capacity, the data node can divide the data packet to be written into multiple data packets to be written according to the preset upper limit of the capacity.

[0040] In step S104, the data node allocates a disk array for the data packet to be written. The disk array corresponds to multiple disks.

[0041] In some embodiments, multiple disks in a data node are divided into a preset number of partitions. For example, each disk is divided into 4 partitions. Each disk selects a partition to form a disk array. For example, the disk array can be in RAID0 form. Each disk array is formatted as a local file system and mounted to a local directory, and these local directories are used as data directories of the data node. Furthermore, the data node can allocate a disk array for the data packet to be written according to the data directory. For example, the data node can allocate a disk array for the data packet to be written according to the principle of load balancing.

[0042] like Figure 2 This is an application example of partitioning a disk array. Figure 2As shown in the figure, the data node includes four disks, sda, sdb, sdc, and sdd. Each disk is divided into four partitions. For example, sda is divided into sda1, sda2, sda3, and sda4. The four partitions corresponding to each row belong to different disks and form a disk array. For example, sda1, sdb1, sdc1, and sdd1 form the disk array md0 (RAID0 can be implemented through md in the Linux Kernel), corresponding to the data directory / data0. Figure 2 As shown, multiple disks in a data node can be combined into a stripe device, and a disk array consisting of a row of disk partitions is used as a stripe device to store data in the form of stripes.

[0043] In step S106, the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disks corresponding to the disk array.

[0044] In some embodiments, a disk array can be used as a stripe device and divided into multiple stripes. Each stripe contains multiple storage slices (Chunks), and different storage slices in a stripe belong to different disks. It can also be understood that each disk partition in a disk array is divided into smaller storage units, namely storage slices, and each disk selects a storage slice to form a stripe.

[0045] In some embodiments, the data node allocates storage space in the disk array for the data packet to be written according to the size of the data packet to be written. In the case where the storage space corresponds to multiple storage slices, the data node divides the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space. For example, the data node allocates continuous storage addresses for the data packet to be written according to the data packet to be written. The allocated address may contain one or more storage slices, or even storage space in one or more stripes. In actual application, the data node may allocate discontinuous storage addresses for the data packet to be written. For example, the remaining continuous storage space does not meet the size of the data packet to be written. In this case, it is more likely that the allocated address contains storage space in multiple stripes. In the case where the storage space corresponds to multiple storage slices, the data packet to be written is allocated into multiple data slices according to each storage slice corresponding to the allocated address. The data slice is less than or equal to the remaining space of the corresponding storage slice. The capacity of the storage slice is less than or equal to the amount of data read and written by a disk at a single time.

[0046] When the size of the data packet to be written exceeds the threshold, different storage slices in the same stripe can be preferentially allocated to the data packet to be written. This allows the data packet to be written to be scattered in different disks for parallel writing, thereby improving writing efficiency. Figure 3 FIG. 4 shows an application example in which data packets to be written are stored in a striped form. Figure 3 As shown, the file to be written is divided into multiple data packets, and then divided into multiple data slices, chunk0, chunk1, chunk2, etc. A disk array RAID0 contains multiple stripes, strip0, strip1, etc. Ideally, the four data slices in a data packet to be written are stored in four different storage slices in a stripe, and the four different storage slices are located in different disks. That is, the storage location of the packet on the RAID0 device is aligned with the strip and is divided into four chunks, which are located on four disks respectively. Therefore, the packet is divided into four IOs, and reads and writes four disks at the same time. For example, chunk0, chunk1, chunk2, and chunk3 are stored in the storage slices of different disks on strip0.

[0047] The data node sends data slices to the disk array according to the data directory of each disk array, so that each data slice can be written to different disk partitions in the disk array in parallel.

[0048] In the method of the above embodiment, the disk partitions of multiple disks form a disk array, and the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disk partitions in the disk array. In the case where the data packet to be written is large, the method of the above embodiment writes data in parallel by multiple disks, thereby improving the efficiency of data writing. In addition, compared with the prior art that sequentially writes hot data and easily causes the data node to keep writing to individual disks, causing the disk to be continuously busy, the corresponding writing efficiency is reduced, and other disks are in an idle state, the method of the above embodiment can balance the load of each disk, so that the overall efficiency of each disk is improved.

[0049] The solution disclosed in this disclosure is applicable to the application scenario of erasure code. HDFS erasure code (EC) divides large files (usually more than GiB) into multiple data block groups (Block Group). The data block group consists of data blocks (Data Block) and parity blocks (Parity Block). As one of the core components of HDFS, the core function of data node (Data Node) is to store file blocks (Block). Figure 4 Other embodiments of the data storage method disclosed herein are described.

[0050] Figure 4 Flowcharts of other embodiments of the data storage method disclosed in the present invention. Figure 4 As shown, the method of this embodiment includes: steps S402 to S414.

[0051] In step S402, the client applies for a block group from the management node (Name Node).

[0052] When the client fills a block group or just starts writing data, it will request a new block group from the Name Node.

[0053] In step S404, the client receives the block group information sent by the management node, and the information in the block group includes address information of multiple data nodes.

[0054] like Figure 5 As shown, a block group includes multiple blocks, for example, data blocks (Block D0 to Block D3) and check blocks (Block P0 to Block P1). Different blocks correspond to different data node information and will be stored in different data nodes later.

[0055] In step S406, the client divides the data to be written into data units, and calculates corresponding check units according to a set of preset number of data units.

[0056] The data (file) to be written by the client is distributed on multiple data nodes in a stripe layout. Figure 5 As shown in the figure, each block is divided into multiple cells, and a row of cells corresponding to a block group is written to different data nodes respectively, and cells in different rows form a stripe layout. That is, for EC scenarios, data is stored in a stripe layout on each data node, and is also stored in a stripe layout in each data node. This storage form improves the writing efficiency and reduces the cache occupancy.

[0057] The client sets a buffer area, which contains multiple buffers, each corresponding to a cell. Data is first written to multiple buffers in a stripe layout. Each cell corresponds to a data streamer. When a data cell in a stripe is written, the check unit is immediately calculated based on the erasure code and written to the check block buffer.

[0058] In step S408, the client sends the data of the data unit and the check unit to each data node in the form of a data packet to be written according to the address information of each data node.

[0059] like Figure 5 As shown in the figure, a unit is divided into multiple packets, and the cache transmits the packet to the data stream corresponding to the cache and sends it to the corresponding data node. When the data is written and the file is ready to be closed, the last data unit may not be full. At this time, it is necessary to fill the data unit with zeros to generate the check unit of the strip and write it into the check block cache, and then generate data packets and send them to their respective data streams.

[0060] In step S410, for each data node, the data node receives a data packet to be written sent by a client.

[0061] In step S412, the data node allocates a disk array for the data packet to be written.

[0062] In step S414, the data node divides the data packet to be written into multiple data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disks corresponding to the disk array.

[0063] Steps S410 to S414 may refer to the method of the aforementioned embodiment.

[0064] In some embodiments, a data node receives a data read request sent by a client for a data packet to be read; the data node searches for a disk array storing a data block to be read according to a data directory, and searches for a disk storing at least one data slice corresponding to the data packet to be read according to a file system corresponding to the disk array; the data node reads each data slice corresponding to the data packet to be read from each storage disk in parallel.

[0065] The data written by the client is distributed to different data nodes and different disks. The client can query the management node for the data node where the data block to be read is located, and the management node returns the information of the data node where the data block is located to the client. The client sends a data read request to the data node. The data node searches the disk array where each data packet in the data block to be read is stored according to the data directory. In the process of writing data to the disk array, the corresponding file system can record the storage address of the data. Therefore, the data node queries the file system for the storage location of each data slice in the data packet, and then reads each data slice contained in the data packet to be read from each disk in parallel and returns it to the client.

[0066] In the method of the above embodiment, the client distributes the data to be written to each data node in a stripe layout, and writes the data to each disk in a stripe layout in the data node. Both the process of writing to the data node in parallel and writing to the disk in parallel improves the efficiency of writing data and saves cache space. Since the data is stored in a stripe layout, when reading data, it is still read in stripes, and multiple disks of multiple data nodes read in parallel, which further improves the efficiency of reading data.

[0067] The present disclosure also provides a data node, Figure 6 Give a description.

[0068] Figure 6 Figure 1 is a structural diagram of some embodiments of the data node disclosed in the present invention. Figure 6 As shown, the data node 60 of this embodiment includes: a data packet receiving module 602 , a disk array determining module 604 , and a data writing module 606 .

[0069] The data packet receiving module 602 is used to receive a data packet to be written sent by a client.

[0070] The disk array determining module 604 is used to allocate a disk array for the data packet to be written, and the disk array corresponds to multiple disks.

[0071] The data writing module 606 is used to divide the data packet to be written into multiple data slices according to the size of the data packet to be written, and write each data slice in parallel to different disks corresponding to the disk array.

[0072] In some embodiments, the disk array is divided into multiple stripes, each stripe contains multiple storage slices, and different storage slices in a stripe belong to different disks. The data writing module 606 is used to allocate storage space in the disk array for the data packet to be written according to the size of the data packet to be written; and divide the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space.

[0073] In some embodiments, the capacity of the storage slice is less than or equal to the amount of data that can be read and written by a disk at a single time.

[0074] In some embodiments, the data writing module 606 is used to send data slices to the disk array according to the data directories of each disk array, so as to write each data slice to different disks corresponding to the disk array in parallel. The data directory of the disk array is generated after the disk array is formatted as a local file system and mounted to a local directory.

[0075] In some embodiments, the data node 60 further includes: a read request receiving module 608 , a data block search module 610 , and a data block reading module 612 .

[0076] The read request receiving module 608 is used to receive a data read request sent by the client for a data packet to be read.

[0077] The data block search module 610 is used to search the disk array storing the data block to be read according to the data directory, and search the disk storing at least one data slice corresponding to the data packet to be read according to the file system corresponding to the disk array.

[0078] The data block reading module 612 is used to read in parallel each data slice corresponding to the data packet to be read from each storage disk.

[0079] The data nodes in the embodiments of the present disclosure may be implemented by various computing devices or computer systems. Figure 7 as well as Figure 8 Give a description.

[0080] Figure 7 Figure 1 is a structural diagram of some embodiments of the data node disclosed in the present invention. Figure 7 As shown, the data node 70 of this embodiment includes: a memory 710 and a processor 720 coupled to the memory 710 , and the processor 720 is configured to execute the data storage method in any of some embodiments of the present disclosure based on instructions stored in the memory 710 .

[0081] The memory 710 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0082] Figure 8 FIG. 1 is a structural diagram of some other embodiments of the data node disclosed in the present invention. Figure 8 As shown, the data node 80 of this embodiment includes: a memory 810 and a processor 820, which are similar to the memory 710 and the processor 720 respectively. It can also include an input and output interface 830, a network interface 840, a storage interface 850, etc. These interfaces 830, 840, 850 and the memory 810 and the processor 820 can be connected, for example, through a bus 860. Among them, the input and output interface 830 provides a connection interface for input and output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 840 provides a connection interface for various networked devices, for example, it can be connected to a database server or a cloud storage server. The storage interface 850 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0083] The present disclosure also provides a data storage system. Fig. 9 Give a description.

[0084] Fig. 9 FIG. 1 is a structural diagram of some embodiments of the data storage system disclosed in the present invention. Fig. 9 As shown, the system 9 of this embodiment includes: the data node 60 / 70 / 80 of any of the aforementioned embodiments, and a client 92.

[0085] The client 92 is used to receive the block group information sent by the management node, the information in the block group includes the address information of multiple data nodes; divide the data to be written into data units, and calculate the corresponding check units according to a set of preset number of data units; according to the address information of each data node, send the data of the data unit and the check unit to each data node in the form of a data packet to be written. The client 92 can perform the above Figure 4 The steps in the corresponding embodiments.

[0086] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0090] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A data storage method, comprising: The client distributes the data to be written to each data node in parallel in a strip layout; The data node receives the data packet to be written sent by the client; The data node writes the data packets to be written to each disk in parallel in a stripe layout manner; The data node writes the data packets to be written to each disk in parallel in a stripe layout manner, including: The data node allocates a disk array for the data packet to be written, and the disk array corresponds to a plurality of disks; The data node divides the data packet to be written into a plurality of data slices according to the size of the data packet to be written, and writes each data slice in parallel to different disks corresponding to the disk array.

2. The method according to claim 1, wherein: The disk array is divided into a plurality of stripes, each stripe includes a plurality of storage slices, and different storage slices in a stripe belong to different disks.

3. The method according to claim 2, wherein: The data node divides the data packet to be written into a plurality of data slices according to the size of the data packet to be written, including: The data node allocates storage space in the disk array for the data packet to be written according to the size of the data packet to be written; The data node divides the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space.

4. The method according to claim 1, wherein: The step of writing the data slices in parallel to different disk partitions in the disk array comprises: The data node sends the data slices to the disk array according to the data directory of each disk array, so as to write each data slice into different disks corresponding to the disk array in parallel; The data directory of the disk array is generated after the disk array is formatted into a local file system and then mounted to a local directory.

5. The method according to claim 1, further comprising: The data node receives a data read request sent by the client for the data packet to be read; The data node searches for the disk array storing the data block to be read according to the data directory, and searches for the disk storing at least one data slice corresponding to the data packet to be read according to the file system corresponding to the disk array; The data node reads each data slice corresponding to the data packet to be read from each storage disk in parallel.

6. The method according to claim 1, further comprising: The client receives block group information sent by the management node, where the information in the block group includes address information of multiple data nodes; The client divides the data to be written into data units, and calculates corresponding check units according to a set of preset number of data units; The client sends the data of the data unit and the verification unit to each data node in the form of a data packet to be written according to the address information of each data node.

7. A data node, comprising: The data packet receiving module is used to receive the data packet to be written sent by the client; A disk array determining module, used for allocating a disk array for the data packet to be written, wherein the disk array corresponds to a plurality of disks; A data writing module, used for dividing the data packet to be written into a plurality of data slices according to the size of the data packet to be written, and writing each data slice in parallel into different disks corresponding to the disk array; The client is used to distribute the data to be written to each data node in parallel in a stripe layout, and the data node is used to write the data packets to be written to each disk in parallel in a stripe layout.

8. The data node according to claim 7, wherein: The disk array is divided into a plurality of stripes, each stripe includes a plurality of storage slices, and different storage slices in a stripe belong to different disks.

9. The data node according to claim 8, wherein: The data writing module is used to allocate storage space in the disk array for the data packet to be written according to the size of the data packet to be written; and divide the data packet to be written into multiple data slices according to the remaining space of each storage slice corresponding to the storage space.

10. The data node according to claim 7, wherein: The data writing module is used to send the data slices to the disk array according to the data directory of each disk array, so as to write each data slice into different disks corresponding to the disk array in parallel; The data directory of the disk array is generated after the disk array is formatted into a local file system and then mounted to a local directory.

11. The data node according to claim 7, further comprising: A read request receiving module, used for receiving a data read request sent by a client for a data packet to be read; A data block search module, used to search the disk array storing the data block to be read according to the data directory, and search the disk storing at least one data slice corresponding to the data packet to be read according to the file system corresponding to the disk array; The data block reading module is used to read in parallel each data slice corresponding to the data packet to be read from each storage disk.

12. A data node, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the data storage method according to any one of claims 1 to 6 based on instructions stored in the memory.

13. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

14. A data storage system comprising: The data node according to any one of claims 7 to 11; as well as A client is used to receive block group information sent by a management node, wherein the information in the block group includes address information of multiple data nodes; divide the data to be written into data units, and calculate corresponding verification units based on a set of preset number of data units; and send the data of the data units and the data of the verification units to each data node in the form of data packets to be written according to the address information of each data node.

Images