Data storage method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data storage method and device, computer equipment, a readable storage medium and a program product. The data storage method and device are applied to any node in a storage cluster. The method comprises the following steps: when first to-be-stored data sent by a client is received, storing the first to-be-stored data to a first storage pool, sending the first to-be-stored data to other nodes in a storage cluster, receiving a first reply message returned after the other nodes store the first to-be-stored data to a second storage pool, and sending the first reply message to the client, generating a second reply message according to the first reply message, and sending the second reply message to the client; the second reply message is used for indicating that the first to-be-stored data is completely stored. According to the method, the message of whether storage succeeds can be replied to the client only by storing the to-be-stored data in the storage pools corresponding to any node and other nodes, and for the streaming data, the method greatly improves the storage efficiency of the streaming data.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a data storage method, apparatus, computer equipment, readable storage medium and program product. Background Art

[0002] In a distributed storage cluster, efficient and reliable storage of data to be stored has always been a hot topic in the storage field. Since any node in a distributed storage cluster may fail during the data caching process, resulting in data damage and loss, in general, in order to avoid data loss and damage in the event of a failure, a replica cache strategy is generally used when caching data. For example, for the data to be stored that the client transmits to the primary node, it is necessary to confirm that other replica nodes within the redundancy have completed the disk, and then feedback the client with a message that the data has been received, so that the client can continue to transmit the next data to be stored to the primary node to ensure high data availability.

[0003] However, for streaming data to be stored, the above data storage method has the problem of low storage efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a data storage method, apparatus, computer equipment, readable storage medium and program product that can improve the storage efficiency of streaming storage data in response to the above technical problems.

[0005] In a first aspect, the present application provides a data storage method, which is applied to any node in a storage cluster, comprising:

[0006] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0007] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0008] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0009] When performing data storage, the above method not only stores the data to be stored in any node, but also sends it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster. Compared with the method of replying to the client with a message whether the storage is successful after the data to be stored is received and then processed off the disk, the above method greatly improves the processing efficiency of streaming data.

[0010] In one embodiment, the method further comprises:

[0011] Detecting whether the first storage pool after storing the first data to be stored is in a full state;

[0012] When the first storage pool is full, all data stored in the first storage pool is written to a disk and the first storage pool is cleared.

[0013] The data storage method provided in the embodiment of the present application can write all the data in the first storage pool to the disk when the first storage pool is full, so that the first storage pool can continue to store the data to be stored, and provide storage space for the data to be stored in any node next time, thereby solving the problem that the storage pool is full and cannot store data.

[0014] In one embodiment, the first storage pool includes a plurality of storage blocks, and storing the first data to be stored in the first storage pool includes:

[0015] Determine a free memory block among a plurality of memory blocks;

[0016] Dividing the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data;

[0017] According to the division order of the multiple sub-storage data, they are stored in the corresponding free storage blocks in sequence.

[0018] The data storage method provided in the embodiment of the present application first determines the free storage blocks to store the data to be stored based on the free storage blocks, so that the data to be stored can be efficiently stored in the first storage pool; in addition, the data to be stored is divided according to the size of the free storage blocks, and each sub-storage data is sequentially stored in the corresponding free storage blocks based on the division order, so that each sub-storage data can be efficiently stored in each free storage block, thereby avoiding the situation where the storage efficiency is reduced due to insufficient free space in the free storage blocks.

[0019] In one embodiment, the method further comprises:

[0020] A cache mapping table is generated according to the division order of multiple sub-storage data and the position information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

[0021] The data storage method provided in the embodiment of the present application records the division order of each sub-storage data so that when the data in the storage pool is subsequently written to the disk, it is written to the disk based on the division order of each sub-storage data, thereby ensuring the integrity and accuracy of the written data.

[0022] In one embodiment, the generating the second reply message according to the first reply message includes:

[0023] Determine, according to the first reply message returned by each other node, whether the storage progress of each other node and the local node meets the preset progress condition; the preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool;

[0024] If the storage progress of other nodes and the local storage meets the preset progress condition, a second reply message indicating that the storage is successful is generated;

[0025] If the storage progress of other nodes and the local storage does not meet the preset progress condition, a second reply message indicating storage failure is generated.

[0026] In the embodiment of the present application, if at least one of the other nodes and any one node fails to successfully store the first data to be stored in the corresponding storage pool, then any one node generates a second reply message indicating storage failure.

[0027] In one of the embodiments, upon receiving the first data to be stored sent by the client, the method further includes:

[0028] Detect whether there is a storage failure on any node;

[0029] If it exists, a disk write instruction is sent to other nodes in the storage cluster; the disk write instruction is used to instruct other nodes to write the data in the second storage pool to disk;

[0030] If it does not exist, a clearing instruction is sent to other nodes in the storage cluster; clearing is used to instruct other nodes to clear the data in the second storage pool.

[0031] The data storage method provided in the embodiment of the present application can write the data stored on other nodes to disks when a storage failure is detected in a node, thereby ensuring the reliability of the data to be stored.

[0032] In a second aspect, the present application provides a data storage method, which is applied to any other node in a storage cluster, including:

[0033] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0034] Send the first reply message to any node.

[0035] When performing data storage, the above method not only stores the data to be stored in any node, but also sends it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster, and after receiving the data to be stored, it is necessary to process the data to be stored off the disk before replying to the client with a message whether the storage is successful. For streaming data, the above method greatly improves the processing efficiency of streaming data.

[0036] In a third aspect, the present application further provides a data storage device, comprising:

[0037] A storage module, configured to store the first data to be stored in a first storage pool upon receiving the first data to be stored sent by the client, and send the first data to be stored to other nodes in the storage cluster;

[0038] A receiving module, used for receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0039] The sending module is used to send a second reply message to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0040] In a fourth aspect, the present application further provides a data storage device, comprising:

[0041] A receiving module, used for receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0042] The sending module is used to send the first reply message to any node.

[0043] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0044] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0045] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0046] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0047] In a sixth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0048] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0049] Send the first reply message to any node.

[0050] In a seventh aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0051] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0052] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0053] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0054] In an eighth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0055] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0056] Send the first reply message to any node.

[0057] In a ninth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0058] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0059] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0060] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0061] In a tenth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0062] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0063] Send the first reply message to any node.

[0064] The above-mentioned data storage method, device, computer equipment, readable storage medium and program product, when performing data storage, not only store the data to be stored in any node, but also send it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above-mentioned method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster, and after receiving the data to be stored, it is necessary to process the data to be stored off the disk before replying to the client with a message whether the storage is successful. For streaming data, the above-mentioned method greatly improves the processing efficiency of streaming data. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 An application environment diagram of a data storage method in one embodiment;

[0067] Figure 2 A schematic diagram of a flow chart of a data storage method in one embodiment;

[0068] Figure 3 is a schematic diagram of a storage pool and a cache mapping table in one embodiment;

[0069] Figure 4 is a schematic diagram of a cache mapping table in one embodiment;

[0070] Figure 5 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0071] Figure 6 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0072] Figure 7 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0073] Figure 8 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0074] Fig. 9 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0075] Fig.10 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0076] Fig.11 A schematic diagram of a flow chart of a data storage method in another embodiment;

[0077] Fig.12 is a structural block diagram of a data storage device in one embodiment;

[0078] Fig.13 is a structural block diagram of a data storage device in another embodiment;

[0079] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0081] In a distributed storage cluster, efficient and reliable storage of data to be stored has always been a hot topic in the storage field. Since any node in a distributed storage cluster may fail during the data caching process, resulting in data damage and loss, in general, in order to avoid data loss and damage in the event of a failure, a replica cache strategy is generally used when caching data. For example, for the data to be stored that the client transmits to the primary node, it is necessary to confirm that other replica nodes within the redundancy have completed the disk, and then feedback the client with a message that the data has been received, so that the client can continue to transmit the next data to be stored to the primary node to ensure high data availability.

[0082] However, for streaming data to be stored, the above data storage method has the problem of low storage efficiency. This application aims to solve this problem.

[0083] After introducing the background technology of the data storage method provided by the embodiment of the present application, the implementation environment involved in the data storage method provided by the embodiment of the present application will be briefly described below. The data storage method provided by the embodiment of the present application can be applied to Figure 1 In any node 101 in the storage cluster shown, the storage cluster includes one any node 101 and multiple other nodes 102. It should be noted that each node in the storage cluster can be designated as any node 101. In the storage cluster, any node 101 can send a task instruction to each other node 102 to instruct each other node 102 to perform a corresponding task according to the instruction, obtain execution information, and feed the execution information back to any node 101, so that any node 101 can obtain the execution information of each other node 102. Any node 101 can also connect to the client 103 via network communication. When the client 103 needs to store the data to be stored in any node 101, the data to be stored can be sent to any node 101 first. To ensure storage reliability, any node 101 can send the data to be stored to other nodes 102. After other nodes 102 receive and store the data to be stored, they send feedback information to any node. After any node 101 and other nodes 102 have completed the storage of the data to be stored, storage completion information is fed back to the client 103 so that the client 103 can send the next data to be stored to any node 101 for storage.

[0084] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0085] In one embodiment, Figure 2 As shown, a data storage method is provided, which is applied to Figure 1 Taking any node 101 in the storage cluster as an example, the following steps are included:

[0086] S201. When receiving first data to be stored sent by a client, store the first data to be stored in a first storage pool, and send the first data to be stored to other nodes in the storage cluster.

[0087] Among them, the first data to be stored refers to any data to be stored sent by the client to any node in the storage cluster. The first data to be stored can be any data in the client's streaming data cluster. Streaming data refers to data sent by the client to any node in the storage cluster according to certain sending rules within a preset time period.

[0088] It should be noted that each streaming data in the streaming data cluster carries a node identifier in the storage cluster to instruct the client to send the streaming data to the corresponding node in the storage cluster according to the node identifier. For example, the node corresponding to the identifier of the first data to be stored is any node in the storage cluster.

[0089] Among them, other nodes refer to other nodes in the storage cluster except any node, and the node information of other nodes must meet preset conditions, for example, the memory usage of other nodes must be less than a preset usage threshold.

[0090] Among them, the first storage pool refers to a data storage area pre-set for any node. After receiving the data to be stored, any node can store the data to be stored in the first storage pool; it should be noted that the first storage pool may include multiple storage blocks. After receiving the data to be stored, any node can first store the first data to be stored in the first storage block in the first storage pool, and when the first storage block is full and the first data to be stored has not been stored, continue to store the first data to be stored in the second storage block until the first data to be stored is completely stored.

[0091] In an embodiment of the present application, when a client needs to store a batch of streaming data, the client can first determine the first data to be stored in the streaming data, for example, determine the first data in the streaming database as the first data to be stored, and after determining the first data to be stored, send the first data to be stored to any node in the corresponding storage cluster. When any node receives the first data to be stored sent by the client, it can store the first data to be stored in a pre-set first storage pool, and send the first data to be stored to each other node in the storage cluster.

[0092] Optionally, after receiving the first data to be stored sent by the client, any node can directly send the first data to be stored to other nodes in the storage cluster; or, after receiving the first data to be stored sent by the client, any node can first store the first data to be stored in the first storage pool, and then send the first data to be stored to other nodes in the storage cluster.

[0093] Optionally, when a node in the storage cluster is started, an expected number of storage blocks can be allocated to the storage pool in advance, which can save time for re-applying for memory space during system operation and improve the reusability of memory storage blocks.

[0094] Optionally, each node in the storage cluster can initialize its corresponding storage pool and set corresponding storage rules for the storage pool, which is called the storage pool model. The storage pool model provides functions such as application and return of storage blocks, and regular inspection of long-term immobile storage blocks, so as to facilitate the implementation of streaming data transmission based on storage block copies during the storage process. Storage pool model see Figure 3 , Figure 3 The storage pool in is the storage pool corresponding to one node. Multiple storage blocks are pre-set in the storage pool, namely Figure 3 The box marked with 4M in the figure refers to a storage block of 4M in size. When the data stream file is transmitted to the node and needs to be stored, a certain number of storage blocks can be borrowed from the corresponding storage pool to store the data. After the data is stored on the disk, the data in the storage block can be released and returned to the storage pool.

[0095] Optionally, the storage pool model includes a cache mapping table, which stores the mapping relationship between the target data stream and the storage block in the storage pool. Through the cache mapping table, the cache block where the target data stream is located can be easily and accurately addressed and the copy data can be restored. The cache mapping table is shared among the cluster nodes to prevent information asymmetry.

[0096] Optional, cache mapping table see Figure 4 Among them, file1 represents data stream 1, and the arrow corresponding to file1 indicates that file1 is stored in three storage blocks of 4M in size. The order of these three storage blocks is the order of the data in file1, so the positions of these three storage blocks cannot be changed arbitrarily; file2 represents data stream 2, and the arrow corresponding to file2 indicates that file2 is stored in a storage block of 4M in size; file3 represents data stream 3, and the arrow corresponding to file3 indicates that file3 is stored in a storage block of 4M in size.

[0097] S202: Receive a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generate a second reply message according to the first reply message.

[0098] The first reply message is used to indicate whether the other nodes have successfully stored the first data to be stored, and the second reply message is used to indicate whether any node and each other node have successfully stored the first data to be stored.

[0099] In an embodiment of the present application, after the first node sends the first data to be stored to each other node, each other node may send a first reply message to any node after receiving the first data to be stored, or each other node may send a first reply message to any node after receiving the first data to be stored and storing the first data to be stored in the second storage pool. Then, after receiving the first reply message sent by each other node, any node integrates the first reply messages and the message of whether any node successfully stores the first data to be stored to obtain a second reply message.

[0100] For example, if the first reply message of each other node indicates that each other node has successfully stored the first data to be stored, and any node has also successfully stored the first data to be stored, then the second reply message generated by any node is the successful storage of the first data to be stored; for another example, if the first reply message of any other node among the other nodes indicates that any other node has not successfully stored the first data to be stored, and any node has successfully stored the first data to be stored, then the second reply message generated by any node is that any other node has not successfully stored the first data to be stored. In this case, any node may send the first data to be stored to any other node again until any other node successfully stores the first data to be stored.

[0101] S203: Send a second reply message to the client.

[0102] The second reply message is used to indicate that the first data to be stored has been stored.

[0103] In the embodiment of the present application, after the second reply message is obtained as described above, a second reply message may be sent to the client to indicate that any node and other nodes have successfully stored the first data to be stored.

[0104] The data storage method provided in the embodiment of the present application is applied to any node in the storage cluster. When receiving the first data to be stored sent by the client, the first data to be stored is stored in the first storage pool, and the first data to be stored is sent to each other node in the storage cluster. Then, the first reply message returned by each other node after storing the first data to be stored in the second storage pool is received, and a second reply message is generated according to the first reply message, and the second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored. When performing data storage, the above method not only stores the data to be stored in any node, but also sends it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster. Compared with the method in which the data to be stored needs to be processed after the data to be stored is received and then the message of whether the storage is successful is replied to the client, for streaming data, the above method greatly improves the processing efficiency of streaming data.

[0105] In one embodiment, Figure 2 Based on the embodiment shown, Figure 5 As shown, the above method also includes:

[0106] S204: Detect whether the first storage pool after storing the first data to be stored is in a full state.

[0107] In an embodiment of the present application, after any node completes storage of the first data to be stored through its corresponding first storage pool, a monitoring device installed on the first storage pool can be used to detect whether the first storage pool after storing the first data to be stored is in a full state.

[0108] S205: When the first storage pool is full, write all data stored in the first storage pool to a disk and clear the first storage pool.

[0109] In an embodiment of the present application, when it is detected that the first storage pool is full, all data stored in the first storage pool are written to disk, and after all data in the first storage pool is written to disk, the first storage pool is emptied; or, after part of the data in the first storage pool is written to disk, the corresponding part of the data in the first storage pool is emptied, and after another part of the data in the first storage pool is written to disk, the corresponding other part of the data in the first storage pool is emptied, until all the data in the first storage pool is written to disk, that is, all the data in the first storage pool is emptied.

[0110] The data storage method provided in the embodiment of the present application can write all the data in the first storage pool to the disk when the first storage pool is full, so that the first storage pool can continue to store the data to be stored, and provide storage space for the data to be stored in any node next time, thereby solving the problem that the storage pool is full and cannot store data.

[0111] In one embodiment, Figure 2 or Figure 5 Based on the embodiment shown, the first storage pool includes a plurality of storage blocks, and the process of storing the first data to be stored in the first storage pool can be described as follows: Figure 6 As shown, the above S201 “storing the first data to be stored in the first storage pool” includes:

[0112] S301. Determine a free storage block among multiple storage blocks.

[0113] The first storage pool includes multiple storage blocks.

[0114] In an embodiment of the present application, when the first data to be stored sent by the client is received, a storage block that meets the preset condition can be first determined from multiple storage blocks in the storage pool, and the storage block that meets the preset condition is determined as a free storage block. Optionally, the storage block that meets the preset condition refers to a storage block whose memory occupancy rate meets a preset threshold, for example, a storage block whose memory occupancy rate is less than 5%.

[0115] S302 . Divide the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data.

[0116] In an embodiment of the present application, after the multiple free storage blocks are determined as described above, the available size of each free storage block can be determined, and the first data to be stored can be divided according to the available size of the free storage block to obtain multiple divided sub-storage data, and the division order of each sub-storage data can be recorded to determine the entire first data to be stored based on the division order of each sub-storage data.

[0117] For example, the above S301 determines 3 free storage blocks, the available space of the first free storage block is 10M, the available space of the second free storage block is 6M, the available space of the third free storage block is 8M, and the data size of the first data to be stored is 20M. Then, based on the principle that the more available space, the higher the storage level, the first data to be stored can be divided into three parts of 10M, 8M and 2M based on the first storage block, the third storage block and the second storage block respectively.

[0118] S303 , storing the plurality of sub-storage data in corresponding free storage blocks in sequence according to the division order.

[0119] In the embodiment of the present application, after the plurality of sub-storage data of the first data to be stored are obtained, the respective storage data can be stored in the corresponding free storage block in sequence according to the division order of the plurality of sub-storage data.

[0120] Continuing with the above example, after dividing the first data to be stored into three parts of 10M, 8M and 2M, the first 10M of the data to be stored can be stored in the first free storage block, the middle 8M of the data to be stored can be stored in the second free storage block, and the last 2M of the data to be stored can be stored in the third free storage block.

[0121] The data storage method provided in the embodiment of the present application first determines the free storage blocks to store the data to be stored based on the free storage blocks, so that the data to be stored can be efficiently stored in the first storage pool; in addition, the data to be stored is divided according to the size of the free storage blocks, and each sub-storage data is sequentially stored in the corresponding free storage blocks based on the division order, so that each sub-storage data can be efficiently stored in each free storage block, thereby avoiding the situation where the storage efficiency is reduced due to insufficient free space in the free storage blocks.

[0122] In addition, the present application also provides a method for storing first data to be stored using multiple free storage blocks in a storage pool, including: determining a free storage block (at least one) among multiple storage blocks in the storage pool, and receiving the first data to be stored based on the first free storage block among the multiple free storage blocks (if there is only one free storage block, then the first free storage block is this free storage block, if there are multiple free storage blocks, then any one of the multiple free storage blocks can be used as the first free storage block), and after the first free storage block completes the storage, determining whether the storage of the first data to be stored is completed, if the storage of the first data to be stored is not completed, continuing to receive the first data to be stored based on other storage blocks in the first storage pool except the first storage block, and stopping the storage operation after the other storage blocks complete the storage of the first data to be stored, and when the other storage blocks have not completed the storage of the first data to be stored, returning to continue to determine other free storage blocks in the first storage pool to continue to store the first data to be stored until the storage of the first data to be stored is completed, and recording the storage order of the first data to be stored in each free storage block, so that the first data to be stored can be subsequently processed on the disk based on the storage order.

[0123] In one embodiment, Figure 6 Based on the embodiment shown, Figure 7 As shown, the above method also includes:

[0124] S304: Generate a cache mapping table according to the division order of the plurality of sub-storage data and the location information of the free storage blocks corresponding to each sub-storage data.

[0125] The cache mapping table records the corresponding relationship between data and storage blocks, and the corresponding relationship between data partition numbers and storage block positions. In addition, the cache mapping table also includes the corresponding relationship between any node and other nodes.

[0126] In an embodiment of the present application, after the first data to be stored is divided to obtain multiple sub-storage data, the division order of the multiple sub-storage data can be recorded, and after the multiple sub-storage data are stored in the corresponding free storage blocks in sequence according to the division order, the location information of the free storage blocks corresponding to each sub-storage data can be recorded, and based on the division order of the multiple sub-storage data and the location information of the free storage blocks corresponding to each sub-storage data, a cache mapping table including the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions is generated.

[0127] For example, a first data to be stored of size 10M is divided into three parts of sub-storage data, the first part is 5M of sub-storage data, the second part is 2M of sub-storage data, and the third part is 3M of sub-storage data, and the 5M of sub-storage data corresponds to the first free storage block, the 2M of sub-storage data corresponds to the second free storage block, and the 3M of sub-storage data corresponds to the third free storage block, then the 5M of sub-storage data is determined as data 1, the 2M of sub-storage data is determined as data 2, and the 3M of sub-storage data is determined as data 3, and the position of the first free storage block is determined as position 1, the position of the second free storage block is determined as position 2, and the position of the third storage block is determined as position 3, then the correspondence between the data partition number and the storage position is determined as data 1 corresponds to position 1, data 2 corresponds to position 2, and data 3 corresponds to position 3.

[0128] The data storage method provided in the embodiment of the present application records the division order of each sub-storage data so that when the data in the storage pool is subsequently written to the disk, it is written to the disk based on the division order of each sub-storage data, thereby ensuring the integrity and accuracy of the written data.

[0129] In one embodiment, Figure 2 or Figure 5 Based on the illustrated embodiment, the process of generating the second reply message according to the first reply message can be described as follows: Figure 8 As shown, the above S202 “generating a second reply message according to the first reply message” includes:

[0130] S401. Determine whether the storage progress of each other node and the local node meets the preset progress condition based on the first reply message returned by each other node. If the storage progress of each other node and the local node meets the preset progress condition, execute S402. If the storage progress of each other node and the local node does not meet the preset progress condition, execute S403.

[0131] The preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool.

[0132] In an embodiment of the present application, after any of the above-mentioned nodes receives the first reply message returned by each other node, it is determined whether each other node has successfully stored the first data to be stored in the second storage pool, and whether any node has successfully stored the first data to be stored in the first storage pool, based on the first reply message returned by each other node. If each other node has successfully stored the first data to be stored in the second storage pool, and any node has successfully stored the first data to be stored in the first storage pool, then the following S402 is executed; if at least one of the other nodes and any node has failed to successfully store the first data to be stored in the corresponding storage pool, then S403 is executed.

[0133] S402: Generate a second reply message indicating successful storage.

[0134] The second reply message of successful storage refers to a message that any node and all other nodes have successfully stored the first data to be stored.

[0135] In the embodiment of the present application, if all other nodes successfully store the first data to be stored in the second storage pool, and after any node successfully stores the first data to be stored in the first storage pool, any node generates a second reply message indicating successful storage.

[0136] S403: Generate a second reply message indicating storage failure.

[0137] Among them, the second reply message of storage failure is a message that at least one node among any node and other nodes fails to successfully store the first data to be stored, and the second reply message of storage failure includes the node identifier of the node that failed to successfully store the first data to be stored and the reason for the failure to successfully store the first data to be stored.

[0138] In the embodiment of the present application, if at least one of the other nodes and any one node fails to successfully store the first data to be stored in the corresponding storage pool, then any one node generates a second reply message indicating storage failure.

[0139] Furthermore, any node can determine the node that failed to store the data successfully based on the node identifier of the node that failed to store the first data to be stored in the second reply message of the storage failure, and resend the first data to be stored to the node that failed to store the data successfully, so that the node stores the first data to be stored again, until all other nodes successfully store the first data to be stored in the second storage pool, and after any node successfully stores the first data to be stored in the first storage pool, return to the above S402.

[0140] The data storage method provided in the embodiment of the present application generates a second reply message of successful storage only when it is ensured that all other nodes and any node have successfully stored the first data to be stored, so as to feed back the second reply message of successful storage to the client, and the client can send the next data to be stored to the node, thereby ensuring the successful storage of the data to be stored while improving the data storage efficiency.

[0141] In one embodiment, Figure 2 or Figure 5 Based on the embodiment shown, Fig. 9 As shown, when receiving the first data to be stored sent by the client, the method further includes:

[0142] S206. Detect whether there is a storage failure in any node. If so, execute S207; if not, execute S208.

[0143] The storage failure includes the free space of the node being less than the minimum free space occupancy rate, or the node disk failure, or the node storage pool failure, etc.

[0144] In the embodiment of the present application, each node in the storage cluster may be pre-installed with a fault detection device, and the fault detection device may detect in real time or periodically whether a node has a storage fault.

[0145] S207: Send a disk placement instruction to other nodes in the storage cluster.

[0146] The disk write instruction is used to instruct other nodes to write the data in the second storage pool to disk.

[0147] In the embodiment of the present application, if a node has a storage failure, any node needs to send a disk write instruction to each other node in the storage cluster to instruct each other node in the storage cluster to write the data in the corresponding second storage pool to disk.

[0148] S208: Send a clear instruction to other nodes in the storage cluster.

[0149] The clearing is used to instruct other nodes to clear the data in the second storage pool.

[0150] In the embodiment of the present application, if there is no storage failure in the node, any node needs to send a clear instruction to each other node in the storage cluster to instruct each other node in the storage cluster to clear the data in the corresponding second storage pool.

[0151] The data storage method provided in the embodiment of the present application can write the data stored on other nodes to disks when a storage failure is detected in a node, thereby ensuring the reliability of the data to be stored.

[0152] In one embodiment, when receiving the first data to be stored sent by the client, the above method also includes: detecting whether there is a storage failure in the node; if so, determining the target node in the storage cluster, and sending the first data to be stored and the task instruction to the target node; the task instruction is used to instruct the target node to store the first data to be stored in the third storage pool for storage, and undertake the distributed storage task of the node.

[0153] In one embodiment, Fig.10 As shown, a data storage method is provided, which is applied to Figure 1 Taking any other node 102 in the storage cluster as an example, the method includes the following steps:

[0154] S501, receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in a second storage pool, and generating a first reply message after completing the storage.

[0155] In an embodiment of the present application, after any node in the above-mentioned storage cluster sends the first data to be stored to any other node, any other node receives the first data to be stored sent by any node in the storage cluster, and stores the first data to be stored in the second storage pool corresponding to any other node, and generates a first reply message after completing the storage.

[0156] S502: Send a first reply message to any node.

[0157] In the embodiment of the present application, after the first reply message is obtained as described above, the first reply message can be sent to any node to inform any node that any other node has successfully stored the first data to be stored.

[0158] The data storage method provided in the embodiment of the present application is applied to any other node in the storage cluster, receives the first data to be stored sent by any node in the storage cluster, and stores the first data to be stored in the second storage pool, generates a first reply message after the storage is completed, and sends the first reply message to any node. When performing data storage, the above method not only stores the data to be stored in any node, but also sends it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster, and after receiving the data to be stored, it is necessary to process the data to be stored off the disk before replying to the client with a message whether the storage is successful. For streaming data, the above method greatly improves the processing efficiency of streaming data.

[0159] In one embodiment, it can be detected whether the second storage pool is full after storing the first data to be stored. If the second storage pool is full after storing the first data to be stored, all data stored in the second storage pool are written to disk and the second storage pool is cleared.

[0160] In one embodiment, Fig.11 As shown, a data storage method is provided, including:

[0161] S10. When receiving first data to be stored sent by the client, storing the first data to be stored in a first storage pool;

[0162] S11, sending the first data to be stored to other nodes in the storage cluster;

[0163] S12, receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0164] S13, sending the first reply message to any node;

[0165] S14, receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool;

[0166] S15. Generate a second reply message according to the first reply message;

[0167] S16. Send a second reply message to the client indicating that the first data to be stored has been stored.

[0168] When performing data storage, the above method not only stores the data to be stored in any node, but also sends it to other nodes for storage via any node, thereby ensuring the reliability of data storage; in addition, the above method only needs to store the data to be stored in the storage pool corresponding to any node and other nodes, and can reply to the client with a message whether the storage is successful, so as to instruct the client to send the next data to be stored to the node in the storage cluster, and after receiving the data to be stored, it is necessary to process the data to be stored off the disk before replying to the client with a message whether the storage is successful. For streaming data, the above method greatly improves the processing efficiency of streaming data.

[0169] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0170] Based on the same inventive concept, the embodiment of the present application also provides a data storage device for implementing the data storage method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more data storage device embodiments provided below can refer to the limitations on the data storage method above, and will not be repeated here.

[0171] In an exemplary embodiment, Fig.12 As shown, a data storage device is provided, comprising: a storage module 10, a receiving module 11 and a sending module 12, wherein:

[0172] The storage module 10 is used for storing the first data to be stored in the first storage pool upon receiving the first data to be stored sent by the client, and sending the first data to be stored to other nodes in the storage cluster.

[0173] The receiving module 11 is used to receive a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generate a second reply message according to the first reply message.

[0174] The sending module 12 is used to send a second reply message to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0175] In an exemplary embodiment, the above device further includes: a detection module and a clearing module, wherein:

[0176] A detection module, used for detecting whether the first storage pool after storing the first data to be stored is in a full state;

[0177] The clearing module is used to write all data stored in the first storage pool to a disk and clear the first storage pool when the first storage pool is full.

[0178] In an exemplary embodiment, the first storage pool includes a plurality of storage blocks, and the storage module 10 includes: a determination unit, a division unit and a storage unit, wherein:

[0179] A determination unit, specifically used to determine a free storage block among a plurality of storage blocks;

[0180] A division unit, specifically used to divide the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data;

[0181] The storage unit is specifically used to store the plurality of sub-storage data in the corresponding free storage blocks in sequence according to the division order of the plurality of sub-storage data.

[0182] In an exemplary embodiment, the above-mentioned storage module 10 also includes: a generation unit, which is specifically used to generate a cache mapping table according to the division order of multiple sub-storage data and the location information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

[0183] In an exemplary embodiment, the receiving module 11 includes: a determining unit, a first generating unit and a second generating unit, wherein:

[0184] A determination unit, specifically used to determine whether the storage progress of each other node and the local node meets a preset progress condition according to the first reply message returned by each other node; the preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool;

[0185] A first generating unit, specifically configured to generate a second reply message indicating successful storage when the storage progress of each other node and the local node meets a preset progress condition;

[0186] The second generating unit is specifically configured to generate a second reply message indicating storage failure when the storage progress of other nodes and the local storage progress does not meet a preset progress condition.

[0187] In an exemplary embodiment, when receiving the first data to be stored sent by the client, the above device further includes: a detection module, a disk storage module and a clearing module, wherein:

[0188] A detection module is used to detect whether any node has a storage failure;

[0189] The disk placement module is used to send a disk placement instruction to other nodes in the storage cluster when a storage failure occurs in any node; the disk placement instruction is used to instruct other nodes to place the data in the second storage pool on disk;

[0190] The clearing module is used to send clearing instructions to other nodes in the storage cluster when there is no storage failure in any node; the clearing is used to instruct other nodes to clear the data in the second storage pool.

[0191] In an exemplary embodiment, Fig.13 As shown, a data storage device is provided, comprising:

[0192] The receiving module 20 and the sending module 21 include:

[0193] The receiving module 20 is used to receive the first data to be stored sent by any node in the storage cluster, store the first data to be stored in the second storage pool, and generate a first reply message after the storage is completed;

[0194] The sending module 21 is used to send the first reply message to any one of the nodes.

[0195] Each module in the above data storage device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0196] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.14As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be stored. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data storage method is implemented.

[0197] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0198] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0199] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0200] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0201] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0202] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0203] Detecting whether the first storage pool after storing the first data to be stored is in a full state;

[0204] When the first storage pool is full, all data stored in the first storage pool is written to a disk and the first storage pool is cleared.

[0205] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0206] Determine a free memory block among a plurality of memory blocks;

[0207] Dividing the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data;

[0208] According to the division order of the multiple sub-storage data, they are stored in the corresponding free storage blocks in sequence.

[0209] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0210] A cache mapping table is generated according to the division order of multiple sub-storage data and the position information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

[0211] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0212] Determine, according to the first reply message returned by each other node, whether the storage progress of each other node and the local node meets the preset progress condition; the preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool;

[0213] If the storage progress of other nodes and the local node meets the preset progress condition, a second reply message indicating successful storage is generated;

[0214] If the storage progress of other nodes and the local storage does not meet the preset progress condition, a second reply message indicating storage failure is generated.

[0215] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0216] Detect whether there is a storage failure on any node;

[0217] If it exists, a disk write instruction is sent to other nodes in the storage cluster; the disk write instruction is used to instruct other nodes to write the data in the second storage pool to disk;

[0218] If it does not exist, a clearing instruction is sent to other nodes in the storage cluster; clearing is used to instruct other nodes to clear the data in the second storage pool.

[0219] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0220] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0221] Send the first reply message to any node.

[0222] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0223] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0224] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0225] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0226] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0227] Detecting whether the first storage pool after storing the first data to be stored is in a full state;

[0228] When the first storage pool is full, all data stored in the first storage pool is written to a disk and the first storage pool is cleared.

[0229] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0230] Determine a free memory block among a plurality of memory blocks;

[0231] Dividing the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data;

[0232] According to the division order of the multiple sub-storage data, they are stored in the corresponding free storage blocks in sequence.

[0233] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0234] A cache mapping table is generated according to the division order of multiple sub-storage data and the position information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

[0235] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0236] Determine, according to the first reply message returned by each other node, whether the storage progress of each other node and the local node meets the preset progress condition; the preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool;

[0237] If the storage progress of other nodes and the local storage meets the preset progress condition, a second reply message indicating that the storage is successful is generated;

[0238] If the storage progress of other nodes and the local storage does not meet the preset progress condition, a second reply message indicating storage failure is generated.

[0239] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0240] Detect whether there is a storage failure on any node;

[0241] If it exists, a disk write instruction is sent to other nodes in the storage cluster; the disk write instruction is used to instruct other nodes to write the data in the second storage pool to disk;

[0242] If it does not exist, a clearing instruction is sent to other nodes in the storage cluster; clearing is used to instruct other nodes to clear the data in the second storage pool.

[0243] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0244] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0245] Send the first reply message to any node.

[0246] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0247] When receiving the first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to each other node in the storage cluster;

[0248] receiving a first reply message returned by each other node after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message;

[0249] A second reply message is sent to the client; the second reply message is used to indicate that the first data to be stored has been stored.

[0250] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0251] Detecting whether the first storage pool after storing the first data to be stored is in a full state;

[0252] When the first storage pool is full, all data stored in the first storage pool is written to a disk and the first storage pool is cleared.

[0253] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0254] Determine a free memory block among a plurality of memory blocks;

[0255] Dividing the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data;

[0256] According to the division order of the multiple sub-storage data, they are stored in the corresponding free storage blocks in sequence.

[0257] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0258] A cache mapping table is generated according to the division order of multiple sub-storage data and the position information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

[0259] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0260] Determine, according to the first reply message returned by each other node, whether the storage progress of each other node and the local node meets the preset progress condition; the preset progress condition includes that each other node successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool;

[0261] If the storage progress of other nodes and the local node meets the preset progress condition, a second reply message indicating successful storage is generated;

[0262] If the storage progress of other nodes and the local storage does not meet the preset progress condition, a second reply message indicating storage failure is generated.

[0263] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0264] Detect whether there is a storage failure on any node;

[0265] If it exists, a disk write instruction is sent to other nodes in the storage cluster; the disk write instruction is used to instruct other nodes to write the data in the second storage pool to disk;

[0266] If it does not exist, a clearing instruction is sent to other nodes in the storage cluster; clearing is used to instruct other nodes to clear the data in the second storage pool.

[0267] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0268] Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed;

[0269] Send the first reply message to any node.

[0270] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0271] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0272] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0273] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data storage method, characterized in that: Applied to any node in a storage cluster, the method includes: When receiving first data to be stored sent by the client, storing the first data to be stored in the first storage pool, and sending the first data to be stored to other nodes in the storage cluster; receiving a first reply message returned by each of the other nodes after storing the first data to be stored in the second storage pool, and generating a second reply message according to the first reply message; Send a second reply message to the client; the second reply message is used to indicate that the first data to be stored has been stored.

2. The method according to claim 1, characterized in that The method further comprises: Detecting whether the first storage pool after storing the first data to be stored is in a full state; When the first storage pool is full, all data stored in the first storage pool is written to a disk, and the first storage pool is emptied.

3. The method according to claim 1 or 2, characterized in that: The first storage pool includes a plurality of storage blocks, and storing the first data to be stored in the first storage pool includes: Determining a free storage block among the plurality of storage blocks; Dividing the first data to be stored according to the size of the free storage block to obtain a plurality of sub-storage data; According to the division order of the plurality of sub-storage data, the sub-storage data are sequentially stored in corresponding free storage blocks.

4. The method according to claim 3, characterized in that The method further comprises: A cache mapping table is generated according to the division order of the multiple sub-storage data and the location information of the free storage blocks corresponding to each sub-storage data; the cache mapping table records the correspondence between data and storage blocks, and the correspondence between data division numbers and storage block positions.

5. The method according to claim 1 or 2, characterized in that: The generating a second reply message according to the first reply message includes: Determine, according to the first reply message returned by each of the other nodes, whether the storage progress of each of the other nodes and the local node meets a preset progress condition; the preset progress condition includes that each of the other nodes successfully stores the first data to be stored in the second storage pool, and the local node successfully stores the first data to be stored in the first storage pool; If the storage progress of each of the other nodes and the local node meets the preset progress condition, a second reply message indicating that the storage is successful is generated; If the storage progress of each of the other nodes and the local storage does not meet the preset progress condition, a second reply message indicating storage failure is generated.

6. The method according to claim 1 or 2, characterized in that: When receiving the first data to be stored sent by the client, the method further includes: Detecting whether any of the nodes has a storage failure; If so, sending a disk write instruction to each of the other nodes in the storage cluster; the disk write instruction is used to instruct the other nodes to write the data in the second storage pool to disk; If not, a clearing instruction is sent to each of the other nodes in the storage cluster; the clearing is used to instruct the other nodes to clear the data in the second storage pool.

7. A data storage method, characterized in that: Applied to any other node in the storage cluster, the method includes: Receiving first data to be stored sent by any node in the storage cluster, storing the first data to be stored in the second storage pool, and generating a first reply message after the storage is completed; The first reply message is sent to any one of the nodes.

8. A data storage device, characterized in that: Applied to any node in a storage cluster, the device comprises: A storage module, configured to, upon receiving first data to be stored sent by a client, store the first data to be stored in a first storage pool, and send the first data to be stored to other nodes in the storage cluster; a receiving module, configured to receive a first reply message returned by each of the other nodes after storing the first data to be stored in the second storage pool, and generate a second reply message according to the first reply message; A sending module is used to send a second reply message to the client; the second reply message is used to indicate that the first data to be stored has been stored.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images