Restart processing method of storage system, electronic equipment, medium and product

Abstract

The invention discloses a restart processing method of a storage system, electronic equipment, a medium and a product, and relates to the technical field of electronic equipment.The restart processing method comprises the steps that after the system is restarted, uncompleted write-in operation can be determined according to write-in data information and write-in state marks corresponding to the write-in operation in a write-in operation log; according to the method, the storage system and the storage device, the storage system does not depend on extra hardware support, so that the use cost of the storage system can be reduced, and the flexibility of data processing can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a restart processing method for a storage system, electronic equipment, medium, and product. Background Art

[0002] When updating data in a storage system, if the storage system suddenly loses power, if only part of the data is updated, the data update may be incomplete or erroneous after the storage system restarts.

[0003] Currently, related technologies rely on redundant hardware design to solve the problem of incomplete or erroneous data updates. For example, related technologies use battery backup cache to ensure the continuity of data updates. However, the hardware cost is high, not suitable for large-scale deployment, and has poor flexibility in handling data recovery. Summary of the Invention

[0004] The present application provides a storage system restart processing method, electronic equipment, medium and product, which can reduce the cost of data recovery and improve the convenience and flexibility of data processing.

[0005] This application provides a storage system restart processing method, including:

[0006] After detecting that the storage system is restarted, reading a corresponding write operation log from a preset storage unit of the storage system, wherein the write operation log is generated based on the write operation sent by the user terminal, wherein the write operation log records write data information and a write status mark corresponding to the write operation;

[0007] Determine whether there is an unfinished write operation based on the write status mark in the write operation log;

[0008] If there is an unfinished write operation, obtain the write data information corresponding to the write operation;

[0009] According to the written data information, the write operation is re-executed to update the data corresponding to the written data information into the storage hard disk of the storage system.

[0010] This application also provides a schematic structural diagram of a restart processing device for a storage system, including:

[0011] an acquisition module, configured to read a corresponding write operation log from a preset storage unit of the storage system after detecting that the storage system has been restarted, wherein the write operation log is generated based on a write operation sent by a user terminal, and wherein the write operation log records write data information and a write status flag corresponding to the write operation;

[0012] A processing module, configured to determine whether there is an unfinished write operation based on a write status mark in the write operation log;

[0013] The processing module is further configured to obtain write data information corresponding to the write operation if there is an uncompleted write operation;

[0014] The processing module is further configured to re-execute the write operation according to the write data information so as to update the data corresponding to the write data information into the storage hard disk of the storage system.

[0015] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned storage system restart processing methods when executing the computer program.

[0016] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for restarting a storage system are implemented.

[0017] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned storage system restart processing methods when executed by a processor.

[0018] Through the storage system restart processing method, electronic device, medium and product provided by this application, after the system restarts, the unfinished write operations can be determined through the write operation log and the unfinished write operations can be executed to ensure the integrity of the written data. In addition, this application does not rely on additional hardware support, so it can reduce the use cost of the storage system and improve the flexibility of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a scenario provided for an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of incomplete data update caused by a sudden power outage in an example storage system;

[0022] Figure 3 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 1 ;

[0023] Figure 4 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 2;

[0024] Figure 5 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 3 ;

[0025] Figure 6 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 4 ;

[0026] Figure 7 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 5 ;

[0027] Figure 8 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 6 ;

[0028] Figure 9 The data processing flow of the storage system is shown as an example Figure 1 ;

[0029] Figure 10 The data processing flow of the storage system is shown as an example Figure 2 ;

[0030] Figure 11 This is a flowchart of data processing after the storage system is restarted;

[0031] Figure 12 A schematic diagram of the structure of a restart processing device for a storage system provided in an embodiment of the present application;

[0032] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0033] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0035] Figure 1 A schematic diagram of a scenario provided in an embodiment of the present application, such as Figure 1 As shown, when updating data in a storage system, if the storage system suddenly loses power, if only part of the data is updated, after the storage system restarts, the data update may be incomplete or erroneous. In this case, it is necessary to continue to complete the operation of the data in the new storage system to ensure data integrity.

[0036] for example, Figure 2 The following is a diagram illustrating an example of incomplete data update caused by a sudden power outage in a storage system. If the data stored in the storage hard disk of the storage system is striped data, it is stored in the form of Redundant Array of Independent Disks (RAID 6), where D1-D5 are data blocks corresponding to business data, and P and Q are check blocks corresponding to parity data. Figure 2 As shown, if stripe data D1 is updated to D1' and D2 is updated to D2', and then the system fails, the data D3-D5, as well as the data P and Q, are not updated. This incomplete data update can lead to stripe data errors. Currently, related technologies rely on redundant hardware designs to address incomplete or erroneous data updates. For example, related technologies use battery-backed caches to ensure continuous data updates. However, this hardware is expensive, unsuitable for large-scale deployment, and lacks flexibility in handling data recovery.

[0037] The restart processing method for the storage system provided in the embodiment of the present application can determine the unfinished write operations in the updated storage system through the write operation log, and execute the unfinished write operations to ensure the integrity of the written data. It does not rely on additional hardware support, so it can reduce the use cost of the storage system and improve the flexibility of data processing.

[0038] Figure 3 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 1 ,like Figure 3 As shown, an embodiment of the present application provides a method for restarting a storage system, which is described in detail as follows:

[0039] S301: After detecting that the storage system is restarted, read the corresponding write operation log from the preset storage unit of the storage system, wherein the write operation log is generated based on the write operation sent by the user end, and the write operation log records the write data information and write status mark corresponding to the write operation.

[0040] In combination with the scenario example, the execution subject of the embodiment of the present application can be the server of the storage system. If the data in the storage system needs to be updated, the new data needs to be written into the storage system, so the user terminal sends the write operation of the updated data to the server of the storage system, and the server writes the updated data to the corresponding location of the storage system by executing the write operation. The preset storage unit is used to store the write operation log. The write data information recorded in the write operation log includes information such as the data block address, the check block address, and the operation type, and can also include the updated data that needs to be written and the write progress. For example, in combination with Figure 2 , if the data block address is the address corresponding to D1-D5, the check block address is the address corresponding to P and Q, and the operation type is if the data of D1-D5, P and Q in the storage hard disk needs to be updated, then the write operation can be to write the updated data of D1-D5, P and Q to D1-D5, P and Q of the storage hard disk. The updated data to be written can be the updated data of D1-D5, P and Q, and the write progress can be that the writing of D1 and D2 has been completed. The write status mark indicates whether the updated data has been successfully written to the storage system. If the updated data of D1-D5, P and Q have been written to D1-D5, P and Q, then the write status mark indicates that the writing of the updated data has been completed. Conversely, if the updated data of D1-D5 or P and Q have not been completed to D1-D5 or P and Q, then the write status mark indicates that the writing of the updated data has not been completed.

[0041] S302: Determine whether there is an unfinished write operation based on the write status flag in the write operation log.

[0042] In combination with the scenario example and the above content, whether the write operation is completed can be determined according to the write status mark, thereby determining the unfinished write operation.

[0043] S303: If there is an uncompleted write operation, obtain write data information corresponding to the write operation.

[0044] In combination with the scenario example, if the unfinished write operation is to write the updated data of D1-D5 and P and Q to D1-D5 and P and Q respectively, then the write data information corresponding to the write operation is obtained.

[0045] S304: re-performing the write operation according to the write data information to update the data corresponding to the write data information into the storage hard disk of the storage system.

[0046] In conjunction with the scenario example, the write progress of the write operation can be determined through the write data information corresponding to the write operation, thereby determining that the unfinished portion of the write operation is the update of D3-D5, P, and Q. The updated data corresponding to D3-D5, P, and Q can be determined from the write data information, and then the write operation can be continued to write the updated data corresponding to D3-D5, P, and Q to the corresponding D3-D5, P, and Q of the storage hard disk to complete the write operation.

[0047] Based on the method provided in this example, after the system restarts, the write operation log can be obtained directly from the preset storage unit, and the completed write operation can be determined through the write operation log, so that the unfinished write operation can be continued, thereby improving the flexibility and convenience of processing data after the system restarts.

[0048] Optional, Figure 4 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 2 ,like Figure 4 As shown, S302 includes:

[0049] S401: If the write status mark in the write operation log corresponding to any write operation is a pre-write mark, the write operation is determined to be an uncompleted write operation.

[0050] In combination with the scenario example, the specific form of the pre-write mark can be determined according to the actual situation. For example, the pre-write mark can be "pre-committed". Therefore, if the write status mark of the write operation in the write operation log is "pre-committed", the write operation is determined to be an incomplete write operation.

[0051] S402: If the write status information in the write operation log corresponding to any write operation is a write completion mark, the write operation is determined to be a completed write operation.

[0052] In conjunction with the scenario example, the specific form of the completion mark can also be determined according to the actual situation. For example, the completion mark can be "committed". Therefore, if the updated data of D1-D5 and P and Q have been written to D1-D5 and P and Q respectively, the write status of the write operation in the write operation log can be marked as "committed", and the write operation is determined to be completed.

[0053] Based on the method provided in this example, the incomplete write operations can be accurately obtained through the mark of the write operation to ensure the integrity of the data update.

[0054] Optionally, the storage system is a single-node storage system or a multi-node distributed storage system.

[0055] In this scenario, the storage system can be a single-node storage system or a multi-node distributed storage system. For a single-node storage system, RAID 5 can be used to store data. For a multi-node storage system, RAID 6 can be used to store data.

[0056] The method provided in this example is applicable to different types of storage systems. Therefore, based on the method provided in this example, the integrity of data updates in different types of storage systems can be guaranteed after a restart.

[0057] Optional, Figure 5 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 3 , the storage system is a single-node storage system; Figure 5 As shown, before S301, the method further includes:

[0058] S501: Receive a write request sent by any client.

[0059] In combination with the scenario example, when the storage is a single-node storage system, the write request sent by the user end is received by the only node in the storage system. The data stored in the single-node storage system can be stored in the form of RAID5, that is, only the parity data corresponding to P exists.

[0060] S502: Acquire write data information according to the write request.

[0061] In combination with the scenario example and the above content, the write request includes corresponding write data information, which includes information such as the data block address, the check block address, and the operation type. It can also include the updated data to be written and the write progress. When the write request is first received, the write progress is 0.

[0062] S503: Record the written data information into a write operation log of a preset storage unit, and mark the write status mark in the write operation log as a pre-write mark.

[0063] Based on the scenario example, if the data block address is the address corresponding to D1-D5, the check block address is the address corresponding to P and Q, and the operation type is that if the data of D1-D5 and P in the storage hard disk needs to be updated, the updated data to be written includes the updated data of D1-D5 and P, then the above information will be written to the operation log and the write status will be marked as "pre-committed".

[0064] S504: According to the written data information, a data update operation is performed to update the data corresponding to the written data information to the storage hard disk of the storage system.

[0065] In the scenario example, based on the operation type in the write data information, it can be determined that the write operation to be performed is to write the updated data corresponding to D1-D5 and P to the corresponding D1-D5 and P. Then, the operation of writing the updated data corresponding to D1-D5 and P to the corresponding D1-D5 and P is performed.

[0066] S505: If the data update operation is successfully executed, the write status mark in the write operation log is marked as a write completion mark.

[0067] In conjunction with the scenario example, after the updated data corresponding to D1-D5 and P are successfully written to the corresponding D1-D5 and P, the write status mark is changed from "pre-committed" to "committed".

[0068] Based on the method provided in this example, the purpose of updating data in a single-node storage system can be achieved.

[0069] Optionally, the storage system includes a business data storage hard disk and a verification data storage hard disk corresponding to the node;

[0070] Accordingly, S304 includes:

[0071] The data corresponding to the write data information is rewritten into the business data storage hard disk or the verification data storage hard disk.

[0072] Combined with the scene example, combined with Figure 2 D1-D5 are data blocks in the business data storage hard disk, and the data block corresponding to P is the parity block. The write progress is determined from the write data information to complete the update of D1 and D2. Therefore, the updated data corresponding to D3-D5 and P can be obtained from the write data information. The updated data corresponding to D3-D5 is rewritten to D3-D5 of the business data storage hard disk, and the updated data corresponding to P is rewritten to P of the parity data storage hard disk.

[0073] Based on the method provided in this example, the purpose of updating data in a single-node storage system after restart can be achieved.

[0074] Optional, Figure 6 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 4 ,like Figure 6 As shown, after rewriting the data corresponding to the write data information to the business data storage hard disk, it also includes:

[0075] S601: Determine a check hash value of data corresponding to the written data information.

[0076] Combined with the scenario example, after the updated data corresponding to D3-D5 is rewritten into D3-D5 of the business data storage hard disk, and the updated data corresponding to P is rewritten into P of the verification data storage hard disk, the hash value of each data block D1-D5 and the verification block corresponding to P is determined.

[0077] S602: Reading a historical hash value of data corresponding to the written data information from a preset storage unit of the storage system.

[0078] In combination with the scenario example, each data block D1-D5 and the historical hash value corresponding to the check block corresponding to P are read from the preset storage unit, and the historical hash value corresponding to each data block and check block and the corresponding check hash value are compared to determine whether there is an error in the update of the data block and the check block.

[0079] S603: If the verification hash value does not match the historical hash value, the data corresponding to the written data information is repaired.

[0080] In this scenario, if the checksum hash value corresponding to data block D5 does not match the historical hash value, then D5 can be determined to be a faulty data block. Data block D5 needs to be repaired. Specifically, data block D5 can be repaired using the checksum corresponding to P and data blocks D1-D4.

[0081] Based on the method provided in this example, the faulty data blocks are repaired to ensure the correctness of the data during the update process.

[0082] Optional, Figure 7 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 5 ,The storage system is a multi-node distributed storage system, which includes a master node and multiple storage nodes;

[0083] like Figure 7 As shown, before S301, the method further includes:

[0084] S701: The master node receives a write request sent by any user terminal.

[0085] In combination with the scenario example, when the storage is a multi-node distributed storage system, the master node in the storage system receives the write request sent by the user end. The data stored in the multi-node distributed storage system can be stored in the form of RAID6, that is, there are two sets of parity data corresponding to P and Q.

[0086] S702: The master node obtains write data information according to the write request, and records the write data information into a write operation log of a preset storage unit of the master node.

[0087] In conjunction with the scenario example and the aforementioned content, a write request includes corresponding write data information, including information such as the data block address, the check block address, and the operation type. It may also include the updated data to be written and the write progress. When a write request is first received, the write progress is 0. The master node records the write data information in a write operation log in a preset storage unit.

[0088] S703: The master node synchronizes the write operation log to each storage node through a preset communication protocol, so that each storage node stores the write operation log in a corresponding preset storage unit.

[0089] In combination with the scenario example, the preset communication protocol is the Redundancy Consistency Protocol (RCP), such as the Raft protocol. The Raft protocol is a distributed consistency algorithm. The write operation log is synchronized to each storage node through this protocol. Each storage node includes its own corresponding preset storage unit. Each storage node can store the obtained write operation log in the corresponding preset storage unit.

[0090] S704: The master node divides the data corresponding to the written data information into multiple data update operations according to the written data information, and sends the multiple data update operations to each storage node.

[0091] Combined with the scenario example, the master node can split the data to be written into multiple data update operations according to the corresponding data to be written in the write data information, and send the split data update operations to different storage nodes respectively, so that each storage node can perform the write operations on the data to be written in parallel.

[0092] S705: Each storage node marks the write status of the corresponding data update operation as a pre-write mark in the corresponding write operation log, and after completing the corresponding data update operation, marks the write status of the corresponding data update operation as a completed write mark in the corresponding write operation log.

[0093] Combined with the scenario example, for each storage node, after receiving the data update operation sent by the master node, the write status of the data update operation is marked as "uncommitted" in the write operation log. After completing the corresponding data update operation, the write status of the data update operation is marked as "committed" in the write operation log.

[0094] S706: Each storage node synchronously writes a mark to the master node.

[0095] In combination with the scenario example, for each storage node, after completing the corresponding data update operation, the "committed" write status mark is synchronized to the master node at the same time.

[0096] S707: After receiving the write completion mark synchronized by each storage node, the master node marks the write status of the written data information as the write completion mark in the write operation log.

[0097] In combination with the scenario example, after the master node obtains the "committed" write status mark synchronized by each storage node, it updates the write status mark of the write data information corresponding to the write request to "committed" in the write operation log.

[0098] S708: The master node synchronizes the completion mark of the written data information to each storage node.

[0099] In combination with the scenario example, the master node finally synchronizes the "committed" write status mark of the write data information corresponding to the write request to each storage node, so that each storage node is synchronously aware that the write request has been completed.

[0100] Based on the method provided in this example, the purpose of updating data in a multi-node distributed storage system can be achieved. Through the cooperation between the master node and the storage node, the integrity of the system data writing is guaranteed, and the flexibility of the system data processing can be improved.

[0101] Optionally, each storage node includes a corresponding business data storage hard disk and a verification data storage hard disk;

[0102] Accordingly, according to the written data information, re-performing the write operation to update the data corresponding to the written data information to the storage hard disk of the storage system includes:

[0103] After a storage node crashes and restarts, the operation log and missing data are synchronously written from the storage nodes that have not crashed to the corresponding business data storage hard disk or verification data storage hard disk.

[0104] In combination with the scenario example, if storage node A goes down and the data update operation that storage node A is responsible for cannot be completed by storage node A, if storage node B takes over the data update operation that storage node A is responsible for, then during the period when storage node A is down, storage node B processes the data update operation that storage node A is responsible for, and stores the progress of the data update operation that storage node A is responsible for in the write operation log corresponding to storage node B itself. After storage node A is back online, the data that storage node A missed during the downtime is the progress of the completed data update operation processed by storage node B and the updated written data, so storage node A can synchronize the progress of the data update operation completed by storage node B and the updated written data during the downtime from storage node B. If the missing data is the business data corresponding to D3-D5, the missing data will be synchronized to the business data storage hard disk. If the missing data is the parity data corresponding to P and Q, the missing data will be synchronized to the parity data storage hard disk.

[0105] Based on the method provided in this example, if any storage node goes down, it can be taken over by another node that is not down. After the storage node recovers, the missing data can be synchronized from the taken-over node to ensure that the system can still operate normally even if the node goes down.

[0106] Optional, Figure 8 Schematic diagram of the process of restarting the storage system provided in the embodiment of the present application Figure 6 ,like Figure 8 As shown, after synchronously writing the operation log and the missing data from the storage node that has not crashed to the corresponding business data storage hard disk or the verification data storage hard disk, it also includes:

[0107] S801: Determine a check hash value of data corresponding to the written data information.

[0108] Combined with the scenario example, after the updated data corresponding to D3-D5 is rewritten into D3-D5 of the business data storage hard disk, and the updated data corresponding to P and Q is rewritten into P and Q of the verification data storage hard disk, the hash value of each data block D1-D5 and the verification block corresponding to P and Q is determined.

[0109] S802: Reading a historical hash value of data corresponding to the written data information from a preset storage unit of the storage system.

[0110] In combination with the scenario example, each data block D1-D5 and the historical hash value corresponding to the check blocks corresponding to P and Q are read from the preset storage unit, and the historical hash value corresponding to each data block and check block and the corresponding check hash value are compared to determine whether there are errors in the update of the data block and the check block.

[0111] S803: If the verification hash value does not match the historical hash value, the data corresponding to the written data information is repaired.

[0112] In this scenario, if the checksum hash value corresponding to data block D5 does not match the historical hash value, then D5 can be determined to be a faulty data block. Data block D5 needs to be repaired. Specifically, data block D5 can be repaired using the checksums corresponding to P and Q and data blocks D1-D4.

[0113] Based on the method provided in this example, the faulty data blocks are repaired to ensure the correctness of the data during the update process.

[0114] Optionally, the preset storage unit is a persistent memory, which is used to store write operation logs, historical hash values and related hot data, where the related hot data is data whose access frequency exceeds a preset limit; the storage hard disk is used to store related cold data, where the cold data is data whose access frequency does not exceed a preset limit.

[0115] Based on the scenario examples, the access frequency can be determined based on actual conditions and can be a preset value. Persistent Memory (PMEM) combines the high speed of memory with storage persistence. It is used to store relevant hot data such as write operation logs, historical hash values, and metadata. Metadata manages write operation logs and historical hash values. PMEM's low latency accelerates access and improves data processing efficiency. The storage hard drive can be a solid-state drive (SSD). A hybrid design of PMEM and SSD can reduce storage system costs.

[0116] Figure 9 The data processing flow of the storage system is shown as an example Figure 1 ,like Figure 9 As shown, the client sends a write request to the storage system's master node. The master node records the write data corresponding to the write request in a write operation log and synchronizes the write operation log to all storage nodes. The persistent memory (PMEM) in the storage system is used to store hot data, while the SSD is used to store cold data. After the storage system restarts, the master node and each storage node work together to continue writing the updated data in the write request. After completing the data write, the master node verifies the fault block and check block stored in the SSD to identify the faulty block and repair it.

[0117] Figure 10 The data processing flow of the storage system is shown as an example Figure 2 ,like Figure 10As shown, when the system has a single node, the system stores the write operation log corresponding to the write request in PMEM, and records all write operations involved in the write request as "pre-committed". It then executes each "pre-committed" write operation to write the corresponding data to the SSD, and updates the write operation to "committed" after completing the write operation. When the system has multiple nodes, that is, a multi-node distributed storage system, each storage node completes the write operation marked as "pre-committed" in the write operation log, and updates the write operation mark to "committed" after completion. When the write operation mark in the write operation log corresponding to each storage node is all "committed", the processing of the write request ends.

[0118] Figure 11 This is a flowchart of data processing after the storage system is restarted. Figure 11 As shown, after the system restarts, the write operation log is read to determine whether there is a "pre-committed" write operation. If there is a "pre-committed" write operation, the "pre-committed" write operation is re-executed to write the corresponding data to the storage device, obtaining the relevant data blocks and check blocks. The data blocks and check blocks are then verified using hash values to obtain data blocks or check blocks with inconsistent hash values, i.e., faulty data blocks or check blocks. After the faulty data blocks or check blocks are successfully repaired, the faulty data blocks or check blocks are updated.

[0119] The restart processing method for the storage system provided in the embodiment of the present application can determine the unfinished write operations in the updated storage system through the write operation log, and execute the unfinished write operations to ensure the integrity of the written data. It does not rely on additional hardware support, so it can reduce the use cost of the storage system and improve the flexibility of data processing.

[0120] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0121] Figure 12 A schematic diagram of a restart processing device for a storage system provided in an embodiment of the present application is shown in FIG. Figure 12 Shown, including:

[0122] The acquisition module 121 is configured to read a corresponding write operation log from a preset storage unit of the storage system after detecting that the storage system has been restarted, wherein the write operation log is generated based on the write operation sent by the user terminal, and the write operation log records the written data information and write status flag corresponding to the write operation;

[0123] The processing module 122 is used to determine whether there is an unfinished write operation based on the write status mark in the write operation log;

[0124] The processing module 122 is further configured to obtain write data information corresponding to the write operation if there is an uncompleted write operation;

[0125] The processing module 122 is further configured to re-execute the write operation according to the write data information to update the data corresponding to the write data information into the storage hard disk of the storage system.

[0126] Optionally, the processing module 122 is specifically configured to determine that a write operation is an incomplete write operation if the write status mark in the write operation log corresponding to any write operation is a pre-write mark;

[0127] The processing module 122 is further configured to determine that a write operation is a completed write operation if the write status information in the write operation log corresponding to any write operation is a write completion mark.

[0128] Optionally, the storage system is a single-node storage system or a multi-node distributed storage system.

[0129] Optionally, the storage system is a single-node storage system;

[0130] The processing module 122 is further configured to receive a write request sent by any user terminal;

[0131] The processing module 122 is further configured to obtain write data information according to the write request;

[0132] The processing module 122 is further configured to record the written data information into a write operation log of a preset storage unit, and mark the write status in the write operation log as a pre-write mark;

[0133] The processing module 122 is further configured to execute a data update operation to update the data corresponding to the written data information to the storage hard disk of the storage system according to the written data information;

[0134] The processing module 122 is further configured to mark the write status in the write operation log as a write completion mark if the data update operation is successfully executed.

[0135] Optionally, the storage system includes a business data storage hard disk and a verification data storage hard disk corresponding to the node;

[0136] The processing module 122 is further configured to rewrite the data corresponding to the write data information into the business data storage hard disk or the verification data storage hard disk.

[0137] Optionally, the processing module 122 is further configured to determine a check hash value of data corresponding to the written data information;

[0138] The processing module 122 is further configured to read a historical hash value of data corresponding to the written data information from a preset storage unit of the storage system;

[0139] The processing module 122 is further configured to repair the data corresponding to the written data information if the verification hash value does not match the historical hash value.

[0140] Optionally, the storage system is a multi-node distributed storage system, which includes a master node and multiple storage nodes;

[0141] The processing module 122 is further configured to receive a write request sent by any user terminal;

[0142] The processing module 122 is further configured to obtain write data information according to the write request, and record the write data information in a write operation log of a preset storage unit of the master node;

[0143] The processing module 122 is further configured to synchronize the write operation log to each storage node through a preset communication protocol, so that each storage node stores the write operation log in a corresponding preset storage unit;

[0144] The processing module 122 is further configured to split the data corresponding to the written data information into multiple data update operations according to the written data information, and send the multiple data update operations to each storage node;

[0145] The processing module 122 is further configured to mark the write status of the corresponding data update operation as a pre-write mark in the corresponding write operation log, and after completing the corresponding data update operation, mark the write status of the corresponding data update operation as a complete write mark in the corresponding write operation log;

[0146] The processing module 122 is further configured to synchronously write a completion mark to the master node;

[0147] The processing module 122 is further configured to mark the write status of the written data information as a write completion mark in the write operation log after receiving the write completion mark synchronized by each storage node;

[0148] The processing module 122 is further configured to synchronize a write completion mark of the written data information to each storage node.

[0149] Optionally, each storage node includes a corresponding business data storage hard disk and a verification data storage hard disk;

[0150] The processing module 122 is further configured to synchronously write the operation log and missing data from the storage nodes that have not crashed to the corresponding business data storage hard disk or verification data storage hard disk after the storage node crashes and restarts.

[0151] Optionally, the processing module 122 is further configured to determine a check hash value of data corresponding to the written data information;

[0152] The processing module 122 is further configured to read a historical hash value of data corresponding to the written data information from a preset storage unit of the storage system;

[0153] The processing module 122 is further configured to repair the data corresponding to the written data information if the verification hash value does not match the historical hash value.

[0154] Optionally, the preset storage unit is a persistent memory, and the persistent memory is used to store write operation logs, historical hash values, and related hot data, where the related hot data is data whose access frequency exceeds a preset limit;

[0155] The storage hard disk is used to store relevant cold data, wherein the cold data is data whose access frequency does not exceed a preset limit.

[0156] For descriptions of features in the embodiment corresponding to the restart processing device of the storage system provided in this embodiment, reference can be made to the relevant descriptions of the embodiment corresponding to the restart processing method of the storage system, and will not be repeated here.

[0157] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 13 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus.

[0158] During the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that the at least one processor 501 executes the above-mentioned restart processing method embodiment of the storage system.

[0159] The specific implementation process of the processor 501 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0160] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the application may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0161] The memory may include a high-speed memory (Random Access Memory, referred to as RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0162] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0163] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned storage system restart processing method embodiments when running.

[0164] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0165] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned storage system restart processing method embodiments are implemented.

[0166] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned storage system restart processing method embodiments are implemented.

[0167] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] The above is a detailed introduction to the restart processing method, electronic device, medium and product of a storage system provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for restarting a storage system, characterized in that: include: After detecting that the storage system is restarted, reading a corresponding write operation log from a preset storage unit of the storage system, wherein the write operation log is generated based on the write operation sent by the user terminal, and the write operation log records write data information and a write status flag corresponding to the write operation; Determining whether there is an unfinished write operation based on the write status mark in the write operation log; If there is an unfinished write operation, obtaining the write data information corresponding to the write operation; According to the write data information, the write operation is re-executed to update the data corresponding to the write data information into the storage hard disk of the storage system.

2. The method according to claim 1, characterized in that The determining whether there is an unfinished write operation according to the write status mark in the write operation log includes: If the write status mark in the write operation log corresponding to any write operation is a pre-write mark, then the write operation is determined to be an incomplete write operation; If the write status information in the write operation log corresponding to any write operation is a write completion mark, it is determined that the write operation is a completed write operation.

3. The method according to claim 1, characterized in that The storage system is a single-node storage system; Before detecting that the storage system is restarted, the method further includes: Receive a write request from any client; Acquiring write data information according to the write request; Recording the written data information into a write operation log of a preset storage unit, and marking the write status mark in the write operation log as a pre-write mark; According to the written data information, executing a data update operation to update the data corresponding to the written data information to the storage hard disk of the storage system; If the data update operation is successfully executed, the write status mark in the write operation log is marked as a write completion mark.

4. The method according to claim 3, characterized in that The storage system includes a business data storage hard disk and a verification data storage hard disk corresponding to the node; Correspondingly, re-performing the write operation according to the write data information to update the data corresponding to the write data information to the storage hard disk of the storage system includes: The data corresponding to the write data information is rewritten into the business data storage hard disk or the verification data storage hard disk.

5. The method according to claim 4, characterized in that After rewriting the data corresponding to the write data information into the business data storage hard disk, the method further includes: Determine a check hash value of data corresponding to the written data information; Reading a historical hash value of the data corresponding to the written data information from a preset storage unit of the storage system; If the verification hash value and the historical hash value do not match, the data corresponding to the written data information is repaired.

6. The method according to claim 1, characterized in that The storage system is a multi-node distributed storage system, and the multi-node distributed storage system includes a master node and multiple storage nodes; Before detecting that the storage system is restarted, the method further includes: The master node receives write requests from any client. The master node obtains write data information according to the write request, and records the write data information in a write operation log of a preset storage unit of the master node; The master node synchronizes the write operation log to each storage node through a preset communication protocol, so that each storage node stores the write operation log in a corresponding preset storage unit; The master node divides the data corresponding to the written data information into multiple data update operations according to the written data information, and sends the multiple data update operations to each storage node; Each storage node marks the write status of the corresponding data update operation as a pre-write mark in the corresponding write operation log, and after completing the corresponding data update operation, marks the write status of the corresponding data update operation as a complete write mark in the corresponding write operation log; Each storage node completes the write mark synchronously to the master node; The master node, after receiving the write completion mark synchronized by each storage node, marks the write status of the write data information as the write completion mark in the write operation log; The master node synchronizes the write completion mark of the written data information to each storage node.

7. The method according to claim 6, characterized in that Each storage node includes a corresponding hard disk for storing business data and a hard disk for storing verification data; Correspondingly, re-performing the write operation according to the write data information to update the data corresponding to the write data information to the storage hard disk of the storage system includes: After a storage node crashes and restarts, the operation log and missing data are synchronously written from the storage nodes that have not crashed to the corresponding business data storage hard disk or verification data storage hard disk.

8. The method according to claim 7, characterized in that After synchronously writing the operation logs and missing data from the storage nodes that have not crashed to the corresponding business data storage hard disk or verification data storage hard disk, the following steps are also included: Determine the check hash value of the data corresponding to the written data information; Reading a historical hash value of data corresponding to the written data information from a preset storage unit of the storage system; If the verification hash value does not match the historical hash value, the data corresponding to the written data information is repaired.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the storage system restart processing method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the storage system restart processing method according to any one of claims 1 to 8 are implemented.

Images