Data reconstruction method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a data reconstruction method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a data reconstruction scene is determined according to a data reconstruction command; wherein the data reconstruction scene comprises an online disk changing scene and an object storage service management scene; when the data reconstruction scene is the online disk replacement scene, determining a fault hard disk and acquiring identification information and key information of a fault object storage service corresponding to the fault hard disk; when the fault hard disk is replaced by a new hard disk, creating a new object storage service corresponding to the new hard disk by using the identification information and the key information; and starting the new object storage service to complete data reconstruction. By applying the technical scheme provided by the invention, redundant data reconstruction can be avoided in the disk changing process, so that successful execution of data reconstruction can be effectively ensured.

Description

Technical Field

[0001] The present application relates to the technical field of cluster services, and in particular to a data reconstruction method, and also to a data reconstruction device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In a distributed cluster system, data reconstruction will be triggered after a hard disk failure, and the data of the failed hard disk will be reconstructed onto the hard disks of other OSDs (Object-based Storage Devices). In order to prevent the OSD hard disk from being filled up after data reconstruction, the data reconstruction process will be controlled by a threshold. If the threshold is exceeded, data reconstruction will be stopped. Therefore, when a hard disk fails when the water level is high, data reconstruction may not be completed. To avoid this situation, in the related technology, data reconstruction is prohibited when the cluster is in a high water level state, and data reconstruction can only be started after the failed hard disk is replaced.

[0003] The current disk replacement solution is to first delete the OSD of the failed hard disk from the cluster topology (CrushMap), the OSD table (OSDMap), and the authorization table (AuthMap) in sequence; then recreate the OSD and add it to the OSD table, cluster topology, and authorization table in sequence. Obviously, such steps are not only cumbersome and redundant, but most importantly, they will also cause the PG (Placement Group) members of the OSD that is not faulty to change. This reconstruction is completely unnecessary and redundant. Without this part of the reconstruction, the data water level will not exceed the threshold during the reconstruction process, thereby causing the data reconstruction failure.

[0004] Therefore, how to avoid redundant data reconstruction during the disk replacement process to effectively ensure the successful execution of data reconstruction is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] The purpose of the present application is to provide a data reconstruction method, which can avoid unnecessary data reconstruction during the disk replacement process to effectively ensure the successful execution of data reconstruction; another purpose of the present application is to provide a data reconstruction device, electronic device, computer-readable storage medium and computer program product, all of which have the above-mentioned beneficial effects.

[0006] In a first aspect, the present application provides a data reconstruction method, comprising:

[0007] Determining a data reconstruction scenario according to the data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario;

[0008] When the data reconstruction scenario is the online disk replacement scenario, determining the failed hard disk and obtaining identification information and key information of the failed object storage service corresponding to the failed hard disk;

[0009] When the failed hard disk is replaced with a new hard disk, a new object storage service corresponding to the new hard disk is created using the identification information and the key information;

[0010] Start the new object storage service to complete data reconstruction.

[0011] Optionally, when the failed hard disk is replaced with a new hard disk, creating a new object storage service corresponding to the new hard disk by using the identification information and the key information includes:

[0012] When the failed hard disk is replaced with the new hard disk, determining the storage status of the cluster system to which the failed hard disk belongs; wherein the storage status includes a high water level status and a non-high water level status;

[0013] When the storage state is the high water level state, entering a waiting state until the storage state is converted to the non-high water level state;

[0014] When the storage state is the non-high water level state, the new object storage service corresponding to the new hard disk is created using the identification information and the key information.

[0015] Optionally, obtaining identification information of a faulty object storage service corresponding to the faulty hard disk includes:

[0016] Obtaining a hard disk identification of the faulty hard disk;

[0017] A first query instruction carrying the hard disk identifier is sent to the object storage service table of the cluster system to which the faulty hard disk belongs, so as to obtain identification information of the faulty object storage service corresponding to the faulty hard disk.

[0018] Optionally, obtaining key information of a faulty object storage service corresponding to the faulty hard disk includes:

[0019] Obtaining a hard disk identification of the faulty hard disk;

[0020] A second query instruction carrying the hard disk identifier is sent to the authorization table of the cluster system to which the failed hard disk belongs, so as to obtain key information of the failed object storage service corresponding to the failed hard disk.

[0021] Optionally, the data reconstruction method further includes:

[0022] When the faulty hard disk is replaced with the new hard disk, obtaining the first identification information stored in the data partition of the faulty hard disk and saving it to a preset space;

[0023] Accordingly, the new object storage service is started to complete data reconstruction, including:

[0024] Starting the new object storage service and obtaining the second identification information stored in the data partition of the new hard disk;

[0025] When the second identification information is consistent with the first identification information, data reconstruction is completed using the new object storage service;

[0026] When the second identification information is inconsistent with the first identification information, exit the new object storage service.

[0027] Optionally, when the failed hard disk is replaced with a new hard disk, creating a new object storage service corresponding to the new hard disk by using the identification information and the key information includes:

[0028] When the failed hard disk is replaced with the new hard disk, the file system and database file corresponding to the new hard disk are created using the identification information and the key information to create the new object storage service corresponding to the new hard disk;

[0029] Accordingly, the new object storage service is started to complete data reconstruction, including:

[0030] Determine whether the new object storage service has authentication authority;

[0031] If the new object storage service does not have the authentication authority, adding the authentication authority to the new object storage service;

[0032] If the new object storage service has the authentication authority, initializing the new object storage service;

[0033] Mounting the database file to the file system and opening the database file to start the new object storage service;

[0034] The new object storage service is used to reconstruct data.

[0035] Optionally, determining a data reconstruction scenario according to the data reconstruction command includes:

[0036] Parsing the data reconstruction command to determine whether the data reconstruction command contains an online disk replacement identifier;

[0037] When the data reconstruction command does not include the online disk replacement identifier, determining that the data reconstruction scenario is the object storage service management scenario;

[0038] When the data reconstruction command includes the online disk replacement identifier, it is determined that the data reconstruction scenario is the online disk replacement scenario.

[0039] In a second aspect, the present application further discloses a data reconstruction device, comprising:

[0040] A determination module, used to determine a data reconstruction scenario according to a data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario;

[0041] An acquisition module, used for determining a faulty hard disk and acquiring identification information and key information of a faulty object storage service corresponding to the faulty hard disk when the data reconstruction scenario is the online disk replacement scenario;

[0042] A creation module, used for creating a new object storage service corresponding to the new hard disk by using the identification information and the key information when the failed hard disk is replaced with a new hard disk;

[0043] The startup module is used to start the new object storage service to complete data reconstruction.

[0044] In a third aspect, the present application further discloses an electronic device, comprising:

[0045] Memory for storing computer programs;

[0046] A processor is used to implement the steps of any one of the data reconstruction methods described above when executing the computer program.

[0047] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the data reconstruction methods described above are implemented.

[0048] In a fifth aspect, the present invention further discloses a computer program product, including a computer program / instruction, which implements the steps of any one of the data reconstruction methods described above when executed by a processor.

[0049] The present application provides a data reconstruction method, including: determining a data reconstruction scenario according to a data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario; when the data reconstruction scenario is the online disk replacement scenario, determining a faulty hard disk and obtaining identification information and key information of a faulty object storage service corresponding to the faulty hard disk; when the faulty hard disk is replaced with a new hard disk, creating a new object storage service corresponding to the new hard disk using the identification information and the key information; and starting the new object storage service to complete data reconstruction.

[0050] Applying the technical solution provided by the present application, both online disk replacement scenarios and object storage service management scenarios (such as object storage service deployment and expansion scenarios, etc.) will trigger data reconstruction. However, the latter will not trigger unnecessary data reconstruction. Therefore, when triggering data reconstruction, the data reconstruction scenario judgment can be performed first; further, in the online disk replacement scenario, the identification information and key information of the faulty object storage service corresponding to the faulty hard disk (i.e., the hard disk that needs to be replaced) can be obtained respectively during the disk replacement process, and after the disk replacement is completed, the identification information and key information of the faulty object storage service are directly used to create a new object storage service corresponding to the new hard disk, ensuring that the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, and completing data reconstruction based on the new object storage service. Obviously, since the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, it is possible to effectively avoid changes in the placement group members of the non-faulty object storage service, thereby avoiding unnecessary data reconstruction, causing the cluster high water level state, and then causing the problem of data reconstruction failure. It can be seen from this that the technical solution can avoid redundant data reconstruction during the disk replacement process, and thus effectively ensure the successful execution of data reconstruction.

[0051] In one embodiment of the present application, during the disk replacement process, the first identification information stored in the data partition of the faulty hard disk can also be synchronously obtained and saved. After the subsequent new object storage service is created and started, the second identification information stored in the data partition of the new hard disk can be obtained and compared with the first identification information. If the two are consistent, the new object storage service can continue to operate normally. Otherwise, the new object storage service is directly exited, thereby realizing the verification function of the object storage service, which can effectively avoid the problem of inconsistent data after disk replacement due to disk replacement errors, and further enhance the data security during the disk replacement process.

[0052] The data reconstruction device, electronic device, computer-readable storage medium and computer program product provided in this application also have the above-mentioned technical effects, which will not be described in detail in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0054] Figure 1A flowchart of a data reconstruction method provided in this application;

[0055] Figure 2 A flowchart of an online disk replacement method provided by this application;

[0056] Figure 3 A flowchart of a method for creating an object storage service provided by this application;

[0057] Figure 4 A flowchart of a method for starting an object storage service provided in this application;

[0058] Figure 5 A schematic diagram of the structure of a data reconstruction device provided by the present application;

[0059] Figure 6 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0060] The core of the present application is to provide a data reconstruction method, which can avoid unnecessary data reconstruction during the disk replacement process to effectively ensure the successful execution of data reconstruction; another core of the present application is to provide a data reconstruction device, electronic device, computer-readable storage medium and computer program product, all of which have the above-mentioned beneficial effects.

[0061] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0062] An embodiment of the present application provides a data reconstruction method.

[0063] Please refer to Figure 1 , Figure 1 This is a flow chart of a data reconstruction method provided in the present application. The data reconstruction method may include the following S101 to S104.

[0064] S101: Determine a data reconstruction scenario according to a data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario.

[0065] This step is intended to determine the data reconstruction scenario. It is understandable that both the online disk replacement scenario and the object storage service management scenario (such as the object storage service deployment scenario and the capacity expansion scenario, etc.) will trigger data reconstruction, however, the latter will not trigger unnecessary data reconstruction, therefore, when triggering data reconstruction, the data reconstruction scenario can be determined first, and the data reconstruction method provided in the embodiment of the present application is applied to the online disk replacement scenario.

[0066] In one embodiment of the present application, determining a data reconstruction scenario according to a data reconstruction command may include:

[0067] Parsing the data reconstruction command to determine whether the data reconstruction command contains an online disk replacement identifier;

[0068] When the data reconstruction command does not include the online disk replacement identifier, it is determined that the data reconstruction scenario is the object storage service management scenario;

[0069] When the data reconstruction command includes an online disk replacement identifier, it is determined that the data reconstruction scenario is an online disk replacement scenario.

[0070] The embodiment of the present application provides an implementation method for determining the data reconstruction scenario according to the data reconstruction command, that is, it can be implemented based on the online disk replacement identifier. Specifically, a function entry is provided in the corresponding management software, and there are deployment pages and expansion pages on the web page. The automatic disk replacement function relies on inserting a new disk and calling the disk replacement script through the system's UDEV (a mechanism for managing devices in user mode) event to enter the disk replacement process. Based on this, an online disk replacement identifier can be added to the disk replacement module. The existence of this identifier indicates an online disk replacement scenario, otherwise it is an object storage service management scenario. In one possible implementation, the online disk replacement identifier is specifically a parameter disk_replace. When the parameter disk_replace is added to the data reconstruction command, it is determined that the current data reconstruction scenario is an online disk replacement scenario, otherwise it is an object storage service management scenario.

[0071] S102: When the data reconstruction scenario is an online disk replacement scenario, determine the failed hard disk and obtain identification information and key information of the failed object storage service corresponding to the failed hard disk.

[0072] This step aims to identify the faulty hard disk and obtain relevant information. Specifically, when it is determined that the current data reconstruction scenario is an online disk replacement scenario, it means that a disk replacement operation is currently in progress, replacing the faulty hard disk with a new normal hard disk. On this basis, the faulty hard disk can be first identified, and the identification information (UUID) and key information (keyring) of the faulty object storage service corresponding to the faulty hard disk can be obtained, so as to create a new hard disk OSD based on the relevant information of the faulty disk OSD, so as to effectively ensure the consistency of the identification information and key information of the new and old hard disk OSDs.

[0073] In one embodiment of the present application, obtaining identification information of a faulty object storage service corresponding to a faulty hard disk may include:

[0074] Get the hard disk ID of the faulty hard disk;

[0075] A first query instruction carrying a hard disk identifier is sent to the object storage service table of the cluster system to which the faulty hard disk belongs, so as to obtain identification information of the faulty object storage service corresponding to the faulty hard disk.

[0076] The embodiment of the present application provides a method for obtaining the identification information of a faulty disk OSD, that is, the hard disk identification of the faulty hard disk can be used to query and obtain the information from the object storage service table (OSDMap) of the cluster system to which the faulty hard disk belongs. The hard disk identification can specifically be a hard disk ID.

[0077] In one embodiment of the present application, key information of a failed object storage service corresponding to a failed hard disk may be obtained, including:

[0078] Get the hard disk ID of the faulty hard disk;

[0079] A second query instruction carrying the hard disk identifier is sent to the authorization table of the cluster system to which the failed hard disk belongs, so as to obtain key information of the failed object storage service corresponding to the failed hard disk.

[0080] The embodiment of the present application provides a method for obtaining the key information of the OSD of a faulty disk, that is, the key information can be obtained by querying the authorization table (AuthMap) of the cluster system to which the faulty disk belongs using the hard disk identification of the faulty hard disk. Similarly, the hard disk identification can specifically be the hard disk ID.

[0081] S103: When the failed hard disk is replaced with a new hard disk, a new object storage service corresponding to the new hard disk is created using the identification information and the key information.

[0082] This step is intended to achieve the creation of a new object storage service after the hard disk is replaced. Specifically, after replacing the faulty hard disk with a new hard disk, the identification information and key information of the faulty object storage service corresponding to the faulty hard disk can be used to create a new object storage service corresponding to the new hard disk, so as to effectively ensure the consistency between the faulty object storage service corresponding to the faulty hard disk and the new object storage service corresponding to the new hard disk. It can be understood that since the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, it is possible to effectively avoid changes in the placement group members of the non-faulty object storage service, thereby avoiding unnecessary data reconstruction that causes a high water level state in the cluster and then causes data reconstruction failure.

[0083] In one embodiment of the present application, when a failed hard disk is replaced with a new hard disk, creating a new object storage service corresponding to the new hard disk using identification information and key information may include:

[0084] When the failed hard disk is replaced with a new hard disk, determining the storage state of the cluster system to which the failed hard disk belongs; wherein the storage state includes a high water level state and a non-high water level state;

[0085] When the storage state is in a high water level state, it enters a waiting state until the storage state is converted to a non-high water level state;

[0086] When the storage state is not in the high water mark state, a new object storage service corresponding to the new hard disk is created using the identification information and the key information.

[0087] In order to effectively ensure the stable execution of data reconstruction operations, the storage status of the current cluster system can also be monitored synchronously during the disk replacement process to avoid executing the creation operation of the new object storage service in the high water level state. The high water level state means that the amount of data stored in the current cluster system has reached or exceeded a certain threshold, and the non-high water level state means that the amount of data stored in the current cluster system has not reached the threshold.

[0088] Based on this, in a possible implementation, when a faulty hard disk is replaced with a new hard disk, determining the storage state of the cluster system to which the faulty hard disk belongs may include: when the faulty hard disk is replaced with a new hard disk, obtaining the data storage capacity of the cluster system to which the faulty hard disk belongs; when the data storage capacity reaches a preset threshold, determining that the storage state of the cluster system to which the faulty hard disk belongs is a high water level state; when the data storage capacity does not reach the preset threshold, determining that the storage state of the cluster system to which the faulty hard disk belongs is a non-high water level state. Of course, the specific value of the preset threshold does not affect the implementation of this technical solution, and can be set by technical personnel according to actual conditions, and this application does not limit this.

[0089] S104: Start a new object storage service to complete data reconstruction.

[0090] This step aims to achieve data reconstruction based on the new object storage service. That is, after the new object storage service is created, the new object storage service can be used to achieve data reconstruction of the current cluster system.

[0091] In one embodiment of the present application, when a failed hard disk is replaced with a new hard disk, creating a new object storage service corresponding to the new hard disk using identification information and key information may include:

[0092] When the failed hard disk is replaced with a new hard disk, the identification information and key information are used to create a file system and database file corresponding to the new hard disk to create a new object storage service corresponding to the new hard disk;

[0093] Accordingly, starting a new object storage service to complete data reconstruction may include:

[0094] Determine whether the new object storage service has authentication permissions;

[0095] If the new object storage service does not have authentication permissions, add authentication permissions for the new object storage service;

[0096] If the new object storage service has authentication permissions, the new object storage service is initialized;

[0097] Mount the database file to the file system and open the database file to start the new object storage service;

[0098] Use the new object storage service to reconstruct data.

[0099] The embodiments of the present application provide a method for creating and starting an object storage service. Specifically, during the creation of a new object storage service, a file system and a database file (DB file) corresponding to a new hard disk can be created based on the identification information and key information of the failed object storage service; during the startup of the new object storage service, the new object storage service can be initialized first, and then permission setting, file mounting, and file opening operations can be performed in sequence to achieve the normal operation of the new object storage service. Among them, the authentication authority set by the new object storage service can specifically be an auth authority.

[0100] It can be seen that the data reconstruction method provided by the embodiment of the present application, both the online disk replacement scenario and the object storage service management scenario (such as the object storage service deployment and expansion scenario, etc.) will trigger data reconstruction, however, the latter will not trigger unnecessary data reconstruction, therefore, when triggering data reconstruction, the data reconstruction scenario judgment can be performed first; further, in the online disk replacement scenario, the identification information and key information of the faulty object storage service corresponding to the faulty hard disk (i.e., the hard disk that needs to be replaced) can be obtained respectively during the disk replacement process, and after the disk replacement is completed, the identification information and key information of the faulty object storage service are directly used to create a new object storage service corresponding to the new hard disk, ensuring that the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, and completing data reconstruction based on the new object storage service. Obviously, since the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, it is possible to effectively avoid changes in the placement group members of the non-faulty object storage service, thereby avoiding the problem of unnecessary data reconstruction causing the high water level state of the cluster and then causing data reconstruction failure. It can be seen from this that the technical solution can avoid redundant data reconstruction during the disk replacement process, and thus effectively ensure the successful execution of data reconstruction.

[0101] Based on the above embodiments:

[0102] In one embodiment of the present application, the data reconstruction method may further include:

[0103] When the failed hard disk is replaced with a new hard disk, the first identification information stored in the data partition of the failed hard disk is obtained and saved to a preset space;

[0104] Accordingly, starting a new object storage service to complete data reconstruction may include:

[0105] Start the new object storage service and obtain the second identification information stored in the data partition of the new hard disk;

[0106] When the second identification information is consistent with the first identification information, the data reconstruction is completed using the new object storage service;

[0107] When the second identification information is inconsistent with the first identification information, exit the new object storage service.

[0108] The data reconstruction method provided in the embodiment of the present application can further realize the verification function of the object storage service. Specifically, during the disk replacement process, the first identification information stored in the data partition of the faulty hard disk can be obtained at the same time and saved to the preset space; after completing the creation and startup of the new object storage service, the second identification information stored in the data partition of the new hard disk can be obtained, and the second identification information can be compared and analyzed with the first identification information to determine whether the two are consistent. If the two are consistent, the new object storage service can continue to operate normally to realize data reconstruction, otherwise the new object storage service can be directly exited and the corresponding alarm prompt can be output.

[0109] It can be seen that in the embodiment of the present application, during the disk replacement process, the first identification information stored in the data partition of the faulty hard disk can also be synchronously obtained and saved. After the subsequent new object storage service is created and started, the second identification information stored in the data partition of the new hard disk can be obtained, and it can be compared and analyzed with the first identification information. If the two are consistent, the new object storage service can continue to operate normally. Otherwise, the new object storage service is directly exited, thereby realizing the verification function of the object storage service, which can effectively avoid the problem of inconsistent data after disk replacement due to disk replacement errors, and further enhance the data security during the disk replacement process.

[0110] Based on the above embodiments, an embodiment of the present application provides another data reconstruction method.

[0111] First, please refer to Figure 2 , Figure 2This is a flowchart of an online disk replacement method provided by this application. It is understandable that cluster deployment, expansion and disk replacement will create OSDs. Without affecting the deployment and expansion processes, only the disk replacement process is modified (the modified content is Figure 2 The process content marked with shadow in the figure) can recreate the OSD without deleting the failed disk OSD. Accordingly, the implementation process of online disk replacement is as follows:

[0112] (1) A new parameter disk_replace is added to the disk replacement module. When this parameter is added, it indicates a hard disk replacement scenario. When this parameter is not added, it indicates an OSD deployment or expansion scenario.

[0113] (2) Create a control partition (100M), which is used to store the identification information of the newly created OSD.

[0114] (3) Create a data partition, which is used to store actual data.

[0115] Taking HDD (Hard Disk Drive) as an example, an OSD manages an HDD. The control partition is the first partition of the HDD. This partition will be formatted into an xfs file system and mounted with a directory. The OSD's UUID, whoami and other identification information will be saved in the directory. The data partition is the second partition of the HDD. This partition will not be formatted into a file system, but will store data as a raw partition.

[0116] (4) Determine whether there is a disk_replace parameter. If there is no disk_replace parameter, continue the original process to create related files.

[0117] (5) If the disk_replace parameter is used, you need to first query the UUID of the OSD of the failed disk and record it.

[0118] When OSD is started, MON (Monitor, the cluster's monitor daemon) will check whether the OSD's UUID is consistent with the one in OSDMap, so it is necessary to use the UUID in OSDMap to create OSD, otherwise the OSD cannot be started after creation. The original process is to delete the faulty disk OSD from OSDMap and then create a new one. When creating, since the UUID in OSDMap is newly applied, there is no mismatch problem.

[0119] (6) Create related files, such as fsid, magic, etc.

[0120] (7) Determine whether there is a disk_replace parameter. If there is no disk_replace parameter, continue the original process to create the relevant OSD.

[0121] (8) If the disk_replace parameter is used, you need to first query the keyring information of the failed disk OSD and generate a new keyring file.

[0122] When OSD starts, MON will check whether the keyring information of OSD is consistent with that in AuthMap. If they are inconsistent, the state of OSD cannot be set to up. Therefore, the keyring information saved by MON can be saved to the hard disk of OSD. The original process is to delete the faulty disk OSD from AuthMap and then create a new one. When creating, since the keyring information in AuthMap is newly applied, there is no mismatch problem.

[0123] (9) Use OSD related commands to create an OSD. The UUID and keyring information obtained in (5) and (8) need to be passed as parameters to the OSD creation command. In this way, the newly created OSD will save the UUID and send it to the MON as a startup parameter when it is started.

[0124] Furthermore, in the automatic disk replacement scheme in the related art, the disk replacement process is controlled by the management software. The faulty disk is unplugged from the tray or hard disk slot and a new disk is inserted. When the new disk is inserted, the disk replacement process automatically begins. The management software will perform a series of automated processes and eventually create the new disk as an OSD. However, in order to further improve performance, the cluster system sometimes creates the OSD's DB file on NVME (non-volatile memory storage), and the data device is still deployed on the HDD. In other words, the OSD will manage both the DB files deployed on NVME and the data deployed on the HDD. With this deployment method, when either the NVME disk or the HDD disk fails, only the corresponding faulty disk needs to be replaced. However, if it is discovered that the wrong disk was replaced during the disk replacement process, unplugging the new disk and inserting the old disk will cause data inconsistency. Based on this, for Figure 2 The OSD creation and startup process shown in the figure has been further modified in this application, that is, a new OSD verification function has been added to effectively avoid the problem of data inconsistency after disk replacement due to misoperation.

[0125] 1. Please refer to Figure 3 , Figure 3 A flowchart of a method for creating an object storage service provided by this application ( Figure 3 The process contents marked with shadows are modified contents), and the implementation process is as follows:

[0126] (1) Create a file system.

[0127] (2) Create a DB file.

[0128] (3) Open the bluefs file (small file system).

[0129] (4) Get block_uuid.

[0130] Here, the UUID (first identification information) of the data partition in the HDD is obtained and recorded in the label, with the key being block_uuid and the value being UUID. It can be understood as saving the UUID of the data partition in the HDD into the variable block_uuid in the label.

[0131] (5) Write block_uuid to the label of the db partition.

[0132] 2. Please refer to Figure 4 , Figure 4 A flowchart of a method for starting an object storage service provided in this application ( Figure 4 The process contents marked with shadows are modified contents), and the implementation process is as follows:

[0133] (1) OSD initialization.

[0134] (2) Mount the DB file to the file system.

[0135] (3) Open the DB file.

[0136] (4) Open the bluefs file.

[0137] (5) Obtain the UUID (second identification information) in the data partition on the HDD of the OSD.

[0138] (6) Compare the UUID obtained in step (5) with block_uuid.

[0139] (7) If the comparison is consistent, 0 is returned; if it is inconsistent, EIO (error message) is returned.

[0140] (8) If step (7) returns EIO, the OSD process exits.

[0141] Obviously, this technical solution can prevent the problem of data inconsistency that occurs when inserting an old disk during or after a disk change.

[0142] 1. After replacing the HDD, insert the old HDD.

[0143] Based on the technical solution provided by this application, when the HDD disk is replaced, the UUID of the data partition in the new HDD disk will be recorded in the label of the DB partition in the NVME disk. After the old HDD disk is reinserted, the UUID of the data partition in the old HDD disk is the UUID of the old disk. Then, after the OSD is started, the block_uuid read from the label of the DB partition in the NVME is the UUID of the new disk, which does not match the UUID of the old disk, so the OSD process will exit.

[0144] 2. After replacing the NVME disk, insert the old NVME disk.

[0145] When replacing the NVME disk, the data on the old HDD disk will also be redone, and the UUID of the data partition in the old HDD disk will also be refreshed. Based on the technical solution provided by this application, when replacing the NVME disk, the UUID of the data partition of the old HDD disk will be recorded in the label of the DB partition on the new NVME disk. After reinserting the previous old NVME disk, the block_uuid of the HDD saved in the label of the DB partition in the old NVME disk does not match the UUID of the old HDD after refreshing, so the OSD process will exit.

[0146] It can be seen that the data reconstruction method provided by the embodiment of the present application, both the online disk replacement scenario and the object storage service management scenario (such as the object storage service deployment and expansion scenario, etc.) will trigger data reconstruction, however, the latter will not trigger unnecessary data reconstruction, therefore, when triggering data reconstruction, the data reconstruction scenario judgment can be performed first; further, in the online disk replacement scenario, the identification information and key information of the faulty object storage service corresponding to the faulty hard disk (i.e., the hard disk that needs to be replaced) can be obtained respectively during the disk replacement process, and after the disk replacement is completed, the identification information and key information of the faulty object storage service are directly used to create a new object storage service corresponding to the new hard disk, ensuring that the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, and completing data reconstruction based on the new object storage service. Obviously, since the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, it is possible to effectively avoid changes in the placement group members of the non-faulty object storage service, thereby avoiding the problem of unnecessary data reconstruction causing the high water level state of the cluster and then causing data reconstruction failure. It can be seen from this that the technical solution can avoid redundant data reconstruction during the disk replacement process, and thus effectively ensure the successful execution of data reconstruction.

[0147] An embodiment of the present application provides a data reconstruction device.

[0148] Please refer to Figure 5 , Figure 5This is a structural schematic diagram of a data reconstruction device provided by the present application, and the data reconstruction device may include:

[0149] Determining module 1, used to determine the data reconstruction scenario according to the data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario;

[0150] Acquisition module 2, used for determining the faulty hard disk and acquiring identification information and key information of the faulty object storage service corresponding to the faulty hard disk when the data reconstruction scenario is an online disk replacement scenario;

[0151] Creation module 3, used to create a new object storage service corresponding to the new hard disk using identification information and key information when the failed hard disk is replaced with a new hard disk;

[0152] The startup module 4 is used to start the new object storage service to complete data reconstruction.

[0153] It can be seen that the data reconstruction device provided by the embodiment of the present application, both the online disk replacement scenario and the object storage service management scenario (such as the object storage service deployment and expansion scenario, etc.) will trigger data reconstruction, however, the latter will not trigger unnecessary data reconstruction, therefore, when triggering data reconstruction, the data reconstruction scenario judgment can be performed first; further, in the online disk replacement scenario, the identification information and key information of the faulty object storage service corresponding to the faulty hard disk (i.e., the hard disk to be replaced) can be obtained respectively during the disk replacement process, and after the disk replacement is completed, the identification information and key information of the faulty object storage service are directly used to create a new object storage service corresponding to the new hard disk, ensuring that the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, and completing data reconstruction based on the new object storage service. Obviously, since the identification information and key information of the faulty storage service are consistent with the identification information and key information of the new object storage service, it is possible to effectively avoid changes in the placement group members of the non-faulty object storage service, thereby avoiding the problem of unnecessary data reconstruction causing the high water level state of the cluster and then causing data reconstruction failure. It can be seen from this that the technical solution can avoid redundant data reconstruction during the disk replacement process, and thus effectively ensure the successful execution of data reconstruction.

[0154] In one embodiment of the present application, the above-mentioned creation module 3 can be specifically used to determine the storage status of the cluster system to which the faulty hard disk belongs when a faulty hard disk is replaced with a new hard disk; wherein the storage status includes a high water level state and a non-high water level state; when the storage status is a high water level state, a waiting state is entered until the storage status is converted to a non-high water level state; when the storage status is a non-high water level state, a new object storage service corresponding to the new hard disk is created using identification information and key information.

[0155] In one embodiment of the present application, the above-mentioned acquisition module 2 can be specifically used to obtain the hard disk identification of the faulty hard disk; send a first query instruction carrying the hard disk identification to the object storage service table of the cluster system to which the faulty hard disk belongs, so as to obtain the identification information of the faulty object storage service corresponding to the faulty hard disk.

[0156] In one embodiment of the present application, the above-mentioned acquisition module 2 can be specifically used to obtain the hard disk identification of the faulty hard disk; send a second query instruction carrying the hard disk identification to the authorization table of the cluster system to which the faulty hard disk belongs, so as to obtain the key information of the faulty object storage service corresponding to the faulty hard disk.

[0157] In one embodiment of the present application, the data reconstruction device may further include a saving module, which is used to obtain the first identification information stored in the data partition of the failed hard disk and save it to a preset space when the failed hard disk is replaced with a new hard disk;

[0158] Correspondingly, the above-mentioned startup module 4 can be specifically used to start a new object storage service and obtain the second identification information stored in the data partition of the new hard disk; when the second identification information is consistent with the first identification information, the new object storage service is used to complete data reconstruction; when the second identification information is inconsistent with the first identification information, the new object storage service is exited.

[0159] In one embodiment of the present application, the creation module 3 may be specifically used to create a file system and database file corresponding to the new hard disk using identification information and key information when a failed hard disk is replaced with a new hard disk, so as to create a new object storage service corresponding to the new hard disk;

[0160] Accordingly, the startup module 4 can be specifically used to determine whether the new object storage service has authentication authority; if the new object storage service does not have authentication authority, then add authentication authority for the new object storage service; if the new object storage service has authentication authority, then initialize the new object storage service; mount the database file to the file system, and open the database file to start the new object storage service;

[0161] Use the new object storage service to reconstruct data.

[0162] In one embodiment of the present application, the above-mentioned determination module 1 can be specifically used to parse the data reconstruction command to determine whether the data reconstruction command contains an online disk swap identifier; when the data reconstruction command does not contain an online disk swap identifier, the data reconstruction scenario is determined to be an object storage service management scenario; when the data reconstruction command contains an online disk swap identifier, the data reconstruction scenario is determined to be an online disk swap scenario.

[0163] For an introduction to the device provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0164] An embodiment of the present application provides an electronic device.

[0165] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application, and the electronic device may include:

[0166] A memory 11, used for storing computer programs;

[0167] The processor 10 can implement the steps of any one of the above-mentioned data reconstruction methods when being used to execute a computer program.

[0168] like Figure 6 , which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0169] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0170] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in the embodiment of the data reconstruction method.

[0171] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0172] Determine a data reconstruction scenario according to the data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario;

[0173] When the data reconstruction scenario is an online disk replacement scenario, determine the failed hard disk and obtain the identification information and key information of the failed object storage service corresponding to the failed hard disk;

[0174] When the failed hard disk is replaced with a new one, a new object storage service corresponding to the new hard disk is created using the identification information and key information;

[0175] Start a new object storage service to complete data reconstruction.

[0176] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0177] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0178] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0179] Of course, it should be noted that Figure 6 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0180] An embodiment of the present application provides a computer-readable storage medium.

[0181] The computer-readable storage medium provided in the embodiment of the present application stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned data reconstruction methods can be implemented.

[0182] Among them, the computer-readable storage medium can be any available medium that can be stored in a computer or a data storage device such as a server or data center that includes one or more available media integrated. For example, it can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape, etc.), an optical medium (such as a DVD) or a semiconductor medium (such as a solid-state hard disk) and other media that can store computer program codes.

[0183] For an introduction to the computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0184] An embodiment of the present application provides a computer program product.

[0185] The computer program product provided in the embodiments of the present application includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of any of the above-mentioned data reconstruction methods can be implemented.

[0186] Specifically, in the above embodiments, all or part of them can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of them can be implemented in the form of a computer program product.

[0187] Among them, the computer program product may include one or more computer programs / instructions, and when the computer program / instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application may be generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0188] For an introduction to the computer program product provided in the embodiments of the present application, please refer to the above method embodiments, and the present application will not elaborate on them here.

[0189] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0190] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0191] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0192] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A data reconstruction method, characterized in that: include: Determining a data reconstruction scenario according to the data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario; When the data reconstruction scenario is the online disk replacement scenario, determining the failed hard disk and obtaining identification information and key information of the failed object storage service corresponding to the failed hard disk; When the failed hard disk is replaced with a new hard disk, a new object storage service corresponding to the new hard disk is created using the identification information and the key information; Start the new object storage service to complete data reconstruction.

2. The data reconstruction method according to claim 1, characterized in that: When the failed hard disk is replaced with a new hard disk, a new object storage service corresponding to the new hard disk is created using the identification information and the key information, including: When the failed hard disk is replaced with the new hard disk, determining the storage status of the cluster system to which the failed hard disk belongs; wherein the storage status includes a high water level status and a non-high water level status; When the storage state is the high water level state, entering a waiting state until the storage state is converted to the non-high water level state; When the storage state is the non-high water level state, the new object storage service corresponding to the new hard disk is created using the identification information and the key information.

3. The data reconstruction method according to claim 1, characterized in that: Obtaining identification information of the faulty object storage service corresponding to the faulty hard disk includes: Obtaining a hard disk identification of the faulty hard disk; A second query instruction carrying the hard disk identifier is sent to the object storage service table of the cluster system to which the faulty hard disk belongs, so as to obtain key information of the faulty object storage service corresponding to the faulty hard disk.

4. The data reconstruction method according to claim 1, characterized in that: Obtaining key information of the faulty object storage service corresponding to the faulty hard disk includes: Obtaining a hard disk identification of the faulty hard disk; A first query instruction carrying the hard disk identification is sent to the authorization table of the cluster system to which the faulty hard disk belongs, so as to obtain identification information of the faulty object storage service corresponding to the faulty hard disk.

5. The data reconstruction method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the faulty hard disk is replaced with the new hard disk, obtaining the first identification information stored in the data partition of the faulty hard disk and saving it to a preset space; Accordingly, the new object storage service is started to complete data reconstruction, including: Starting the new object storage service and obtaining the second identification information stored in the data partition of the new hard disk; When the second identification information is consistent with the first identification information, data reconstruction is completed using the new object storage service; When the second identification information is inconsistent with the first identification information, exit the new object storage service.

6. The data reconstruction method according to claim 1, characterized in that: When the failed hard disk is replaced with a new hard disk, a new object storage service corresponding to the new hard disk is created using the identification information and the key information, including: When the failed hard disk is replaced with the new hard disk, the file system and database file corresponding to the new hard disk are created using the identification information and the key information to create the new object storage service corresponding to the new hard disk; Accordingly, the new object storage service is started to complete data reconstruction, including: Determine whether the new object storage service has authentication authority; If the new object storage service does not have the authentication authority, adding the authentication authority to the new object storage service; If the new object storage service has the authentication authority, initializing the new object storage service; Mounting the database file to the file system and opening the database file to start the new object storage service; The new object storage service is used to reconstruct data.

7. The data reconstruction method according to claim 1, characterized in that: Determine the data reconstruction scenario based on the data reconstruction command, including: Parsing the data reconstruction command to determine whether the data reconstruction command contains an online disk replacement identifier; When the data reconstruction command does not include the online disk replacement identifier, determining that the data reconstruction scenario is the object storage service management scenario; When the data reconstruction command includes the online disk replacement identifier, it is determined that the data reconstruction scenario is the online disk replacement scenario.

8. A data reconstruction device, characterized in that: include: A determination module, used to determine a data reconstruction scenario according to a data reconstruction command; wherein the data reconstruction scenario includes an online disk replacement scenario and an object storage service management scenario; An acquisition module, used for determining a faulty hard disk and acquiring identification information and key information of a faulty object storage service corresponding to the faulty hard disk when the data reconstruction scenario is the online disk replacement scenario; A creation module, used for creating a new object storage service corresponding to the new hard disk by using the identification information and the key information when the failed hard disk is replaced with a new hard disk; The startup module is used to start the new object storage service to complete data reconstruction.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data reconstruction method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data reconstruction method according to any one of claims 1 to 7 are implemented.

Images