Database migration method, device, apparatus and storage medium

Abstract

The application relates to the technical field of computers and discloses a database migration method, device and equipment and a storage medium, which comprises the following steps: obtaining an original storage directory of target data; identifying the storage position of the target data under the original storage directory; when the storage position represents that the target data is stored in a database to be migrated in, updating the data storage directory of the database to be migrated in based on the original storage directory; and when the storage position represents that the target data is not stored in the database to be migrated in, migrating the target data to the database to be migrated in and updating the data storage directory of the database to be migrated in based on the original storage directory. The application can migrate the data of the database partition on the non-volatile memory standard hard disk back to the mechanical hard disk, reduces the data volume on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when the non-volatile memory standard hard disk is replaced, and increasing the reliability of the storage system.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a database migration method, apparatus, device, and storage medium. Background Technology

[0002] Distributed storage systems store data in units of object storage services. The underlying object storage service manages the data through a storage system, writing data directly to hard disk drives (HDDs). The database can reside on the HDD along with the data, or, for performance reasons, high-speed storage can be placed on non-volatile memory (NDRAM) standard hard drives, with some high-speed storage reserved on the HDDs. Furthermore, to fully utilize the performance of NDRAM standard hard drives and for cost savings, database partitions from multiple data disks are typically placed on the same NDRAM standard hard drive.

[0003] However, since non-volatile memory standard hard drives have a limited lifespan, when the lifespan of the non-volatile memory standard hard drive is reached, a large amount of metadata information has been stored in the database. At this time, when the distributed data storage system replaces the non-volatile memory standard hard drive, data recovery is required. The more metadata there is, the longer the recovery will take. Summary of the Invention

[0004] In view of this, the present invention provides a database migration method, apparatus, device and storage medium to solve the problem that when replacing non-volatile memory standard hard disks in existing distributed data storage systems, it takes a long time to recover metadata, and once the recovery is interrupted, it is easy to cause metadata recovery errors, thereby increasing the risk of replacing non-volatile memory standard hard disks.

[0005] In a first aspect, the present invention provides a database migration method, which includes: obtaining the original storage directory of target data; identifying the storage location of the target data under the original storage directory; when the storage location indicates that the target data is stored in the database to be migrated, updating the data storage directory of the database to be migrated based on the original storage directory; when the storage location indicates that the target data is not stored in the database to be migrated, migrating the target data to the database to be migrated, and updating the data storage directory of the database to be migrated based on the original storage directory. Through the above process, data of the database partition on the non-volatile memory standard hard disk can be migrated back to the mechanical hard disk, reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0006] In one optional implementation, updating the data storage directory of the database to be migrated based on the original storage directory includes:

[0007] Obtain the directory index of the target data and the original storage directory;

[0008] Update the directory index to match the indexes of the target data and data storage directories.

[0009] In one alternative implementation, before migrating the target data to the database to be migrated, the method further includes:

[0010] The validity of the original storage directory and the data storage directory is determined to obtain the validity determination result;

[0011] When the validity determination result indicates that the original storage directory and the data storage directory exist and are not invalid, the step of migrating the target data to the database to be migrated is executed.

[0012] When the validity determination result indicates that the original storage directory and data storage directory do not exist or are invalid, a migration error message is output and the migration of the target data is stopped.

[0013] In one alternative implementation, migrating the target data to the database to be migrated includes:

[0014] Obtain the space capacity of the first storage space occupied by the target data before migration, and the space capacity of the second storage space allocated to the database to be migrated;

[0015] The capacity of the first storage space is compared with that of the second storage space, and based on the capacity comparison results, the target data is migrated to the database to be migrated.

[0016] In one optional implementation, based on the capacity comparison results, the target data is migrated to the database to be migrated, including:

[0017] When the capacity comparison result indicates that the space capacity of the first storage space is greater than or equal to the space capacity of the second storage space, the target data will be migrated to the database to be migrated.

[0018] When the capacity comparison result indicates that the capacity of the first storage space is less than that of the second storage space, storage space is requested from the database to be migrated based on the capacity of the first storage space.

[0019] In one optional implementation, after migrating the target data to the database to be migrated and updating the data storage directory of the database to be migrated based on the original storage directory, the method further includes:

[0020] Retrieve the first content value corresponding to the target data in the original storage directory;

[0021] Calculate the second content value of the target data in the data storage directory, and compare the second content value with the first content value;

[0022] If the second content value is different from the first content value, it is determined that the target data in the original storage directory is different from the target data in the data storage directory.

[0023] If the second content value is the same as the first content value, it is determined that the target data in the original storage directory is the same as the target data in the data storage directory.

[0024] In one optional implementation, after determining that the target data in the original storage directory is the same as the target data in the data storage directory, the method further includes:

[0025] Perform read and write tests on the target data to obtain test results;

[0026] When the test results indicate that the target data read / write was successful, the database to be migrated is marked as invalid.

[0027] When the test results indicate that the target data read / write operation failed, the target data is migrated again.

[0028] Secondly, this invention provides a database migration device, which mainly includes: a directory acquisition module, a location identification module, a directory update module, and a data migration module. The directory acquisition module is used to acquire the original storage directory of the target data; the location identification module is used to identify the storage location of the target data under the original storage directory; the directory update module is used to update the data storage directory of the database to be migrated to based on the original storage directory when the storage location indicates that the target data is stored in the database to be migrated; the data migration module is used to migrate the target data to the database to be migrated to when the storage location indicates that the target data is not stored in the database to be migrated, and update the data storage directory of the database to be migrated to based on the original storage directory. Through the above process, data of the database partition on the non-volatile memory standard hard disk can be migrated back to the mechanical hard disk, reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0029] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the database migration method of the first aspect or any corresponding embodiment described above.

[0030] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the database migration method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0031] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of an application environment according to an embodiment of the present invention;

[0033] Figure 2 This is a flowchart illustrating the database migration method according to an embodiment of the present invention;

[0034] Figure 3 This is a flowchart illustrating another database migration method according to an embodiment of the present invention;

[0035] Figure 4 This is a flowchart illustrating another database migration method according to an embodiment of the present invention;

[0036] Figure 5 This is a flowchart illustrating another database migration method according to an embodiment of the present invention;

[0037] Figure 6 This is a structural block diagram of the database migration device according to an embodiment of the present invention;

[0038] Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] The terms "first" and "second" in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses. The term "multiple" in this invention can mean at least two, for example, two, three, or more, and the embodiments of this invention are not limited thereto.

[0041] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided by an embodiment of the present invention. The diagram includes a storage server 100, which may include a display 101, a processor 102, and a memory 103. The storage server 100 can communicate with a storage management server 200 via a network 300. The storage management server 200 can provide services (such as management services) to computing programs installed on clients. A database 201 can be set up on or independently of the storage management server 200 to provide data storage services to the storage management server 200. Furthermore, the storage management server 200 may run a processing engine 202, which can be used to execute steps performed by the storage management server 200.

[0042] Optionally, the storage server 100 may be, but is not limited to, a terminal capable of computing data, such as a mobile terminal (e.g., a tablet), a laptop, or a PC (Personal Computer). The network may include, but is not limited to, a wireless network or a wired network. The wireless network includes Bluetooth, Wi-Fi (Wireless Fidelity), and other networks that enable wireless communication. The wired network may include, but is not limited to, a wide area network (WAN), a metropolitan area network (MAN), or a storage management server cluster. The storage management server 200 may be, but is not limited to, any hardware device capable of computing.

[0043] Furthermore, in this embodiment, the database migration method described above can also be applied to, but is not limited to, a powerful independent processing device without requiring data interaction. For example, the processing device can be, but is not limited to, a powerful terminal device; that is, the various operations in the database migration method described above can be integrated into a single independent processing device. The above is merely an example, and this embodiment does not impose any limitations on it.

[0044] Optionally, in this embodiment, the database migration method described above can be executed by the storage management server 200, by the storage server 100, or jointly by both the storage management server 200 and the storage server 100. Alternatively, the database migration method of this embodiment can be executed by a client installed on the storage server 100.

[0045] According to an embodiment of the present invention, a database migration method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0046] This embodiment provides a database migration method that can be used in the aforementioned storage server. The storage server includes a target hard disk and a non-volatile memory standard hard disk. The non-volatile memory standard hard disk has at least one database partition, such as the database to be migrated. The target hard disk (such as a mechanical hard disk) also has a reserved portion of database storage space, such as the database to be migrated. Figure 2 This is a flowchart of a database migration method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0047] Step S201: Obtain the original storage directory of the target data.

[0048] In this embodiment, the original storage directory of the target data is obtained so that the target data stored in the original storage directory can be migrated to the target hard disk.

[0049] In one optional implementation, the hard drive information of the non-volatile memory standard hard drive is first monitored. Before the non-volatile memory standard hard drive reaches its lifespan, or when the amount of data stored on the non-volatile memory standard hard drive (such as target data) exceeds a threshold, a database migration instruction is generated. Based on this database migration instruction, the storage service (based on the non-volatile memory standard hard drive and the target hard drive) is stopped, and it is confirmed that the storage service has been stopped. Then, a hard drive detection command is used to confirm that the target hard drive is in normal condition, that the non-volatile memory standard hard drive is accessible, and that the storage server is within a fault domain. Next, a directory retrieval command is used to retrieve the original storage directory stored in the database to be migrated, which is used to store the target data.

[0050] Taking files as the target data as an example, the object storage service is first stopped using a stop command. After confirming that the service has stopped, an environment check is performed. This includes verifying the target hard drive's status is normal using a hard drive test command, confirming that the non-volatile memory standard hard drive is accessible, and confirming that the storage server has not exceeded the fault domain. After completing the environment check, a directory retrieval command is added to obtain the directory information of the files in the database records to be migrated. The execution flow of this command is as follows: it opens the file system of the storage server, retrieves the file information under the ultra-high-speed space storage directory, high-speed space storage directory, and slow space storage directory recorded by the file system, and outputs the file name and number of files in each directory in the migration interface. If the read fails, the corresponding error message is output. It can be understood that the original storage directory can be a collection of multiple space storage directories.

[0051] Step S202: Identify the storage location of the target data in the original storage directory.

[0052] In this embodiment, by identifying the storage location of the target data in the original storage directory, it is possible to determine whether to migrate the target data or update the index directory of the target data based on the storage location of the target data, thereby avoiding repeated migration or incomplete migration of the target data.

[0053] Step S203: When the storage location indicates that the target data is stored in the database to be migrated, update the data storage directory of the database to be migrated based on the original storage directory.

[0054] In this embodiment, when the target data is to be migrated to the database from its original storage directory, the data storage directory of the database to be migrated is updated based on the original storage directory. This updates the target data storage directory and avoids data read / write errors that occur after the database migration to the non-volatile memory standard hard disk is completed, due to the failure to migrate the target data directory.

[0055] In one optional implementation, the original storage directory of the target data is first checked. If the target data is stored in the original storage directory, but the target data is actually stored in the database to be migrated, the data storage directory of the database to be migrated is updated based on the original storage directory to update the directory index of the target data and the original storage directory to the index of the target data and the data storage directory, and the contents of the original storage directory corresponding to the database to be migrated are deleted.

[0056] In specific implementation, based on step S201, if a file is stored in the slow space storage directory and that file is located in the database to be migrated, compression commands are added or removed to compress the file stored in the slow space storage directory into the database to be migrated. The index of the file and the slow space storage directory is updated to the data storage directory of the database to be migrated. This data storage directory can be the high-speed space storage directory of the high-speed storage space in the database to be migrated. If a file is stored in the slow space storage directory but that file is located in the database to be migrated, the file system of the storage server in the distributed storage system and the database to be migrated are opened. The connection interface between the database to be migrated and the file system is called to update the directory index information of the file in the slow space storage directory. The directory index of the file in the slow space storage directory is updated to the high-speed space storage directory of the high-speed storage space in the database to be migrated. If execution fails, the process stops and an error message is output on the migration interface.

[0057] Furthermore, it can be checked whether the directory index of the files in the slow space storage directory has been completely updated to the high-speed space storage directory of the high-speed storage space in the database to be migrated. If the directory index of the files in the slow space storage directory has been completely updated to the high-speed space storage directory of the high-speed storage space in the database to be migrated, the target data not stored in the database to be migrated can be migrated to the database to be migrated until the migration of the database to be migrated is completed. If there are files in the slow space storage directory whose directory index update failed, the migration process of migrating the target data not stored in the database to be migrated to the database to be migrated will be stopped and an error message will be output in the migration interface.

[0058] Step S204: When the storage location indicates that the target data is not stored in the database to be migrated, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory.

[0059] In this embodiment, when the target data is not stored in the database to be migrated from the original storage directory, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory to achieve the migration of the target data. This enables the data of the database to be migrated from the non-volatile memory standard hard disk to the target hard disk, reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0060] In one optional implementation, the original storage directory of the target data is first checked. If the target data is stored in the original storage directory and the target data is stored in the database to be migrated, the target data is migrated to the database to be migrated. The data storage directory of the database to be migrated is updated based on the original storage directory so that the index of the target data is updated to the data storage directory. The contents of the original storage directory of the database to be migrated are then deleted.

[0061] In specific implementation, based on step S201, a data migration instruction is first added to migrate all files stored on the non-volatile memory standard hard disk under the high-speed space storage directory and the slow space storage directory to the target hard disk and update the directory index of the files.

[0062] Further, the original storage directory and data storage directory are first checked. If the directory does not exist or is invalid and inaccessible, an error message is output on the migration interface, and the migration service stops. A directory not existing includes being deleted or corrupted. Then, the capacity used by files stored on the non-volatile memory standard hard disk under the current high-speed and slow-speed storage directories is obtained, along with the capacity allocated to the data storage directory on the target hard disk. It is determined whether the allocated space in the data storage directory is sufficient. If not, the data storage directory requests space again. If sufficient space cannot be allocated, the database migration process stops, and an error message is output on the migration interface. Finally, the file system service retrieves all files in the database to be migrated, i.e., reads files stored on the non-volatile memory standard hard disk under the high-speed and high-speed storage directories, writes the read files to the target hard disk, updates the directory index information of the files in the database to be migrated, and updates the log file of the database to be migrated after all files have been migrated, marking the partition of the database to be migrated on the non-volatile memory standard hard disk as invalid. If an error occurs during the process of reading or writing files from the non-volatile memory standard hard disk to the target hard disk, the migration action stops, and an error message is output.

[0063] The database migration method provided in this embodiment first obtains the original storage directory of the target data to facilitate the migration of the target data stored in the original storage directory to the target hard disk. It then identifies the storage location of the target data in the original storage directory to determine whether to migrate the target data itself or update its index directory, avoiding duplicate or incomplete migrations. When the target data is in the storage location of the database to be migrated from the original storage directory, the data storage directory of the database to be migrated is updated based on the original storage directory, thus updating the target data storage directory and preventing data read / write errors that might occur after the database migration to a non-volatile memory standard hard disk is completed, due to the target data directory not being migrated. Finally, when the target data is not stored in the database to be migrated from the original storage directory, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory, thus achieving the migration of the target data. Therefore, this invention can realize the migration of data to be migrated to the database from the non-volatile memory standard hard disk back to the target hard disk (such as a mechanical hard disk), reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0064] This embodiment provides a database migration method that can be used in the aforementioned storage server. The storage server includes a target hard disk and a non-volatile memory standard hard disk. The non-volatile memory standard hard disk has at least one database partition, such as the database to be migrated. The target hard disk also has a reserved portion of database storage space, such as the database to be migrated. Figure 3 This is a flowchart of a database migration method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0065] Step S301: Obtain the original storage directory of the target data.

[0066] In this embodiment, the original storage directory of the target data is obtained so that the target data stored in the original storage directory can be migrated to the target hard disk.

[0067] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0068] Step S302: Identify the storage location of the target data in the original storage directory.

[0069] In this embodiment, by identifying the storage location of the target data in the original storage directory, it is possible to determine whether to migrate the target data or update the index directory of the target data based on the storage location of the target data, thereby avoiding repeated migration or incomplete migration of the target data.

[0070] Step S303: When the storage location indicates that the target data is stored in the database to be migrated, update the data storage directory of the database to be migrated based on the original storage directory.

[0071] Specifically, step S303 above includes:

[0072] Step S3031: Obtain the directory index of the target data and the original storage directory.

[0073] In this embodiment, when the target data is to be migrated to the database from its original storage location, the directory index of the target data and the original storage directory is obtained so as to obtain the original directory index information of the target data.

[0074] Step S3032: Update the directory index to the index of the target data and data storage directory.

[0075] In this embodiment, when the target data is to be migrated to the database from its original storage location, the directory index of the target data and the original storage directory is updated to the index of the target data and the data storage directory. This updates the target data storage directory and avoids data read / write errors that occur after the database migration to the non-volatile memory standard hard disk is completed, due to the failure of the target data directory to be migrated.

[0076] Step S304: When the storage location indicates that the target data is not stored in the database to be migrated, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory.

[0077] In this embodiment, when the target data is not stored in the database to be migrated from the original storage directory, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory to achieve the migration of the target data. This enables the data of the database to be migrated from the non-volatile memory standard hard disk to the target hard disk, reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0078] The database migration method provided in this embodiment first obtains the original storage directory of the target data to facilitate the migration of the target data stored in the original storage directory to the target hard disk. It then identifies the storage location of the target data in the original storage directory to determine whether to migrate the target data itself or update its index directory, avoiding duplicate or incomplete migrations. When the target data is in the storage location of the database to be migrated from the original storage directory, the data storage directory of the database to be migrated is updated based on the original storage directory, thus updating the target data storage directory and preventing data read / write errors that might occur after the database migration to a non-volatile memory standard hard disk is completed, due to the target data directory not being migrated. Finally, when the target data is not stored in the database to be migrated from the original storage directory, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory, thus achieving the migration of the target data. Therefore, this invention can realize the migration of data to be migrated to the database from the non-volatile memory standard hard disk back to the target hard disk (such as a mechanical hard disk), reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumption when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0079] This embodiment provides a database migration method that can be used in the aforementioned storage server. The storage server includes a target hard disk and a non-volatile memory standard hard disk. The non-volatile memory standard hard disk has at least one database partition, such as the database to be migrated. The target hard disk also has a reserved portion of database storage space, such as the database to be migrated. Figure 4 This is a flowchart of a database migration method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0080] Step S401: Obtain the original storage directory of the target data.

[0081] In this embodiment, the original storage directory of the target data is obtained so that the target data stored in the original storage directory can be migrated to the target hard disk.

[0082] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0083] Step S402: Identify the storage location of the target data in the original storage directory.

[0084] In this embodiment, by identifying the storage location of the target data in the original storage directory, it is possible to determine whether to migrate the target data or update the index directory of the target data based on the storage location of the target data, thereby avoiding repeated migration or incomplete migration of the target data.

[0085] Step S403: When the storage location indicates that the target data is stored in the database to be migrated, update the data storage directory of the database to be migrated based on the original storage directory.

[0086] In this embodiment, when the target data is to be migrated to the database from its original storage directory, the data storage directory of the database to be migrated is updated based on the original storage directory. This updates the target data storage directory and avoids data read / write errors that occur after the database migration to the non-volatile memory standard hard disk is completed, due to the failure to migrate the target data directory.

[0087] Please see details Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0088] Step S404: When the storage location indicates that the target data is not stored in the database to be migrated, the validity of the original storage directory and the data storage directory is determined to obtain the validity determination result.

[0089] In this embodiment, by determining the validity of the original storage directory and the data storage directory when the target data in the original storage directory is not stored in the database to be migrated, a criterion is provided for whether the target data should be migrated in the future.

[0090] Step S405: When the validity determination result indicates that the original storage directory and the data storage directory exist and are not invalid, the target data is migrated to the database to be migrated.

[0091] In this embodiment, the target data is migrated to the database to be migrated when the original storage directory and data storage directory exist and are not invalid, thus avoiding the situation where the target data migration fails due to the loss or invalidation of the original storage directory and data storage directory.

[0092] Step S406: When the validity determination result indicates that the original storage directory and data storage directory do not exist or are invalid, output migration error information and stop the migration of target data.

[0093] In this embodiment, when the original storage directory and the data storage directory do not exist or are invalid, a migration error message is output and the migration of the target data is stopped, so as to avoid the problem of server crashing caused by forced data migration.

[0094] Understandably, by migrating the target data to the database to be migrated when it is not stored in the original storage directory, and updating the data storage directory of the database to be migrated based on the original storage directory, the target data can be migrated. This allows the data of the database to be migrated from the non-volatile memory standard hard disk to the target hard disk, reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing the data recovery time consumed when replacing the non-volatile memory standard hard disk and increasing the reliability of the storage system.

[0095] Please see details Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0096] The database migration method provided in this embodiment first obtains the original storage directory of the target data to facilitate the migration of the target data stored in the original storage directory to the target hard disk. It then identifies the storage location of the target data in the original storage directory to determine whether to migrate the target data itself or update its index directory, avoiding duplicate or incomplete migrations. Finally, when the target data is located at the desired database location in the original storage directory, the data storage directory of the database to be migrated is updated based on the original storage directory. This update of the target data storage directory prevents subsequent migrations after the database migration to the non-volatile memory standard hard disk is completed. This invention addresses the issue of data read / write errors occurring due to the target data's directory not being migrated. By verifying the validity of the original storage directory and data storage directory when the target data is not stored in the database to be migrated, a criterion is provided for determining whether to migrate the target data. Furthermore, by migrating the target data to the database to be migrated when the original storage directory and data storage directory exist and are not invalid, migration failures due to the loss or invalidation of the original storage directory and data storage directory are avoided. Finally, by outputting migration error information and stopping the migration of the target data when the original storage directory and data storage directory are absent or invalid, the invention prevents server crashes caused by forced data migration. Therefore, this invention can migrate data from a non-volatile memory standard hard disk to the target hard disk (such as a mechanical hard disk), reducing the amount of data on the non-volatile memory standard hard disk, thereby reducing data recovery time during non-volatile memory standard hard disk replacement and increasing the reliability of the storage system.

[0097] This embodiment provides a database migration method that can be used in the aforementioned storage server. The storage server includes a target hard disk and a non-volatile memory standard hard disk. The non-volatile memory standard hard disk has at least one database partition, such as the database to be migrated. The target hard disk also has a reserved portion of database storage space, such as the database to be migrated. Figure 5This is a flowchart of a database migration method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0098] Step S501: Obtain the original storage directory of the target data.

[0099] In this embodiment, the original storage directory of the target data is obtained so that the target data stored in the original storage directory can be migrated to the target hard disk.

[0100] Please see details Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0101] Step S502: Identify the storage location of the target data in the original storage directory.

[0102] In this embodiment, by identifying the storage location of the target data in the original storage directory, it is possible to determine whether to migrate the target data or update the index directory of the target data based on the storage location of the target data, thereby avoiding repeated migration or incomplete migration of the target data.

[0103] Step S503: When the storage location indicates that the target data is not stored in the database to be migrated, the validity of the original storage directory and the data storage directory is determined to obtain the validity determination result.

[0104] In this embodiment, by determining the validity of the original storage directory and the data storage directory when the target data in the original storage directory is not stored in the database to be migrated, a criterion is provided for whether the target data should be migrated in the future.

[0105] Step S504: When the validity determination result indicates that the original storage directory and the data storage directory exist and are not invalid, the target data is migrated to the database to be migrated.

[0106] In this embodiment, the target data is migrated to the database to be migrated when the original storage directory and data storage directory exist and are not invalid, thus avoiding the situation where the target data migration fails due to the loss or invalidation of the original storage directory and data storage directory.

[0107] Specifically, step S504 above includes:

[0108] Step S5041: Obtain the space capacity of the first storage space occupied by the target data before migration, and the space capacity of the second storage space allocated to the database to be migrated.

[0109] In this embodiment, the space capacity of the first storage space occupied by the target data before migration and the space capacity of the second storage space allocated to the database to be migrated are obtained so that the target data can be migrated to the database by comparing the space capacity of the first storage space with that of the second storage space.

[0110] Step S5042: Compare the space capacity of the first storage space with the space capacity of the second storage space, and based on the capacity comparison results, migrate the target data to the database to be migrated.

[0111] In this embodiment, the space capacity of the first storage space is compared with that of the second storage space, and based on the capacity comparison result, the target data is migrated to the database to be migrated, thereby realizing the migration of the target data.

[0112] In one optional implementation, when the capacity comparison result indicates that the capacity of the first storage space is greater than or equal to the capacity of the second storage space, the target data is migrated to the database to be migrated; when the capacity comparison result indicates that the capacity of the first storage space is less than the capacity of the second storage space, storage space is requested from the database to be migrated based on the capacity of the first storage space.

[0113] In one optional implementation, after migrating the target data to the database to be migrated and updating the data storage directory of the database to be migrated based on the original storage directory, a first content value corresponding to the target data in the original storage directory can be obtained; a second content value defining the content of the target data in the data storage directory can be calculated, and the second content value can be compared with the first content value; if the second content value is different from the first content value, it is determined that the target data in the original storage directory and the target data in the data storage directory are different; if the second content value is the same as the first content value, it is determined that the target data in the original storage directory and the target data in the data storage directory are the same. The calculation of the first and second content values ​​can be obtained through hash calculation or through a data similarity comparison algorithm. It is understood that by comparing the first and second content values, the consistency of the target data before and after migration can be judged, avoiding data loss or garbled characters during the data migration process, which could lead to subsequent data read / write errors or inconsistencies.

[0114] In one optional implementation, after determining that the target data in the original storage directory is the same as the target data in the data storage directory, read / write tests can be performed on the target data to obtain test results. When the test results indicate that the target data read / write is successful, the database to be migrated is marked as invalid; when the test results indicate that the target data read / write fails, the target data is migrated again. The read / write test on the target data can be a continuous data read / write test or a random data read / write test. It is understood that by marking the database to be migrated as invalid when the test results indicate that the target data read / write is successful, the reuse of the database to be migrated in the non-volatile memory standard hard disk can be avoided, thus providing the necessary conditions for subsequent replacement of the non-volatile memory standard hard disk. By migrating the target data again when the test results indicate that the target data read / write fails, the migration of the database to be migrated is ultimately achieved.

[0115] Step S505: When the validity determination result indicates that the original storage directory and data storage directory do not exist or are invalid, output migration error information and stop the migration of target data.

[0116] In this embodiment, when the original storage directory and the data storage directory do not exist or are invalid, a migration error message is output and the migration of the target data is stopped, so as to avoid the problem of server crashing caused by forced data migration.

[0117] The database migration method provided in this embodiment first obtains the original storage directory of the target data to facilitate the migration of the target data stored in the original storage directory to the target hard drive. It then identifies the storage location of the target data in the original storage directory to determine whether to migrate the target data itself or update its index directory, avoiding duplicate or incomplete migrations. When the target data is not stored in the database to be migrated from the original storage directory, the validity of the original storage directory and data storage directory is determined, providing a criterion for subsequent migration. Finally, if the original storage directory and data storage directory are absent or invalid, migration error information is output, and the migration of the target data is stopped to avoid server crashes caused by forced data migration. Therefore, this invention can migrate data from a non-volatile memory standard hard drive (NDRAM) back to the target hard drive (e.g., a mechanical hard drive), reducing the amount of data on the NDRAM and thus reducing data recovery time during NDRAM hard drive replacement, increasing the reliability of the storage system.

[0118] This embodiment also provides a database migration apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0119] This embodiment provides a database migration device, such as... Figure 6 As shown, it includes:

[0120] The directory retrieval module 601 is used to retrieve the original storage directory of the target data.

[0121] The location identification module 602 is used to identify the storage location of the target data in the original storage directory.

[0122] The directory update module 603 is used to update the data storage directory of the database to be migrated based on the original storage directory when the storage location indicates that the target data is stored in the database to be migrated.

[0123] In some alternative implementations, the directory update module 603 includes:

[0124] The index acquisition unit is used to acquire the directory index of the target data and the original storage directory.

[0125] The index update unit is used to update the directory index to the index of the target data and data storage directory.

[0126] The data migration module 604 is used to migrate the target data to the database to be migrated when the storage location indicates that the target data is not stored in the database to be migrated, and to update the data storage directory of the database to be migrated based on the original storage directory.

[0127] In some alternative implementations, the data migration module 604 includes:

[0128] The validity determination unit is used to determine the validity of the original storage directory and the data storage directory to obtain the validity determination result.

[0129] The data migration unit is used to migrate the target data to the database to be migrated when the validity determination result indicates that the original storage directory and the data storage directory exist and are not invalid.

[0130] Optionally, when migrating target data to the database to be migrated, first obtain the space capacity of the first storage space occupied by the target data before migration, and the space capacity of the second storage space allocated to the database to be migrated; compare the space capacity of the first storage space with the space capacity of the second storage space, and based on the capacity comparison result, migrate the target data to the database to be migrated.

[0131] Specifically, when the capacity comparison result indicates that the capacity of the first storage space is greater than or equal to the capacity of the second storage space, the target data is migrated to the database to be migrated; when the capacity comparison result indicates that the capacity of the first storage space is less than the capacity of the second storage space, storage space is requested from the database to be migrated based on the capacity of the first storage space.

[0132] The stop unit is used to output migration error information and stop the migration of target data when the validity determination result indicates that the original storage directory and data storage directory do not exist or are invalid.

[0133] In some alternative embodiments, the above-described apparatus further includes:

[0134] The consistency determination module is used to obtain the first content value corresponding to the target data in the original storage directory; calculate the second content value of the content defined by the target data in the data storage directory, and compare the second content value with the first content value; if the second content value is different from the first content value, it is determined that the target data in the original storage directory and the target data in the data storage directory are different; if the second content value is the same as the first content value, it is determined that the target data in the original storage directory and the target data in the data storage directory are the same.

[0135] The read / write test module is used to perform read / write tests on the target data to obtain test results. When the test results indicate that the target data read / write is successful, the database to be migrated is marked as invalid. When the test results indicate that the target data read / write fails, the target data is migrated again.

[0136] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0137] In this embodiment, the database migration device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0138] This invention also provides a computer device having the above-described features. Figure 6 The database migration device shown.

[0139] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a storage server array, a group of blade storage servers, or a multiprocessor system). Figure 7 Take a processor 10 as an example.

[0140] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0141] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0142] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, server clusters, mobile communication networks, and combinations thereof.

[0143] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0144] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0145] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0146] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A database migration method, characterized in that, The method includes: The hard disk information of the non-volatile memory standard hard disk is monitored. Before the non-volatile memory standard hard disk reaches its life cycle, or when the amount of data stored in the non-volatile memory standard hard disk exceeds a threshold, a database migration instruction is generated. Based on the database migration instruction, stop the storage service based on the non-volatile memory standard hard disk and confirm that the storage service has been stopped; If the hard disk test command confirms that the mechanical hard disk is in normal condition, the non-volatile memory standard hard disk is accessible, and the storage server is in the fault domain, then the directory retrieval command is used to retrieve the original storage directory of the target data, as well as the validity of the original storage directory and the data storage directory of the database to be migrated. The target data and the original storage directory are stored on the non-volatile memory standard hard disk. Identify the storage location of the target data in the original storage directory; When the storage location indicates that the target data is stored in the database to be migrated, the data storage directory of the database to be migrated is updated based on the original storage directory. The database to be migrated and the data storage directory are stored on a mechanical hard disk. When the storage location indicates that the target data is not stored in the database to be migrated, the target data is migrated to the database to be migrated, and the data storage directory of the database to be migrated is updated based on the original storage directory; The step of updating the data storage directory of the database to be migrated based on the original storage directory includes: Obtain the directory index of the target data and the original storage directory; Update the directory index to match the index of the target data and the data storage directory.

2. The method according to claim 1, characterized in that, Before migrating the target data to the database to be migrated, the method further includes: The validity of the original storage directory and the data storage directory is determined to obtain the validity determination result; When the validity determination result indicates that the original storage directory and the data storage directory exist and are not invalid, the step of migrating the target data to the database to be migrated is executed. When the validity determination result indicates that the original storage directory and the data storage directory do not exist or are invalid, a migration error message is output, and the migration of the target data is stopped.

3. The method according to claim 2, characterized in that, The process of migrating the target data to the database to be migrated includes: Obtain the space capacity of the first storage space occupied by the target data before migration, and the space capacity of the second storage space allocated to the database to be migrated; The space capacity of the first storage space is compared with that of the second storage space, and based on the capacity comparison result, the target data is migrated to the database to be migrated.

4. The method according to claim 3, characterized in that, The process of migrating the target data to the database to be migrated, based on the capacity comparison results, includes: When the capacity comparison result indicates that the space capacity of the first storage space is greater than or equal to the space capacity of the second storage space, the target data is migrated to the database to be migrated. When the capacity comparison result indicates that the capacity of the first storage space is less than the capacity of the second storage space, a storage space application is made to the database to be migrated based on the capacity of the first storage space.

5. The method according to any one of claims 1 to 4, characterized in that, After migrating the target data to the database to be migrated to, and updating the data storage directory of the database to be migrated to based on the original storage directory, the method further includes: Obtain the first content value corresponding to the target data in the original storage directory; Calculate the second content value of the content defined by the target data in the data storage directory, and compare the second content value with the first content value; If the second content value is different from the first content value, it is determined that the target data in the original storage directory is different from the target data in the data storage directory; If the second content value is the same as the first content value, it is determined that the target data in the original storage directory is the same as the target data in the data storage directory.

6. The method according to claim 5, characterized in that, After determining that the target data in the original storage directory is the same as the target data in the data storage directory, the method further includes: Perform read and write tests on the target data to obtain test results; When the test results indicate that the target data read / write was successful, the database to be migrated is marked as invalid. When the test results indicate that the target data read / write has failed, the target data is migrated again.

7. A database migration device, characterized in that, The device, used in a database, includes: The directory acquisition module monitors the hard disk information of the non-volatile memory standard hard disk. Before the non-volatile memory standard hard disk reaches its life cycle, or when the amount of data stored on the non-volatile memory standard hard disk exceeds a threshold, a database migration instruction is generated. Based on the database migration instruction, the storage service based on the non-volatile memory standard hard disk is stopped and the storage service is confirmed to have stopped. The hard disk detection command confirms that the mechanical hard disk is in normal condition, that the non-volatile memory standard hard disk is accessible, and that the storage server is within the fault domain. Then, the directory acquisition instruction retrieves the original storage directory of the target data, as well as the validity of the original storage directory and the data storage directory of the database to be migrated. The target data and the original storage directory are stored on the non-volatile memory standard hard disk. The location identification module is used to identify the storage location of the target data in the original storage directory; The directory update module is used to update the data storage directory of the database to be migrated based on the original storage directory when the storage location indicates that the target data is stored in the database to be migrated. The database to be migrated and the data storage directory are stored on a mechanical hard disk. The data migration module is used to migrate the target data to the database to be migrated when the storage location indicates that the target data is not stored in the database to be migrated, and to update the data storage directory of the database to be migrated based on the original storage directory. The step of updating the data storage directory of the database to be migrated based on the original storage directory includes: Obtain the directory index of the target data and the original storage directory; Update the directory index to match the index of the target data and the data storage directory.

8. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 6.

Images