File migration method and device
By creating a file directory on a new server and updating the system database path, combined with Rclone's parallel transmission and sharding processing, the data integrity and system reliability during file migration are greatly improved, and the problem of reading failure during file migration is solved. , ensure the normal operation of the system.
Patent Information
- Application Number
- CN202310645277.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-03
AI Technical Summary
During the iteration of the software development system version, when the file storage server is aging, the existing tool Rclone cannot synchronously modify the file storage path in the system database, resulting in file reading failure.
Provide a file migration method and device. By creating a file directory on a new server, determining and updating the storage path in the system database, ensuring that the file can be successfully read after being migrated to the new server, Rclone uses data synchronization tool to transfer file blocks in parallel. Judge file integrity and process large files shards if necessary.
Ensure that after the file is migrated to the new server, the system can successfully read the file according to the updated storage path, avoid reading failures, and ensure that the system is running normally.
Smart Images

Figure CN119066029A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of file processing technology, and in particular to a file migration method and device. Background Art
[0002] In the process of software development system version iteration, you may encounter such scenarios: due to the aging of the file storage server involved in the system or other reasons, you want to migrate the files on this server to one or more new file storage servers to ensure the normal operation of the system.
[0003] Nowadays, Rclone is usually used to support migration operations between file storage. Rclone is a command-line program for managing files on cloud storage. It packages the interfaces of major cloud storage providers to provide a unified access interface. Rclone can support more than 40 cloud storage products, including object storage, enterprise and consumer file storage, services, and standard transmission protocols. At the same time, in order to protect the reliability of data, Rclone can retain timestamps and always verify files.
[0004] However, although Rclone can operate object storage, upload objects, pull objects, and migrate between object storages, it cannot meet the requirement of synchronously modifying the file storage path in the system database when migrating files, that is, changing the old path to the path under the new file storage server. Then when the system reads files, it will still obtain files from the system database according to the old file storage path, resulting in the system failing to find the file and reading it. For this reason, an effective optimization solution is urgently needed. Summary of the invention
[0005] In view of this, the present application provides a file migration method and device, so that when migrating the target file to the new server environment, the old storage path recorded in the system database is updated using the new storage path of the target file on the new server, so as to automatically complete the migration operation of the target file, thereby ensuring that no matter which new server the target file is migrated to, the target file can be successfully read according to the new storage path updated in the system database, so as not to affect the normal operation of the system.
[0006] In order to achieve the above objectives, this application mainly provides the following technical solutions:
[0007] The first aspect of the present application provides a file migration method, the method comprising:
[0008] Determine the target files to be migrated;
[0009] Acquire a first storage path corresponding to the target file from a system database, where the first storage path is used to represent a current storage location of the target file in the first server;
[0010] Creating a file directory on the second server to which the target file is migrated;
[0011] Determining, according to the file directory, a second storage path required for migrating the target file;
[0012] The target file is migrated to the second server according to the second storage path and the first storage path recorded in the system database is updated using the second storage path to complete the migration operation of the target file.
[0013] In some modified implementation manners of the first aspect of the present application, migrating the target file to the second server according to the second storage path includes:
[0014] Determine whether the size of the target file is greater than a preset threshold;
[0015] If yes, migrating the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path;
[0016] If not, the target file is migrated to the second server using the data synchronization tool Rclone according to the second storage path.
[0017] In some modified implementation manners of the first aspect of the present application, the migrating the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path includes:
[0018] Divide the target file into a plurality of file blocks according to a preset number, and add a unique label to each file block;
[0019] According to the second storage path, using a plurality of the data synchronization tools Rclone to support the migration of each of the file blocks to the second server in parallel;
[0020] According to the labels corresponding to the file blocks, the file blocks migrated to the second server are spliced to obtain the target file.
[0021] In some modified implementation manners of the first aspect of the present application, after the file blocks migrated to the second server are spliced together according to the labels corresponding to the file blocks to obtain the target file, the method further includes:
[0022] Based on whether the labels between the file blocks on the second server are continuous, determining whether the target file migrated to the second server is complete; and / or,
[0023] Based on the labels corresponding to the respective file blocks on the second server, it is counted whether the number of the labels is consistent with the preset number, so as to determine whether the target file migrated to the second server is complete.
[0024] In some modified implementations of the first aspect of the present application, if it is determined that the target file migrated to the second server is incomplete, the migration operation is determined to have failed, and a prompt message indicating a migration exception is triggered.
[0025] In some modified implementation modes of the first aspect of the present application, the method further includes:
[0026] Before performing the file migration operation, calculating first summary information corresponding to the target file stored on the first server;
[0027] After performing the file migration operation to obtain the migration file to be verified, calculating second summary information of the migration file stored on the second server;
[0028] Based on comparing whether the second summary information is consistent with the first summary information, verifying whether the target file is completely migrated to the second server.
[0029] A second aspect of the present application provides a file migration device, the device comprising:
[0030] A first determining unit, used to determine a target file to be migrated;
[0031] an acquisition unit, configured to acquire a first storage path corresponding to the target file from a system database, wherein the first storage path is used to represent a current storage location of the target file in the first server;
[0032] A creating unit, used for creating a file directory on the second server to which the target file is migrated;
[0033] A second determining unit, configured to determine, according to the file directory, a second storage path required for migrating the target file;
[0034] a migration unit, configured to migrate the target file to the second server according to the second storage path;
[0035] An updating unit is used to update the first storage path recorded in the system database by using the second storage path to complete the migration operation of the target file.
[0036] In some modified implementations of the second aspect of the present application, the migration unit includes:
[0037] A judging module, used to judge whether the size of the target file is greater than a preset threshold;
[0038] a migration module, configured to migrate the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path when it is determined that the size of the target file is greater than a preset threshold;
[0039] When it is determined that the size of the target file is not greater than a preset threshold, the migration module uses the data synchronization tool Rclone to migrate the target file to the second server according to the second storage path.
[0040] In some modified implementation modes of the second aspect of the present application, the migration module includes:
[0041] A division submodule, used to divide the target file into a plurality of file blocks according to a preset number, and to add a unique label to each file block;
[0042] A processing submodule, configured to support the migration of each file block to the second server in parallel by using a plurality of the data synchronization tools Rclone according to the second storage path;
[0043] A splicing submodule is used to splice the file blocks migrated to the second server to obtain the target file according to the labels corresponding to the file blocks.
[0044] In some modified implementation manners of the second aspect of the present application, after the target file is obtained by splicing the file blocks migrated to the second server according to the labels corresponding to the file blocks, the migration module further includes:
[0045] a check submodule, configured to determine whether the target file migrated to the second server is complete based on whether the labels between the file blocks on the second server are continuous; and / or,
[0046] The check submodule is further used to count whether the number of the numbers is consistent with the preset number based on the numbers corresponding to the file blocks on the second server, so as to determine whether the target file migrated to the second server is complete.
[0047] In some modified implementations of the second aspect of the present application, the migration module further includes: an exception handling submodule;
[0048] The exception handling submodule is used to determine that the migration operation fails and trigger a migration exception prompt message if it is determined that the target file migrated to the second server is incomplete.
[0049] In some modified implementation modes of the second aspect of the present application, the device further includes:
[0050] a calculation unit, configured to calculate first summary information corresponding to the target file stored on the first server before performing a file migration operation;
[0051] The calculation unit is further configured to calculate second summary information of the migration file stored on the second server after performing the file migration operation to obtain the migration file to be verified;
[0052] A verification unit is used to verify whether the target file is completely migrated to the second server based on comparing whether the second summary information is consistent with the first summary information.
[0053] A third aspect of the present application provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the file migration method as described above.
[0054] A fourth aspect of the present application provides an electronic device, the device comprising at least one processor, and at least one memory and a bus connected to the processor;
[0055] Wherein, the processor and the memory communicate with each other through the bus;
[0056] The processor is used to call the program instructions in the memory to execute the file migration method as described above.
[0057] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:
[0058] The present application provides a file migration method and device. After determining the target file to be migrated, the present application obtains the storage path of the target file from the system database. The storage path is used to indicate the storage location of the target file on the old server. Based on the purpose of file migration, the present application creates a file directory on the new server based on the migration requirements, which is used to store the migrated target file under this file directory. After the target file is migrated from the old server to the new server, the present application also simultaneously modifies the old storage path recorded in the system database to a new storage path, so that the system database can read the required file based on the new storage path when calling the target file requirement. Compared with the prior art, the optimized solution for file migration provided by the present application ensures that no matter which new server the target file is migrated to, the target file can be successfully read according to the new storage path updated in the system database, so as not to affect the normal operation of the system.
[0059] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0061] Figure 1 A flow chart of a file migration method provided in an embodiment of the present application;
[0062] Figure 2 A flowchart of another file migration method provided in an embodiment of the present application;
[0063] Figure 3 A block diagram of a file migration device provided in an embodiment of the present application;
[0064] Figure 4 A block diagram of another file migration device provided in an embodiment of the present application;
[0065] Figure 5 An electronic device for file migration provided in an embodiment of the present application. DETAILED DESCRIPTION
[0066] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.
[0067] Now, in the process of software development system version iteration, there is a scenario where the files on the file storage server involved in the system are aging or other reasons, and the files on this server need to be migrated to one or more new file storage servers. The inventor found that if the Rclone tool is used to implement the migration operation between file storages, but the file storage path in the system database is not synchronously modified during file migration, when the system reads the file, if the file is still obtained from the system database according to the old file storage path, the system will not be able to find the file and the reading will fail.
[0068] To this end, the inventor has conducted research and found that in order to ensure that the system does not fail to read the file in such a scenario, the present application embodiment provides a file migration method, such as Figure 1 , the present application embodiment provides the following specific steps:
[0069] 101. Determine the target files to be migrated.
[0070] 102. Obtain a first storage path corresponding to the target file from a system database, where the first storage path is used to represent a current storage location of the target file in the first server.
[0071] In the embodiments of the present application, the terms "first" and "second" are used to refer to different servers and storage paths of target files on different servers. The "first server" refers to the old server where the target file is stored before the target file migration is performed, and the "second server" refers to the new server where the target file is located after the target file migration is completed.
[0072] In addition, the file migration scenario implemented by the embodiment of the present application may be to migrate the target file on an old server to one or more new servers.
[0073] 103. Create a file directory on the second server to which the target file is to be migrated.
[0074] 104. Determine a second storage path required for migrating the target file according to the file directory.
[0075] In an embodiment of the present application, as in steps 103-104, in order to perform preparatory work before file migration, the size and type of the target files to be migrated can be taken into consideration, and the required file directory can be created on the new server to which the files are migrated to store the migrated target files.
[0076] 105. Migrate the target file to the second server according to the second storage path and use the second storage path to update the first storage path recorded in the system database to complete the migration operation of the target file.
[0077] In the embodiment of the present application, not only is it necessary to realize the migration of files from an old server to a new server, but it is also necessary to determine that when reading the target file from the system database, it is necessary to know the new storage path to replace the old storage path. Therefore, the embodiment of the present application is to obtain the target file on the new server, and the storage path of the target file in the system database is updated before the file migration operation is recognized as complete.
[0078] As described above, the embodiment of the present application provides a file migration method. After determining the target file to be migrated, the embodiment of the present application obtains the storage path of the target file from the system database. The storage path is used to indicate the storage location of the target file on the old server. Based on the purpose of file migration, the present application creates a file directory on the new server based on the migration requirements, which is used to store the migrated target file under this file directory. After the target file is migrated from the old server to the new server, the embodiment of the present application also simultaneously modifies the old storage path recorded in the system database to the new storage path, so that the system database can read the required file based on the new storage path when calling the target file requirement. Compared with the prior art, the optimized solution for file migration provided by the embodiment of the present application ensures that no matter which new server the target file is migrated to, the target file can be successfully read according to the new storage path updated in the system database, so as not to affect the normal operation of the system.
[0079] In order to explain the above embodiment in more detail, the present application embodiment also provides another file migration method, such as Figure 2 As shown, the present application embodiment provides the following specific steps:
[0080] 201. Determine the target file to be migrated.
[0081] 202. Obtain a first storage path corresponding to the target file from a system database, where the first storage path is used to represent a current storage location of the target file in the first server.
[0082] 203. Create a file directory on the second server to which the target file is to be migrated.
[0083] 204. Determine a second storage path required for migrating the target file according to the file directory.
[0084] For the explanation of steps 201-204 in the embodiment of the present application, please refer to steps 101-104, which will not be repeated here.
[0085] In addition, when performing the file migration operation, the embodiment of the present application also provides corresponding implementation plans according to the different volumes of the target files to achieve diverse processing of file migration, as shown in the following steps 205, 206a and 206b-208b.
[0086] 205. Determine whether the size of the target file is greater than a preset threshold.
[0087] The embodiment of the present application uses the preset threshold as the boundary and provides two different file migration schemes, such as step 206a. If the size of the target file is not greater than the preset threshold, the data synchronization tool Rclone can be used to copy and transfer the target file from the old server to the new server. However, if the size of the target file is greater than the preset threshold, considering the inevitable inherent shortcomings of network data transmission, the preferred implementation scheme, such as steps 206b-208b, divides the target file into multiple file blocks and copies and transfers them to the new server in parallel.
[0088] 206a. If it is determined that the size of the target file is not greater than the preset threshold, the target file is migrated to the second server using the data synchronization tool Rclone according to the second storage path.
[0089] 206b. If it is determined that the size of the target file is greater than a preset threshold, the target file is divided into a plurality of file blocks according to a preset number, and a unique label is added to each file block.
[0090] 207b. According to the second storage path, multiple data synchronization tools Rclone are used to support the migration of each file block to the second server in parallel.
[0091] 208b. According to the labels corresponding to the file blocks, the file blocks migrated to the second server are spliced to obtain the target file.
[0092] In the embodiment of the present application, the above steps 206b-208b are adopted to divide the entire target file into multiple file blocks. Based on the working principle that one data synchronization tool Rclone can support the copying and transmission of one file block, multiple third-party servers are used to provide multiple data synchronization tools Rclone to achieve parallel support for the synchronous copying and transmission of multiple file blocks to the new server, thereby improving the processing efficiency of copying and transmitting the target file to the new server.
[0093] In addition, to ensure that the files copied and transferred on the new server are consistent with the target files, the embodiment of the present application also provides the following two verification schemes, as follows:
[0094] One verification scheme is: based on whether the labels between the file blocks on the second server are continuous, it is determined whether the target file migrated to the second server is complete.
[0095] Another verification scheme is: based on the labels corresponding to the file blocks on the second server, counting the number of labels to see whether it is consistent with a preset number, so as to determine whether the target file migrated to the second server is complete.
[0096] Based on the above two verification schemes, if the target file migrated to the second server is determined to be complete, the operation of copying and transferring the target file from the old server to the new server is determined to be successful. However, if the target file migrated to the second server is determined to be incomplete, the migration operation is determined to have failed, and a migration exception prompt message is triggered.
[0097] In addition, in the preferred implementation scheme, since the network needs to support the transmission of the target file between the first server and the second server, the security and integrity of the target file copy transmission (whether it has been tampered with or the file is missing) should be further considered. Accordingly, the embodiment of the present application also provides the following verification implementation scheme, as follows:
[0098] Before performing a file migration operation, first summary information corresponding to a target file stored on a first server is calculated; after performing the file migration operation to obtain a migration file to be verified, second summary information of the migration file stored on a second server is calculated; and based on comparing the second summary information with the first summary information to see whether they are consistent, it is verified whether the target file is completely migrated to the second server.
[0099] For example, a hash value algorithm is used to calculate summary information, and when the target file is copied and transmitted from the old server to the new server, the summary information calculated on the old server will also be carried. After the migration file to be verified is obtained on the new server, the summary information of the migration file is calculated for comparison. If the two are consistent, it indicates that the target file is copied and transmitted to the second server completely without tampering or loss.
[0100] Furthermore, as a response to the above Figure 1 , Figure 2In order to realize the method shown in the figure, the embodiment of the present application provides a file migration device. The device embodiment corresponds to the aforementioned method embodiment. For the sake of ease of reading, the present device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. The device is used to migrate the target file from the old server to the new server and update the storage path recorded in the system database at the same time. Specifically, Figure 3 As shown, the device comprises:
[0101] A first determining unit 31 is used to determine a target file to be migrated;
[0102] An acquiring unit 32 is configured to acquire a first storage path corresponding to the target file from a system database, wherein the first storage path is used to represent a current storage location of the target file in the first server;
[0103] A creating unit 33, configured to create a file directory on the second server to which the target file is migrated;
[0104] A second determining unit 34 is used to determine a second storage path required for migrating the target file according to the file directory;
[0105] A migration unit 35, configured to migrate the target file to the second server according to the second storage path;
[0106] The updating unit 36 is configured to update the first storage path recorded in the system database by using the second storage path to complete the migration operation on the target file.
[0107] Further, such as Figure 4 As shown, the migration unit 35 includes:
[0108] A judging module 351 is used to judge whether the size of the target file is greater than a preset threshold;
[0109] A migration module 352, configured to migrate the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path when it is determined that the size of the target file is greater than a preset threshold;
[0110] The migration module 352 also uses the data synchronization tool Rclone to migrate the target file to the second server according to the second storage path when it is determined that the size of the target file is not greater than a preset threshold.
[0111] Further, such as Figure 4 As shown, the migration module 352 includes:
[0112] The division submodule 3521 is used to divide the target file into a plurality of file blocks according to a preset number, and to add a unique label to each file block;
[0113] A processing submodule 3522 is used to support the migration of each file block to the second server in parallel by using a plurality of the data synchronization tools Rclone according to the second storage path;
[0114] The splicing submodule 3523 is used to splice the file blocks migrated to the second server to obtain the target file according to the labels corresponding to the file blocks.
[0115] Further, such as Figure 4 As shown, after the target file is obtained by splicing the file blocks migrated to the second server according to the labels corresponding to the file blocks, the migration module 352 further includes:
[0116] A check submodule 3524 is used to determine whether the target file migrated to the second server is complete based on whether the labels between the file blocks on the second server are continuous; and / or,
[0117] The check submodule 3524 is further used to count whether the number of the labels is consistent with the preset number based on the labels corresponding to the file blocks on the second server, so as to determine whether the target file migrated to the second server is complete.
[0118] Further, such as Figure 4 As shown, the migration module 352 also includes: an exception handling submodule 3525;
[0119] The exception handling submodule 3525 is used to determine that the migration operation fails and trigger a migration exception prompt message if it is determined that the target file migrated to the second server is incomplete.
[0120] Further, such as Figure 4 As shown, the device also includes:
[0121] A calculation unit 37, configured to calculate first summary information corresponding to the target file stored on the first server before performing the file migration operation;
[0122] The calculation unit 37 is further configured to calculate second summary information of the migration file stored on the second server after performing the file migration operation to obtain the migration file to be verified;
[0123] The verification unit 38 is configured to verify whether the target file is completely migrated to the second server based on comparing whether the second summary information is consistent with the first summary information.
[0124] In summary, the embodiment of the present application provides a file migration method and device. After determining the target file to be migrated, the embodiment of the present application provides two different migration schemes according to the size of the target file to improve the processing efficiency of file migration. After the target file is migrated from the old server to the new server, the present application also modifies the old storage path recorded in the system database to the new storage path at the same time, so that the system database can read the required file based on the new storage path when calling the target file demand. Compared with the prior art, the optimized solution for file migration provided by the present application ensures that no matter which new server the target file is migrated to, the target file can be successfully read according to the new storage path updated in the system database, so as not to affect the normal operation of the system.
[0125] The file migration device includes a processor and a memory. The first determination unit, acquisition unit, creation unit, second determination unit, migration unit and update unit are all stored in the memory as program units. The processor executes the program units stored in the memory to implement corresponding functions.
[0126] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to automatically complete the migration of the target file by using the new storage path of the target file on the new server while performing the migration of the target file to the new server environment, thereby ensuring that no matter which new server the target file is migrated to, the target file can be successfully read according to the new storage path updated in the system database, so as not to affect the normal operation of the system.
[0127] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the file migration method is implemented.
[0128] An embodiment of the present application provides a processor, which is used to run a program, wherein the file migration method is executed when the program is running.
[0129] The present application embodiment provides an electronic device 4, such as Figure 5 As shown, the device includes at least one processor 41, and at least one memory 42 and a bus 43 connected to the processor 41; wherein the processor 41 and the memory 42 communicate with each other through the bus 43; the processor 41 is used to call the program instructions in the memory 42 to execute the above-mentioned file migration method.
[0130] The device in this article can be a server, PC, etc.
[0131] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the steps of the above file migration method.
[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0133] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.
[0134] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
[0135] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0136] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0138] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A file migration method, It is characterized in that The method comprises: Determine the target files to be migrated; Acquire a first storage path corresponding to the target file from a system database, where the first storage path is used to represent a current storage location of the target file in the first server; Creating a file directory on the second server to which the target file is migrated; Determining, according to the file directory, a second storage path required for migrating the target file; The target file is migrated to the second server according to the second storage path and the first storage path recorded in the system database is updated using the second storage path to complete the migration operation of the target file.
2. The method according to claim 1, It is characterized in that The step of migrating the target file to the second server according to the second storage path includes: Determine whether the size of the target file is greater than a preset threshold; If yes, migrating the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path; If not, the target file is migrated to the second server using the data synchronization tool Rclone according to the second storage path.
3. The method according to claim 2, It is characterized in that The step of migrating the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path includes: Divide the target file into a plurality of file blocks according to a preset number, and add a unique label to each file block; According to the second storage path, using a plurality of the data synchronization tools Rclone to support the migration of each of the file blocks to the second server in parallel; According to the labels corresponding to the file blocks, the file blocks migrated to the second server are spliced to obtain the target file.
4. The method according to claim 3, It is characterized in that After the target file is obtained by splicing the file blocks migrated to the second server according to the labels corresponding to the file blocks, the method further includes: Based on whether the labels between the file blocks on the second server are continuous, determining whether the target file migrated to the second server is complete; and / or, Based on the labels corresponding to the respective file blocks on the second server, it is counted whether the number of the labels is consistent with the preset number, so as to determine whether the target file migrated to the second server is complete.
5. The method according to claim 4, It is characterized in that If it is determined that the target file to be migrated to the second server is incomplete, the migration operation is determined to have failed, and a migration abnormality prompt message is triggered.
6. The method according to any one of claims 1 to 5, It is characterized in that The method further comprises: Before performing the file migration operation, calculating first summary information corresponding to the target file stored on the first server; After performing the file migration operation to obtain the migration file to be verified, calculating second summary information of the migration file stored on the second server; Based on comparing whether the second summary information is consistent with the first summary information, verifying whether the target file is completely migrated to the second server.
7. A file migration device, It is characterized in that The device comprises: A first determining unit, used to determine a target file to be migrated; an acquisition unit, configured to acquire a first storage path corresponding to the target file from a system database, wherein the first storage path is used to represent a current storage location of the target file in the first server; A creating unit, used for creating a file directory on the second server to which the target file is migrated; A second determining unit, configured to determine, according to the file directory, a second storage path required for migrating the target file; a migration unit, configured to migrate the target file to the second server according to the second storage path; An updating unit is used to update the first storage path recorded in the system database by using the second storage path to complete the migration operation of the target file.
8. The device according to claim 7, It is characterized in that The migration unit comprises: A judging module, used to judge whether the size of the target file is greater than a preset threshold; a migration module, configured to migrate the target file to the second server based on transmitting the plurality of file blocks into which the target file is fragmented to the second server according to the second storage path when it is determined that the size of the target file is greater than a preset threshold; When it is determined that the size of the target file is not greater than a preset threshold, the migration module uses the data synchronization tool Rclone to migrate the target file to the second server according to the second storage path.
9. A storage medium, It is characterized in that The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the file migration method according to any one of claims 1 to 6.
10. An electronic device, It is characterized in that The device includes at least one processor, and at least one memory and a bus connected to the processor; Wherein, the processor and the memory communicate with each other through the bus; The processor is used to call the program instructions in the memory to execute the file migration method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Data migration and correction method and system
CN107037978A
Data migration method and device and medium
CN110019208A
Virtual data center backup method and device based on cloud management platform
CN112181720A
Data migration method, device, electronic equipment and readable storage medium
CN113836116A
File migration method and device, storage medium and electronic equipment
CN114461573A
