A file migration method, device, equipment and readable storage medium

Abstract

The application discloses a file migration method and device in the computer technology field, and a readable storage medium. The application can automatically construct a source end mounting command according to source access information of a source file system, and automatically execute the source end mounting command to realize automatic mounting of the source file system; automatically construct a destination end mounting command according to destination access information of a destination file system, and automatically execute the destination end mounting command to realize automatic mounting of the destination file system; and then decouple the scanning action and the migration action by using a scanning thread pool, a target queue and a migration thread, so that the migration efficiency can be improved, the scanning action and the migration action can be prevented from affecting each other during the migration process, and the migration success rate and reliability can be improved. The scheme reduces the complexity of the file migration operation, and reduces the cost and risk of manual intervention through automatic mounting, automatic scanning and automatic migration.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a file migration method, apparatus, device, and readable storage medium. Background Technology

[0002] Currently, NFS (Network File System) enables connection and access between business hosts and storage devices. In practical applications, as business needs change, it may be necessary to upgrade or replace old storage systems with new ones. Therefore, it is necessary to migrate the NFS file system from one storage device to another. This migration process usually requires a series of manual operations, such as mounting NFS shared paths. These operations are labor-intensive and time-consuming, and prone to errors, affecting the efficiency and reliability of file migration. NFS is a network protocol for file sharing.

[0003] Therefore, how to improve the efficiency and reliability of file migration is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a file migration method, apparatus, device, and readable storage medium to improve the efficiency and reliability of file migration. The specific solution is as follows:

[0005] Firstly, this application provides a document migration method, including:

[0006] Construct a source-end mount command based on the source access information of the source file system, and mount the source file system by executing the source-end mount command;

[0007] Construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command;

[0008] The source file system is scanned using a scanning thread pool. If it is determined that a scanned file does not exist in the destination file system, the scanned file is added to the target queue.

[0009] A migration thread is assigned to each file in the target queue, and the corresponding migration thread is used to migrate each file in the target queue to the destination file system.

[0010] Optionally, it also includes:

[0011] Obtain the source access information and the destination access information input by the user through the human-computer interaction interface.

[0012] Optionally, the step of using a scanning thread pool to scan the source file system, and if it is determined that a scanned file does not exist in the destination file system, adding the scanned file to the target queue, includes:

[0013] Select the target scan thread from the scan thread pool;

[0014] Determine the root directory of the files in the source file system;

[0015] The root directory of the file is determined as the directory to be scanned;

[0016] The target scanning thread is invoked to scan the directory to be scanned, and files in the directory to be scanned that are not present in the target file system are added to the target queue.

[0017] If the directory to be scanned has a subdirectory, then the subdirectory of the directory to be scanned is taken as the directory to be scanned. Another scanning thread is selected from the scanning thread pool as the target scanning thread, and the steps of calling the target scanning thread to scan the directory to be scanned and adding files in the directory to be scanned that are not present in the target file system to the target queue are executed until the directory to be scanned has no subdirectory.

[0018] Optionally, it also includes:

[0019] If the directory to be scanned does not have a subdirectory, a message indicating that the scan has ended will be displayed.

[0020] Optionally, the step of scanning the source file system using a scanning thread pool includes:

[0021] For the source file system, construct a directory tree according to the directory hierarchy;

[0022] A scanning thread is selected for each directory node in the directory tree from the scanning thread pool, and the corresponding scanning thread is used to scan each directory node in the directory tree.

[0023] Optionally, determining that the scanned file does not exist in the destination file system includes:

[0024] If the metadata of the scanned file is not stored in the destination file system, then it is determined that the scanned file is not stored in the destination file system.

[0025] Optionally, if the metadata of the scanned file is stored in the destination file system, then it is determined that the scanned file is stored in the destination file system.

[0026] Optionally, it also includes:

[0027] If it is determined that the scanned file exists in the target file system, then the scanned file is skipped and not added to the target queue.

[0028] Optionally, it also includes:

[0029] For each file, determine whether the full-text checksum of the current file matches the checksum of the corresponding file in the destination file system;

[0030] If they match, skip the current file and do not migrate the current file;

[0031] If there is a discrepancy, the current file will be migrated to the destination file system using the corresponding migration thread.

[0032] Optionally, it also includes:

[0033] Real-time monitoring of the current device's bandwidth utilization and / or processor resource utilization;

[0034] Adjust the number of migration threads based on bandwidth utilization and / or processor resource utilization.

[0035] Optionally, the number of migration threads can be adjusted based on bandwidth utilization and / or processor resource utilization, including:

[0036] If the bandwidth utilization and / or processor resource utilization exceed the preset threshold, the number of migration threads is increased; otherwise, the number of migration threads is decreased.

[0037] Secondly, this application provides a document migration apparatus, comprising:

[0038] The source-side mount module is used to construct a source-side mount command based on the source access information of the source file system, and mount the source file system by executing the source-side mount command;

[0039] The destination mount module is used to construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command;

[0040] The scanning module is used to scan the source file system using a scanning thread pool. If it is determined that the scanned file does not exist in the destination file system, the scanned file is added to the target queue.

[0041] The migration module is used to allocate a migration thread to each file in the target queue and use the corresponding migration thread to migrate each file in the target queue to the destination file system.

[0042] Optionally, it also includes:

[0043] The access information acquisition module is used to acquire the source access information and the destination access information input by the user based on the human-computer interaction interface.

[0044] Optionally, the scanning module is specifically used for:

[0045] Select the target scan thread from the scan thread pool;

[0046] Determine the root directory of the files in the source file system;

[0047] The root directory of the file is determined as the directory to be scanned;

[0048] The target scanning thread is invoked to scan the directory to be scanned, and files in the directory to be scanned that are not present in the target file system are added to the target queue.

[0049] If the directory to be scanned has a subdirectory, then the subdirectory of the directory to be scanned is taken as the directory to be scanned. Another scanning thread is selected from the scanning thread pool as the target scanning thread, and the steps of calling the target scanning thread to scan the directory to be scanned and adding files in the directory to be scanned that are not present in the target file system to the target queue are executed until the directory to be scanned has no subdirectory.

[0050] Optionally, the scanning module is specifically used for:

[0051] If the directory to be scanned does not have a subdirectory, a message indicating that the scan has ended will be displayed.

[0052] Optionally, the scanning module is specifically used for:

[0053] For the source file system, construct a directory tree according to the directory hierarchy;

[0054] A scanning thread is selected for each directory node in the directory tree from the scanning thread pool, and the corresponding scanning thread is used to scan each directory node in the directory tree.

[0055] Optionally, the scanning module is specifically used for:

[0056] If the metadata of the scanned file is not stored in the destination file system, then it is determined that the scanned file is not stored in the destination file system; if the metadata of the scanned file is stored in the destination file system, then it is determined that the scanned file is stored in the destination file system.

[0057] Optionally, the scanning module is further configured to:

[0058] If it is determined that the scanned file exists in the target file system, then the scanned file is skipped and not added to the target queue.

[0059] Optionally, the migration module is also used for:

[0060] For each file, determine whether the full-text checksum of the current file matches the checksum of the corresponding file in the destination file system; if they match, skip the current file and do not migrate it; if they do not match, use the corresponding migration thread to migrate the current file to the destination file system.

[0061] Optionally, it also includes:

[0062] The thread adjustment module is used to collect the current device's bandwidth utilization and / or processor resource utilization in real time; and adjust the number of migration threads based on the bandwidth utilization and / or processor resource utilization.

[0063] Optionally, the thread adjustment module is specifically used for:

[0064] If the bandwidth utilization and / or processor resource utilization exceed the preset threshold, the number of migration threads is increased; otherwise, the number of migration threads is decreased.

[0065] Thirdly, this application provides an electronic device, comprising:

[0066] Memory, used to store computer programs;

[0067] A processor for executing the computer program to implement the aforementioned disclosed file migration method.

[0068] Fourthly, this application provides a readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned disclosed file migration method.

[0069] As can be seen from the above scheme, this application provides a file migration method, including: constructing a source-end mount command based on the source access information of the source file system, and mounting the source file system by executing the source-end mount command; constructing a destination-end mount command based on the destination access information of the destination file system, and mounting the destination file system by executing the destination-end mount command; scanning the source file system using a scanning thread pool, and if it is determined that the scanned file does not exist in the destination file system, adding the scanned file to the target queue; allocating a migration thread for each file in the target queue, and using the corresponding migration thread to migrate each file in the target queue to the destination file system.

[0070] As can be seen, this application can automatically construct source-side mount commands based on the source file system's source access information and automatically execute these commands to automatically mount the source file system; similarly, it can automatically construct and execute destination-side mount commands based on the destination file system's destination access information to automatically mount the destination file system. Furthermore, by utilizing a scan thread pool, target queue, and migration thread, the scanning and migration actions are decoupled, which not only improves migration efficiency but also reduces the mutual interference between scanning and migration actions during the migration process, thereby increasing the migration success rate and reliability. This solution reduces the complexity of file migration operations through automatic mounting, automatic scanning, and automatic migration, and reduces the cost and risk of manual intervention.

[0071] Correspondingly, the file migration device, equipment, and readable storage medium provided in this application also have the above-mentioned technical effects. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0073] Figure 1 This is a flowchart of a document migration method disclosed in this application;

[0074] Figure 2 This is a schematic diagram of a migration system disclosed in this application;

[0075] Figure 3 This is a schematic diagram of a catalog scanning process disclosed in this application;

[0076] Figure 4 This is a schematic diagram illustrating the adjustment of the number of concurrent threads disclosed in this application;

[0077] Figure 5 This is a schematic diagram of a document migration device disclosed in this application;

[0078] Figure 6 This is a schematic diagram of an electronic device disclosed in this application;

[0079] Figure 7 A server architecture diagram provided in this application;

[0080] Figure 8 A terminal structure diagram provided for this application. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0082] Currently, migrating an NFS network file system from one storage device to another requires a series of manual operations, such as mounting NFS shared paths. These operations are time-consuming and labor-intensive, and prone to errors, affecting the efficiency and reliability of file migration. Therefore, this application provides a file migration solution that improves the efficiency and reliability of file migration.

[0083] See Figure 1 As shown in the figure, this application discloses a file migration method, including:

[0084] S101. Construct a source-end mount command based on the source access information of the source file system, and mount the source file system by executing the source-end mount command.

[0085] S102. Construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command.

[0086] In this embodiment, a human-computer interaction interface can be provided for user operation, allowing the user to input source access information of the source file system and destination access information of the destination file system through the human-computer interaction interface. Therefore, in one implementation, it further includes: obtaining the source access information and destination access information input by the user based on the human-computer interaction interface. The execution order of S101 and S102 can be interchanged, or they can be executed simultaneously. The source access information and destination access information may include: IP address, file storage path, etc.

[0087] S103. Use the scanning thread pool to scan the source file system. If it is determined that the scanned file does not exist in the destination file system, add the scanned file to the target queue.

[0088] In this embodiment, the scanning thread pool has multiple pre-set scanning threads. In one implementation, the scanning thread pool is used to scan the source file system. If it is determined that a scanned file does not exist in the destination file system, the scanned file is added to the target queue. This includes: selecting a target scanning thread in the scanning thread pool; determining the root directory of the file in the source file system; determining the root directory as the directory to be scanned; calling the target scanning thread to scan the directory to be scanned, and adding files in the directory to be scanned that do not exist in the destination file system to the target queue; if the directory to be scanned has a subdirectory, then the subdirectory of the directory to be scanned is taken as the directory to be scanned, another scanning thread is selected as the target scanning thread in the scanning thread pool, and the steps of calling the target scanning thread to scan the directory to be scanned and adding files in the directory to be scanned that do not exist in the destination file system to the target queue are executed, until the directory to be scanned has no subdirectory. If the directory to be scanned has no subdirectory, a prompt message indicating that the scan has ended is displayed.

[0089] In one implementation, scanning the source file system using a scanning thread pool includes: constructing a directory tree for the source file system according to the directory hierarchy; selecting a scanning thread for each directory node in the directory tree in the scanning thread pool, and using the corresponding scanning thread to scan each directory node in the directory tree.

[0090] This embodiment uses one scanning thread to scan one directory, and different scanning threads can scan different directories simultaneously, thus improving scanning efficiency. For each scanned file, it is first determined whether it exists in the target file system. If the scanned file already exists in the target file system, the scanned file is skipped and not added to the target queue. If the scanned file does not exist in the target file system, the scanned file is added to the target queue. The situation where a scanned file does not exist in the target file system includes two cases: (1) the scanned file does not exist in the target file system; (2) the scanned file exists in the target file system, but the file's metadata in the target file system is inconsistent with the scanned file's metadata. File metadata includes, for example, modification time and file size.

[0091] In one implementation, determining that a scanned file is not present in the destination file system includes: if the metadata of the scanned file is not present in the destination file system, then determining that the scanned file is not present in the destination file system. The situation where the metadata of the scanned file is not present in the destination file system includes two scenarios: (1) the metadata of the scanned file cannot be found in the destination file system; (2) the metadata of the scanned file can be found in the destination file system, but the metadata of that file in the destination file system is inconsistent with the metadata of the scanned file. File metadata includes, for example, modification time and file size.

[0092] Accordingly, if the metadata of the scanned file is stored in the destination file system, then it is determined that the scanned file is stored in the destination file system. In one embodiment, the method further includes: if it is determined that the scanned file is stored in the destination file system, then the scanned file is skipped and not added to the target queue.

[0093] S104. Assign a migration thread to each file in the target queue, and use the corresponding migration thread to migrate each file in the target queue to the destination file system.

[0094] This embodiment utilizes one migration thread to migrate one file, and different migration threads can migrate different files simultaneously, thus improving migration efficiency. In one implementation, the method further includes: for each file, determining whether the full-text checksum of the current file matches the checksum of the corresponding file in the destination file system; if they match, the current file is skipped and not migrated; if they do not match, the corresponding migration thread is used to migrate the current file to the destination file system.

[0095] In this embodiment, for each file to be migrated, it first determines whether its full-text checksum matches the corresponding file checksum in the destination file system. If the full-text checksum of the current file matches the corresponding file checksum in the destination file system, the current file is skipped and not migrated; if they do not match, the migration action is performed on the current file. By comparing the full-text checksums of the files, duplicate migrations can be avoided and migration can be ensured without omissions; compared with partial file checksums, full-text checksums are more reliable. The full-text checksum of a file is, for example, the hash value of the full text of the file. Among them, the inconsistency between the full-text checksum of the current file and the corresponding file checksum in the destination file system includes two situations: (1) the current file does not exist in the destination file system; (2) the current file exists in the destination file system, but the full-text checksum of the file in the destination file system is inconsistent with the full-text checksum of the file in the source file system.

[0096] In one implementation, the method further includes: real-time collection of the current device's bandwidth utilization and / or processor resource utilization; and adjusting the number of migration threads based on the bandwidth utilization and / or processor resource utilization. Processor resource utilization may include, for example, CPU utilization. The method can determine whether to adjust the number of migration threads simply by judging whether the bandwidth utilization or processor resource utilization exceeds limits, or it can perform a weighted calculation on the bandwidth utilization and processor resource utilization, and determine whether to adjust the number of migration threads based on the weighted calculation result.

[0097] In one example, the number of migration threads is adjusted based on bandwidth utilization and / or processor resource utilization, including: increasing the number of migration threads if bandwidth utilization and / or processor resource utilization exceed a preset threshold; otherwise, decreasing the number of migration threads.

[0098] As can be seen, this embodiment can automatically construct and execute source-side mount commands based on the source file system's source access information to automatically mount the source file system; similarly, it can automatically construct and execute destination-side mount commands based on the destination file system's destination access information to automatically mount the destination file system. Furthermore, by utilizing a scan thread pool, target queue, and migration thread, the scanning and migration actions are decoupled, which not only improves migration efficiency but also reduces the mutual interference between scanning and migration actions during the migration process, thereby increasing the migration success rate and reliability. This solution reduces the complexity of file migration operations through automatic mounting, automatic scanning, and automatic migration, thereby reducing the cost and risk of manual intervention.

[0099] The following section uses NFS file system migration as an example to illustrate the specific implementation of the inventive concept of this application. A migration system implemented with the inventive concept of this application is installed on any computer device. This migration system allows users to configure source and destination access information, scans source files to form a queue of files to be transferred, and migrates the files in the queue to the destination.

[0100] Please see Figure 2 The system internally encapsulates and integrates NFS shared directory mount commands. During migration, you only need to fill in the source and destination access information in the migration task configuration interface provided by the system. Subsequently, the migration system will automatically generate and execute NFS mount commands to mount the remote file directory to the local machine.

[0101] After the NFS file directory is mounted locally, multiple scanning threads (scanners) are started to scan from the root directory of the source file down level by level. When any file is encountered, it is first checked whether the file exists on the destination. If it does not exist, it is directly added to the transfer queue. If it already exists, it needs to be compared according to the checking rules to check whether the metadata (such as modification time, size, etc.) of the file on the source and destination are consistent. If they are consistent, it is skipped. If they are inconsistent, the file is added to the transfer queue.

[0102] The process of scanning the file directory level by level is as follows: Figure 3As shown in the diagram, a scanning thread first scans the root directory, obtaining several subdirectories and files. For each file, it first checks if it exists on the destination. If it doesn't exist, it's added directly to the transfer queue. If it already exists, the modification time and size of the source and destination files are checked and compared. If they match, the file is skipped; otherwise, it's added to the transfer queue. For each subdirectory, other idle scanning threads scan it, obtaining the next-level directories and files. Similarly, files are checked and compared before being added to the migration queue. The scanning of directories continues until all files and directories have been scanned.

[0103] During the scanning process, files in the transfer queue are migrated concurrently using multi-threading. A set of migration threads (workers) is pre-created, and a migration thread is assigned to each file in the queue in order, thus migrating files concurrently. Specifically, when a file needs to be migrated, the migration thread pool retrieves a file from the queue and assigns it to an idle migration thread, thereby migrating the file from the source storage device to the destination storage device. To ensure the consistency of migrated files, the hash verification of the same file at the source and destination is compared. If the two are inconsistent, the migration is performed, ensuring the integrity and consistency of the migrated files.

[0104] Throughout the migration process, scanning and migration threads run concurrently, with the number of scanning threads being twice the number of migration threads. This ensures that there are no idle migration threads before the scanning is complete, fully leveraging the advantages of multi-core CPUs to improve migration efficiency and speed.

[0105] In addition, the migration system includes an automatic adjustment mechanism for the migration thread pool, allowing the number of migration threads to be adjusted in real time based on system resource usage, thereby achieving higher migration efficiency. Please see [link / reference]. Figure 4 During the migration process, system resources are monitored, such as CPU core usage and migration network speed. When CPU utilization falls below the 80% threshold or migration network speed falls below the 80% bandwidth threshold, the maximum number of threads in the thread pool is dynamically adjusted, adding two migration threads at a time. When CPU utilization or migration network speed exceeds the 80% threshold, the maximum number of concurrent threads in the thread pool is dynamically reduced. This maximizes the utilization of system resources while improving the efficiency of file migration.

[0106] After all files have been migrated, the NFS unmount directory command will be executed automatically to cancel the mounts and port connections on both the source and destination sides without manual intervention.

[0107] This embodiment can automatically mount the source and destination directories, automatically scan and inspect files, and automatically migrate and transfer data. The access information for the source and destination directories is input by the user, greatly reducing the complexity of the data migration operation and saving manpower. Based on the access information provided by the user, the migration system can mount the shared file directories of the remote source and destination storage to the local directory of the server where the migration system resides. The file scanning module uses multi-threading to scan the source directory to obtain the files to be migrated and place them in a data queue to be transferred. Then, the migration task execution module starts multiple migration threads to migrate the files based on the data queue. After the data migration is complete, the migration system can also automatically unmount the local NFS shared directory.

[0108] As can be seen, this embodiment automatically mounts the NFS shared directories of the source and destination storage, eliminating the need for manual directory mounting pre-operations. During the migration process, a multi-threaded recursive scanning of the file directory and a multi-migration thread mechanism separate file transfer and file scanning, allowing them to proceed simultaneously, thus improving efficiency and performance. Furthermore, sharing the same transfer queue among multiple threads enables concurrent multi-threaded transmission, reducing waiting time and significantly increasing file transfer speed. The multi-threaded recursive scanning mechanism improves the scanning speed of massive file directories, and the use of a cache queue to store scanned files allows for simultaneous scanning and migration, further enhancing migration efficiency. After file migration is complete, the migration threads automatically perform data verification to ensure the integrity and reliability of the migrated data.

[0109] Furthermore, this embodiment employs a flexible concurrency strategy to control the number of concurrent threads. The number of concurrent migration threads is dynamically adjusted based on changes in system load and hardware resources, maximizing the utilization of system CPU resources to achieve higher migration efficiency and improve the efficiency and security of NFS file migration.

[0110] The following describes a file migration device provided by an embodiment of this application. The file migration device described below can be referred to in conjunction with other embodiments described herein.

[0111] See Figure 5 As shown in the figure, this application discloses a file migration device, including:

[0112] The source-side mount module 501 is used to construct a source-side mount command based on the source access information of the source file system, and mount the source file system by executing the source-side mount command;

[0113] The destination mount module 502 is used to construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command;

[0114] The scanning module 503 is used to scan the source file system using the scanning thread pool. If it is determined that the scanned file does not exist in the destination file system, the scanned file is added to the target queue.

[0115] Migration module 504 is used to allocate a migration thread to each file in the target queue and use the corresponding migration thread to migrate each file in the target queue to the destination file system.

[0116] In one implementation, it further includes:

[0117] The access information acquisition module is used to acquire source access information and destination access information input by the user based on the human-computer interaction interface.

[0118] In one implementation, the scanning module is specifically used for:

[0119] Select the target scan thread from the scan thread pool;

[0120] Determine the root directory of the file system in the source file system;

[0121] Set the root directory of the file as the directory to be scanned;

[0122] Call the target scanning thread to scan the directory to be scanned, and add the files in the directory to be scanned that do not exist in the target file system to the target queue;

[0123] If the directory to be scanned has a subdirectory, then the subdirectory of the directory to be scanned is taken as the directory to be scanned. Another scanning thread is selected from the scanning thread pool as the target scanning thread, and the target scanning thread is called to scan the directory to be scanned. Files in the directory to be scanned that do not exist in the target file system are added to the target queue, until the directory to be scanned has no subdirectories.

[0124] In one implementation, the scanning module is specifically used for:

[0125] If the directory to be scanned does not have a subdirectory, a message indicating that the scan has ended will be displayed.

[0126] In one implementation, the scanning module is specifically used for:

[0127] For the source file system, construct a directory tree according to the directory hierarchy;

[0128] Select a scanning thread for each directory node in the directory tree from the scanning thread pool, and use the corresponding scanning thread to scan each directory node in the directory tree.

[0129] In one implementation, the scanning module is specifically used for:

[0130] If the metadata of the scanned file is not stored in the destination file system, then it is determined that the scanned file is not stored in the destination file system; if the metadata of the scanned file is stored in the destination file system, then it is determined that the scanned file is stored in the destination file system.

[0131] In one implementation, the scanning module is further used for:

[0132] If it is determined that the scanned file exists in the target file system, then the scanned file is skipped and not added to the target queue.

[0133] In one implementation, the migration module is further configured to:

[0134] For each file, check if the full-text checksum of the current file matches the checksum of the corresponding file in the destination file system. If they match, skip the current file and do not migrate it. If they do not match, use the corresponding migration thread to migrate the current file to the destination file system.

[0135] In one implementation, it further includes:

[0136] The thread adjustment module is used to collect the current device's bandwidth utilization and / or processor resource utilization in real time; and adjust the number of migration threads based on the bandwidth utilization and / or processor resource utilization.

[0137] In one implementation, the thread adjustment module is specifically used for:

[0138] If the bandwidth utilization and / or processor resource utilization exceed the preset threshold, the number of migration threads is increased; otherwise, the number of migration threads is decreased.

[0139] For more detailed information on the working process of each module and unit in this embodiment, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0140] As can be seen, this embodiment decouples the scanning and migration actions, which not only improves migration efficiency but also reduces the mutual interference between the scanning and migration actions during the migration process, thereby increasing the migration success rate and reliability. This solution reduces the complexity of file migration operations through automatic mounting, automatic scanning, and automatic migration, minimizing the cost and risk of manual intervention.

[0141] The following describes an electronic device provided by an embodiment of this application. The electronic device described below can be referred to in conjunction with other embodiments described herein.

[0142] See Figure 6 As shown in the figure, an embodiment of this application discloses an electronic device, including:

[0143] Memory 601 is used to store computer programs;

[0144] Processor 602 is configured to execute the computer program to implement the method disclosed in any of the above embodiments.

[0145] Furthermore, embodiments of this application also provide an electronic device. The aforementioned electronic device can be, for example,... Figure 7 The server 50 shown can also be as follows: Figure 8 Terminal 60 is shown. Figure 7 and Figure 8 These are all diagrams illustrating the structure of an electronic device according to an exemplary embodiment. The content in the diagrams should not be considered as any limitation on the scope of this application.

[0146] Figure 7 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 50 may specifically include: at least one processor 51, at least one memory 52, a power supply 53, a communication interface 54, an input / output interface 55, and a communication bus 56. The memory 52 stores a computer program, which is loaded and executed by the processor 51 to implement the relevant steps in monitoring the published application disclosed in any of the foregoing embodiments.

[0147] In this embodiment, the power supply 53 is used to provide operating voltage for each hardware device on the server 50; the communication interface 54 can create a data transmission channel between the server 50 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 55 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0148] In addition, the memory 52, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include operating system 521, computer program 522, and data 523, etc., and the storage method can be temporary storage or permanent storage.

[0149] The operating system 521 manages and controls the various hardware devices on the server 50 and the computer program 522 to enable the processor 51 to perform operations and processing on the data 523 in the memory 52. ​​The operating system 521 can be Windows Server, Netware, Unix, Linux, etc. The computer program 522, in addition to including a computer program capable of performing the application distribution monitoring method disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 523 may include application update information and other data, as well as application developer information.

[0150] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal 60 may specifically include, but is not limited to, a smartphone, tablet computer, laptop computer, or desktop computer.

[0151] Typically, the terminal 60 in this embodiment includes a processor 61 and a memory 62.

[0152] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0153] The memory 62 may include one or more computer-readable storage media, which may be non-transitory. The memory 62 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 62 is used to store at least the following computer program 621, which, after being loaded and executed by the processor 61, is capable of implementing the relevant steps in the monitoring method for the published application executed on the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 62 may also include an operating system 622 and data 623, and the storage method may be temporary or permanent storage. The operating system 622 may include Windows, Unix, Linux, etc. The data 623 may include, but is not limited to, application update information.

[0154] In some embodiments, the terminal 60 may further include a display screen 63, an input / output interface 64, a communication interface 65, a sensor 66, a power supply 67, and a communication bus 68.

[0155] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on terminal 60 and may include more or fewer components than shown.

[0156] The following describes a readable storage medium provided in an embodiment of this application. The readable storage medium described below can be referred to in conjunction with other embodiments described herein.

[0157] A readable storage medium is provided for storing a computer program, wherein the computer program, when executed by a processor, implements the file migration method disclosed in the foregoing embodiments. The readable storage medium is a computer-readable storage medium, which serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon include an operating system, computer programs, and data, and the storage method can be temporary or permanent storage.

[0158] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0159] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of readable storage medium known in the art.

[0160] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A file migration method, characterized in that, include: Construct a source-end mount command based on the source access information of the source file system, and mount the source file system by executing the source-end mount command; Construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command; The source file system is scanned using a scanning thread pool. If it is determined that a scanned file does not exist in the destination file system, the scanned file is added to the target queue. Assign a migration thread to each file in the target queue, and use the corresponding migration thread to migrate each file in the target queue to the destination file system; The step of scanning the source file system using a scanning thread pool includes: For the source file system, construct a directory tree according to the directory hierarchy; A scanning thread is selected for each directory node in the directory tree from the scanning thread pool, and the corresponding scanning thread is used to scan each directory node in the directory tree; Specifically, for each file, it is determined whether the full-text checksum of the current file matches the checksum of the corresponding file in the destination file system; if they match, the current file is skipped and not migrated; if they do not match, the current file is migrated to the destination file system using the corresponding migration thread.

2. The method according to claim 1, characterized in that, Also includes: Obtain the source access information and the destination access information input by the user through the human-computer interaction interface.

3. The method according to claim 1, characterized in that, The step of scanning the source file system using a scanning thread pool, and if it is determined that a scanned file does not exist in the destination file system, then adding the scanned file to the target queue includes: Select the target scan thread from the scan thread pool; Determine the root directory of the files in the source file system; The root directory of the file is determined as the directory to be scanned; The target scanning thread is invoked to scan the directory to be scanned, and files in the directory to be scanned that are not present in the target file system are added to the target queue. If the directory to be scanned has a subdirectory, then the subdirectory of the directory to be scanned is taken as the directory to be scanned. Another scanning thread is selected from the scanning thread pool as the target scanning thread, and the steps of calling the target scanning thread to scan the directory to be scanned and adding files in the directory to be scanned that are not present in the target file system to the target queue are executed until the directory to be scanned has no subdirectory.

4. The method according to claim 3, characterized in that, Also includes: If the directory to be scanned does not have a subdirectory, a message indicating that the scan has ended will be displayed.

5. The method according to claim 1, characterized in that, The determination that the scanned file does not exist in the destination file system includes: If the metadata of the scanned file is not stored in the destination file system, then it is determined that the scanned file is not stored in the destination file system.

6. The method according to claim 5, characterized in that, If the metadata of the scanned file is stored in the destination file system, then it is determined that the scanned file is stored in the destination file system.

7. The method according to claim 1, characterized in that, Also includes: If it is determined that the scanned file exists in the target file system, then the scanned file is skipped and not added to the target queue.

8. The method according to any one of claims 1 to 7, characterized in that, Also includes: Real-time monitoring of the current device's bandwidth utilization and / or processor resource utilization; Adjust the number of migration threads based on bandwidth utilization and / or processor resource utilization.

9. A file migration device, characterized in that, include: The source-side mount module is used to construct a source-side mount command based on the source access information of the source file system, and mount the source file system by executing the source-side mount command; The destination mount module is used to construct a destination mount command based on the destination access information of the destination file system, and mount the destination file system by executing the destination mount command; The scanning module is used to scan the source file system using a scanning thread pool. If it is determined that the scanned file does not exist in the destination file system, the scanned file is added to the target queue. The migration module is used to allocate a migration thread to each file in the target queue and use the corresponding migration thread to migrate each file in the target queue to the destination file system; Specifically, the scanning module is used to: construct a directory tree according to the directory hierarchy for the source file system; select a scanning thread for each directory node in the directory tree from the scanning thread pool, and use the corresponding scanning thread to scan each directory node in the directory tree; The migration module is used to: for each file, determine whether the full-text checksum of the current file is consistent with the checksum of the corresponding file in the destination file system; if they are consistent, skip the current file and do not migrate the current file; if they are inconsistent, use the corresponding migration thread to migrate the current file to the destination file system.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images