Data migration method and apparatus, electronic device, storage medium, and program product

Abstract

The present disclosure relates to the technical field of data storage, and provides a data migration method and apparatus, an electronic device, a storage medium, and a program product. The method comprises: first, monitoring a file manifest in an object storage service of source data to determine an incremental file index, wherein the file manifest comprises file identifiers of files stored by the object storage service, and the incremental file index comprises a file identifier of an incremental file in the file manifest; and then, reading incremental data corresponding to the file identifier of the incremental file from a source data address, and writing the incremental data into a destination data address. In the present embodiment, by means of monitoring a file manifest, incremental data can be quickly identified and migrated to a destination data address, thereby reducing data migration time and meeting a high timeliness requirement for data migration.

Description

Data migration methods, apparatus, electronic devices, storage media and software products

[0001] This disclosure claims priority to Chinese Patent Application No. 202411845705.3, filed on December 13, 2024, entitled “Data Migration Method, Apparatus, Electronic Device, Storage Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of data storage technology, and in particular to a data migration method, apparatus, electronic device, storage medium, and program product. Background Technology

[0003] Data migration refers to the process of moving data from one storage system or platform to another. This process can involve different types of data sources and destinations, such as migrating from a local server to cloud storage, from one cloud storage service to another, or from an old system to a new system. To ensure the continuous availability of the migration service, incremental data from the source needs to be migrated to the destination as quickly as possible.

[0004] In related technologies, multiple full migrations are performed, with each migration involving a full scan and migration of all files. However, to migrate a small amount of incremental data, a full scan of the source data is required, which is time-consuming and unsuitable for scenarios with large amounts of source data and high timeliness requirements. Summary of the Invention

[0005] This disclosure provides a data migration method, apparatus, electronic device, storage medium, and program product to shorten data migration time and meet the high timeliness requirements of data migration.

[0006] In a first aspect, embodiments of this disclosure provide a data migration method, comprising: monitoring a file list in an object storage service of source data; determining an incremental file index, wherein the file list includes file identifiers of files stored in the object storage service; the incremental file index includes file identifiers of incremental files in the file list; reading incremental data corresponding to the file identifiers of incremental files from the source data address; and writing the incremental data to the destination data address.

[0007] Secondly, embodiments of this disclosure provide a data migration apparatus, comprising: a monitoring module for monitoring a file list in an object storage service for source data, determining an incremental file index, wherein the file list includes file identifiers of files stored in the object storage service; and the incremental file index includes file identifiers of incremental files in the file list; and a migration module for reading incremental data corresponding to the file identifiers of incremental files from the source data address and writing it to the destination data address.

[0008] Thirdly, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods described above when executing the computer program.

[0009] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0010] Fifthly, embodiments of this disclosure provide a computer program product, which includes a computer program that, when executed by a processor, implements any of the methods described above.

[0011] This disclosure has the following advantages:

[0012] This disclosure provides a data migration method, apparatus, electronic device, storage medium, and program product. First, it monitors a file list in the object storage service of the source data to determine an incremental file index. The file list includes file identifiers of files stored in the object storage service; the incremental file index includes file identifiers of incremental files in the file list. Then, it reads the incremental data corresponding to the file identifiers of the incremental files from the source data address and writes the incremental data to the destination data address. In this embodiment, by monitoring the file list, incremental data can be quickly identified and migrated to the destination data address, reducing data migration time and meeting the high timeliness requirements of data migration.

[0013] The above description is only an overview of the technical solution of this disclosure. In order to better understand the technical means of this disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other objects, features and advantages of this disclosure more obvious and understandable, specific embodiments of this disclosure are given below. Attached Figure Description

[0014] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0015] Figure 1 is a schematic diagram of an application scenario of a data migration method according to an embodiment of the present disclosure.

[0016] Figure 2 is a flowchart of a data migration method according to an embodiment of the present disclosure.

[0017] Figure 3 is a flowchart of a data migration method according to an embodiment of the present disclosure.

[0018] Figure 4 is a structural block diagram of a data migration apparatus according to an embodiment of the present disclosure.

[0019] Figure 5 is a block diagram of an electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0020] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this disclosure. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0021] To facilitate understanding of the technical solutions of the embodiments of this disclosure, the related technologies of the embodiments of this disclosure are described below. The following related technologies are optional solutions and can be combined with the technical solutions of the embodiments of this disclosure in any way, and all of them fall within the protection scope of the embodiments of this disclosure.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0023] To facilitate rapid migration of incremental data, related technologies often involve creating message queues. For each incremental file, a migration service subscribes to and consumes these messages to perform the data migration. However, relying on message queues for data migration presents several challenges: Firstly, it limits application scenarios. If the source object storage service does not support event notifications, incremental messages cannot be generated, hindering data migration. Secondly, it poses data risks, as event notifications provided by the source object storage service are often unreliable, leading to message loss and potentially missed migration data.

[0024] The data migration method provided in this disclosure first monitors the file list in the object storage service of the source data to determine the incremental file index. The file list includes file identifiers of files stored in the object storage service; the incremental file index includes file identifiers of incremental files in the file list. Then, the incremental data corresponding to the file identifiers of the incremental files is read from the source data address and written to the destination data address. In this disclosure, by monitoring the file list, incremental data can be quickly identified and migrated to the destination data address, reducing data migration time and meeting the high timeliness requirements of data migration. Moreover, the object storage service supports generating file lists, preventing data omissions and improving the reliability and practicality of data migration.

[0025] Figure 1 is a schematic diagram of an application scenario of the data migration method provided in this disclosure. As shown in Figure 1, the data migration method in this embodiment is implemented through a migration service, in which a migration task is created and executed to achieve data migration from the source data address to the destination data address.

[0026] The object storage service shown in Figure 1 is the object storage service for the source data. The object storage service can generate a file inventory. Bucket A is the storage space for the source data, and `inventory_id / ` is an identifier used to uniquely identify the file inventory. The file inventory includes a `manifest.json` file and files in the `data / ` directory. According to user configuration, the required file inventory can be generated at each preset time period. The `manifest.json` file shown in Figure 1 is a summary file of the file inventory, recording the inventory file list and inventory metadata. The inventory file list includes the bucket identifier and the identifier of the stored file (e.g., object key name), and the inventory metadata includes information describing the rows and columns in the inventory file list. The directories `2024-09-01T16-00Z / ` and `2024-09-01T18-00Z / ` each correspond to a `manifest.json` file, representing a summary file of the file inventory generated at different times. The files `1.csv.gv`, `2.csv.gv`, ... stored in the `data / ` directory represent the metadata of the inventory files in the inventory file list.

[0027] Configure the source and destination data addresses. Configuring the source data address includes: configuring the storage space for the source data to be migrated, as shown in Figure 1, where the source data storage space is Bucket A; configuring the storage address for the file manifest, i.e., the storage space and directory where the file manifest is located. `inventory_Prefix` is a configuration option used to specify the location of the generated file manifest, allowing the migration service to monitor the file manifest at that location. Configuring the destination data address includes: configuring the destination storage space to be migrated, as shown in Figure 1, where the destination storage space is Bucket B.

[0028] The migration service monitors the file list in the object storage service, specifically monitoring newly added list files. It locates the file list based on `inventory_Prefix`, identifies the file identifiers of the newly added list files, and if there are multiple newly added list files, it increments the file identifiers according to their creation time to obtain an incremental file index, as shown in Figure 1: 1a.txt, 2b.txt, 3c.txt, 4d.txt, 5e.txt. For the current migration task, the migration service enumerates files starting from the current location (e.g., starting from 1.a.txt). Based on the file identifiers 1.a.txt, 2.b.txt…5.e.txt in the incremental file index, it reads the incremental data corresponding to each file identifier from Bucket A and writes it to the destination data address, i.e., the destination storage space is Bucket B, thus realizing the incremental data migration from Bucket A to Bucket B.

[0029] In this embodiment of the disclosure, by monitoring a file list, incremental data can be quickly identified and migrated to the destination data address, avoiding the time-consuming problem of full migration, reducing data migration time, and meeting the high timeliness requirements of data migration. Furthermore, the object storage service supports generating file lists, preventing data omissions and improving the reliability and practicality of data migration.

[0030] This disclosure provides a data migration method. The method in this embodiment can be applied to servers, terminal devices, platforms, devices, etc., which have computing and processing capabilities. The server can be a server cluster or a single server, a server deployed in the cloud, or a local server.

[0031] Figure 2 shows a flowchart of a data migration method according to an embodiment of this disclosure, including:

[0032] Step S201: Monitor the file list in the object storage service of the source data, determine the incremental file index, the file list includes the file identifier of the file stored in the object storage service; the incremental file index includes the file identifier of the incremental file in the file list.

[0033] The source data is stored using object storage services, which store data as objects. Object storage is a storage architecture that stores data as individual objects and accesses these objects through identifiers (e.g., object keys). Object storage can store and retrieve unstructured data, such as text files, images, videos, and audio. Object storage services also have a file inventory generation function, which allows you to identify newly added files by monitoring the file inventory.

[0034] The file identifiers of files stored in the object storage service can be object key names. Incremental file indexes are determined based on the file identifiers of incremental files in the file list.

[0035] Step S202: Read the incremental data corresponding to the file identifier of the incremental file from the source data address, and write the incremental data to the destination data address.

[0036] Obtain the source data address and destination data address. Based on the file identifier of the incremental file, query the data to be migrated from the source data address, i.e., the incremental file. Read the incremental file and write it to the destination data address to realize the migration of incremental data.

[0037] The data migration method provided in this embodiment first monitors the file list in the object storage service of the source data to determine the incremental file index. The file list includes file identifiers of files stored in the object storage service; the incremental file index includes file identifiers of incremental files in the file list. Then, the incremental data corresponding to the file identifiers of the incremental files is read from the source data address and written to the destination data address. In this embodiment, by monitoring the file list, incremental data can be quickly identified and migrated to the destination data address, reducing data migration time and meeting the high timeliness requirements of data migration.

[0038] The following describes the specific implementation process of each step above through various implementation methods:

[0039] In one implementation, before determining the incremental file index from the file list in the object storage service for monitoring source data, the data migration method further includes: configuring the storage address of the configuration file list; and determining the incremental file index from the file list in the object storage service for monitoring source data, including: checking the file list in the storage address according to a preset time period to determine the file identifier of the incremental file; and determining the incremental file index based on the file identifier of the incremental file.

[0040] In practical applications, the storage address of the configuration file list is pre-defined. The file list is checked at this address at a preset time interval to see if any new files have been added. If new files are found, their file identifiers are determined, and these identifiers are used to determine the incremental file index. The file identifier can be a filename, etc.

[0041] Multiple file lists can be configured with different storage addresses according to different data migration needs. The target storage address is determined according to the user's selection instructions, and the file list at the target storage address is monitored.

[0042] In this embodiment, by periodically monitoring the file list through the storage address of the configuration file list, the incremental files can be accurately identified, avoiding the problem of missed migration data that can easily occur when monitoring new files through message queues.

[0043] In one implementation, there are multiple incremental files. Based on the file identifiers of the incremental files, the incremental file index is determined, including: incrementing the file identifiers of the multiple incremental files according to their file generation time to obtain the incremental file index.

[0044] In practical applications, each newly added manifest file is assigned an incrementing number. For example, the first file is numbered 1, the second file is numbered 2, and so on. Information such as the file number, file identifier, and generation time of each newly added file is recorded in an index file.

[0045] In this embodiment, the incrementing number provides a unique identifier for each file, allowing different files to be distinguished even if they have the same filename. Furthermore, the incrementing number arranges the files in chronological order of their creation, facilitating the tracking of the file creation sequence and timeline.

[0046] In one implementation, reading incremental data corresponding to the file identifier of the incremental file from the source data address and writing the incremental data to the destination data address includes: reading incremental data from the source data address in ascending order of the file identifier numbers of the incremental files, recording the read numbers, and writing the incremental data to the destination data address.

[0047] For example, start reading from file number 1, then read file number 2, and so on, recording the reading numbers, reading multiple incremental files, and writing them to the destination data address.

[0048] In this embodiment, incremental numbering allows for easy identification of newly added files since the last synchronization, enabling efficient incremental synchronization and reducing unnecessary full scans. Furthermore, if the migration process is interrupted, synchronization can resume using the last recorded number without restarting the entire process. Incremental numbering avoids repeatedly processing already migrated files, improving overall migration performance.

[0049] In one implementation, reading incremental data corresponding to the file identifier of the incremental file from the source data address and writing the incremental data to the destination data address includes: obtaining the file identifier of a preset category file from the incremental data, reading the data corresponding to the file identifier of the preset category file from the source data address, and writing it to the destination data address.

[0050] The incremental files can include files of multiple categories. These categories can be based on different services, different projects, or different data types. During data migration, not all newly added files can be migrated; instead, files of preset categories can be migrated according to specific needs, thereby meeting different data migration requirements.

[0051] In one implementation, before reading the incremental data corresponding to the file identifier of the incremental file from the source data address and writing it to the destination data address, the data migration method further includes configuring the source data address and the destination data address.

[0052] The source data address can be the storage space for the source data, and the destination data address can be the storage space for the destination data, such as a storage bucket. The source data address and destination data address can be pre-configured. After a new file is detected, the data is read directly from the configured source data address and written to the configured destination data address.

[0053] Optionally, multiple source data addresses and multiple destination data addresses can be pre-configured, and users can select from these multiple source data addresses and multiple destination data addresses according to their selection instructions.

[0054] This disclosure provides a data migration method. The method in this embodiment can be applied to servers, terminal devices, platforms, devices, etc., which have computing and processing capabilities. The server can be a server cluster or a single server, a server deployed in the cloud, or a local server.

[0055] Figure 3 shows a flowchart of a data migration method according to an embodiment of this disclosure, including:

[0056] Step S301: Configure the storage address, source data address, and destination data address of the configuration list.

[0057] The storage address, source data address, and destination data address of the file list can include storage space, such as a storage bucket.

[0058] Step S302: Check the file list in the storage address according to the preset time period to determine the file identifier of the incremental file. The file list includes the file identifier of the file stored in the object storage service.

[0059] The storage address of the configuration file list is pre-defined. The file list is checked in the storage address at a preset time period to check if there are any newly added files. If there are newly added files, the file identifier of the incremental file is determined. The file identifier can be a file name, etc.

[0060] Step S303: The file identifiers of multiple incremental files are incremented according to the file generation time to obtain the incremental file index.

[0061] Incrementing numbers provide a unique identifier for each file, allowing different files to be distinguished even if they have the same filename. Furthermore, incrementing numbers arrange files in chronological order of their creation, facilitating the tracking of file creation sequence and timelines.

[0062] Step S304: Read incremental data from the source data address in ascending order of the file identifier numbers of the incremental files in the incremental file index, record the read numbers, and write the incremental data to the destination data address.

[0063] For example, start reading from file number 1, then read file number 2, and so on, recording the reading numbers, reading multiple incremental files, and writing them to the destination data address.

[0064] Optionally, the file identifier of a preset category file can be obtained from the incremental data, the data corresponding to the file identifier of the preset category file can be read from the source data address, and written to the destination data address.

[0065] The incremental files can include files of multiple categories. These categories can be based on different services, different projects, or different data types. During data migration, not all newly added files can be migrated; instead, files of preset categories can be migrated according to specific needs, thereby meeting different data migration requirements.

[0066] In this embodiment of the disclosure, by monitoring a file list, incremental data can be quickly identified and migrated to the destination data address, reducing data migration time and meeting the high timeliness requirements of data migration. Furthermore, the object storage service supports generating file lists, preventing data omissions and improving the reliability and practicality of data migration.

[0067] Corresponding to the application scenarios and methods provided in the embodiments of this disclosure, the embodiments of this disclosure also provide a data migration apparatus. Figure 4 shows a structural block diagram of a data migration apparatus according to an embodiment of this disclosure. The apparatus includes:

[0068] Monitoring module 401 is used to monitor the file list in the object storage service of the source data, determine the incremental file index, the file list includes the file identifier of the file stored in the object storage service; the incremental file index includes the file identifier of the incremental file in the file list;

[0069] The migration module 402 is used to read the incremental data corresponding to the file identifier of the incremental file from the source data address and write it to the destination data address.

[0070] The data migration apparatus provided in this embodiment first monitors the file list in the object storage service of the source data to determine the incremental file index. The file list includes file identifiers of files stored in the object storage service; the incremental file index includes file identifiers of incremental files in the file list. Then, the incremental data corresponding to the file identifiers of the incremental files is read from the source data address and written to the destination data address. In this embodiment, by monitoring the file list, incremental data can be quickly identified and migrated to the destination data address, reducing data migration time and meeting the high timeliness requirements of data migration.

[0071] In one implementation, the data migration device is further configured to: configure the storage address of the file list in the object storage service for monitoring source data before determining the incremental file index; the monitoring module 401 is configured to: check the file list in the storage address according to a preset time period to determine the file identifier of the incremental file; and determine the incremental file index based on the file identifier of the incremental file.

[0072] In one implementation, there are multiple incremental files. When the monitoring module 401 determines the incremental file index based on the file identifier of the incremental file, it is used to: increment the file identifiers of the multiple incremental files according to the file generation time to obtain the incremental file index.

[0073] In one implementation, the migration module 402 is used to: read incremental data from the source data address in ascending order of the file identifier number of the incremental file, record the read number, and write the incremental data to the destination data address.

[0074] In one implementation, the migration module 402 is used to: obtain the file identifier of a preset category file from the incremental data, read the data corresponding to the file identifier of the preset category file from the source data address, and write it to the destination data address.

[0075] In one implementation, the data migration device is further configured to: configure the source data address and the destination data address before reading the incremental data corresponding to the file identifier of the incremental file from the source data address and writing it to the destination data address.

[0076] The functions of each module in this embodiment can be found in the corresponding descriptions in the above methods, and they have corresponding beneficial effects, which will not be repeated here.

[0077] Figure 5 is a block diagram of an electronic device used to implement embodiments of the present disclosure. As shown in Figure 5, the electronic device includes a memory 510 and a processor 520. The memory 510 stores a computer program that can run on the processor 520. When the processor 520 executes the computer program, it implements the methods described in the above embodiments. The number of memories 510 and processors 520 can be one or more.

[0078] The electronic device also includes:

[0079] The communication interface 530 is used to communicate with external devices and exchange and transmit data.

[0080] If the memory 510, processor 520, and communication interface 530 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 5, but this does not indicate that there is only one bus or one type of bus.

[0081] Optionally, in a specific implementation, if the memory 510, processor 520, and communication interface 530 are integrated on a single chip, then the memory 510, processor 520, and communication interface 530 can communicate with each other through an internal interface.

[0082] This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods provided in this disclosure.

[0083] This disclosure provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in this disclosure.

[0084] This disclosure also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the methods provided in this disclosure.

[0085] This disclosure also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0086] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0087] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0088] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0089] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0090] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0091] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0092] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0093] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0094] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0095] The above description is merely an exemplary embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this disclosure, and these should all be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

A data migration method, characterized in that, The method includes: The monitoring source data includes a file list in the object storage service, which determines an incremental file index. The file list includes file identifiers of the files stored in the object storage service, and the incremental file index includes file identifiers of the incremental files in the file list. Read the incremental data corresponding to the file identifier of the incremental file from the source data address, and write the incremental data to the destination data address. The method according to claim 1, characterized in that, Before determining the incremental file index in the file inventory of the monitored source data object storage service, the method further includes: Configure the storage address for the file list; The file list in the object storage service containing the monitored source data is used to determine the incremental file index, including: The file list in the storage address is checked according to a preset time period to determine the file identifier of the incremental file; The incremental file index is determined based on the file identifier of the incremental file. The method according to claim 2, characterized in that, The number of incremental files is multiple, and the process of determining the incremental file index based on the file identifier of the incremental file includes: The file identifiers of the multiple incremental files are incremented according to the file generation time to obtain the incremental file index. The method according to claim 3, characterized in that, The step of reading the incremental data corresponding to the file identifier of the incremental file from the source data address and writing the incremental data to the destination data address includes: The incremental data is read from the source data address in ascending order of the file identifier numbers of the incremental files, and the read number is recorded. The incremental data is then written to the destination data address. The method according to claim 4, characterized in that, The method further includes: If an interruption occurs during the process of writing the incremental data to the destination data address, the data migration will continue according to the last recorded number. The method according to claim 1, characterized in that, The incremental files include files of multiple categories, which are categorized according to any of the following: services, projects, or data types. The method according to any one of claims 1-4, characterized in that, The step of reading the incremental data corresponding to the file identifier of the incremental file from the source data address and writing the incremental data to the destination data address includes: Obtain the file identifier of the preset category file from the incremental data, read the data corresponding to the file identifier of the preset category file from the source data address, and write it to the destination data address. The method according to any one of claims 1-4, characterized in that, Before reading the incremental data corresponding to the file identifier of the incremental file from the source data address and writing it to the destination data address, the method further includes: Configure the source data address and the destination data address. The method according to claim 1, characterized in that, Prior to monitoring the file inventory in the object storage service that contains the source data, the method further includes: Configure different storage addresses for multiple lists of files; The file list in the object storage service for the monitored source data includes: The target storage address is determined from the different storage addresses according to the user's selection instructions, and the file list at the target storage address is monitored. A data migration device, characterized in that, The device includes: The monitoring module is used to monitor the file list in the object storage service of the source data, determine the incremental file index, wherein the file list includes the file identifier of the file stored in the object storage service; and the incremental file index includes the file identifier of the incremental file in the file list. The migration module is used to read the incremental data corresponding to the file identifier of the incremental file from the source data address and write it to the destination data address. The apparatus according to claim 10, characterized in that, The device is also used for: Before determining the incremental file index in the file list in the object storage service of the monitored source data, configure the storage address of the file list; The monitoring module is used for: The file list in the storage address is checked according to a preset time period to determine the file identifier of the incremental file; The incremental file index is determined based on the file identifier of the incremental file. The apparatus according to claim 11, characterized in that, The number of incremental files is multiple, and the monitoring module is used for: The file identifiers of the multiple incremental files are incremented according to the file generation time to obtain the incremental file index. The apparatus according to claim 12 is characterized in that, The migration module is used for: The incremental data is read from the source data address in ascending order of the file identifier numbers of the incremental files, and the read number is recorded. The incremental data is then written to the destination data address. The apparatus according to claim 13 is characterized in that, The device is also used for: If an interruption occurs during the process of writing the incremental data to the destination data address, the data migration will continue according to the last recorded number. The apparatus according to claim 10, characterized in that, The incremental files include files of multiple categories, which are categorized according to any of the following: services, projects, or data types. The apparatus according to any one of claims 10-13 is characterized in that, The migration module is used for: Obtain the file identifier of the preset category file from the incremental data, read the data corresponding to the file identifier of the preset category file from the source data address, and write it to the destination data address. The apparatus according to any one of claims 10-13 is characterized in that, The device is also used for: Before monitoring the file inventory in the object storage service for the source data, configure different storage addresses for multiple file inventories; The monitoring module is used for: The target storage address is determined from the different storage addresses according to the user's selection instructions, and the file list at the target storage address is monitored. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1-9.

Images