Data migration method in multi-tenant mode based on MinIO technology

Abstract

The invention particularly relates to a data migration method in a multi-tenant mode based on a MinIO technology. According to the data migration method in the multi-tenant mode based on the MinIO technology, tenant space data needing to be migrated is backed up, and a container service CKE, a public component, an upper-layer data governance application module, a monitoring center and a message center for data governance are deployed and configured under a K8s cluster; data is exported from a source data end and imported into a target data end, and meanwhile, the data file importing progress is stored into a Redis database; and establishing a data migration retry mechanism, importing lost or inaccessible data, recovering the data to a corresponding position, verifying the migrated data, and ensuring that the tenant can normally use the service after data migration. According to the data migration method in the multi-tenant mode based on the MinIO technology, large-capacity data can be rapidly transmitted in a distributed environment, safety is high, good expandability and customizability are achieved, and different data processing requirements and scenes can be easily met.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a data migration method based on MinIO technology in a multi-tenant mode. Background Art

[0002] In the digital economy era, data migration has become a key step for enterprises to optimize their data architecture, improve data processing capabilities, and ensure business continuity and data security. Enterprises need to migrate data from one storage system to another for a variety of reasons, such as cost-effectiveness, performance improvement, system upgrade, centralized data management, or cloud service integration. The demand for customer data migration services continues to grow, which is closely related to the rapid development of technologies such as cloud computing, big data, artificial intelligence, and large models. Due to the large amount of business data and business processes in the existing systems, manual migration of multi-tenant spaces has many problems such as cumbersome migration, long cycle, data security and integrity.

[0003] In order to solve the above problems, the present invention proposes a data migration method in a multi-tenant mode based on MinIO technology. Summary of the invention

[0004] In order to make up for the defects of the prior art, the present invention provides a simple and efficient data migration method in a multi-tenant mode based on MinIO technology.

[0005] The present invention is achieved through the following technical solutions:

[0006] A data migration method in a multi-tenant mode based on MinIO technology, characterized in that it includes the following steps:

[0007] Step S1: Data backup

[0008] Before migration, back up the tenant space data that needs to be migrated to prevent data loss or damage during the migration process;

[0009] Step S2: Data migration

[0010] Deploy and configure the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster;

[0011] Export data from the source data end, process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database;

[0012] The target data end imports data, obtains file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database;

[0013] Step S3: Data recovery

[0014] Establish a data migration retry mechanism on the target data end and provide an exception log viewing and analysis tool. If lost or inaccessible data is found during the data import process, start the data migration retry mechanism, restore the imported data to the corresponding location, and make necessary configurations and adjustments.

[0015] Step S4: Data verification

[0016] Use business logic to verify whether the migrated data is complete, accurate, and consistent to ensure that tenants can use the service normally after data migration.

[0017] In the step S2, life cycle management LCM (Life Cycle Management), log monitoring and analysis system LMA, MinIO distributed object storage system and distributed key value storage system ETCD components are installed and deployed in the K8s cluster;

[0018] Public components include but are not limited to identity and access management IAM (Identity and Access Management), Nginx proxy server, API gateway APIG, open source relational database management system MariaDB and Elasticsearch engine;

[0019] The upper-level data governance application modules include but are not limited to the management center, data integration unit, data modeling unit, data quality unit, data labeling unit, data security unit, data service unit, and data visualization unit;

[0020] The management center is used to provide unified workspace, data resources, scheduling resources, role and functional authority management functions;

[0021] Container service CKE, common components, and upper-layer data governance application modules are all in HA high-availability mode to ensure the stability of the K8s cluster and maintain the continuity of platform services.

[0022] In step S2, when data is exported from the source data end, the steps are as follows:

[0023] Step S201: The data migration entry is the management center, and the workspace workSpaceId and the data migration module modelName to which data needs to be exported are determined;

[0024] Step S202: According to the batchID of this data migration and the parameters related to the tenant space ID, asynchronously call the data export interface of each data governance module, download the data files of all data export modules under the corresponding batch from the MinIO distributed object storage system in sequence, and compress the files downloaded from the MinIO distributed object storage system under the batch, and upload the compressed files to the storage bucket named after the management center in the MinIO distributed object storage system;

[0025] Generate a Key based on the batch ID and data export module name modelName in the tenant space, and store the data export file in the MinIO distributed object storage system; at the same time, cache this Key to the Redis server so that the data migration progress and exception log can be viewed in real time based on the Redis server Key;

[0026] In the step S202, the CompletableFuture.allOf() method is used to asynchronously call each data management module to export the data interface.

[0027] Step S203, loop through each data export module to export and return result sets;

[0028] If the data export module returns the result status as completed or failed, the data export result, data volume execCount, unique identification code docID (MinIO distributed object storage system returns a unique ID), export completion time and file path filePath information are stored in the database to achieve data migration management;

[0029] If the data export module returns the result status as in progress, the amount of data exported by the current data module execCount, the export message msg, the unique identification code docID (the MinIO distributed object storage system returns a unique ID) and the file path filePath are obtained from the Redis database according to the key value;

[0030] Step S204: store the data export progress, export data volume, export file name, MinIO distributed object storage system and daily log information into the database; obtain the data export progress from the data migration management list or Redis database according to the import progress.

[0031] In step S2, the target data end imports data, and the steps are as follows:

[0032] Step S211: The data migration entry is the management center, determine the workspace workSpaceId where data needs to be imported, and select the file to be imported;

[0033] Step S212: automatically parse the imported file during the file upload process, upload the parsed file to the storage bucket of the MinIO distributed object storage system, and return a unique identification code docID;

[0034] Step S213: after parsing the import file, asynchronously call the data import data interface of each data management module;

[0035] In the step S213, the CompletableFuture.allOf() method is used to asynchronously call each data management module to import the data interface.

[0036] Step S214, loop through each data export module to import data and return a result set;

[0037] If the data import module returns the result status as completed or failed, the data import result, data volume execCount, unique identification code docID (MinIO distributed object storage system returns a unique ID), import completion time and file path filePath information are stored in the database to achieve data migration management;

[0038] If the data import module returns the result status as in progress, the amount of data imported by the current data module execCount, the import message msg, the unique identification code docID (the MinIO distributed object storage system returns a unique ID) and the file path filePath are obtained from the Redis database according to the key value;

[0039] Step S215: store the data import progress, import data volume, import file name, MinIO distributed object storage system, and daily log information into the database; obtain the data import progress from the data migration management list or Redis database according to the import progress.

[0040] A data migration system based on MinIO technology in multi-tenant mode, characterized by:

[0041] The data backup module is responsible for backing up the tenant space data that needs to be migrated before migration to prevent data loss or damage during the migration process;

[0042] The data migration module is responsible for deploying and configuring the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster, and exporting data from the source data end and importing data to the target data end;

[0043] When exporting data from the source data end, first process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database;

[0044] When the target data end imports data, it obtains the file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database;

[0045] The data recovery module is responsible for establishing a data migration retry mechanism on the target data end and providing tools for viewing and analyzing abnormal logs. If lost or inaccessible data is found during the data import process, the data migration retry mechanism is activated to restore the imported data to the corresponding location and make necessary configurations and adjustments.

[0046] The data verification module is responsible for verifying whether the migrated data is complete, accurate, and consistent through business logic to ensure that tenants can use the service normally after data migration.

[0047] A data migration device in a multi-tenant mode based on MinIO technology, characterized in that it includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method when executing the computer program.

[0048] A readable storage medium, characterized in that: a computer program is stored on the readable storage medium, and the computer program implements the above method when executed by a processor.

[0049] The beneficial effects of the present invention are: the data migration method based on MinIO technology in multi-tenant mode can quickly transmit large-capacity data in a distributed environment, can store structured, unstructured and semi-structured data, and can also provide security features such as data encryption and access control, and obtain the migration progress of each module in real time. It has good scalability and customizability, and can easily adapt to different data processing needs and scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Attached Figure 1 Schematic diagram of the data export method of the present invention.

[0052] Attached Figure 2Schematic diagram of the data import method of the present invention. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0054] MinIO is an open source object storage service that is compatible with the Amazon S3 cloud storage service interface. It is suitable for storing large-capacity structured, unstructured and other types of data, and has the advantages of high performance, comprehensive SDK support, and comprehensive development documentation. Redis is a high-performance key-value database that supports master-slave synchronization.

[0055] Data migration between multi-tenant modes based on MinIO supports data export, data import and other capabilities. MinIO is used as an intermediate station for data security migration, and the data migration progress and daily logs are cached in Redis. The data migration progress can be viewed in real time to meet customer needs and business scenarios.

[0056] Data migration planning is an important step in ensuring that data is moved safely, completely, and efficiently from one system or storage device to another.

[0057] In order to achieve data migration across tenant spaces, back up tenant space data before data migration, arrange data migration time reasonably, reduce business interruption time, and do a good job in data security and privacy protection. This ensures that data is migrated from the existing system or storage device to the new system or storage device, while ensuring data integrity, security and business continuity.

[0058] The data migration method based on MinIO technology in multi-tenant mode includes the following steps:

[0059] Step S1: Data backup

[0060] Before migration, back up the tenant space data that needs to be migrated to prevent data loss or damage during the migration process;

[0061] Data backup refers to the regular copying of important data of an enterprise or organization to another storage medium to protect the data from accidental deletion, loss, damage or other disasters. The data that needs to be backed up includes important files, databases, applications and operating systems. The backup data should be stored in a safe and reliable place, and a quick recovery test method should be set up to ensure the effectiveness of the backup.

[0062] Step S2: Data migration

[0063] Deploy and configure the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster;

[0064] Export data from the source data end, process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database;

[0065] The target data end imports data, obtains file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database;

[0066] In step S2, MinIO and Redis clusters are built and deployed under the K8s cluster, including the installation and deployment of life cycle management LCM (Life Cycle Management), log monitoring and analysis system LMA, MinIO distributed object storage system and distributed key-value storage system ETCD components;

[0067] Public components include but are not limited to identity and access management IAM (Identity and Access Management), Nginx proxy server, API gateway APIG, open source relational database management system MariaDB and Elasticsearch engine;

[0068] The upper-level data governance application modules include but are not limited to the management center, data integration unit, data modeling unit, data quality unit, data labeling unit, data security unit, data service unit, and data visualization unit;

[0069] The management center is used to provide unified workspace, data resources, scheduling resources, role and functional authority management functions;

[0070] Container service CKE, common components, and upper-layer data governance application modules are all in HA high-availability mode to ensure the stability of the K8s cluster and maintain the continuity of platform services.

[0071] In step S2, when data is exported from the source data end, the steps are as follows:

[0072] Step S201: The data migration entry is the management center, and the workspace workSpaceId and the data migration module modelName to which data needs to be exported are determined;

[0073] Step S202: According to the batchID of this data migration and the parameters related to the tenant space ID, asynchronously call the data export interface of each data governance module, download the data files of all data export modules under the corresponding batch from the MinIO distributed object storage system in sequence, and compress the files downloaded from the MinIO distributed object storage system under the batch, and upload the compressed files to the storage bucket named after the management center in the MinIO distributed object storage system;

[0074] Generate a Key based on the batch ID and data export module name modelName in the tenant space, and store the data export file in the MinIO distributed object storage system; at the same time, cache this Key to the Redis server so that the data migration progress and exception log can be viewed in real time based on the Redis server Key;

[0075] In the step S202, the CompletableFuture.allOf() method is used to asynchronously call each data management module to export the data interface.

[0076] The following is the core code for asynchronous concurrent calls:

[0077] for(String service:serviceList){

[0078] CompletableFuture <ResponseBeanDTO <object>>

[0079] future=schedulerTaskService.getExportExecCount(service,token,workspaceId,batchId);

[0080] futures.add(future);

[0081] data.put(service,future);

[0082] }

[0083] CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();

[0084] Among them, serviceList is the set of modules to be migrated, getExportExecCount is the core business logic exported by each module, and CompletableFuture.allOf(f1,f2,f3) is completed only when all CompletableFutures are completed.

[0085] Step S203, loop through each data export module to export and return result sets;

[0086] Query the current data migration progress from the data migration management progress table. If the data export module returns the result status as completed or failed, store the data export result, data volume execCount, unique identification code docID (MinIO distributed object storage system returns a unique ID), export completion time and file path filePath information in the database to implement data migration management;

[0087] If the data export module returns the result status as in progress, the amount of data exported by the current data module execCount, the export message msg, the unique identification code docID (the MinIO distributed object storage system returns a unique ID) and the file path filePath are obtained from the Redis database according to the key value;

[0088] Step S204: store the data export progress, export data volume, export file name, MinIO distributed object storage system and daily log information into the database; obtain the data export progress from the data migration management list or Redis database according to the import progress.

[0089] In step S2, the target data end imports data, and the steps are as follows:

[0090] Step S211: The data migration portal is the management center. Determine the workspace workSpaceId (i.e., tenant space) where data needs to be imported, and select the file to be imported.

[0091] Step S212, data migration in multi-tenant mode, in order to ensure the isolation of multi-tenant resources in the cluster, the data resources under each tenant are different, and the data import file and export file are automatically parsed for data migration.

[0092] During the upload process, the imported file is automatically parsed and uploaded to the bucket of the MinIO distributed object storage system, returning a unique identification code docID.

[0093] Step S213: after parsing the import file, asynchronously call the data import data interface of each data management module;

[0094] In the step S213, the CompletableFuture.allOf() method is used to asynchronously call each data management module to import the data interface.

[0095] The following is the core code for asynchronous concurrent calls:

[0096] for(String serviceType:fileTypelist){

[0097] transferDetailWithBLOBs.setId(UUIDTool.generateMixString(32));

[0098] transferDetailWithBLOBs.setBatchId(batchId);

[0099] transferDetailWithBLOBs.setServiceType(serviceType);

[0100] transferDetailWithBLOBs.setCreator(AuthenticationProvider.getUserId());

[0101] transferDetailWithBLOBs.setCreateTime(DateUtil.getCurrentDateTime());

[0102] transferDetailService.insert(transferDetailWithBLOBs);

[0103] if(serviceType.contains(ServiceTypeEnum.DATAMODEL.getCode())){

[0104] CompletableFuture<ResponseBeanDTO <object>>datamodel=schedulerTaskService.getImportExecCount(ServiceTypeEnum.DATAMODEL.getCode(),fileP ath,workSpaceId,batchId,importInfo,token);

[0105] futures.add(datamodel);

[0106] }

[0107] iParam = new HashMap();

[0108] }

[0109] / / Asynchronous call to join each module

[0110] CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();

[0111] Among them, fileTypelist is the set of data modules to be imported, getImportExecCount is the core business logic imported by each module, and CompletableFuture.allOf(f1,f2,f3) can only be completed when all CompletableFutures are completed.

[0112] Step S214, loop through each data export module to import data and return a result set;

[0113] During the file upload process, the files containing the module name in the import file are looped through, and the module files are uploaded one by one, and other automatic parsing processes are performed.

[0114] If the data import module returns the result status as completed or failed, the data import result, data volume execCount, unique identification code docID (MinIO distributed object storage system returns a unique ID), import completion time and file path filePath information are stored in the database to achieve data migration management;

[0115] If the data import module returns the result status as in progress, the amount of data imported by the current data module execCount, the import message msg, the unique identification code docID (the MinIO distributed object storage system returns a unique ID) and the file path filePath are obtained from the Redis database according to the key value;

[0116] Step S215: store the data import progress, import data volume, import file name, MinIO distributed object storage system, and daily log information into the database; obtain the data import progress from the data migration management list or Redis database according to the import progress.

[0117] Step S3: Data recovery

[0118] Establish a data migration retry mechanism on the target data end and provide an exception log viewing and analysis tool. If lost or inaccessible data is found during the data import process, start the data migration retry mechanism, restore the imported data to the corresponding location, and make necessary configurations and adjustments.

[0119] Step S4: Data verification

[0120] Use business logic to verify whether the migrated data is complete, accurate, and consistent to ensure that tenants can use the service normally after data migration.

[0121] The data migration system based on the MinIO technology multi-tenant mode is characterized by:

[0122] The data backup module is responsible for backing up the tenant space data that needs to be migrated before migration to prevent data loss or damage during the migration process;

[0123] The data migration module is responsible for deploying and configuring the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster, and exporting data from the source data end and importing data to the target data end;

[0124] When exporting data from the source data end, first process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database;

[0125] When the target data end imports data, it obtains the file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database;

[0126] The data recovery module is responsible for establishing a data migration retry mechanism on the target data end and providing tools for viewing and analyzing abnormal logs. If lost or inaccessible data is found during the data import process, the data migration retry mechanism is activated to restore the imported data to the corresponding location and make necessary configurations and adjustments.

[0127] The data verification module is responsible for verifying whether the migrated data is complete, accurate, and consistent through business logic to ensure that tenants can use the service normally after data migration.

[0128] The data migration device in the multi-tenant mode based on MinIO technology includes a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the above method when executing the computer program.

[0129] The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0130] Compared with the existing technology, this data migration method based on MinIO technology in multi-tenant mode has the following characteristics:

[0131] First, based on MinIO's high-performance distributed object storage service technology, it can achieve distributed high-availability HA deployment.

[0132] Second, it provides an error correction code mechanism to ensure the integrity of storage resources and has high availability and reliability.

[0133] Third, it supports the migration of data files in multiple batches and multiple modules across tenant spaces, and can support the data export and import processing of all or part of the modules in a custom tenant space.

[0134] Fourth, provide a customized mode for a batch data export module under a tenant, and the data import file can be automatically parsed according to the module name.

[0135] Fifth, data migration progress and results can be obtained through the intelligent combination of Redis database and local database, and data migration progress and log information can be viewed in real time.

[0136] The embodiment described above is only one specific implementation of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.< / object> < / object>

Claims

1. A data migration method based on MinIO technology in multi-tenant mode, characterized by: The following steps are involved: Step S1: Data backup Before migration, back up the tenant space data that needs to be migrated to prevent data loss or damage during the migration process; Step S2: Data migration Deploy and configure the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster; Export data from the source data end, process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database; The target data end imports data, obtains file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database; Step S3: Data recovery Establish a data migration retry mechanism on the target data end and provide an exception log viewing and analysis tool. If lost or inaccessible data is found during the data import process, start the data migration retry mechanism, restore the imported data to the corresponding location, and make necessary configurations and adjustments. Step S4: Data verification Use business logic to verify whether the migrated data is complete, accurate, and consistent to ensure that tenants can use the service normally after data migration.

2. According to the data migration method based on MinIO technology multi-tenant mode in claim 1, it is characterized by: In the step S2, lifecycle management LCM, log monitoring and analysis system LMA, MinIO distributed object storage system and distributed key-value storage system ETCD components are installed and deployed in the K8s cluster; Common components include but are not limited to identity and access management IAM, Nginx proxy server, API gateway APIG, open source relational database management system MariaDB and Elasticsearch engine; The upper-level data governance application modules include but are not limited to the management center, data integration unit, data modeling unit, data quality unit, data labeling unit, data security unit, data service unit, and data visualization unit; The management center is used to provide unified workspace, data resources, scheduling resources, role and functional authority management functions; Container service CKE, common components, and upper-layer data governance application modules are all in HA high-availability mode to ensure the stability of the K8s cluster and maintain the continuity of platform services.

3. According to claim 1, the data migration method based on MinIO technology in multi-tenant mode is characterized in that: In step S2, when data is exported from the source data end, the steps are as follows: Step S201: The data migration entry is the management center, and the workspace workSpaceId and the data migration module modelName to which data needs to be exported are determined; Step S202: According to the batchID of this data migration and the parameters related to the tenant space ID, asynchronously call the data export interface of each data governance module, download the data files of all data export modules under the corresponding batch from the MinIO distributed object storage system in sequence, and compress the files downloaded from the MinIO distributed object storage system under the batch, and upload the compressed files to the storage bucket named after the management center in the MinIO distributed object storage system; Generate a Key based on the batch ID and data export module name modelName in the tenant space, and store the data export file in the MinIO distributed object storage system; at the same time, cache this Key to the Redis server so that the data migration progress and exception log can be viewed in real time based on the Redis server Key; Step S203, loop through each data export module to export and return result sets; If the data export module returns a result status of completed or failed, the data export result, data volume execCount, unique identification code docID, export completion time and file path filePath information are stored in the database to implement data migration management; If the data export module returns the result status as in progress, the amount of data exported by the current data module execCount, the export message msg, the unique identification code docID and the file path filePath are obtained from the Redis database according to the key value; Step S204: store the data export progress, export data volume, export file name, MinIO distributed object storage system and daily log information into the database; obtain the data export progress from the data migration management list or Redis database according to the import progress.

4. According to claim 3, the data migration method based on MinIO technology in multi-tenant mode is characterized in that: In the step S202, the CompletableFuture.allOf() method is used to asynchronously call each data management module to export the data interface.

5. According to the data migration method based on MinIO technology multi-tenant mode in claim 1, it is characterized by: In step S2, the target data end imports data, and the steps are as follows: Step S211: The data migration entry is the management center, determine the workspace workSpaceId where data needs to be imported, and select the file to be imported; Step S212: automatically parse the imported file during the file upload process, upload the parsed file to the storage bucket of the MinIO distributed object storage system, and return a unique identification code docID; Step S213: after parsing the import file, asynchronously call the data import data interface of each data management module; Step S214, loop through each data export module to import data and return a result set; If the data import module returns a result status of completed or failed, the data import result, data volume execCount, unique identification code docID, import completion time and file path filePath information are stored in the database to implement data migration management; If the data import module returns the result status as in progress, the amount of data imported by the current data module execCount, the import message msg, the unique identification code docID and the file path filePath are obtained from the Redis database according to the key value; Step S215: store the data import progress, import data volume, import file name, MinIO distributed object storage system, and daily log information into the database; obtain the data import progress from the data migration management list or Redis database according to the import progress.

6. According to claim 5, the data migration method based on MinIO technology in multi-tenant mode is characterized in that: In the step S213, the CompletableFuture.allOf() method is used to asynchronously call each data management module to import the data interface.

7. A data migration system based on MinIO technology in multi-tenant mode, characterized by: include: The data backup module is responsible for backing up the tenant space data that needs to be migrated before migration to prevent data loss or damage during the migration process; The data migration module is responsible for deploying and configuring the data governance container service CKE, common components, upper-layer data governance application modules, monitoring center, and message center in the K8s cluster, and exporting data from the source data end and importing data to the target data end; When exporting data from the source data end, first process the data export file, compress the exported data file and upload it to the MinIO distributed object storage system, and store the data file export progress in the Redis database; When the target data end imports data, it obtains the file information in the MinIO distributed object storage system according to the unique identification code docID of the imported file, and stores the data file import progress in the Redis database; The data recovery module is responsible for establishing a data migration retry mechanism on the target data end and providing tools for viewing and analyzing abnormal logs. If lost or inaccessible data is found during the data import process, the data migration retry mechanism is activated to restore the imported data to the corresponding location and make necessary configurations and adjustments. The data verification module is responsible for verifying whether the migrated data is complete, accurate, and consistent through business logic to ensure that tenants can use the service normally after data migration.

8. A data migration device based on MinIO technology in multi-tenant mode, characterized by: The method comprises a memory and a processor; the memory is used to store a computer program, and the processor is used to implement the method according to any one of claims 1 to 6 when executing the computer program.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images