Method for migrating OSS from single cloud to multiple clouds

Abstract

The invention relates to the field of data migration, and discloses a method for migrating an OSS from a single cloud to multiple clouds, which is used for solving the problem that a migrated target cloud platform cannot be autonomously selected when the OSS is migrated from the single cloud to the multiple clouds, and comprises a migration method selection module, a multi-cloud environment selection module, a data preprocessing module and a data migration implementation module, the method comprises the following steps: selecting a migration method according to a data volume and a migration demand, obtaining a migration index through calculation, selecting a multi-cloud environment according to the migration index, cleaning and arranging data to be migrated, deleting unnecessary data and objects, classifying and marking the data, and migrating the data according to a formulated migration plan. The working content of developers is effectively reduced, the data migration efficiency is improved, and the data can be accurately migrated to the optimal cloud platform during multi-cloud migration.

Description

Technical Field

[0001] The present invention relates to the field of data migration, and more specifically to a method for migrating OSS from a single cloud to multiple clouds. Background Art

[0002] Migrating from a single cloud to multiple clouds refers to the process of migrating an enterprise's original applications and data from a single cloud platform to multiple cloud platforms. In a multi-cloud environment, enterprises can run and deploy applications on different cloud platforms, improve reliability and performance, and reduce risks and costs.

[0003] The existing migration method abstracts the object storage service interfaces and interface behaviors of different clouds, and developers make judgments and choices on which cloud to deploy the application. However, this migration method may lead to incorrect selection of the migration cloud platform due to human misjudgment, resulting in program instability and increased costs.

[0004] In view of the above problems, the present invention proposes a solution. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method for migrating OSS from a single cloud to multiple clouds to solve the problems existing in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for migrating OSS from a single cloud to multiple clouds includes a migration method selection module, a multi-cloud environment selection module, a data preprocessing module, and a data migration implementation module. The modules are connected through signals. The steps of data processing between the modules are as follows:

[0008] Migration method selection module, used to select a migration method based on data volume and migration requirements;

[0009] A multi-cloud environment selection module is used to obtain a migration index by calculation and select a multi-cloud environment according to the migration index;

[0010] Data preprocessing module, used to clean and organize the data to be migrated, delete unnecessary data and objects, and classify and label the data;

[0011] The data migration implementation module is used to migrate data according to the formulated migration plan, including migrating data from single-cloud OSS to multi-cloud OSS and configuring access control and security policies of multi-cloud OSS.

[0012] Preferably, the migration index calculation step is:

[0013] The staff collects statistics on all target cloud providers and the distance between the target cloud providers and the given data center.

[0014] The silhouette coefficient method is used to determine the number of clusters, and the K-means clustering method is used to perform clustering based on the distance between the target cloud provider and the given data center;

[0015] According to the final clusters obtained by clustering, the distance coefficient of the target cloud provider is calculated;

[0016] Use the cloud cost calculator to calculate the migration cost of the target cloud provider calculated by the staff and record it as the cost coefficient;

[0017] Obtain the storage capacity, data read / write speed, and throughput of the target cloud service provider through official documents and information provided by the target cloud service provider, and calculate the storage performance coefficient of the target cloud service provider through storage capacity, read / write speed, and throughput;

[0018] The migration index is calculated by weighted summation based on the distance coefficient, cost coefficient, and storage performance coefficient of the target cloud provider.

[0019] Preferably, the clustering steps using the K-means clustering method with the distance between the target cloud provider and the given data center as a characteristic are:

[0020] The distance between each target cloud provider and a given data center is used as a data point, and the silhouette coefficient method is used to determine the number of clusters;

[0021] According to the number of clusters determined by the silhouette coefficient method, randomly select data points as the initial cluster centers, use the Euclidean distance method to calculate the distance from each data point to the initial cluster center, and assign the data point to the cluster with the closest distance;

[0022] Calculate new cluster centers for the initial clusters;

[0023] Repeat the above clustering steps until the cluster centers of the obtained clusters no longer change significantly.

[0024] Preferably, the distance coefficient calculation step is to calculate the proportion of each cluster center in the sum of all cluster centers according to the cluster centers of the final cluster cluster obtained, as the weight coefficient of the cluster cluster, and record the weight coefficient as the distance coefficient.

[0025] Preferably, the multi-cloud environment is selected according to the migration index by sorting the target cloud providers counted by the staff from high to low according to their migration index, and selecting the target cloud providers from high to low according to the OSS migration requirements.

[0026] Preferably, the steps of cleaning and arranging the data to be migrated are:

[0027] Delete invalid data, correct errors, remove duplicates, and format data uniformly;

[0028] Determine how to map raw data in a multi-cloud environment;

[0029] Use data compression technology to compress data, use encryption technology to encrypt data, and back up the original data.

[0030] Preferably, the steps of configuring access control and security policy of multi-cloud OSS are:

[0031] Select the storage type and configure appropriate access control policies based on the importance and access frequency of the data;

[0032] Set up security authentication mechanisms, such as username and password, key pair, STS temporary authorization, change passwords or keys regularly, and enable two-factor authentication;

[0033] For sensitive data, use the data encryption service provided by OSS to encrypt and store the data;

[0034] Enable the OSS access log function to record and audit all access behaviors to OSS, helping to identify potential security risks and abnormal behaviors.

[0035] Configure regular backup and recovery strategies for data in Multi-Cloud OSS and store backup data on independent storage media.

[0036] Technical effects and advantages of the present invention:

[0037] The migration method is selected based on the data volume and migration requirements. The migration index is calculated and the multi-cloud environment is selected based on the migration index. The data to be migrated is cleaned and organized, unnecessary data and objects are deleted, and the data is classified and labeled. The data is migrated according to the established migration plan, which effectively reduces the workload of developers, improves the efficiency of data migration, and ensures that data can be accurately migrated to the optimal cloud during multi-cloud migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The figure is a flow chart of calculating the migration index of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are only examples. The method for migrating OSS from a single cloud to multiple clouds involved in the present invention is not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0040] The present invention provides a method for migrating OSS from a single cloud to multiple clouds, including a migration method selection module, a multi-cloud environment selection module, a data preprocessing module, and a data migration implementation module. The modules are connected by signals, and the steps of data processing between the modules are as follows:

[0041] Migration method selection module, used to select a migration method based on data volume and migration requirements;

[0042] A multi-cloud environment selection module is used to obtain a migration index by calculation and select a multi-cloud environment according to the migration index;

[0043] Data preprocessing module, which cleans and organizes the data to be migrated, removes unnecessary data and objects, and categorizes and tags the data for more efficient processing during the migration process;

[0044] The data migration implementation module is used to migrate data according to the formulated migration plan, including migrating data from single-cloud OSS to multi-cloud OSS and configuring access control and security policies of multi-cloud OSS;

[0045] In this embodiment, it should be specifically noted that the multi-cloud environment allows expansion on multiple cloud platforms according to business needs. By selecting an appropriate migration method, it is possible to ensure that OSS can be seamlessly expanded in a multi-cloud environment, thereby better supporting growing data needs. Multi-cloud storage solutions generally have higher availability and reliability because data can be redundantly stored across multiple cloud platforms. By selecting the correct migration method, high availability and reliability of data can be ensured, thereby improving the performance and stability of the application. By migrating data to a multi-cloud environment, it can be dynamically allocated between different cloud platforms according to business needs and resource utilization, which can better utilize cloud computing resources, optimize cost-effectiveness, and avoid over-investment or insufficient resources.

[0046] The steps of selecting the migration method are:

[0047] Choose a suitable multi-cloud strategy based on business needs and goals. If you need to store data on multiple cloud platforms, you can choose a federated cloud storage or distributed storage solution.

[0048] Understand the complexity of migration. For simple migration scenarios, you can directly export data from a single cloud to multiple clouds. However, for large-scale migrations or migrations involving complex data structures, more advanced migration methods may be required.

[0049] It is critical to ensure that the multi-cloud environment complies with relevant regulations and security standards. When choosing a migration approach, consider the needs for data encryption, access control, and auditing.

[0050] Develop a detailed migration plan based on the assessment results and business needs, including selecting appropriate migration tools, determining the migration schedule and resource requirements, and developing rules for data conversion and formatting;

[0051] Before the formal migration, the migration method should be fully tested and verified. Through testing, the stability of the migration process and the integrity of the data can be ensured, and potential problems can be discovered and resolved in a timely manner.

[0052] In this embodiment, it should be specifically explained that Figure 1 As shown, the steps for calculating the migration index are:

[0053] The staff collects statistics on all target cloud providers and the distance between the target cloud providers and the given data center.

[0054] The silhouette coefficient method is used to determine the number of clusters, and the K-means clustering method is used to perform clustering based on the distance between the target cloud provider and the given data center. The silhouette coefficient method is a method for evaluating clustering effects. It combines the two factors of cohesion and separation and can be used to evaluate the impact of different algorithms or different operating modes on clustering results based on the same original data. The K-means clustering method is a commonly used unsupervised learning algorithm for dividing a data set into K different clusters.

[0055] According to the final clusters obtained by clustering, the distance coefficient of the target cloud provider is calculated;

[0056] Using a cloud cost calculator to calculate the migration cost of the target cloud provider calculated by the staff, and recording it as a cost coefficient. The cloud cost calculator is a tool used to help users estimate and compare the total cost required to use cloud services, and can help users understand the infrastructure, storage, bandwidth and other related costs required to run a cloud environment;

[0057] Obtain the storage capacity, data read / write speed, and throughput of the target cloud service provider through official documents and information provided by the target cloud service provider, and calculate the storage performance coefficient of the target cloud service provider;

[0058] The migration index is calculated by weighted summation based on the distance coefficient, cost coefficient, and storage performance coefficient of the target cloud provider. The calculation formula is: Wherein, MG represents the migration index, JL represents the distance coefficient, the higher the distance coefficient is, the farther the distance between the target cloud provider and the given data center is, and data migration needs to be transmitted through the network. If the distance between the data center and the cloud provider is farther, the bandwidth required for data transmission will increase, and the delay may also increase. The migration index is lower, CO represents the cost coefficient, the higher the cost coefficient is, the higher the migration cost is, and the migration index is lower. ST represents the storage performance coefficient. Devices with higher storage performance coefficients usually have faster read and write speeds, which means that data can be migrated in a shorter time. The higher the storage performance coefficient is, the higher the migration index is. a1, a2, and a3 are represented as weight coefficients of the distance coefficient JL, the cost coefficient CO, and the storage performance coefficient ST, and this embodiment does not specifically calculate the specific values ​​of a1, a2, and a3.

[0059] The staff conducts statistics on all target cloud providers so that they understand the target cloud providers' technical architecture, platform functions, API interfaces, and compatibility with existing systems. This information helps to evaluate whether cloud services can meet business needs and whether technical transformation or customized development is required. The staff also understands the target cloud providers' customer support system, after-sales service, and technical training. During the implementation of the multi-cloud strategy, good customer support and after-sales service can provide necessary help and guarantees, thereby improving the reliability and success rate of the migration process.

[0060] The K-means clustering method is used to cluster the distance between the target cloud provider and the given data center as a feature, and the clustering steps are as follows:

[0061] The distance between each target cloud provider and a given data center is used as a data point, and the silhouette coefficient method is used to determine the number of clusters. For example, the number of clusters determined is three;

[0062] Randomly select three data points as the initial cluster centers, use the Euclidean distance method to calculate the distance from each data point to the initial cluster center, and assign the data point to the cluster with the closest distance. For example, if the data point data is [50, 100, 70, 80, 90, 40, 66, 110, 60], select [50, 100, 70] as the initial cluster center, and use the Euclidean distance method to calculate the distance from the data point

[80] to the initial cluster center. Then the data point

[80] is assigned to the initial cluster 3;

[0063] Calculate the new cluster center for the initial cluster. For example, the initial cluster 3 is [70, 80, 66]. Calculate its new cluster center using the following formula: Then 72 is the new cluster center of cluster 3;

[0064] Repeat the above clustering steps until the cluster centers of the obtained clusters no longer change significantly.

[0065] The distance coefficient calculation step is to calculate the distance coefficient according to the obtained cluster center of the final cluster. For example, if the final clusters are [50, 40, 60], [100, 110, 90], and [70, 80, 66], then the final cluster center is [50, 100, 66]. Then the weight coefficients JL of the three clusters are calculated as follows: And the weight coefficient is recorded as the distance coefficient.

[0066] The storage performance coefficient is calculated by collecting the storage capacity, data read and write speed, and throughput of the cloud service provider to calculate the storage performance coefficient. The calculation formula is: Among them, ST represents the storage performance coefficient, CT represents the storage capacity of the target cloud service provider, SP represents the read and write speed, and TG represents the throughput. The storage capacity, read and write speed, and throughput are all calculated in megabytes. The larger the coefficient, the better the storage performance.

[0067] Sort the migration index from high to low, and select the target cloud service provider according to the migration index from high to low based on the migration requirements.

[0068] In this embodiment, it should be specifically noted that when migrating large amounts of data, deleting unnecessary files and objects can significantly reduce the size of the data and the migration time. By deleting duplicate, erroneous or inaccurate data, the accuracy and reliability of the migrated data can be improved. Deleting unnecessary data can save storage space at the target location. By cleaning up unnecessary data, network bandwidth occupancy and transmission time during data migration can be reduced. In some cases, deleting sensitive or illegal data is necessary to ensure compliance with relevant regulations and policies during the migration process. By cleaning up and organizing data, the data organization efficiency at the target location and subsequent data maintenance work can be improved.

[0069] The steps for cleaning and organizing the data to be migrated are:

[0070] Delete invalid data, correct errors, and remove duplicates to ensure data quality and accuracy;

[0071] Format data uniformly for unified processing and analysis in multi-cloud environments;

[0072] Before migration, determine how to map the original data in the multi-cloud environment to achieve data consistency and integrity;

[0073] Use data compression technology to compress data in order to reduce migration time and network bandwidth usage;

[0074] Data is encrypted using encryption technology to prevent data leakage and unauthorized access, and original data is backed up to prevent data loss and damage.

[0075] In this embodiment, it should be specifically noted that by distributing data on multiple cloud platforms, the availability and reliability of data can be improved. Even if a cloud platform fails or a disaster occurs, other cloud platforms can still provide data access and services to ensure business continuity. Multi-cloud strategies can reduce the cost of data storage and processing. Multi-cloud environments can provide better scalability. As the amount of data and access volume grows, storage and processing capabilities can be easily expanded on different cloud platforms to meet growing demand. By distributing data on multiple cloud platforms, the risk of data leakage and single point failure can be reduced. In addition, a multi-cloud environment can provide more security options and encryption technologies to improve data security. A multi-cloud environment can provide more choices and flexibility.

[0076] Enterprises can choose the most suitable cloud platform and storage solution based on business needs and performance requirements, and easily migrate data and applications between different cloud platforms. A multi-cloud environment can promote innovation and competitiveness. By leveraging the advantages and special services of different cloud platforms, enterprises can develop, deploy and manage applications and services faster, thereby better meeting customer needs and improving competitiveness.

[0077] When developers inject the correct implementation, operation and maintenance personnel need to purchase machines from different cloud service providers to install the environment, write different environment variables, such as -Dcloud=tencent and -Dcloud=alibaba, and encapsulate SDKs for different cloud object storage services. In the SDK, the Tencent Cloud implementation is used for the environment variable -Dcloud=tencent, and the Alibaba Cloud implementation is used for -Dcloud=alibaba, and so on.

[0078] When running applications on multiple clouds, traffic can be switched arbitrarily. To avoid the problem of inconsistent object storage, operation and maintenance personnel can add several environment variables, such as -Dcloud=tencent_sync_alibaba and -Dcloud=alibaba_sync_huawei. When the SDK sets the environment variable to -Dcloud=tencent_sync_alibaba, the read strategy is to read Tencent Cloud. If it cannot be read, it reads Alibaba Cloud and writes to Tencent Cloud asynchronously. This method actually treats Tencent Cloud's object storage as the primary storage and Alibaba Cloud's object storage as the backup storage.

[0079] The steps for configuring access control and security policies for multi-cloud OSS are:

[0080] Choose the appropriate storage type based on the importance and access frequency of the data, such as standard storage, infrequent access storage, archive storage, and cold archive storage;

[0081] In multi-cloud OSS, appropriate access control policies need to be configured to control access rights to data;

[0082] Set up security authentication mechanisms, such as username and password, key pair, STS temporary authorization, change passwords or keys regularly, and enable two-factor authentication;

[0083] For sensitive data, use the data encryption service provided by OSS to encrypt and store the data to protect the security of the data during transmission and storage;

[0084] Enable the OSS access log function to record and audit all access behaviors to OSS, helping to identify potential security risks and abnormal behaviors.

[0085] Configure regular backup and recovery policies for data in multi-cloud OSS to prevent data loss or accidental deletion, and store backup data on independent storage media.

[0086] The standard storage, low-frequency access storage, archive storage, and cold archive storage are different types of data storage. Standard storage is suitable for frequently accessed data, such as application code, logs, pictures, audio, video, etc. Standard storage has high reliability and high performance, can quickly respond to user requests, and ensure data availability and integrity. At the same time, the price of standard storage is relatively high. Low-frequency access storage is suitable for business scenarios that require infrequent access but also require fast access to data when needed. Low-frequency access storage is suitable for scenarios that do not require high data access speed but need to be preserved for a long time. Archive storage is a form of data storage for infrequent access but needs to be preserved for a long time. The main purpose of archive storage is to provide enterprises with long-term data retention and compliance support by reducing storage costs. Archive storage is usually used to store backup data, old files, data archiving and other usage scenarios. It does not require high access timeliness but needs to be retained for a long time. Cold archive storage is the transfer of data to lower-cost storage media or systems, such as tapes, disks, or cloud storage services. Cold archive data is usually not accessed frequently, but can be restored when needed. Cold archive is a storage method for long-term data preservation, usually for archiving or backing up data. Compared with standard storage, cold archive has a lower access frequency, but the data retention time is longer.

[0087] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0088] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for migrating OSS from a single cloud to multiple clouds, characterized in that: It includes a migration method selection module, a multi-cloud environment selection module, a data preprocessing module, and a data migration implementation module. The modules are connected through signals. The steps of data processing between the modules are as follows: Migration method selection module, used to select a migration method based on data volume and migration requirements; A multi-cloud environment selection module is used to obtain a migration index by calculation and select a multi-cloud environment according to the migration index; Data preprocessing module, used to clean and organize the data to be migrated, delete unnecessary data and objects, and classify and label the data; The data migration implementation module is used to migrate data according to the formulated migration plan, including migrating data from single-cloud OSS to multi-cloud OSS and configuring access control and security policies of multi-cloud OSS.

2. The method for migrating OSS from a single cloud to multiple clouds according to claim 1, characterized in that: The steps for calculating the migration index are as follows: The staff collects statistics on all target cloud providers and the distance between the target cloud providers and the given data center. The silhouette coefficient method is used to determine the number of clusters, and the K-means clustering method is used to perform clustering based on the distance between the target cloud provider and the given data center; According to the final clusters obtained by clustering, the distance coefficient of the target cloud provider is calculated; Use the cloud cost calculator to calculate the migration cost of the target cloud provider calculated by the staff and record it as the cost coefficient; Obtain the storage capacity, data read / write speed, and throughput of the target cloud service provider through official documents and information provided by the target cloud service provider, and calculate the storage performance coefficient of the target cloud service provider through storage capacity, read / write speed, and throughput; The migration index is calculated by weighted summation based on the distance coefficient, cost coefficient, and storage performance coefficient of the target cloud provider.

3. The method for migrating OSS from a single cloud to multiple clouds according to claim 2, characterized in that: The K-means clustering method is used to cluster the distance between the target cloud provider and the given data center as a feature, and the clustering steps are as follows: The distance between each target cloud provider and a given data center is used as a data point, and the silhouette coefficient method is used to determine the number of clusters; According to the number of clusters determined by the silhouette coefficient method, randomly select data points as the initial cluster centers, use the Euclidean distance method to calculate the distance from each data point to the initial cluster center, and assign the data point to the cluster with the closest distance; Calculate new cluster centers for the initial clusters; Repeat the above clustering steps until the cluster centers of the obtained clusters no longer change significantly.

4. The method for migrating OSS from a single cloud to multiple clouds according to claim 2, characterized in that: The distance coefficient calculation step is to calculate the proportion of each cluster center in the sum of all cluster centers according to the cluster center of the final cluster cluster obtained, as the weight coefficient of the cluster cluster, and record the weight coefficient as the distance coefficient.

5. The method for migrating OSS from a single cloud to multiple clouds according to claim 1, characterized in that: The selection of the multi-cloud environment according to the migration index is to sort the target cloud providers counted by the staff from high to low according to their migration index, and select the target cloud providers from high to low according to the OSS migration requirements.

6. The method for migrating OSS from a single cloud to multiple clouds according to claim 1, characterized in that: The steps for cleaning and organizing the data to be migrated are: Delete invalid data, correct errors, remove duplicates, and format data uniformly; Determine how to map raw data in a multi-cloud environment; Use data compression technology to compress data, use encryption technology to encrypt data, and back up the original data.

7. The method for migrating OSS from a single cloud to multiple clouds according to claim 1, characterized in that: The steps for configuring access control and security policies for multi-cloud OSS are: Select the storage type and configure appropriate access control policies based on the importance and access frequency of the data; Set up security authentication mechanisms, such as username and password, key pair, STS temporary authorization, change passwords or keys regularly, and enable two-factor authentication; For sensitive data, use the data encryption service provided by OSS to encrypt and store the data; Enable the OSS access log function to record and audit all access behaviors to OSS, helping to identify potential security risks and abnormal behaviors. Configure regular backup and recovery strategies for data in Multi-Cloud OSS and store backup data on independent storage media.

Images