Dynamic migration method for cloud platform database

Abstract

The invention discloses a dynamic migration method for a cloud platform database, which relates to the technical field of database migration and comprises the following steps of: substituting the size of collected data, access frequency and residual storage space into a dynamic migration priority algorithm unit of the cloud platform database to calculate a migration priority index of each database; and then sorting the databases from high to low according to the migration priorities, preferentially selecting a part of databases with high migration priorities as the first batch of migration objects according to actual demands and resource limiting factors, and carrying out migration. The method has the advantages that the multiple databases are sequenced according to the priorities, and the dynamic corresponding migration is performed according to the sequencing of the priorities, so that the operation efficiency is improved, and the user cost is reduced; in addition, the cost in the migration process can be visually displayed and substituted into calculation to continuously and dynamically optimize the migration method so as to improve the applicability of cloud platform database migration, and the overall method has the advantages of being efficient and flexible.

Description

Technical Field

[0001] The present invention relates to the technical field of database migration, and particularly to a method for dynamically migrating a cloud platform database. Background Technique

[0002] With the rapid development of technology, cloud platform databases have become important tools for data storage and processing. With the continuous expansion of databases and the growth of data, cloud platform databases are facing increasing challenges. When the database of a certain server overflows, it is necessary to migrate the database. Due to the complexity of the data, when performing data migration, multiple databases need to be migrated.

[0003] However, the existing database migration methods currently in use have poor migration efficiency during implementation, often requiring a large amount of time and resources for data migration, and unable to make dynamic optimizations in cloud platform database migration. They can only migrate multiple databases according to set indicators, and thus have a poor level of intelligence. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for dynamically migrating a cloud platform database, which solves the problems raised in the above background technique.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A method for dynamically migrating a cloud platform database, including the following steps: Data collection: Collect the data size, access frequency, remaining storage space, and network bandwidth values of each database; Dynamic migration: Specifically including: First, preset the weight factor A1, and then substitute the collected data size, access frequency, and remaining storage space into the cloud platform database dynamic migration priority algorithm unit to calculate the migration priority index of each database; Next, sort the databases according to the migration priority from high to low. According to actual requirements and resource limitation factors, preferentially select a part of the databases with high migration priority as the first batch of migration objects and perform migration; After that, substitute the migration priority index of the database and the data size of the database into the cloud platform database dynamic migration cost evaluation algorithm unit to calculate the migration cost value of the corresponding database; And, substitute the migration priority index of the database and the database migration cost value into the database post-migration performance cost algorithm unit for operation to calculate the performance value after database migration; Finally, the calculated performance value after database migration is the growth value of A1 in the cloud platform database dynamic migration priority algorithm unit. This value is fed back for calculation to adjust the value of A1. Through iteration, it continuously affects the migration priority index of each database and the migration cost value of the database until all databases are migrated; Summary output: When all database migrations are completed, count the migration time and cost index, and then organize the migration results and the values during the migration process into a report and upload it to the control terminal for users to view.

[0006] Optionally, a data acquisition module, a computer, a terminal output module, a monitoring and management unit, a database storage service device, a temporary data storage and backup unit, and a database dynamic migration reaction module are adopted; The data acquisition module is used to collect and organize the data size, access frequency, remaining storage space, and network bandwidth values of i databases, and upload the data to the computer. The database dynamic migration reaction module in the computer is used to calculate the collected database information, calculate the corresponding priority index of the database in turn, and sort multiple databases from high to low according to the priority relationship. After the sorting is completed, the database with a higher priority is migrated first, and the performance data after migration is collected after one migration to calculate the migration cost value of the database. The performance value after database migration is calculated together with the migration cost value and the migration priority index of the database. Targeted migration is carried out through the priority, and calculating through the migration cost value can facilitate quickly understanding the economic cost and corresponding improvement and decision-making. The performance value after database migration can intuitively reflect the effect and efficiency of migration, and this performance value can be fed back to the calculation process of the database priority index to continuously iterate and optimize, thereby continuously affecting the output value of the database dynamic migration reaction module to reflect the purpose of dynamic migration; The database dynamic migration reaction module includes: a cloud platform database dynamic migration priority algorithm unit, a cloud platform database dynamic migration cost evaluation algorithm unit, and a database post-migration performance cost algorithm unit.

[0007] Optionally, the cloud platform database dynamic migration priority algorithm unit is as follows: Set the base value of the weight factor A1 to A3; ; ; Where: is the migration priority index of the i-th database; is the size value of the i-th database; is the access frequency value of the i-th database; is the remaining space of the cloud storage device storing the i-th database; X is the weight of the database size value; A2 is calculated by the performance cost algorithm unit after database migration and is used to dynamically adjust the incremental value of the weight factor A1; and are the weight factors of the access frequency value and the remaining space respectively.

[0008] Optionally, the cloud platform database dynamic migration cost evaluation algorithm unit is as follows: ; where: is the migration cost value of the i-th database; is the comprehensive index value of the i-th database size priority; is the network bandwidth value for migrating the i-th database; is the fixed migration cost value, and the migration cost includes: labor cost and downtime.

[0009] Optionally, the performance cost algorithm unit after database migration is as follows: ; where: A2 is the performance value after database migration and is also used as the feedback value in the calculation process; is the performance value after the i-th database migration, and the performance value is the corresponding time after the i-th database migration; At the first loop after initial setting, A3 is A1, and A3 is the base value of A1.

[0010] Optionally, the monitoring and management unit acts on the database dynamic migration reaction module to monitor the migration progress and resource consumption in real time during the database migration process, and display the monitored values to the user through the computer and the terminal output module.

[0011] Optionally, the database storage service device is used to store and manage the database to be migrated and improve the data access function, and store the data used by the database dynamic migration reaction module.

[0012] Optionally, the temporary data storage and backup unit is used to store the temporary data and log files during the migration process to ensure the integrity and recoverability of the data.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: Through the migration priority algorithm unit, the present invention can prioritize the sorting of multiple databases, accurately calculate based on the actual size value, access frequency value and remaining space information, and then quickly calculate the priority of each database, sort them, and migrate and perform corresponding processing on the database with a high priority in the first place, thereby reducing the cost increase for users. Moreover, in the cloud platform database dynamic migration cost evaluation algorithm unit, the comprehensive cost value after migration is further calculated. Through this cost value, users can intuitively observe the migration effect and cost situation, and then conveniently and quickly adjust the method steps; At the same time, the performance cost algorithm unit after database migration can dynamically adjust the dynamic migration method of the cloud platform database, and the summary of the two algorithm units enables the overall algorithm unit system to continuously feedback and iterate, achieving excellent improvement effects, and can also intuitively display the migration effect. In summary, the three algorithms used in this migration method are beneficial to the total migration method itself, and the three algorithms are interrelated and intertwined, forming a complete dynamic migration system method, which can have the dual effects of high efficiency and economy during the migration process, and this method is more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a method flow chart for the dynamic migration of the cloud platform database; Figure 2 It is a schematic diagram of the overall structure of the cloud platform database dynamic migration method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0016] Regarding the dynamic migration method of the cloud platform database, different from the existing migration methods that mostly migrate multiple databases according to certain set indicators, such as migrating through limited indicators such as the size and usage of multiple databases, the applicability of this migration method is relatively limited and cannot be dynamically optimized, thus resulting in poor migration efficiency during implementation. However, this dynamic migration method conducts a comprehensive ranking of databases for multiple groups of related data, precisely calculates based on the size value, access frequency value, and remaining space information of the database to achieve a comprehensive priority ranking, and can also calculate the migration cost and performance changes in real time during the migration process. Moreover, it can continuously optimize and adjust the overall algorithm system according to the actual situation to adapt to the dynamic traction of more databases. This method is more advantageous and effective compared to the existing database migration methods.

[0017] Example 1. Please refer to Figures 1 to 2 , this example provides a dynamic migration method for the cloud platform database, including the following steps: Data collection: Collect the data size, access frequency, remaining storage space, and network bandwidth values of each database; Dynamic migration: Specifically includes: First, preset the weight factor A1, and then substitute the collected data size, access frequency, and remaining storage space into the dynamic migration priority algorithm unit of the cloud platform database to calculate the migration priority index of each database; Next, sort the databases in descending order according to the migration priority. According to the actual requirements and resource limitation factors, preferentially select a part of the databases with high migration priority as the first batch of migration objects and perform the migration; After that, substitute the migration priority index of the database and the data size of the database into the dynamic migration cost evaluation algorithm unit of the cloud platform database to calculate the migration cost value of the corresponding database; And, substitute the migration priority index of the database and the migration cost value of the database into the performance cost algorithm unit after database migration for operation to calculate the performance value after database migration; Finally, the calculated performance value after database migration is the growth value of A1 in the dynamic migration priority algorithm unit of the cloud platform database. Feedback calculation is performed on this value to adjust the value of A1. Through iteration, it continuously affects the migration priority index of each database and the migration cost value of the database until all databases are migrated; Summary and output: When all database migrations are completed, count the migration time and cost index, and then organize the migration results and the values during the migration process into a report and upload it to the control terminal for users to view.

[0018] Please refer toFigures 1 to 2 , a cloud platform database dynamic migration method, which adopts a data collection module, a computer, a terminal output module, a monitoring and management unit, a database storage service device, a temporary data storage and backup unit, and a database dynamic migration reaction module; The data collection module is used to collect and collate the data size, access frequency, remaining storage space, and network bandwidth values of i databases, and upload the data to the computer. The database dynamic migration reaction module in the computer is used to calculate the collected database information, and sequentially calculate the priority index of the corresponding database, and sort multiple databases from high to low according to the priority relationship. After the sorting is completed, the database with a higher priority is migrated first, and the performance data after migration is collected after one migration to calculate the migration cost value of the database, and the performance value after database migration is calculated together with the migration cost value and the migration priority index of the database. Targeted migration is carried out through the priority, and the calculation of the migration cost value can facilitate the quick understanding of the economic cost for corresponding improvement and decision-making. The performance value after database migration can intuitively reflect the effect and efficiency of the migration, and can feedback this performance value to the calculation process of the database priority index to continuously iterate and optimize, thereby continuously affecting the output value of the database dynamic migration reaction module to reflect the purpose of dynamic migration; The database dynamic migration reaction module includes: a cloud platform database dynamic migration priority algorithm unit, a cloud platform database dynamic migration cost evaluation algorithm unit, and a database post-migration performance cost algorithm unit.

[0019] In this embodiment: In this cloud platform database dynamic migration method, three algorithm formulas are used for data calculation. Through the cloud platform database dynamic migration priority algorithm unit, multiple databases can be prioritized. Precise calculations are performed based on information such as actual size values, access frequency values, and remaining space. Then, the priority of each database can be quickly calculated and sorted. Subsequently, according to this algorithm, databases with high priorities can be migrated and corresponding processing can be carried out immediately to reduce the increased costs for users. Moreover, in the cloud platform database dynamic migration cost evaluation algorithm unit, the comprehensive cost value after migration is further calculated. It not only has a correlation with the cloud platform database dynamic migration priority algorithm unit but also can be related to the performance changes before and after migration. Thus, through this cost value, users can intuitively observe the migration effect and cost situation, and then conveniently and quickly adjust the method steps. The database migration post-performance cost algorithm unit can dynamically adjust this cloud platform database dynamic migration method, as well as summarize the above two algorithm units. This not only enables the overall algorithm unit system to continuously feedback and iterate, achieving excellent improvement effects, reducing the increased costs for users with a certain amount of resources, but also can intuitively display the migration effect. Therefore, the three algorithms used in this migration method are all beneficial to the overall method itself. And the three algorithms are interrelated and intertwined, forming a complete dynamic migration method, which can achieve both high efficiency and economy during the migration process. Different from the fixed migration system method, this method is more flexible. And the cloud platform database dynamic migration priority algorithm unit, the cloud platform database dynamic migration cost evaluation algorithm unit, and the database migration post-performance cost algorithm unit have logical and numerical calculation correlations with each other, and have the effect of iterating and optimizing the overall calculation. Therefore, this migration method is more practical and convenient.

[0020] Please refer to Figures 1 to 2 , the cloud platform database dynamic migration priority algorithm unit is as follows: Set the base value of the weight factor A1 as A3; ; ; Where: is the migration priority index of the i-th database; is the size value of the i-th database; is the access frequency value of the i-th database; is the remaining space of the cloud storage device storing the i-th database; X is the weight of the database size value; A2 is calculated by the performance cost algorithm unit after database migration and is used to dynamically adjust the increment value of the weight factor A1; and are the weight factors of the access frequency value and the remaining space respectively.

[0021] In this embodiment: During the dynamic migration of the cloud platform database, there may be multiple databases. If the multiple databases are treated equally and migrated in a fixed order, there may be a situation where the migration of the database with a high utilization rate, that is, a high priority, is slow or stagnant, which will cause the user to lose time costs. Therefore, this algorithm unit first sorts the priorities of multiple databases, conducts comprehensive analysis and calculation based on various influencing factors, can accurately evaluate the important priorities of multiple databases, and then ensures that high-priority data can be processed immediately, thereby reducing the cost increase of users.

[0022] Please refer to Figures 1 to 2 , the dynamic migration cost evaluation algorithm unit of the cloud platform database is as follows: ; Among them: is the migration cost value of the i-th database; is the size priority comprehensive index value of the i-th database; is the network bandwidth value used to migrate the i-th database; is a fixed migration cost value, and the migration cost includes: labor cost and downtime.

[0023] In this embodiment: This algorithm unit can evaluate the migration cost of the database, and then cooperate with the migration priority, network bandwidth and estimated migration time to evaluate the migration cost, comprehensively evaluate the economic cost of different migration schemes for users, thereby facilitating users to make corresponding improvements and adjustments, and also facilitating users to intuitively observe the efficiency and effect of the migration method. And the business interruption cost caused by the migration time is also considered, so the calculated migration cost value is more accurate.

[0024] Please refer to Figures 1 to 2 , the performance cost algorithm unit after database migration is as follows: ; Among them: A2 is the performance value after database migration and also serves as the feedback value in the calculation process; is the performance value after the migration of the i-th database, and the performance value is the corresponding time after the migration of the i-th database; When A3 is A1 in the first cycle after the initial setting, A3 is the base value of A1.

[0025] In this embodiment: First The calculation can play a role in dynamically adjusting A1, which helps to optimize the migration order and ensure that high-performance and high-priority databases are migrated first. It can also evaluate the performance of the migrated database, thereby reflecting the cost performance of the database after migration. Furthermore, the numerical values involved in the operation in the migration method can be adjusted according to the actual situation, so as to achieve the purpose of optimizing the migration effect, which helps to achieve the optimal migration effect within limited resources. And through the iteration and feedback between the algorithm unit and the dynamic migration priority algorithm unit of the cloud platform database, the purpose of dynamic migration can be achieved for this migration method, and the overall computing system has the characteristic of continuous iteration. The applicability of the fixed-form migration system is poor. This algorithm unit can change and optimize dynamically to a certain extent, and thus has better adaptability. This algorithm unit migration system method can consider the performance changes after migration. By calculating the relationship between the cost changes and performance changes after migration, it can provide users with feedback on the migration effect, which helps users make immediate adjustments according to the feedback results, and thus has better practicality. And through real-time feedback and dynamic adjustment of the weight factor, the present invention can adapt to the dynamic changes of the cloud platform database and achieve flexible migration and expansion.

[0026] In the specific implementation process, using the three algorithm units in this method can perform dynamic migration on multiple databases. First, the migration priority algorithm unit can sort multiple databases comprehensively according to priority, which is more accurate and effective than the existing form of sorting according to set indicators. Then, the cloud platform database dynamic migration cost evaluation algorithm unit substitutes the values calculated by the migration priority algorithm unit and the values involved in the calculation and comprehensively calculates the cost value of the corresponding database migration. Through this cost value, users can intuitively observe the migration effect and the actual situation of the cost, so as to facilitate and quickly adjust the method steps. Finally, the performance cost algorithm unit after database migration can integrate the results of the cloud platform database dynamic migration priority algorithm unit and the cloud platform database dynamic migration cost evaluation algorithm unit and perform further operations, on the one hand, to reflect the performance value of the database migration so that users can fully master the comparison situation and performance changes before and after the database migration, and it can also optimize and iterate the system composed of the three algorithms to achieve the purpose of continuous improvement and adjustment, which helps to be applicable to the dynamic migration of databases in a variety of different situations.

[0027] Embodiment 2: On the basis of the above embodiment: Please refer to Figure 1 andFigure 2 The monitoring and management unit acts on the database dynamic migration reaction module, and is used to monitor the migration progress and resource consumption in real time during the database migration process, and display the monitored values to the user through the computer and the terminal output module.

[0028] In this embodiment: By monitoring multiple data before, after and during the migration, it is convenient for the user to observe the data changes and the migration situation in real time, and it is also convenient for the normal operation of this migration method system, and has excellent monitoring and control effects.

[0029] The database storage service device is used to store and manage the database to be migrated and improve the data access function, and store the data used by the database dynamic migration reaction module.

[0030] In this embodiment: It can well preserve and protect the data to be migrated, and avoid the adverse situation of data loss before and after the migration.

[0031] The temporary data storage and backup unit is used to store the temporary data and log files during the migration process to ensure the integrity and recoverability of the data.

[0032] In this embodiment: It can play a role in protection and insurance, avoid the irreversible impact caused by data loss during the migration process, and thus the stability of this migration method is better.

[0033] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cloud platform database dynamic migration method, characterized in that: The following steps are involved: Data collection: collect the data size, access frequency, remaining storage space and network bandwidth values ​​of each database; Dynamic migration: specifically includes: First, the weight factor A1 is preset, and the collected data size, access frequency, and remaining storage space are substituted into the cloud platform database dynamic migration priority algorithm unit to obtain the migration priority index of each database; Next, sort the databases from high to low according to their migration priority. Based on demand and resource constraints, select some databases with high migration priority as the first batch of migration objects and migrate them. Afterwards, the migration priority index of the database and the data size of the database are substituted into the cloud platform database dynamic migration cost evaluation algorithm unit to obtain the migration cost value of the corresponding database; And, the migration priority index of the database and the database migration cost value are substituted into the database migration performance cost algorithm unit for operation to calculate the performance value of the database after migration; Finally, the calculated performance value of the database after migration is the growth value of A1 in the cloud platform database dynamic migration priority algorithm unit. This value is fed back to adjust the value of A1, and the migration priority index of each database and the migration cost value of the database are continuously affected in an iterative manner until all databases are migrated. Summary output: When all databases are migrated, the migration time and cost index are calculated, and then the migration results and the values ​​during the migration process are compiled into a report and uploaded to the control terminal for users to view.

2. The cloud platform database dynamic migration method according to claim 1, characterized in that: The invention adopts a data acquisition module, a computer, a terminal output module, a monitoring management unit, a database storage service device, a temporary data storage backup unit and a database dynamic migration response module.

3. The cloud platform database dynamic migration method according to claim 2, characterized in that: The data acquisition module is used to collect and organize the data size, access frequency, remaining storage space and network bandwidth values ​​of i databases, and upload the data to the computer. The database dynamic migration reaction module in the computer is used to calculate the collected database information, and calculate the priority index of the corresponding database in turn, and sort the multiple databases from high to low according to the priority relationship. After the sorting is completed, the database with a higher priority is migrated first, and the performance data after migration is collected after one migration to calculate the migration cost value of the database, and the performance value of the database after migration is calculated together with the migration cost value and the migration priority index of the database. Targeted migration is carried out according to the priority. The calculation of the migration cost value can facilitate the quick understanding of the economic cost and the corresponding improvement and decision-making, and the performance value of the database after migration can intuitively reflect the effect and efficiency of the migration, and the performance value can be fed back to the calculation process of the database priority index to continuously iterate and optimize, thereby continuously affecting the output value of the database dynamic migration reaction module, so as to reflect the purpose of dynamic migration; The database dynamic migration reaction module includes: a cloud platform database dynamic migration priority algorithm unit, a cloud platform database dynamic migration cost evaluation algorithm unit and a database post-migration performance cost algorithm unit; The cloud platform database dynamic migration priority algorithm unit is as follows: Set the base value of weight factor A1 to A3; ； ； in: is the migration priority index of the ith database; is the size of the i-th database; is the access frequency value of the i-th database; is the remaining space of the cloud storage device storing the i-th database; X is the weight of the database size value; A2 is calculated by the performance cost algorithm unit after database migration and is used to dynamically adjust the incremental value of weight factor A1; and They are the access frequency value and the weight factor of the remaining space respectively.

4. The cloud platform database dynamic migration method according to claim 3, characterized in that: The cloud platform database dynamic migration cost evaluation algorithm unit is as follows: ； in: is the migration cost value of the i-th database; is the value of the priority comprehensive index of the size of the i-th database; is the network bandwidth value used to migrate the i-th database; It is a fixed migration cost value. The migration cost includes: labor cost and downtime.

5. The cloud platform database dynamic migration method according to claim 4, characterized in that: The performance cost algorithm unit after database migration is as follows: ； Among them: A2 is the performance value after database migration, also used as Feedback value during calculation; is the performance value after the migration of the i-th database, and the performance value is the corresponding time after the migration of the i-th database; In the first cycle after the initial setting, A3 is equal to A1, and A3 is the basic value of A1.

6. The cloud platform database dynamic migration method according to claim 2, characterized in that: The monitoring management unit acts on the database dynamic migration response module, and is used to monitor the migration progress and resource consumption in real time during the database migration process, and display the monitored values ​​to the user through the computer and the terminal output module.

7. The cloud platform database dynamic migration method according to claim 3, characterized in that: The database storage service device is used to store and manage the database to be migrated and improve the data access function, and store the data used by the database dynamic migration reaction module.

8. The cloud platform database dynamic migration method according to claim 3, characterized in that: The temporary data storage backup unit is used to store temporary data and log files during the migration process to ensure the integrity and recoverability of the data.

Images