Data migration method, computer readable storage medium and electronic equipment

Abstract

The invention discloses a data migration method of a database, a computer readable storage medium and electronic equipment. The method comprises the steps of obtaining resource use information and a resource use proportion of a current database; determining a data migration scheme of the current database according to the resource use information when the resource use ratio of the first specified time period is greater than the resource upper limit threshold value; performing data migration preheating on the target database according to the data migration scheme, and detecting a preheating state of target migration data in the data migration scheme in the target database; according to the preheating state of the target migration data, an associated application of the target migration data is accessed to a target database; therefore, the resource use information of the service request is reasonably estimated, a data migration scheme is automatically determined, reasonable allocation of database resources is ensured, the resource utilization rate of equipment is improved, data migration preheating is automatically performed according to a data migration method, and the problem of time delay sensitive application alarm caused by cold start of a database during data migration is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data migration, and in particular, relates to a data migration method, a computer-readable storage medium, and an electronic device. Background Art

[0002] A non-distributed database is one where data is stored on a single physical machine, rather than distributed across multiple nodes or servers. Compared to a distributed database, a non-distributed database does not have the ability to shard data across machines, but it reduces the cost of network communication and its response is often faster. In the database field, data migration refers to the process of transferring data from one database to another. In the database field, database splitting (sometimes also called database sharding or partitioning) refers to splitting a large database into multiple smaller, independent parts to improve performance, scalability, and management efficiency. Unbalanced use of database resources often leads to a significant increase in the difficulty of operation and maintenance, so non-distributed databases need to be split.

[0003] In related technologies, the splitting schemes designed manually based on experience are often not balanced enough, and face the risk of wasting resources or splitting again; or after the data is migrated to the target database, since the target database does not have a large number of application requests, it faces the problem of cold start, and some applications with large request volumes or latency-sensitive applications will generate alarms; or, the execution plan of the target database after migration is often different from that of the original database; all of the above problems will lead to the problem of system performance degradation during a period of data migration.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The object of the present invention is to provide a data migration method, a computer-readable storage medium and an electronic device, so as to solve the problem of poor system performance after data migration in the related art.

[0006] According to one aspect of an embodiment of the present application, a data migration method is provided, the method comprising:

[0007] Get the resource usage information and resource usage ratio of the current database;

[0008] When the resource usage ratio in the first specified time period is greater than the resource upper limit threshold, determining a data migration plan for the current database according to the resource usage information;

[0009] Preheat the target database for data migration according to the data migration plan, and detect the preheating status of the target migration data in the data migration plan in the target database;

[0010] According to the preheating status of the target migration data, the associated applications of the target migration data are connected to the target database.

[0011] In some embodiments, a data migration plan for a current database is determined based on resource usage information, including: searching a resource pool for a target device that matches the current database based on resource configuration information of the current database; if a target device that matches the current database exists, pre-locking the resources of the target device; determining a request cycle for the current database based on a request volume detection indicator of the current database in a second specified time period; and determining a data migration plan based on resource usage information within the request cycle.

[0012] In some embodiments, a data migration plan is determined based on resource usage information within a request cycle, including: within the request cycle, using the preset middleware of the current database to audit the business requests of the current device; recording the request source information and database client information within the request cycle time; writing the request source information and database client information to a designated database; performing load aggregation calculations according to preset dimensions to obtain an execution plan for the query statement on the designated device; and determining a data migration plan based on the execution plan for the query statement on the designated device.

[0013] In some embodiments, the preset dimensions include databases and request sources; based on the execution plan of the query statement on the specified device, a data migration plan is determined, including: based on the execution plan of the query statement on the specified device, the resource cost corresponding to the query statement is calculated; the resource costs are summarized according to the preset dimensions to obtain a first resource cost consumed by the request in each database and a second resource cost for each request source; when the number of requests corresponding to the partitioning element is less than or equal to the preset number, each candidate migration plan is traversed using a first algorithm to determine the data migration plan; the first resource cost in the data migration plan tends to be smaller, and the second resource cost is less than or equal to the preset threshold.

[0014] In some embodiments, data migration is preheated for a target database according to a data migration plan, and a preheated state of target migration data in the data migration plan in the target database is detected, including: obtaining a target device corresponding to the data migration plan; deploying a target database on the target device; the configuration of the target database is consistent with the configuration of the current database; according to the data migration plan, the target migration data is imported from the current database to the target database, and a synchronization relationship between the current database and the target database is established; when the communication condition of the target device reaches a preset condition, migration preheating of the target database is performed; and the preheated state of the target migration data in the data migration plan in the target database is detected.

[0015] In some embodiments, when the communication condition of the target device reaches a preset condition, the migration preheating of the target database is performed; the preheating status of the target migration data in the data migration scheme in the target database is detected, including: when the communication condition of the target device reaches a preset condition, turning on the gateway switch of the target device; sending a read request corresponding to the target migration data to the target database and the current database; detecting the first execution information of the read request in the target database and the second execution information of the current database; and determining the preheating status of the target database based on the first execution information and the second execution information.

[0016] In some embodiments, the preheating status of the target database is determined according to the first execution information and the second execution information, including: detecting whether there is an error only on the target database according to the first execution information and the second execution information; if there is an error only on the target database, issuing an alarm corresponding to the error to prompt the user that the target database has not completed preheating; obtaining a first time consumption ratio corresponding to the target database and a second time consumption ratio corresponding to the current database when running a preset event; when the difference between the first time consumption ratio and the second time consumption ratio is less than a ratio threshold and there is no error, determining that the target database has completed preheating.

[0017] In some embodiments, based on the preheating status of the target migration data, the associated application of the target migration data is connected to the target database, including: when the target database completes preheating, updating the database address to which the associated application of the target migration data is connected; re-issuing the associated application of the target migration data; and reclaiming user permissions on the current database.

[0018] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data migration method provided by any embodiment of the present application is implemented.

[0019] According to one aspect of an embodiment of the present application, an electronic device is provided, including: a processor; a memory for storing executable instructions of the processor; the processor executes the executable instructions to enable the electronic device to implement the data migration method provided by any embodiment of the present application.

[0020] In the technical solution of the present application, the resource usage information and resource usage ratio of the current database are obtained; when the resource usage ratio of the first specified time period is greater than the resource upper limit threshold, the data migration plan of the current database is determined according to the resource usage information; the target database is preheated for data migration according to the data migration plan, and the preheating status of the target migration data in the data migration plan in the target database is detected; according to the preheating status of the target migration data, the associated applications of the target migration data are connected to the target database; thereby reasonably estimating the resource usage information of the business request, automatically determining the data migration plan, ensuring the reasonable allocation of database resources and improving the resource utilization rate of the equipment, and automatically preheating the data migration according to the data migration plan, thereby improving the problem of delay-sensitive application alarms caused by database cold start during data migration.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 The flowchart of the data migration method provided in one embodiment of the present application is schematically shown.

[0024] Figure 2 The diagram schematically shows a time series variation diagram of a request volume detection indicator in a certain time period provided by an embodiment of the present application.

[0025] Figure 3 A frequency domain variation diagram of a request volume detection indicator provided in an embodiment of the present application is schematically shown.

[0026] Figure 4 The diagram schematically shows the relationship between the application, gateway and database provided in one embodiment of the present application.

[0027] Figure 5 The structural block diagram of an electronic device provided by an embodiment of the present application is schematically shown.

[0028] Figure 6 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0030] In addition, the features, structures or characteristics described in the present application may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the present application.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0033] The load balancing and data splitting issues of non-distributed relational databases have always been a major difficulty that plagues operations and developers, because the uneven use of database resources often leads to a significant increase in the difficulty of operations. When migrating data in traditional non-distributed relational databases, technical personnel communicate with business personnel about requirements and use experience to determine the data growth rate of the corresponding system. However, due to the deviation between customer growth and expectations and changes in business needs, manual estimates of data volume and data modality are usually inaccurate, resulting in an imbalance in database capacity, which in turn leads to problems such as system performance degradation. Figure 1 As shown, the present application provides a data migration method that can be applied to a non-distributed relational database for data migration. The method includes S110 to S140, and the specific process is as follows.

[0034] S110: Obtain resource usage information and resource usage ratio of the current database.

[0035] Specifically, resource usage information includes database capacity information, processor performance indicator information, IO interface and other system resource information. Resource usage ratio refers to the ratio of resources used by a database for business processing and other operations to total system resources. It should be understood that the current database does not refer to only one database, but can be one database or multiple databases; and in order to facilitate the system to manage data, storage resources are usually divided into multiple storage partitions according to attributes such as data purpose, data type or application to which the data belongs.

[0036] S120: When the resource usage ratio in the first specified time period is greater than the resource upper limit threshold, determine a data migration plan for the current database according to the resource usage information.

[0037] Specifically, the first designated period is a period used to measure whether the resource usage ratio meets the preset conditions. For example, the first designated period can be 10 days, 15 days or 30 days, that is, to measure whether the resource usage ratio in a specific period meets the preset conditions, reduce the business interruption caused by the frequent and unplanned data migration, and the problem of failing to timely discover the system performance degradation caused by the shortage of certain database resources due to long-term non-detection. The resource upper limit threshold is a critical value to measure whether the database resources are in a poor performance state. When the resource usage ratio of the database in the first designated period is greater than the resource upper limit threshold, it means that the resources corresponding to the database are in a state of shortage. If data migration is not performed in time, it may cause the failure of operations such as data insertion, update and deletion, the degradation of business query performance, and even the system crash. As mentioned above, the resource usage information reflects the capacity of each database and the performance indicators of the system. The data migration plan of the current database is determined according to the resource usage information, that is, based on the data capacity of the current database and the corresponding business request volume, and the system resources used by the current database when performing related business, the data that needs to be migrated in the current database is comprehensively determined. The data to be migrated and its associated applications determined in this process belong to the data migration plan.

[0038] In another embodiment, when the resource usage ratio of the first specified time period is greater than the resource upper limit threshold, the number of times the resource usage ratio of the first specified time period is greater than the resource upper limit threshold is recorded; when the number reaches the number threshold, the data migration plan of the current database is determined according to the resource usage information. Specifically, this embodiment adds a number judgment on the basis of S120, excludes the situation where the resource usage ratio is greater than the resource upper limit threshold due to accidental factors, and reduces the use of accidental factors as the starting data migration.

[0039] In some embodiments, a data migration plan for a current database is determined based on resource usage information, including: searching a resource pool for a target device that matches the current database based on resource configuration information of the current database; if a target device that matches the current database exists, pre-locking the resources of the target device; determining a request cycle for the current database based on a request volume detection indicator for the current database in a second specified time period; and determining a data migration plan based on resource usage information within the request cycle.

[0040] Specifically, the resource pool refers to the spare device library of the business system, and the devices in the resource pool are candidate devices for deploying new databases. The resource configuration information specifically includes the number of processor (CPU) cores, memory size, disk model, and operating system version used by the current database. The target device that matches the current database, that is, the resource configuration information such as the number of processor cores, memory size, disk model, and operating system version of the target device, meets the resource configuration requirements of the current database. For example, the current database requires the number of processor cores to be at least 8, the memory to be at least 8G, the use of high-speed transmission disks such as SCSI (Small Computer System Interface) or SSD (Solid State Disk), and the operating system to be Ubuntu Server. According to the above resource configuration information, the corresponding target device is searched from the resource pool. If a target device that meets the resource configuration information is found, the resources of the target device are pre-locked. It can be understood that the target device is locked as a device that receives data migration to prevent multiple databases from migrating data to the target device at the same time, resulting in data loss or data errors. The second specified time period is a recent time period of the current database, such as the last 2 months, the last 1 month, or the last 15 days. Through the request volume detection index of the second specified time period, the change pattern of the business request volume over time can be determined, so as to determine the resource estimation cycle of the current database. The request volume detection index is one of the indicators used to measure the resource usage information of the current database. Specifically, the request volume detection index corresponding to the database is the business request volume processed by the database. When performing business queries, the database needs to use the system resources, that is, the request volume detection index can also reflect the amount of system resources used by the database. The request cycle of the current database refers to the cycle of the current database processing business requests. The request cycle can be used to determine the time pattern of the business requests processed by the current database. For example, obtain the request volume detection index for the last month, and use discrete Fourier transform to calculate the frequency information of the request volume. Where x[n] is the nth data point, X[k] is the amplitude and phase information of the kth frequency domain component in the frequency domain, N is the length of the time series data (the total number of data points), k represents the kth frequency component, and j is the imaginary unit. Represents the component of the rotation phase. Figure 2 The following is a time series diagram of the request volume detection indicator in a certain period of time. Figure 3 The figure shows the frequency domain variation diagram of the request volume detection indicator. The maximum amplitude Xmax in the frequency domain can be determined, and the minimum frequency value with a phase greater than 1 / 4Xmax is selected, and the inverse of the frequency value is taken as the period E. The data migration plan refers to the plan for migrating the data of the current database to the target database, including which data in the current database needs to be migrated to the target database, and which application corresponding to the current database needs to access the target database. The resource usage information within the request cycle can reflect which data in the current database occupies more system resources when querying during the request cycle, resulting in a decrease in system performance, so as to determine the data migration plan in a targeted manner.

[0041] In some embodiments, a data migration plan is determined based on resource usage information within a request cycle, including: within the request cycle, using the preset middleware of the current database to audit the business requests of the current device; recording the request source information and database client information within the request cycle time; writing the request source information and database client information to a designated database; performing load aggregation calculations according to preset dimensions to obtain an execution plan for the query statement on the designated device; and determining a data migration plan based on the execution plan for the query statement on the designated device.

[0042] Specifically, because the total business request volume, processor performance consumption, IO interface and other resource consumption of the system need to be considered for comprehensive judgment during data migration, and the database capacity and business request volume can be directly counted from each database, but resources such as processors and IO interfaces are system shared resources and cannot be quantitatively counted from each database. By counting the processor performance consumption, IO interface and other resource consumption of business queries on the database through the middleware in the database system, a more accurate resource consumption valuation can be obtained. It should be understood that the middleware is a type of software between the application system and the system software, which is responsible for receiving the query request from the client, processing the request according to a certain strategy, and then forwarding the request to the back-end database server. The middleware can intercept, analyze, optimize and forward the query request, wherein the query optimizer is usually used to further perform logical optimization and physical optimization on the syntax analysis tree obtained by the analyzer to generate the execution plan of the query request. Therefore, within the request cycle, the preset middleware can be used to audit the business request initiated by the request source to the current database, so as to analyze and obtain the request source information and database client information within the request cycle. The request source information and database client information are written to the specified database as the preset dimension for load summary calculation. That is, the preset middleware is used to summarize and calculate the load according to the dimension of the request source information or the database client information. For example, the request source of request A is user 1, and the database of the data queried by request A is on client a; the request source of request B is user 2, and the database of the data queried by request B is also on client a. If the requests are summarized according to the request source information, request A and request B belong to different groups; if the requests are summarized according to the database client information, request A and request B belong to the same group. It should be understood that the above-mentioned request source information and database client information are only part of the relevant information of the request, and other information of the request can also be used as the preset dimension of the load summary calculation. Then the middleware performs load statistics, logical optimization and physical optimization on the query request, and can analyze and obtain the execution plan of the query statement (i.e., the query request) on the specified device. The specified device can be a standby machine in the resource pool, which is a standby computer device prepared to avoid failure or other problems of the host. The standby machine usually needs to be synchronized with the host, which is the device where the current database in this application is located. The execution plan includes the data range that the application wants to obtain, and multiple execution paths for obtaining data. However, the resource consumption of different execution paths is not the same. The resource consumption of different execution paths on a specified device is obtained through the preset middleware optimizer, so as to estimate the resource consumption value of the query request, and finally determine the corresponding data migration plan based on the resource consumption value.

[0043] In some embodiments, the preset dimensions include databases and request sources; based on the execution plan of the query statement on the specified device, a data migration plan is determined, including: based on the execution plan of the query statement on the specified device, the resource cost corresponding to the query statement is calculated; the resource costs are summarized according to the preset dimensions to obtain a first resource cost consumed by the request in each database and a second resource cost for each request source; when the number of requests corresponding to the partitioning element is less than or equal to the preset number, each candidate migration plan is traversed using a first algorithm to determine the data migration plan; the first resource cost in the data migration plan tends to be smaller, and the second resource cost is less than or equal to the preset threshold.

[0044] Specifically, according to the execution plan of the query statement on the specified device, the execution path of the query statement is selected from the execution plan, and the corresponding resource cost can be estimated according to the length of the execution path. The resource costs of all query statements in the request cycle are summarized according to the preset dimensions. For example, by summarizing according to the database, the first resource costs of different databases can be obtained. By summarizing according to the request source, the second resource cost corresponding to the query request from different request sources can be obtained. After summarizing the first resource cost of each database and the second resource cost of each request source, the preset algorithm can be used to calculate the data migration plan. The preset algorithm includes a backtracking algorithm, a greedy algorithm, and other feasible algorithms. In this embodiment, the first algorithm can be a backtracking algorithm. When the number of requests corresponding to the partitioning element is less than or equal to the preset number, it means that the number of requests is small, and the corresponding calculation process is also relatively simple. The backtracking algorithm can also obtain the calculation result quickly when the calculation amount is small. In different candidate migration schemes, data can be migrated to different locations of the target database. The resource cost when the data is distributed in different locations is calculated through a backtracking algorithm to determine the resource cost corresponding to each candidate migration scheme. Finally, according to the resource cost conditions: the first resource cost tends to be smaller, and the second resource cost is less than or equal to the preset threshold, a suitable data migration scheme is selected from the candidate migration schemes. Specifically, the first resource cost is the processor performance consumption, IO interface and other resource consumption requested in each database. The second resource cost of each request source refers to the disk usage rate of each device. The preset threshold of the second resource cost is the usage threshold at which the disk space of each device is about to be in a scarce state. When the disk usage of a device is greater than the preset threshold, it means that the disk space of the device is about to enter a scarce state. Therefore, when determining the data migration plan, it is necessary to ensure that the second resource cost is less than or equal to the preset threshold to reduce the problem of disk space shortage after the data migration is completed.

[0045] S130 , preheating the target database for data migration according to the data migration plan, and detecting the preheating status of the target migration data in the data migration plan in the target database.

[0046] Specifically, preheating the target database for data migration means pre-reading the target migration data on the target database to determine whether the settings of the target database are normal, so as to reduce the impact of the cold start of the target database on the normal business of the application. The preheating status of the target database includes the completed preheating status and the incomplete preheating status. If the target database is in the completed preheating status, it means that the read request for the target migration data is executed normally, the settings of the target database are correct, and the business requests of the application can be executed normally; otherwise, it means that there are incorrect settings in the target database, etc., and an alarm investigation is required.

[0047] In some embodiments, data migration is preheated for a target database according to a data migration plan, and a preheated state of target migration data in the data migration plan in the target database is detected, including: obtaining a target device corresponding to the data migration plan; deploying a target database on the target device; the configuration of the target database is consistent with the configuration of the current database; according to the data migration plan, the target migration data is imported from the current database to the target database, and a synchronization relationship between the current database and the target database is established; when the communication condition of the target device reaches a preset condition, migration preheating of the target database is performed; and the preheated state of the target migration data in the data migration plan in the target database is detected.

[0048] Specifically, deploy the target database on the target device, and set the configuration of the target database to be consistent with the configuration of the current database, so that the alarms of the target database can be found in the subsequent preheating process. Establish a synchronization relationship between the current database and the target database, and perform migration preheating of the target database when the communication conditions of the target device meet the preset conditions, all in order to reduce the impact of alarms caused by network delays on the preheating results and ensure the accuracy of the preheating results.

[0049] In some embodiments, when the communication condition of the target device reaches a preset condition, the migration preheating of the target database is performed; the preheating status of the target migration data in the data migration scheme in the target database is detected, including: when the communication condition of the target device reaches a preset condition, turning on the gateway switch of the target device; sending a read request corresponding to the target migration data to the target database and the current database; detecting the first execution information of the read request in the target database and the second execution information of the current database; and determining the preheating status of the target database based on the first execution information and the second execution information.

[0050] Specifically, Figure 4As shown, application 1, application 2, application 3 and application 4 are all connected to the current database instance for read and write services, corresponding to database d1, database d2, database d3 and database d4 respectively. It is assumed that the target migration data is part of the data in database d3 and database d4. When the communication condition of the target device reaches the preset condition, the gateway switch of the target device is turned on, and the read and write requests of application 1 and application 2 are still sent to the current database instance, while the write requests and part of the read requests of application 3 and application 4 are sent to the current database instance, and the read requests corresponding to the target migration data in application 3 and application 4 need to be sent to the target database instance. The target database is preheated for data migration by executing part of the read requests of application 3 and application 4. Part of the read requests of application 3 and application 4 are sent to the current database and the target database for execution, then the first execution information of these read requests in the target database and the second execution information of the current database can be obtained respectively, wherein the first execution information is the execution time and execution alarm corresponding to the execution plan of the read request in the target database. Similarly, the second execution information is the execution time and execution alarm corresponding to the execution plan of the read request in the current database. Then, by comparing the first execution information and the second execution information, the warm-up status of the target database can be determined.

[0051] In some embodiments, the preheating status of the target database is determined according to the first execution information and the second execution information, including: detecting whether there is an error only on the target database according to the first execution information and the second execution information; if there is an error only on the target database, issuing an alarm corresponding to the error to prompt the user that the target database has not completed preheating; obtaining a first time consumption ratio corresponding to the target database and a second time consumption ratio corresponding to the current database when running a preset event; when the difference between the first time consumption ratio and the second time consumption ratio is less than a ratio threshold, and there is no error only on the target database, determining that the target database has completed preheating.

[0052] Specifically, the first execution information packet tests the execution alarm of the read request on the target database, and the second execution information includes the execution alarm of the test read request on the current database. The data migration warm-up process is actually a test process for the target database, so the test read request refers to a read request sent to the current database and the target database at the same time. If there is an error only on the target database, it means that there is a problem in the data query process of the target database, and the target database is in an uncompleted warm-up state. By running the preset events on the target database and the current database respectively, the running time T1 of the target database and the running time T2 of the current database corresponding to the preset events can be obtained. Assuming that m preset events are run, the number of cases p of the running time T1 of the target database exceeding the specified time and the number of cases q of the running time T2 of the current database exceeding the specified time are obtained, and the first time consumption ratio p / m and the second time consumption ratio q / m are calculated. When the difference between the first time consumption ratio p / m and the second time consumption ratio q / m is less than the ratio threshold and there is no error only on the target database, it is determined that the target database has completed the warm-up. In another embodiment, the specified time includes multiple times, such as 5ms, 10ms and 30ms, etc., and multiple execution time ratios are used to measure whether the execution status of the target database meets the available state, thereby improving the problem of delay-sensitive application alarms caused by cold start of the target database.

[0053] S140 . Connecting the associated applications of the target migration data to the target database according to the preheating status of the target migration data.

[0054] Specifically, the preheating status includes completed preheating and incomplete preheating. When the target database is preheated, it means that the target database can ensure normal business execution and the execution time meets the requirements of time-sensitive applications. Therefore, the associated application of the target migration data is connected to the target database so that the associated application can perform business requests on the target database.

[0055] In the technical solution of the present application, the resource usage information and resource usage ratio of the current database are obtained; when the resource usage ratio of the first specified time period is greater than the resource upper limit threshold, the data migration plan of the current database is determined according to the resource usage information; the target database is preheated for data migration according to the data migration plan, and the preheating status of the target migration data in the data migration plan in the target database is detected; according to the preheating status of the target migration data, the associated applications of the target migration data are connected to the target database, so as to reasonably estimate the resource usage information of the business request, realize automatic determination of the data migration plan, ensure reasonable allocation of database resources and improve the resource utilization rate of the equipment, and automatically preheat the data migration according to the data migration method, so as to improve the problem of delay-sensitive application alarm caused by database cold start during data migration.

[0056] In some embodiments, based on the preheating status of the target migration data, the associated application of the target migration data is connected to the target database, including: when the target database completes preheating, updating the database address to which the associated application of the target migration data is connected; re-issuing the associated application of the target migration data; and reclaiming user permissions on the current database.

[0057] Specifically, Figure 4 As shown, assuming that application 1 and application 2 are associated with the current database, and application 3 and application 4 are changed from being associated with the current database to being associated with the target database, then the connection database address of application 3 and application 4 needs to be changed from the current database address to the target database address, so that after data migration, the read and write requests of application 3 and application 4 are sent to the target database instead of the current database. Re-issuing the application associated with the target migration data is to update the database connection information of the application associated with the target migration data. Reclaiming user permissions on the current database is to avoid data leakage and other problems caused by unreclaimed permissions.

[0058] The electronic equipment of this application is introduced below, such as Figure 5 As shown, the present application provides an electronic device 500, which includes: a processor 510 and a memory 520, the memory 520 is used to store executable instructions of the processor; the processor 510 executes the executable instructions to enable the electronic device to implement the data migration method provided by any embodiment of the present application.

[0059] Specifically, the data migration method provided by the present application is stored in the memory 520 of the electronic device, and the data migration method is executed by the processor 510 to obtain the resource usage information and resource usage ratio of the current database; when the resource usage ratio of the first specified time period is greater than the resource upper limit threshold, the data migration plan of the current database is determined according to the resource usage information; the target database is preheated for data migration according to the data migration plan, and the preheating status of the target migration data in the data migration plan in the target database is detected; according to the preheating status of the target migration data, the associated application of the target migration data is connected to the target database, so as to reasonably estimate the resource usage information of the business request, realize automatic determination of the data migration plan, ensure reasonable allocation of database resources and improve the resource utilization rate of the equipment, and automatically preheat the data migration according to the data migration method to improve the problem of delay-sensitive application alarm caused by database cold start during data migration.

[0060] It should be known that the specific implementation content of the electronic device in the present application has been explained in detail in the corresponding method embodiment and will not be repeated here.

[0061] Figure 6 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0062] It should be noted that Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0063] like Figure 6 As shown, the computer system 600 includes a processor 601, which can be a CPU (Central Processing Unit) or an MCU (Microcontroller Unit). The processor 601 can perform various appropriate actions and processes according to the program stored in the read-only memory 602 (Read-Only Memory, ROM) or the program loaded from the storage part 608 to the random access memory 603 (Random Access Memory, RAM). In the random access memory 603, various programs and data required for system operation are also stored. The processor 601, the read-only memory 602 and the random access memory 603 are connected to each other through a bus 604. The input / output interface 605 (Input / Output interface, i.e., I / O interface) is also connected to the bus 604.

[0064] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0065] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, various functions defined in the system of the present application are executed.

[0066] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0067] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0068] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0069] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the implementation method according to the present application.

[0070] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0071] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data migration method, characterized in that: include: Get the resource usage information and resource usage ratio of the current database; When the resource usage ratio in the first specified time period is greater than a resource upper limit threshold, determining a data migration plan for the current database according to the resource usage information; Preheating the target database for data migration according to the data migration plan, and detecting the preheating status of the target migration data in the data migration plan in the target database; According to the preheating state of the target migration data, an application associated with the target migration data is connected to the target database.

2. The data migration method according to claim 1, characterized in that: The determining the data migration scheme of the current database according to the resource usage information includes: According to the resource configuration information of the current database, searching in the resource pool whether there is a target device matching the current database; If there is a target device matching the current database, pre-locking the resources of the target device; Determining a request cycle of the current database according to a request volume detection indicator of the current database in a second specified time period; The data migration plan is determined according to the resource usage information within the request cycle.

3. The migration method according to claim 2, characterized in that: The determining the data migration scheme according to the resource usage information in the request cycle includes: Auditing the service request of the current device using the preset middleware of the current database during the request cycle; Recording the request source information and database client information within the request cycle time; Writing the request source information and the database client information into a designated database; Perform load aggregation calculation according to preset dimensions to obtain the execution plan of the query statement on the specified device; The data migration solution is determined based on an execution plan of the query statement on the designated device.

4. The migration method according to claim 3, characterized in that: The preset dimensions include a database and a request source; and the determining the data migration solution based on the execution plan of the query statement on the designated device includes: Calculating the resource cost corresponding to the query statement based on the execution plan of the query statement on the designated device; Summarizing the resource costs according to a preset dimension to obtain a first resource cost consumed by a request in each database and a second resource cost of each request source; When the number of requests corresponding to the partitioning element is less than or equal to a preset number, a first algorithm is used to traverse each candidate migration plan to determine the data migration plan; the first resource cost in the data migration plan tends to be smaller, and the second resource cost is less than or equal to a preset threshold.

5. The migration method according to claim 1, characterized in that: Preheating the target database for data migration according to the data migration scheme, and detecting the preheating status of the target migration data in the data migration scheme in the target database, includes: Obtaining a target device corresponding to the data migration solution; Deploy a target database on the target device; the configuration of the target database is consistent with the configuration of the current database; According to the data migration plan, import the target migration data from the current database to the target database, and establish a synchronization relationship between the current database and the target database; When the communication condition of the target device reaches a preset condition, preheating the migration of the target database is performed; The preheating state of the target migration data in the data migration solution in the target database is detected.

6. The migration method according to claim 5, characterized in that: When the communication condition of the target device reaches a preset condition, performing migration preheating of the target database; Detecting a preheating state of the target migration data in the data migration solution in the target database includes: When the communication condition of the target device reaches a preset condition, turning on the gateway switch of the target device; Sending a read request corresponding to the target migration data to the target database and the current database; Detecting first execution information of the read request in the target database and second execution information of the current database; A preheating state of the target database is determined according to the first execution information and the second execution information.

7. The migration method according to claim 6, characterized in that: The step of determining the preheating state of the target database according to the first execution information and the second execution information includes: According to the first execution information and the second execution information, detecting whether there is an error reported only on the target database; If there is an error only on the target database, an alarm corresponding to the error is issued to remind the user that the target database has not completed preheating; Acquire a first time consumption ratio corresponding to the target database and a second time consumption ratio corresponding to the current database when running a preset event; When the difference between the first time consumption ratio and the second time consumption ratio is less than the ratio threshold and no error is reported, it is determined that the target database has completed preheating.

8. The migration method according to claim 1, characterized in that: The step of connecting the associated application of the target migration data to the target database according to the preheating state of the target migration data includes: When the target database is preheated, updating the database address to which the associated application of the target migration data is connected; Re-publishing the application associated with the target migration data; Revokes the user's permissions on the current database.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data migration method described in any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor executes the executable instructions to enable the electronic device to implement the data migration method as described in any one of claims 1 to 8.

Images