Data migration method and device and electronic equipment

Abstract

The invention discloses a data migration method and device and electronic equipment. The method comprises the steps of obtaining operation information and statement information of a first database in a target time period; calculating a first running memory required for running the first database according to the running information and the statement information; a second running memory is determined according to the first running memory, a target server is selected according to the second running memory, and the target server is used for running a second database; and obtaining a table establishment statement and target data of the first database, establishing a second database in the target server according to the table establishment statement, and migrating the target data into the second database. By means of the method and device, the problem that in the related technology, an idle server is directly selected to operate the transformed database, and consequently the server operation memory is wasted is solved.

Description

Technical Field

[0001] This application relates to the field of big data, and in particular, to a data migration method, apparatus, and electronic device. Background Art

[0002] With the gradual acceleration of the pace of information technology innovation transformation, the financial industry has started to use self-developed databases to replace existing databases to complete database transformation. However, in the process of database transformation in the prior art, idle servers are usually directly selected to run the database. In this case, the operating memory of the selected server may be much larger than the actual memory required by the database, so it is easy to cause waste of the server operating memory.

[0003] Regarding the problem of waste of server operating memory caused by directly selecting an idle server to run the transformed database in the related art, no effective solution has been proposed yet. Summary of the Invention

[0004] The main purpose of this application is to provide a data migration method, apparatus, and electronic device to solve the problem of waste of server operating memory caused by directly selecting an idle server to run the transformed database in the related art.

[0005] To achieve the above object, according to one aspect of this application, a data migration method is provided. The method includes: obtaining the operation information and statement information of the first database within a target time period; calculating the first operating memory required to operate the first database according to the operation information and statement information; determining a second operating memory according to the first operating memory, and selecting a target server according to the second operating memory, where the target server is used to operate the second database; obtaining the table creation statements and target data of the first database, constructing the second database in the target server according to the table creation statements, and migrating the target data to the second database.

[0006] Optionally, obtaining the operation information and statement information of the first database within a target time period includes: obtaining the maximum business concurrency of the first database within the target time period, and determining the maximum business concurrency as the operation information; obtaining the number of statement sets stored in the first database to obtain a first number, and obtaining the total number of statements in multiple statement sets to obtain a second number; determining the first number and the second number as the statement information.

[0007] Optionally, calculating the first running memory required for running the first database based on the running information and statement information includes: calculating the average number of statement lines in the statement set stored in the first database according to the first quantity and the second quantity; determining the third running memory of the statement set according to the statement set stored in the first database; calculating the running information, the average number of statement lines, the third running memory, the first quantity, the preset influence factor, and the preset parameter to obtain the first running memory.

[0008] Optionally, determining the third running memory of the statement set according to the statement set stored in the first database includes: configuring the first database in a preset server and running the first database to obtain the running data of the first database, where the running memory of the preset server is greater than the running memory required for running the first database; obtaining the running memory required for running each statement set from the running data to obtain a plurality of sub-running memories; adding the plurality of sub-running memories to obtain the third running memory.

[0009] Optionally, determining the second running memory according to the first running memory includes: obtaining a preset comparison table and obtaining the expansion range corresponding to the first running memory from the preset comparison table to obtain the target expansion range; processing the first running memory according to the target expansion range to obtain the second running memory.

[0010] Optionally, selecting a target server according to the second running memory includes: obtaining idle servers from a server cluster to obtain a plurality of idle servers, where the idle running memory of the idle servers is greater than a preset memory value; obtaining idle servers with an idle running memory greater than the second running memory to obtain a plurality of candidate servers, and determining the server with the smallest idle running memory among the plurality of candidate servers as the target server.

[0011] Optionally, constructing a second database in the target server according to the table creation statement and migrating the target data to the second database includes: converting the table creation statement to obtain a converted table creation statement; constructing the second database in the target server through the converted table creation statement; obtaining the storage locations of each sub-data in the target data in the first database, and storing each sub-data in the same storage location in the second database according to the storage locations.

[0012] To achieve the above object, according to another aspect of the present application, a data migration device is provided. The device includes: an acquisition unit for acquiring the operation information and statement information of the first database within a target time period; a calculation unit for calculating the first operating memory required to operate the first database according to the operation information and statement information; a determination unit for determining a second operating memory according to the first operating memory and selecting a target server according to the second operating memory, where the target server is used to operate the second database; and a migration unit for acquiring the table creation statements and target data of the first database, constructing the second database in the target server according to the table creation statements, and migrating the target data to the second database.

[0013] Optionally, the acquisition unit includes: a first acquisition module for acquiring the maximum business concurrency of the first database within the target time period and determining the maximum business concurrency as the operation information; a second acquisition module for acquiring the number of statement sets stored in the first database to obtain a first quantity, and acquiring the total number of lines of statements in multiple statement sets to obtain a second quantity; and a first determination module for determining the first quantity and the second quantity as the statement information.

[0014] Optionally, the calculation unit includes: a first calculation module for calculating the average number of lines of statements in the statement sets stored in the first database according to the first quantity and the second quantity; a second determination module for determining the third operating memory of the statement sets according to the statement sets stored in the first database; and a second calculation module for calculating the operation information, the average number of lines of statements, the third operating memory, the first quantity, a preset influence factor, and a preset parameter to obtain the first operating memory.

[0015] Optionally, the second determination module includes: an operation sub-module for configuring the first database in a preset server and operating the first database to obtain the operation data of the first database, where the operating memory of the preset server is greater than the operating memory required to operate the first database; an acquisition sub-module for acquiring the operating memory required to operate each statement set from the operation data to obtain a plurality of sub-operating memories; and an addition sub-module for adding the plurality of sub-operating memories to obtain the third operating memory.

[0016] Optionally, the determination unit includes: a third acquisition module for acquiring a preset comparison table and acquiring the expansion range corresponding to the first operating memory from the preset comparison table to obtain a target expansion range; and a processing module for processing the first operating memory according to the target expansion range to obtain the second operating memory.

[0017] Optionally, the determination unit includes: a fourth acquisition module, configured to acquire idle servers from the server cluster to obtain a plurality of idle servers, where the idle running memory of the idle servers is greater than a preset memory value; a fifth acquisition module, configured to acquire idle servers whose idle running memory is greater than the second running memory to obtain a plurality of candidate servers, and determine the server with the smallest idle running memory among the plurality of candidate servers as the target server.

[0018] Optionally, the migration unit includes: a conversion module, configured to convert the table creation statement to obtain a converted table creation statement; a construction module, configured to construct a second database in the target server through the converted table creation statement; a sixth acquisition module, configured to acquire the storage locations of the respective sub-data in the target data in the first database, and store the respective sub-data in the same storage locations in the second database according to the storage locations.

[0019] In the embodiments of the present application, by acquiring the operation information and statement information of the first database within a target time period; calculating the first running memory required to operate the first database according to the operation information and statement information; determining the second running memory according to the first running memory, and selecting a target server according to the second running memory, where the target server is used to operate the second database; acquiring the table creation statement and target data of the first database, constructing the second database in the target server according to the table creation statement, and migrating the target data to the second database, the first running memory required to operate the first database is calculated by acquiring the operation information and statement information of the first database, and then the second running memory and the target server are determined, and the construction of the second database and the migration of the target data are completed, solving the problem in the related art that directly selecting an idle server to operate the transformed database causes waste of the server running memory, thereby achieving the technical effects of improving the performance and stability of the database and avoiding waste of the server running memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0021] Figure 1 shows a hardware structure block diagram of a computer terminal for implementing a data migration method;

[0022] Figure 2 is a flowchart of the data migration method according to Embodiment 1 of this application;

[0023] Figure 3 is a flowchart of an optional data migration method provided according to Embodiment 1 of this application;

[0024] Figure 4 It is a schematic diagram of the data migration device provided in Embodiment 2 of the present application;

[0025] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0026] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should be noted that the data migration methods, devices and electronic devices determined by the present disclosure can be used in the big data field, and can also be used in any field other than the big data field. The application fields of the data migration methods, devices and electronic devices determined by the present disclosure are not limited.

[0030] It should be noted that the information collected, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) used in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, complies with relevant laws, regulations, and standards in the relevant regions, adopts necessary confidentiality measures, does not violate public order and good customs, and provides corresponding operation entrances for users to choose to authorize or refuse to use. If the user chooses to refuse, the expert decision-making process will be entered. For example, there is an interface between this system and relevant users or institutions. Before obtaining relevant information, a request for acquisition needs to be sent to the aforementioned users or institutions through the interface, and relevant information can be obtained after receiving the consent information feedback from the aforementioned users or institutions.

[0031] The embodiments or examples of the present disclosure are not exhaustive. They are only illustrations of some embodiments or examples and do not specifically limit the protection scope of the present disclosure. Without contradiction, each step in a certain embodiment or example can be implemented as an independent embodiment, and the steps can be combined arbitrarily. For example, the solution after removing some steps in a certain embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment or example can be exchanged arbitrarily. Additionally, the optional ways or optional examples in a certain embodiment or example can be combined arbitrarily; furthermore, the embodiments or examples can be combined arbitrarily. For example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional ways or optional examples of other embodiments or examples.

[0032] For the convenience of description, some nouns or terms related to the embodiments of the present application are explained as follows:

[0033] Stored procedure: A set of precompiled SQL statements that can be stored in a database and called and executed as a unit. A stored procedure can contain a series of SQL statements, flow control statements, variable declarations and assignments, and other programming elements to achieve specific functions or tasks.

[0034] Embodiment 1

[0035] According to the embodiments of the present application, an embodiment of a data migration method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0036] The method embodiment provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 The hardware block diagram of a computer terminal for implementing a data migration method is shown. As Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (illustrated as 102a, 102b,..., 102n in the figure) (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0037] It should be noted that the above one or more processors 102 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is used for processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data migration method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above data migration method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0040] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0041] Under the above operating environment, this application provides a data migration method as Figure 2 shown. Figure 2 It is a flowchart of the data migration method according to Embodiment 1 of this application.

[0042] Step S201, obtain the operation information and statement information of the first database within the target time period.

[0043] Specifically, during the data migration process, first, the operation information and statement information of the first database within the target time period can be obtained. The first database can be the existing database of a financial institution, and the target time period can be any time period during the normal operation of the first database. The operation information can include the maximum business concurrency of the first database within the target time period, that is, within the target time period, the maximum number of business requests that the first database can support for simultaneous processing. The statement information can include the number of statement sets stored in the first database, and the total number of lines of statements in multiple statement sets. Among them, a statement set refers to a set of statements for a certain stored procedure in the first database, and the total number of lines of statements can be obtained by adding the number of lines of statements of each stored procedure, so that the operation information and statement information of the first database within the target time period can be obtained, and then the operating memory of the first database can be calculated, ensuring the accuracy of calculating the operating memory of the first database.

[0044] Step S202, calculate the first operating memory required to run the first database according to the operation information and statement information.

[0045] Specifically, after obtaining the operation information and statement information of the first database during the target time period, the first operating memory required to operate the first database can be calculated based on the obtained operation information and statement information. First, the average number of statements per statement set stored in the first database can be calculated by the number of statement sets stored in the first database and the total number of statements in the multiple statement sets. The calculation method is to divide the total number of statements in the stored statement sets by the number of statement sets.

[0046] Furthermore, the operating memory required for each statement set in the first database can be obtained, and the total operating memory required for all statement sets can be obtained by addition. Finally, by calculating the operation information, the average number of statements per line, the total operating memory required for the statement sets, the number of statement sets, the preset impact factor, and the preset parameter, the first operating memory can be obtained. The preset impact factor is the number of stored procedures included in a single session of the business system and can be set to the default value of 0.2. The preset parameter can be a parameter for converting storage units. The specific calculation formula is as follows:

[0047] Memory(MB)=C*M*L*P*F / 1024 / 1024 / 1024*1000

[0048] Among them, Memory represents the first operating memory of the first database, with the unit of megabyte (MB). C is the operation information, that is, the maximum business concurrency. M is obtained by calculating the total operating memory required for all statement sets divided by the total number of statements. L represents the average number of statements per line. P represents the number of stored procedure statement sets. F represents the preset impact factor. Through calculation, the first operating memory of the first database is finally obtained.

[0049] Step S203: Determine the second operating memory based on the first operating memory, and select a target server according to the second operating memory, where the target server is used to operate the second database.

[0050] Specifically, after obtaining the first operating memory of the first database, the second operating memory can be determined based on the first operating memory. The second operating memory is the operating memory of the second database, that is, the database after the transformation of the first database. The second database can be a self-developed database of a financial institution, etc. During the process of determining the second operating memory based on the first operating memory, the expansion range corresponding to the first operating memory can be obtained through a preset comparison table, and then the second operating memory can be determined.

[0051] Furthermore, the target server can be selected according to the second operating memory. The target server is an idle server used to run the second database. During the process of selecting the target server, the idle server with an idle operating memory greater than the second operating memory and the smallest idle operating memory itself can be selected as the target server. For example, the second operating memory of the second database is 10G, and there are three idle servers: Server A, Server B, and Server C, and the idle operating memories of the three idle servers are 9G, 12G, and 15G respectively. Since the idle servers with an idle operating memory greater than the second operating memory include Server B and Server C, and the idle operating memory of Server B is the smallest among the two, the target server is finally determined to be Server B with an idle operating memory of 12G, thus reducing the waste of the operating memory of the target server.

[0052] Step S204: Obtain the table creation statement and target data of the first database, construct the second database in the target server according to the table creation statement, and migrate the target data to the second database.

[0053] Specifically, after determining the second operating memory and the target server, the table creation statement and target data of the first database can be obtained. The target data refers to all the data stored in the existing first database. The target data is generally stored in the tables of the first database, and the tables can be created and defined through the table creation statement.

[0054] Furthermore, after obtaining the table creation statement and target data of the first database, since the types of the first database and the second database are different, there may be differences in the syntax of the table creation statements. Therefore, the table creation statement of the first database can be converted according to the actual requirements of the second database, and the second database can be constructed through the converted table creation statement.

[0055] Furthermore, after the second database is set up, first, structure migration can be performed. According to the converted table creation statement, new table structures can be created, including tables, fields, indexes, etc. Finally, by obtaining the storage location of the target data in the first database, the target data can be stored in the same location in the second database, thereby ensuring the integrity and accuracy of the data in the second database and ensuring the stable operation and normal use of the second database.

[0056] The data migration method provided by the embodiments of this application adopts the following steps: obtaining the operation information and statement information of the first database within a target time period; calculating the first operating memory required to run the first database according to the operation information and statement information; determining the second operating memory according to the first operating memory, and selecting a target server according to the second operating memory, where the target server is used to run the second database; obtaining the table creation statements and target data of the first database, constructing the second database in the target server according to the table creation statements, and migrating the target data to the second database. By obtaining the operation information and statement information of the first database, the first operating memory required to run the first database is calculated, and then the second operating memory and the target server are determined, completing the construction of the second database and the migration of the target data, solving the problem in the related art that directly selecting an idle server to run the transformed database causes waste of the server operating memory, thereby achieving the technical effects of improving the performance and stability of the database and avoiding waste of the server operating memory.

[0057] Optionally, in the data migration method provided by the embodiments of this application, obtaining the operation information and statement information of the first database within a target time period includes: obtaining the maximum business concurrency of the first database within the target time period, and determining the maximum business concurrency as the operation information; obtaining the number of statement sets stored in the first database to obtain a first quantity, and obtaining the total number of lines of statements in multiple statement sets to obtain a second quantity; determining the first quantity and the second quantity as the statement information.

[0058] It should be noted that during the data migration process, first, the operation information of the first database within the target time period can be obtained, that is, the maximum business concurrency of the first database within the target time period. The first database can be an existing database of a financial institution, and the target time period can be any time period during the normal operation of the first database. The maximum business concurrency can be obtained through a database performance monitoring tool or the system view of the database.

[0059] Further, after obtaining the operation information of the first database within the target time period, the number of statement sets stored in the first database can be obtained, that is, the number of statement sets of the first database's stored procedures, and it is determined as the first quantity. At the same time, the total number of lines of statements in multiple statement sets can also be obtained and determined as the second quantity. The first quantity and the second quantity can be obtained through a database management tool or by automatically executing query statements. Finally, the first quantity and the second quantity are determined as the statement information.

[0060] In this embodiment, by obtaining the operation information and statement information of the first database within the target time period, the operating memory of the first database can be calculated, ensuring the accuracy of calculating the operating memory of the first database.

[0061] Optionally, in the data migration method provided in the embodiments of the present application, calculating the first operating memory required to operate the first database according to the operating information and statement information includes: calculating the average number of statement lines of the statement set stored in the first database according to the first quantity and the second quantity; determining the third operating memory of the statement set according to the statement set stored in the first database; calculating the operating information, the average number of statement lines, the third operating memory, the first quantity, the preset influence factor, and the preset parameter to obtain the first operating memory.

[0062] It should be noted that after obtaining the operating information and statement information of the first database within the target time period, the average number of statement lines of the statement set stored in the first database can be calculated according to the second quantity and the first quantity in the statement information, that is, dividing the total number of statement lines in the stored statement set by the number of statement sets.

[0063] Further, after obtaining the average number of statement lines of the statement set stored in the first database, the third operating memory of the statement set can be determined according to the statement set stored in the first database, where the third operating memory refers to the total memory required for multiple statement sets during the storage process of the first database, and can be obtained by separately obtaining the operating memory required for the statement set of each stored procedure and adding them up to obtain the third operating memory.

[0064] Further, after obtaining the third operating memory, the operating information, the average number of statement lines, the third operating memory, the first quantity, the preset influence factor, and the preset parameter can be calculated through the following formula to obtain the first operating memory:

[0065] Memory(MB) = C * M * L * P * F / 1024 / 1024 / 1024 * 1000

[0066] Where, Memory represents the first operating memory of the first database, with the unit of megabyte MB, C is the operating information, that is, the maximum business concurrency, M is obtained by calculating the total operating memory required for all statement sets divided by the total number of statement lines, L represents the average number of statement lines, P represents the number of statement sets of the stored procedure, F represents the preset influence factor, and through calculation, the first operating memory of the first database is finally obtained.

[0067] For example, the maximum business concurrency (operating information) of the existing database X used by financial institution A is 100, the total operating memory required for the statement set in the stored procedure (third operating memory) is 1000000000, the total number of statement lines in the statement set in the stored procedure (second quantity) is 20000, the number of stored statement sets (first quantity) is 20, and the preset influence factor is set to the default value of 0.2. Through the following calculation process:

[0068] M = 1000000000 / 20000 * 1000 = 50000000

[0069] L = 20000 / 1000 / 20 = 1

[0070] Memory(MB) = 100 * 50000000 * 1 * 20 * 0.2 / 1024 / 1024 / 1024 * 1000

[0071] Memory(MB) = 18626.45MB

[0072] It is calculated that the first running memory of the first database is about 18G, and then the memory of the transformed database can be determined through the first running memory.

[0073] In this embodiment, the first running memory of the first database is calculated, and then the memory of the transformed database can be determined according to the first running memory, ensuring the stability and accuracy of data migration.

[0074] Optionally, in the data migration method provided in the embodiment of the present application, determining the third running memory of the statement set according to the statement set stored in the first database includes: configuring the first database in a preset server and running the first database to obtain the running data of the first database, where the running memory of the preset server is greater than the running memory required to run the first database; obtaining the running memory required to run each statement set from the running data to obtain a plurality of sub-running memories; adding the plurality of sub-running memories to obtain the third running memory.

[0075] It should be noted that in the process of determining the third running memory of the statement set according to the statement set stored in the first database, first, the first database can be configured in a preset server, where the preset server is an idle server and the running memory of the server is greater than the running memory required to run the first database, that is, greater than the first running memory, so as to ensure that the first database can run normally on the preset server.

[0076] Further, after configuring the first database in the preset server, the first database can be run. A large amount of running data will be generated during the running of the first database. By collecting the running data of the first database within a certain period of time, the running memory required to run each statement set can be obtained from the running data to obtain a plurality of sub-running memories, and then adding the plurality of sub-running memories can obtain the total running memory required for all statement sets of the first database, that is, the third running memory.

[0077] In this embodiment, the third running memory of all statement sets in the first database is obtained through the running data of the first database, so that the first running memory can be calculated based on the third running memory and other metric information, thereby ensuring the accuracy of determining the running memory of the transformed database.

[0078] Optionally, in the data migration method provided in the embodiment of the present application, determining the second running memory according to the first running memory includes: obtaining a preset comparison table, and obtaining the expansion range corresponding to the first running memory from the preset comparison table to obtain a target expansion range; processing the first running memory according to the target expansion range to obtain the second running memory.

[0079] It should be noted that the second running memory refers to the running memory of the transformed database, that is, the second database. In the process of determining the second running memory according to the first running memory, first, a preset comparison table can be obtained. The preset comparison table contains the running memories of different types of source databases and the corresponding expansion ranges for each running memory. For example, the running memory of a C-type source database is 10G, and its corresponding expansion range may be 1G - 3G.

[0080] Further, after obtaining the preset comparison table, according to the calculated first running memory, its corresponding expansion range can be found in the preset comparison table to obtain the target expansion range. Then, the first running memory can be expanded according to the target expansion range, and finally, the second running memory can be obtained. For example, the first running memory of the first database calculated as described above is 18G. Assume that the first database is a B-type database, and the expansion range corresponding to the first running memory in the comparison table is 2G - 4G. The second running memory range of the second database is 20G - 22G. To reduce the waste of the database running memory, the second running memory can be determined as 20G.

[0081] In this embodiment, the second running memory of the second database is determined through the expansion range of the preset comparison table, thereby reducing the waste of the running memory of the transformed database.

[0082] Optionally, in the data migration method provided in the embodiment of the present application, selecting a target server according to the second running memory includes: obtaining idle servers from the server cluster to obtain multiple idle servers, where the idle running memory of the idle servers is greater than a preset memory value; obtaining idle servers with an idle running memory greater than the second running memory to obtain multiple candidate servers, and determining the server with the smallest idle running memory among the multiple candidate servers as the target server.

[0083] It should be noted that after determining the second operating memory of the second database, the target server can be determined based on the second operating memory. First, the servers in the server cluster can be preliminarily screened. The idle servers with idle operating memory greater than the preset memory value in the server cluster can be obtained. The preset memory value can be set as the first operating memory or can be set according to actual requirements, so that the occupied servers or the servers with too small idle operating memory can be filtered out.

[0084] Furthermore, after obtaining multiple idle servers, the idle servers can be secondarily screened. The idle servers with idle operating memory greater than the second operating memory can be obtained to get multiple candidate servers, thus ensuring the normal use of the second database on the server. Then, the server with the smallest idle operating memory among the multiple candidate servers can be determined as the target server. For example, assume that the second operating memory of the second database is 20G, and the idle operating memories of the remaining idle servers after the initial screening are 18G, 21G, and 25G respectively. Since 18G is less than the second operating memory, the candidate servers are the idle servers with idle operating memories of 21G and 25G. Finally, the target server is determined as the server with an idle operating memory of 21G, thus reducing the waste of the operating memory of the target server on the basis of ensuring the stable operation of the second database on the target server.

[0085] In this embodiment, the target server is selected according to the second operating memory, and the server with the smallest idle operating memory greater than the second operating memory is determined as the target server, thus reducing the waste of the operating memory of the target server on the basis of ensuring the stable operation of the second database on the target server.

[0086] Optionally, in the data migration method provided in the embodiment of the present application, constructing the second database in the target server according to the table creation statement and migrating the target data to the second database includes: converting the table creation statement to obtain the converted table creation statement; constructing the second database in the target server through the converted table creation statement; obtaining the storage locations of each sub-data in the target data in the first database, and storing each sub-data at the same storage location in the second database according to the storage locations.

[0087] It should be noted that after determining the target server, the table creation statement and the target data of the first database can be obtained first. Among them, the target data refers to all the data stored in the first database, and the table creation statement is the statement used by the first database to create tables. Then, according to the syntax requirements or other usage requirements of the second database, the table creation statement can be converted to make it adaptable to the second database, and the converted table creation statement can be obtained. Through the converted table creation statement, the second database can be constructed in the target server.

[0088] Further, after the second database is built, the structure migration between the first database and the second database can be carried out. According to the converted table creation statements, a new table structure is created, and then by obtaining the storage location of the target data in the first database, the target data can be stored at the same location in the second database during the data migration process, so as to ensure the integrity of the data in the second database and improve the accuracy of the data migration.

[0089] In this embodiment, the second database is constructed in the target server through the table creation statements, and then the target data is migrated to the second database, ensuring the integrity and accuracy of the data migration.

[0090] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0091] Embodiment 2

[0092] Figure 3 is a flowchart of an optional data migration method provided according to Embodiment 1 of the present application. As Figure 3 shown, the process includes the following steps:

[0093] During the process of migrating data from the first database to the second database, first, the running information and statement information of the first database can be obtained. Among them, the running information includes the maximum business concurrency of the first database during the target time period, and the statement information can include the number of statement sets stored in the first database, as well as the total number of lines of statements in multiple statement sets. By calculating the number of statement sets stored in the first database and the total number of lines of statements in multiple statement sets, the average number of lines of statements in the statement sets stored in the first database can be obtained.

[0094] Further, the total memory of the statement sets can be obtained by obtaining the running memory required for each statement set in the first database and performing an addition calculation. By introducing a preset influence factor and preset parameters, the maximum business concurrency, average number of lines of statements, total memory of the statement sets, and the number of statement sets are calculated, and the running memory of the first database, that is, the first running memory, can be obtained.

[0095] After obtaining the first operating memory, the operating memory of the second database, i.e., the second operating memory, can be determined according to the first operating memory through a preset comparison table, and a target server can be selected according to the second operating memory. The idle server with an idle operating memory greater than the second operating memory and the smallest idle operating memory itself can be selected as the target server. After determining the target server, the second database can be constructed on the target server according to the table creation statement obtained from the first database and converted.

[0096] After the second database is set up, table structure migration can be carried out. According to the converted table creation statement, new table structures, including tables, fields, indexes, etc., can be created in the second database. Finally, by obtaining the storage location of the target data in the first database, the target data can be stored in the same location in the second database, so as to ensure the integrity and accuracy of the data in the second database and ensure the stable operation of the second database.

[0097] It should be noted that during the use of the second database, the situation of insufficient memory may occur as the data continuously increases. In this case, the data can continue to be migrated to a database of the same type as the second database. The second database can be used as the first database, and the above steps can be repeated to determine the new second database and the target server to complete the data migration.

[0098] Embodiment 3

[0099] The embodiment of the present application also provides a data migration device. It should be noted that the data migration device in the embodiment of the present application can be used to execute the data migration method provided by the embodiment of the present application. The data migration device provided by the embodiment of the present application is introduced below.

[0100] According to the embodiment of the present application, a device for implementing the above data migration method is also provided. Figure 4 It is a schematic diagram of the data migration device provided by Embodiment 2 of the present application, as Figure 3 shown. The device includes:

[0101] An obtaining unit 41, configured to obtain the operating information and statement information of the first database within a target time period.

[0102] A calculating unit 42, configured to calculate the first operating memory required to operate the first database according to the operating information and statement information.

[0103] A determining unit 43, configured to determine the second operating memory according to the first operating memory, and select a target server according to the second operating memory, where the target server is used to operate the second database.

[0104] A migration unit 44 is configured to obtain the table creation statements and target data of the first database, build a second database in the target server according to the table creation statements, and migrate the target data to the second database.

[0105] In the data migration device provided by the embodiment of the present application, an acquisition unit 41 is used to acquire the operation information and statement information of the first database within a target time period; a calculation unit 42 calculates a first operating memory required to operate the first database according to the operation information and statement information; a determination unit 43 determines a second operating memory according to the first operating memory, and selects a target server according to the second operating memory, where the target server is used to operate the second database; the migration unit 44 obtains the table creation statements and target data of the first database, builds a second database in the target server according to the table creation statements, and migrates the target data to the second database. By acquiring the operation information and statement information of the first database, the first operating memory required to operate the first database is calculated, and then the second operating memory and the target server are determined, and the construction of the second database and the migration of the target data are completed, solving the problem of waste of server operating memory caused by directly selecting an idle server to operate the transformed database in the related art, thereby achieving the technical effects of improving the performance and stability of the database and avoiding waste of server operating memory.

[0106] Optionally, in the data migration device provided by the embodiment of the present application, the acquisition unit 41 includes: a first acquisition module, configured to acquire the maximum business concurrency of the first database within the target time period, and determine the maximum business concurrency as the operation information; a second acquisition module, configured to acquire the number of statement sets stored in the first database to obtain a first number, and acquire the total number of statements in multiple statement sets to obtain a second number; a first determination module, configured to determine the first number and the second number as the statement information.

[0107] Optionally, in the data migration device provided by the embodiment of the present application, the calculation unit 42 includes: a first calculation module, configured to calculate the average number of statements in the statement sets stored in the first database according to the first number and the second number; a second determination module, configured to determine the third operating memory of the statement sets according to the statement sets stored in the first database; a second calculation module, configured to calculate the operation information, the average number of statements, the third operating memory, the first number, a preset influence factor, and a preset parameter to obtain the first operating memory.

[0108] Optionally, in the data migration device provided in the embodiments of the present application, the second determination module includes: a running sub-module, configured to configure the first database in a preset server and run the first database to obtain the running data of the first database, where the running memory of the preset server is greater than the running memory required to run the first database; an obtaining sub-module, configured to obtain the running memory required to run each statement set from the running data to obtain a plurality of sub-running memories; and an adding sub-module, configured to add the plurality of sub-running memories to obtain a third running memory.

[0109] Optionally, in the data migration device provided in the embodiments of the present application, the determination unit 43 includes: a third obtaining module, configured to obtain a preset comparison table and obtain an expansion range corresponding to the first running memory from the preset comparison table to obtain a target expansion range; and a processing module, configured to process the first running memory according to the target expansion range to obtain a second running memory.

[0110] Optionally, in the data migration device provided in the embodiments of the present application, the determination unit 43 includes: a fourth obtaining module, configured to obtain idle servers from a server cluster to obtain a plurality of idle servers, where the idle running memory of the idle servers is greater than a preset memory value; a fifth obtaining module, configured to obtain idle servers whose idle running memory is greater than the second running memory to obtain a plurality of candidate servers, and determine the server with the smallest idle running memory among the plurality of candidate servers as the target server.

[0111] Optionally, in the data migration device provided in the embodiments of the present application, the migration unit 44 includes: a conversion module, configured to convert a table creation statement to obtain a converted table creation statement; a construction module, configured to construct a second database in the target server through the converted table creation statement; and a sixth obtaining module, configured to obtain the storage locations of each sub-data in the target data in the first database and store each sub-data in the same storage location in the second database according to the storage locations.

[0112] It should be noted here that the above-mentioned obtaining unit 41, calculating unit 42, determination unit 43, and migration unit 44 correspond to steps S201 to S204 in Embodiment 1. The obtaining unit 41, calculating unit 42, determination unit 43, and migration unit 44 have the same implemented examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned module or unit may be a hardware component or a software component stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b,..., 102n). The above-mentioned module may also be a part of the device and may run in the computer terminal 10 provided in Embodiment 1.

[0113] Embodiment 4

[0114] Embodiments of the present application may provide an electronic device, Figure 5 which is a structural block diagram of an electronic device according to an embodiment of the present application. As Figure 5 shown, the electronic device may include: one or more ( Figure 5 only one is shown in the figure) processors 1002, a memory 1004, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0115] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data migration method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned data migration method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided with respect to the processor, and these remote memories may be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0116] The processor may call the information and application programs stored in the memory through a transmission device to perform the following steps: obtaining the operation information and statement information of the first database within a target time period; calculating the first operating memory required to operate the first database according to the operation information and statement information; determining a second operating memory according to the first operating memory, and selecting a target server according to the second operating memory, where the target server is used to operate the second database; obtaining the table creation statement and target data of the first database, constructing the second database in the target server according to the table creation statement, and migrating the target data to the second database.

[0117] The processor may also call the information and application programs stored in the memory through a transmission device to perform the following steps: obtaining the maximum service concurrency of the first database within a target time period, and determining the maximum service concurrency as the operation information; obtaining the number of statement sets stored in the first database to obtain a first quantity, and obtaining the total number of lines of statements in multiple statement sets to obtain a second quantity; determining the first quantity and the second quantity as the statement information.

[0118] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: calculate the average number of lines of statements in the statement set stored in the first database according to the first quantity and the second quantity; determine the third running memory of the statement set according to the statement set stored in the first database; calculate the running information, the average number of lines of statements, the third running memory, the first quantity, the preset influence factor and the preset parameter to obtain the first running memory.

[0119] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: configure the first database in the preset server and run the first database to obtain the running data of the first database, where the running memory of the preset server is greater than the running memory required to run the first database; obtain the running memory required to run each statement set from the running data to obtain multiple sub-running memories; add up the multiple sub-running memories to obtain the third running memory.

[0120] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain the preset comparison table, and obtain the expansion range corresponding to the first running memory from the preset comparison table to obtain the target expansion range; process the first running memory according to the target expansion range to obtain the second running memory.

[0121] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain idle servers from the server cluster to obtain multiple idle servers, where the idle running memory of the idle servers is greater than the preset memory value; obtain the idle servers whose idle running memory is greater than the second running memory to obtain multiple candidate servers, and determine the server with the smallest idle running memory among the multiple candidate servers as the target server.

[0122] The processor can also call the information and application programs stored in the memory through the transmission device to execute the following steps: convert the table creation statement to obtain the converted table creation statement; build the second database in the target server through the converted table creation statement; obtain the storage locations of each sub-data in the target data in the first database, and store each sub-data to the same storage location in the second database according to the storage locations.

[0123] Those of ordinary skill in the art can understand that Figure 5 the structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and a Mobile Internet Device (MID), a PAD, etc. Figure 5 It does not limit the structure of the above electronic device. For example, the electronic device may further include moreFigure 5 more or fewer components shown therein (such as network interfaces, display devices, etc.), or having a configuration different from that Figure 5 shown.

[0124] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware of the terminal device through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, etc.

[0125] Embodiment 5

[0126] An embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the above storage medium may be used to store the program code executed by the data migration method provided in the first embodiment above.

[0127] Optionally, in this embodiment, the above storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0128] The present application also provides a computer program product, which is adapted to execute a program for the steps of the data migration method when executed on a data processing device.

[0129] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0130] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0131] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces, and the indirect couplings or communication connections of the units or modules can be in an electrical or other form.

[0132] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs, etc., which can store program codes.

[0135] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A data migration method, characterized in that: include: Obtaining operation information and statement information of the first database within a target time period; Calculate a first running memory required to run the first database according to the running information and the statement information; Determine a second running memory according to the first running memory, and select a target server according to the second running memory, wherein the target server is used to run a second database; Acquire a table creation statement and target data of the first database, build the second database in the target server according to the table creation statement, and migrate the target data to the second database.

2. The method according to claim 1, characterized in that Obtaining the operation information and statement information of the first database within the target time period includes: Acquire a maximum concurrent business volume of the first database within the target time period, and determine the maximum concurrent business volume as the operation information; Obtaining the number of statement sets stored in the first database to obtain a first number, and obtaining the total number of statement rows in multiple statement sets to obtain a second number; The first number and the second number are determined as the sentence information.

3. The method according to claim 2, characterized in that Calculating the first running memory required for running the first database according to the running information and the statement information includes: Calculate the average number of statement rows in the statement set stored in the first database according to the first number and the second number; Determine a third running memory of the statement set according to the statement set stored in the first database; The running information, the average number of statement lines, the third running memory, the first number, a preset influencing factor and a preset parameter are calculated to obtain the first running memory.

4. The method according to claim 3, characterized in that Determining a third running memory of the statement set according to the statement set stored in the first database includes: configuring the first database in a preset server, and running the first database to obtain operation data of the first database, wherein the operation memory of the preset server is larger than the operation memory required to run the first database; Acquire the running memory required for running each statement set from the running data to obtain multiple sub-running memories; Add the multiple sub-operation memories to obtain the third operation memory.

5. The method according to claim 1, characterized in that Determining the second running memory according to the first running memory includes: Obtain a preset comparison table, and obtain a capacity expansion range corresponding to the first running memory from the preset comparison table to obtain a target capacity expansion range; The first running memory is processed according to the target expansion range to obtain the second running memory.

6. The method according to claim 1, characterized in that Selecting a target server according to the second running memory includes: Obtaining idle servers from the server cluster to obtain multiple idle servers, wherein the idle running memory of the idle servers is greater than a preset memory value; An idle server whose idle running memory is greater than the second running memory is obtained to obtain multiple candidate servers, and a server whose idle running memory is the smallest among the multiple candidate servers is determined as the target server.

7. The method according to claim 1, characterized in that Building the second database in the target server according to the table building statement, and migrating the target data to the second database includes: Converting the table creation statement to obtain a converted table creation statement; Building the second database in the target server by using the converted table building statement; The storage position of each sub-data in the target data in the first database is obtained, and each sub-data is stored in the same storage position in the second database according to the storage position.

8. A data migration device, characterized in that: include: An acquisition unit, used to acquire operation information and statement information of the first database within a target time period; a calculation unit, configured to calculate a first operation memory required to operate the first database according to the operation information and the statement information; a determining unit, configured to determine a second running memory according to the first running memory, and select a target server according to the second running memory, wherein the target server is used to run a second database; The migration unit is used to obtain the table creation statement and target data of the first database, build the second database in the target server according to the table creation statement, and migrate the target data to the second database.

9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the data migration method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the data migration method described in any one of claims 1 to 7 when running.

Images