Task configuration method, device and storage medium

Abstract

The present application provides a task configuration method, device, and storage medium, relating to the field of computer technology, and is used to solve the technical problem that general technologies are difficult to effectively support data migration between different cloud pools. The method is applied to a relay device; the relay device belongs to a task configuration system including a terminal, a relay device, and multiple cloud servers; the cloud server is configured with a proxy service for providing data transmission services; the relay device stores a proxy configuration information set; the method includes: receiving a migration request message sent by the terminal; when the source database instance and the destination database instance are deployed on different cloud servers, based on the proxy configuration information set, configuring the data playback task and the data conversion task in the cloud server to which the destination database instance belongs, and configuring the data extraction task in the cloud server to which the source database instance belongs.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task configuration method, device, and storage medium. Background Art

[0002] When performing data migration between databases, general technologies are mainly implemented in non-cloud pool scenarios or single cloud pool scenarios, and generally require that the source database and the destination database can be directly connected through the Internet Protocol (IP) address.

[0003] However, when faced with data migration tasks between different cloud pools, due to strict network or even physical isolation between different cloud pools and the lack of connectivity between database instances, general technologies are unable to effectively solve the data migration problem between different cloud pools when IP interconnection between the source and destination databases is difficult. Summary of the Invention

[0004] The present application provides a task configuration method, device and storage medium for solving the technical problem that general technologies are difficult to effectively support data migration between different cloud pools.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a task configuration method is provided, which is applied to a relay device; the relay device belongs to a task configuration system including a terminal, a relay device and multiple cloud servers; the cloud server is configured with a proxy service for providing data transmission services; the relay device stores a proxy configuration information set; the proxy configuration information set includes multiple proxy configuration information corresponding one-to-one to the proxy services of multiple cloud servers; the task configuration method includes: receiving a migration request message sent by the terminal; the migration request message is used to request to migrate target data in a source database instance to a destination database instance; when the source database instance and the destination database instance are deployed on different cloud servers, based on the proxy configuration information set, the data playback task and the data conversion task are configured in the cloud server to which the destination database instance belongs, and the data extraction task is configured in the cloud server to which the source database instance belongs.

[0007] Optionally, the migration request message includes instance configuration information of the source database instance and instance configuration information of the destination database instance; based on the proxy configuration information set, the data playback task and the data conversion task are configured in the cloud server to which the destination database instance belongs, and the data extraction task is configured in the cloud server to which the source database instance belongs, the method specifically includes: based on the proxy configuration information set, the instance configuration information of the source database instance and the instance configuration information of the destination database instance, determining the first proxy configuration information and the second proxy configuration information; the first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs; the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs; calling the proxy service corresponding to the second proxy configuration information to configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs; calling the proxy service corresponding to the first proxy configuration information to configure the data extraction task in the cloud server to which the source database instance belongs.

[0008] Optionally, after receiving the migration request message sent by the terminal, the task configuration method also includes: performing a legitimacy check based on preset verification rules to obtain a verification result; the preset verification rules include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether the terminal has data migration permission, and a compatibility verification rule for verifying whether the database instances are compatible.

[0009] Optionally, when the source database instance and the destination database instance are deployed on the same cloud server, the task configuration method further includes: configuring the data playback task, the data conversion task and the data extraction task in the cloud server where the source database instance and the destination database instance are deployed at the same time.

[0010] In a second aspect, a task configuration device is provided, which is applied to a relay device; the relay device belongs to a task configuration system including a terminal, a relay device and a plurality of cloud servers; the cloud server is configured with a proxy service for providing a data transmission service; the relay device stores a proxy configuration information set; the proxy configuration information set includes a plurality of proxy configuration information corresponding one-to-one to the proxy services of the plurality of cloud servers; the task configuration device includes: a receiving unit and a configuration unit

[0011] A receiving unit, configured to receive a migration request message sent by a terminal; the migration request message is used to request migration of target data in a source database instance to a destination database instance;

[0012] A configuration unit is used to configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs, and to configure the data extraction task in the cloud server to which the source database instance belongs, based on the proxy configuration information set, when the source database instance and the destination database instance are deployed on different cloud servers.

[0013] Optionally, the migration request message includes instance configuration information of the source database instance and instance configuration information of the destination database instance; the configuration unit is specifically configured to:

[0014] Determine first proxy configuration information and second proxy configuration information based on the proxy configuration information set, the instance configuration information of the source database instance, and the instance configuration information of the destination database instance; the first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs; the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs;

[0015] Invoke the proxy service corresponding to the second proxy configuration information, and configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs;

[0016] The proxy service corresponding to the first proxy configuration information is called to configure a data extraction task in the cloud server to which the source database instance belongs.

[0017] Optionally, the task configuration device further includes: a verification unit;

[0018] The verification unit is used to perform a legality verification based on preset verification rules to obtain a verification result; the preset verification rules include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether the terminal has data migration permission, and a compatibility verification rule for verifying whether the database instances are compatible.

[0019] Optionally, the configuration unit is further configured to configure the data playback task, the data conversion task, and the data extraction task in a cloud server where both the source database instance and the destination database instance are deployed.

[0020] In a third aspect, a task configuration device is provided, comprising a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the task configuration device is running, the processor executes the computer execution instructions stored in the memory, so that the task configuration device performs the task configuration method as in the first aspect.

[0021] The task configuration device may be a network device, or a portion of a network device, such as a system-on-chip (SoC) within the network device. The SoC is configured to support the network device in implementing the functions described in the first aspect and any possible implementation thereof, such as receiving, determining, and routing data and / or information involved in the task configuration method. The SoC includes a chip, and may also include other discrete components or circuit structures.

[0022] In a fourth aspect, a computer-readable storage medium is provided, comprising computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer is caused to execute the task configuration method of the first aspect.

[0023] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the task configuration device or separately from the processor of the task configuration device, and this application does not limit this.

[0024] In this application, the names of the aforementioned task configuration devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalents.

[0025] These and other aspects of the present application will become more readily apparent from the following description.

[0026] The technical solution provided by this application brings at least the following beneficial effects:

[0027] Based on any of the above aspects, in the present application, since the cloud server in the task configuration system is configured with a proxy service for providing data transmission services, and the relay device in the task configuration system stores multiple proxy configuration information corresponding one-to-one to the proxy services of multiple cloud servers. Therefore, after receiving the migration request message sent by the terminal, the relay device can, when the source database instance and the destination database instance are deployed on different cloud servers, configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs based on the proxy configuration information set, and configure the data extraction task in the cloud server to which the source database instance belongs, so that the target data in the source database instance can be migrated to the destination database instance based on the execution of each task. Therefore, the present application can be used to solve the technical problem that general technology is difficult to effectively support data migration between different cloud pools. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the structure of a task configuration system provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the hardware structure of a relay device provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a task configuration method provided in an embodiment of the present application Figure 1 ;

[0031] Figure 4 A schematic diagram of a task configuration method provided in an embodiment of the present application Figure 2 ;

[0032] Figure 5 A schematic diagram of a task configuration process provided in an embodiment of the present application Figure 1 ;

[0033] Figure 6 A schematic diagram of a task configuration process provided in an embodiment of the present application Figure 2 ;

[0034] Figure 7 A schematic diagram of a task configuration process provided in an embodiment of the present application Figure 3 ;

[0035] Figure 8 A schematic diagram of a task configuration process provided in an embodiment of the present application Figure 4 ;

[0036] Figure 9 A schematic diagram of the structure of a task configuration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0039] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0040] In addition, the terms "including" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules and may also include steps or modules that are not listed.

[0041] In order to facilitate understanding of this application, the relevant elements involved in this application are now described.

[0042] Database technology studies the basic theories and implementation methods of database structure, storage, design, management, and application, and uses these theories to process, analyze, and understand the data in the database. Database technology specifically includes relational database technology and non-relational database technology. For example, relational database technologies represented by MySQL, PostgreSQL, and Oracle, and non-relational database technologies represented by remote dictionary server (Redis) and Mongo DB. A homogeneous database refers to a database in which the source database instance and the destination database instance are exactly the same type. For example, the source database instance and the destination database instance are both MySQL databases. A heterogeneous database refers to a database in which the source database instance and the destination database instance are different types. For example, the source database instance is a MySQL database, while the destination database instance is an Oracle database.

[0043] Data Transmission Service (DTS) is a data migration or synchronization tool that can be used to migrate data between homogeneous and heterogeneous databases. It is suitable for migrating data to a cloud pool (also understood as a cloud server or cloud server cluster), migrating data between cloud pools, and performing data splitting and expansion. Data migration can be categorized into structural migration, full data migration, and incremental data migration based on the data content being migrated. Task-wise, it can be divided into subtasks such as data extraction, data conversion, and data playback.

[0044] Virtualization is a computer resource management technology that can divide and isolate the general hardware resources of a physical computer, such as the central processing unit (CPU) and memory, to provide one or more virtual computers.

[0045] Cloud computing technology, particularly Infrastructure as a Service (IaaS), pools the CPU, memory, and other hardware resources of multiple servers. Users can access computing resources from the pool on demand. Computing resources are delivered quickly through virtual machines or similar methods, while ensuring isolation between computing resources belonging to different users.

[0046] Multi-cloud is a cloud architecture consisting of multiple cloud pools, generally involving more than two public clouds and / or private clouds.

[0047] When performing data migration between databases, general technologies are mainly implemented in non-cloud pool scenarios or single cloud pool scenarios, and generally require that the source database and the destination database can be directly connected through the Internet Protocol (IP) address.

[0048] However, when faced with data migration tasks between different cloud pools, due to strict network or even physical isolation between different cloud pools and the lack of connectivity between database instances, general technologies are unable to effectively solve the data migration problem between different cloud pools when IP interconnection between the source and destination databases is difficult.

[0049] In response to the above problems, an embodiment of the present application provides a task configuration method. In the present application, since the cloud server in the task configuration system is configured with a proxy service for providing data transmission services, and the relay device in the task configuration system stores multiple proxy configuration information corresponding one-to-one to the proxy services of multiple cloud servers. Therefore, after receiving the migration request message sent by the terminal, the relay device can, when the source database instance and the destination database instance are deployed on different cloud servers, configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs based on the proxy configuration information set, and configure the data extraction task in the cloud server to which the source database instance belongs, so that the target data in the source database instance can be migrated to the destination database instance based on the execution of each task. Therefore, the present application can be used to solve the technical problem that general technology is difficult to effectively support data migration between different cloud pools.

[0050] This task configuration method is applicable to the task configuration system. Figure 1 FIG. 1 shows a structure of the task configuration system 100. Figure 1 As shown, the task configuration system 100 includes: a terminal 101, a relay device 102, and multiple cloud servers 103. The relay device 102 can be connected to the terminal 101 for communication, and can also be connected to the multiple cloud servers 103 for communication.

[0051] In one possible approach, the terminal 101 and the relay device 102 may be connected via an external network, and the relay device 102 and the plurality of cloud servers 103 may be connected via an internal network.

[0052] In actual applications, the relay device 102 may also be connected to communicate with multiple terminals 101 .

[0053] For ease of understanding, this application takes the communication connection between a relay device 102 and a terminal 101 as an example for explanation.

[0054] Figure 1 The terminal 101 in the embodiment may be configured with a Hyper Text Transfer Protocol (HTTP) client. The HTTP client may be a service implemented based on a World Wide Web (WEB) browser.

[0055] In one possible approach, terminal 101 may be configured with functional modules such as a display module, an input module, and a communication module. The input module may be an external computer input device such as a mouse and keyboard to support various user operations, such as information editing and sending. The display module may be a device such as an LCD screen, used to display HTTP client interfaces, such as database information editing interfaces. The communication module may be a device such as a transceiver, used to communicate with relay device 102.

[0056] Optionally, Figure 1 The terminal 101 in the figure can be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connection function, or other processing equipment connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a computer with a mobile terminal, or a portable, pocket-sized, handheld, or computer-built-in mobile device that exchanges language and / or data with a radio access network, such as a mobile phone, tablet computer, laptop computer, netbook, or personal digital assistant (PDA). The embodiments of the present application do not impose any restrictions on this.

[0057] Figure 1 The relay device 102 in the embodiment can be configured with an HTTP server, which can be used to provide various data and services to the terminal. Furthermore, the relay device 102 can also be used to manage multiple cloud servers 103.

[0058] Optionally, the relay device 102 may be deployed with a task management module and a distributed control system (DCS) module.

[0059] The task management module is used to dynamically create data migration tasks based on migration task configuration information received from the terminal and schedule them to the corresponding cloud server 103. Data migration tasks are generally divided into three subtasks: data extraction, data conversion, and data playback. The task management function can dynamically schedule these three subtasks to different cloud servers 103 based on the configuration information of the source and destination database instances.

[0060] The DCS module can be deployed in a three-node high-availability configuration to store metadata for data migration tasks. For example, the DCS cluster can be implemented based on Zookeeper or Consul.

[0061] Figure 1 The cloud server 103 in the example can be configured with a proxy service. The proxy service can be implemented based on DTS, exposing a unique access address to the outside through an enterprise information portal (EIP) and a proxy port to solve the network isolation problem between the cloud servers 103.

[0062] In one possible approach, each cloud server 103 can achieve communication connection in the internal network based on proxy service, and each cloud server 103 uses a fully encrypted secure channel to transmit data, thereby ensuring the isolation between each cloud server 103 and the security of user data transmission.

[0063] In one possible approach, the cloud server 103 may be configured with a DTS control module and a DTS execution module. The DTS control module may be used to schedule and manage subtasks within the cloud server 103. The DTS control module may be deployed in a 3-node high-availability configuration.

[0064] The DTS execution module is used to execute and manage specific data migration subtasks and can limit the running resources of subtasks. Since all task metadata is stored in the DCS cluster in the relay device 102, the DTS execution module can adopt a stateless design to achieve horizontal scalability.

[0065] In one possible approach, the cloud server 103 may also be configured with a message queue (MQ), which can be used to implement efficient and reliable information transmission between subtasks within the cloud server 103 .

[0066] Optionally, different cloud servers 103 may be deployed in different locations. Each cloud server 103 is usually strictly physically isolated or network isolated from each other, and the database instances between each cloud server 103 are not interoperable.

[0067] Optionally, the relay device 102 and the cloud server 103 can each be a single server, or a server cluster consisting of multiple servers. In some implementations, the server cluster can also be a distributed cluster. This embodiment of the application does not impose any restrictions on this.

[0068] like Figure 2 FIG2 is a schematic diagram of a hardware structure of a relay device 102 provided in an embodiment of the present application. The relay device 102 includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 can be connected via the bus 24.

[0069] The processor 21 is the control center of the relay device 102 and can be a single processor or a general term for multiple processing elements. For example, the processor 21 can be a CPU or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0070] As an embodiment, the processor 21 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 are shown in the figure.

[0071] The memory 22 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0072] In one possible implementation, the memory 22 may exist independently of the processor 21 and may be connected to the processor 21 via a bus 24 for storing instructions or program codes. When the processor 21 calls and executes the instructions or program codes stored in the memory 22, the task configuration method provided in the following embodiments of the present application can be implemented.

[0073] In another possible implementation, the memory 22 may also be integrated with the processor 21 .

[0074] The communication interface 23 is used to connect the relay device 102 to other devices via a communication network, which may be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0075] The bus 24 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0076] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the relay device 102. Figure 2 In addition to the components shown, relay device 102 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0077] like Figure 3 The figure is a flow chart of a task configuration method provided by an embodiment of the present application. The task configuration method is applied to Figure 1 The relay device 102 in the task configuration system 100 is shown. The task configuration method includes: S301-S302.

[0078] S301: A relay device receives a migration request message sent by a terminal.

[0079] The migration request message may be used to request that target data in a source database instance be migrated to a destination database instance.

[0080] Optionally, the migration request message may include data mapping information. This data mapping information can be used to indicate the migration of target data from the source database instance to the destination database instance. For example, the data mapping information can be used to indicate the migration of data table A from the source database instance to data table B from the destination database instance. Mapping based on database name and table name is supported.

[0081] In one possible approach, the migration request message may include instance configuration information for the source database instance and the destination database instance. The instance configuration information may include the database instance's user account information, password information, the database instance's network access address on the internal network, and the cloud server to which the database instance belongs.

[0082] In one possible approach, the migration request message may also include metadata information about the target data. The target data may be all tables stored in the source database instance, or a subset of tables stored in the source database instance. The metadata information about the target data may be used to specify the data range of the target data.

[0083] In one possible approach, the migration request message may also include metadata information about the data migration task, such as the migration mode of the data migration task (such as structural migration, full migration, or incremental migration), as well as information such as task concurrency and migration rate.

[0084] One possible approach is to combine Figure 1 When the user needs to create a destination database, he can edit the data mapping information, the instance configuration information of the source database instance, the instance configuration information of the destination database instance, the metadata information of the target data, and other related information through the input module configured by the terminal, and perform the data migration operation (for example, click the migration button). In response to the data migration operation performed by the user, the terminal can send a migration request message to the relay device to request that the target data in the source database instance be migrated to the destination database instance. Accordingly, the relay device can receive the migration request message from the terminal. In addition, the relay device can parse the migration request message to obtain relevant information such as the data mapping information, the instance configuration information of the source database instance, the instance configuration information of the destination database instance, and the metadata information of the target data.

[0085] S302. When the source database instance and the destination database instance are deployed on different cloud servers, the relay device configures the data playback task and the data conversion task in the cloud server to which the destination database instance belongs, and configures the data extraction task in the cloud server to which the source database instance belongs, based on the proxy configuration information set.

[0086] It should be noted that the cloud server may be pre-configured with a proxy service for providing data transmission services. The relay device may store a proxy configuration information set. The proxy configuration information set includes multiple proxy configuration information corresponding to the proxy services of multiple cloud servers.

[0087] In one possible approach, the relay device parses the migration request message and obtains the instance configuration information of the source and destination database instances. It then determines the cloud server to which the source and destination database instances belong. The relay device then determines whether the cloud server to which the source and destination database instances belong is the same cloud server.

[0088] When the source database instance and the destination database instance are deployed on different cloud servers, this indicates network isolation between the source and destination database instances. In this case, the relay device can identify the proxy configuration information for the cloud server to which the source database instance belongs, as well as the proxy configuration information for the cloud server to which the destination database instance belongs, from the proxy configuration information set. The relay device can then sequentially configure the data playback task and data conversion task on the cloud server to which the destination database instance belongs, and the data extraction task on the cloud server to which the source database instance belongs.

[0089] Subsequently, the cloud server to which the source database instance belongs can execute a data extraction task to obtain the target data from the source database instance and send the target data to the cloud server to which the destination database instance belongs. Next, the cloud server to which the destination database instance belongs can execute a data conversion task to convert the syntax and data type of the target data. Furthermore, the cloud server to which the destination database instance belongs can execute a data playback task to write the converted target data to the destination database instance.

[0090] In one embodiment, combining Figure 3 In the above S302, when the source database instance and the destination database instance are deployed on different cloud servers, the relay device configures the data playback task and the data conversion task in the cloud server to which the destination database instance belongs, and configures the data extraction task in the cloud server to which the source database instance belongs, based on the proxy configuration information set. Figure 4 As shown, the embodiment of the present application provides an optional implementation method, including: S401-S403.

[0091] S401: The relay device determines first proxy configuration information and second proxy configuration information based on a proxy configuration information set, instance configuration information of a source database instance, and instance configuration information of a destination database instance.

[0092] The first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs, and the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs.

[0093] In one possible approach, the instance configuration information of the source database instance may include the cloud server information to which the source database instance belongs (such as information such as an identifier or access address), and the instance configuration information of the destination database instance may include the cloud server information to which the destination database instance belongs.

[0094] Based on this, after parsing the migration request message to obtain the instance configuration information of the source database instance and the instance configuration information of the destination database instance, the relay device can determine the cloud server information to which the source database instance belongs and the cloud server information to which the destination database instance belongs. The relay device can then determine the proxy configuration information in the proxy configuration information set that corresponds to the cloud server to which the source database instance belongs as the first proxy configuration information. Furthermore, the relay device can determine the proxy configuration information in the proxy configuration information set that corresponds to the cloud server to which the destination database instance belongs as the second proxy configuration information.

[0095] S402: The relay device calls the proxy service corresponding to the second proxy configuration information, and configures a data playback task and a data conversion task in the cloud server to which the destination database instance belongs.

[0096] The proxy service corresponding to the second proxy configuration information can also be understood as the proxy service of the cloud server to which the destination database instance belongs. Data conversion tasks can be used to convert syntax and data types between heterogeneous databases. Data replay tasks can be used to rewrite target data in the destination database instance.

[0097] One possible approach is to combine Figure 1 The relay device can invoke the proxy service corresponding to the second proxy configuration information to send the configuration information for the data playback task and the configuration information for the data conversion task to the DTS control module in the cloud server to which the destination database instance belongs. The configuration information for the data playback task can include information such as the access address and data type of the destination database instance in the cloud server to which it belongs. The configuration information for the data conversion task can include information such as the syntax and data type of the source database instance and the syntax and data type of the destination database instance.

[0098] Next, on the cloud server hosting the target database instance, the DTS control module can create a data playback task based on the data playback task configuration information and set the data playback task to be triggered upon receiving a message from a target message queue. The target message queue can be used to receive messages containing the converted target data. The DTS control module can then schedule the data playback task to the DTS execution module and pass along the data playback task metadata and target message queue access information.

[0099] Next, on the cloud server hosting the target database instance, the DTS control module creates a data conversion task based on the task's configuration information and dispatches it to the DTS execution module. The data conversion task establishes a connection with the target message queue. The DTS execution module then sends the converted target data to the target message queue, triggering the data playback task.

[0100] In one possible approach, after successfully creating a data playback task and a data conversion task, the DTS control module can generate metadata information for the data playback task and the data conversion task, and send the metadata information to the relay device. In response, the relay device can receive the metadata information and store it in the DCS module.

[0101] S403: The relay device calls the proxy service corresponding to the first proxy configuration information, and configures a data extraction task in the cloud server to which the source database instance belongs.

[0102] The proxy service corresponding to the first proxy configuration information may also be understood as the proxy service of the cloud server to which the source database instance belongs.

[0103] Optionally, the data extraction task can be used to extract the library and table structure, full data, and incremental data from the source database instance.

[0104] One possible approach is to combine Figure 1 The relay device can call the proxy service corresponding to the first proxy configuration information and send the configuration information of the data extraction task to the DTS control module in the cloud server to which the source database instance belongs. The configuration information of the data extraction task can include the second proxy configuration information and the data range or filtering conditions of the target data.

[0105] Next, on the cloud server to which the source database instance belongs, the DTS control module can set filtering conditions for the data extraction task based on the data extraction task's configuration information. Furthermore, the DTS control module can establish an encrypted connection channel based on the Transmission Control Protocol (TCP) between the data extraction task and the proxy service corresponding to the second proxy configuration information.

[0106] In one possible approach, after successfully creating a data extraction task, the DTS control module can generate metadata information for the data extraction task and send the metadata information to the relay device. Accordingly, the relay device can receive the metadata information and store it in the DCS module.

[0107] In one possible approach, after storing metadata information of the data playback task, data conversion task, and data extraction task in the DCS module, the relay device may send a task configuration success message to the terminal to remind the user to complete the task configuration related to data migration.

[0108] In one possible approach, after successfully configuring the data extraction, data conversion, and data playback tasks, the target data migration can be completed based on the execution of each task. Specifically, the data extraction task can connect to the source database instance using database connection information and extract the target data from the source database instance. The data extraction task can then be forwarded to the DTS control module via the proxy service of the cloud server to which the target database instance belongs, via the internal network. The DTS control module can then query the message queue corresponding to the data extraction task based on the task configuration information and distribute the target data to the message queue corresponding to the data conversion task.

[0109] Next, the data conversion task performs syntax and data type conversion based on the database types of the source and destination database instances, and transmits the converted target data to the data playback task via a message queue. The data playback task then establishes a connection to the destination database instance and writes the converted target data to the destination database.

[0110] In one embodiment, after the relay device receives the migration request message sent by the terminal, the task configuration method provided by the present application further includes: S501.

[0111] S501: The relay device performs a validity check based on a preset check rule and obtains a check result.

[0112] The preset verification rules may include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether a terminal has data migration permission, and a compatibility verification rule for verifying whether database instances are compatible.

[0113] In one possible approach, the preset validation rules can be pre-set by staff in the relay device. Furthermore, the staff can also provide relevant resource files to support the implementation of the preset validation rules. For example, a resource file containing information about terminals with data migration permissions and a resource file containing matching relationships between compatible database instances can be included.

[0114] In one possible implementation method, after the relay device receives a migration request message from a terminal and parses the migration request message to obtain relevant information such as data mapping information, instance configuration information of the source database instance, instance configuration information of the destination database instance, and metadata information of the target data, it can perform a validity check on each piece of information based on preset verification rules to obtain a verification result.

[0115] Specifically, the relay device can verify whether a connection can be established between the source and destination database instances based on connection verification rules. Based on permission verification rules, the relay device can verify whether the account information provided by the terminal has data migration permissions to the target data. Based on compatibility verification rules, the relay device can verify the compatibility of parameters, system configurations, and plug-ins required for data migration between the source and destination database instances.

[0116] Based on this, the relay device can obtain a verification result indicating that a connection can be established between the source database instance and the destination database instance, or a verification result indicating that a connection cannot be established between the source database instance and the destination database instance. Furthermore, the relay device can obtain a verification result indicating that the terminal has data migration permission, or a verification result indicating that the terminal does not have data migration permission. Furthermore, the relay device can obtain a verification result indicating that the source database instance and the destination database instance are compatible, or a verification result indicating that the source database instance and the destination database instance are not compatible.

[0117] In one embodiment, when the source database instance and the destination database instance are deployed on the same cloud server, the task configuration method provided in the embodiment of the present application further includes: S601.

[0118] S601: The relay device configures a data playback task, a data conversion task, and a data extraction task in a cloud server where both a source database instance and a destination database instance are deployed.

[0119] In one possible approach, when the source database instance and the destination database instance are deployed on the same cloud server, the relay device can indicate that the source database instance and the destination database instance can establish a connection within the same cloud server. In this case, the relay device can determine, from the proxy configuration information set, the proxy configuration information for the cloud server to which the destination database instance belongs, i.e., the second proxy configuration information. The relay device can then invoke the proxy service corresponding to the second proxy configuration information and send the configuration information for the data playback task, the data conversion task, and the data extraction task to the DTS control module in the cloud server to which the destination database instance belongs.

[0120] Next, on the cloud server to which the destination database instance belongs, the DTS control module can sequentially create a data playback task, a data conversion task, and a data extraction task. After successfully creating the data playback task, the DTS control module can generate metadata information for each of these tasks and send it to the relay device. In response, the relay device can receive the metadata information for each of these tasks and store it in the DCS module.

[0121] It should be noted that, unlike the data extraction task created in S403 , the data extraction task created in S601 does not require configuration of the proxy configuration information of the cloud server to which the target database instance belongs.

[0122] In one possible approach, after successfully configuring the data extraction, data conversion, and data playback tasks, the target data can be migrated based on the execution of each task. Specifically, the data extraction task can connect to the source database instance using database connection information and extract the target data from the source database instance. The data extraction task can then distribute the target data to the message queue corresponding to the data conversion task.

[0123] Next, the data conversion task performs syntax and data type conversion based on the database types of the source and destination database instances, and transmits the converted target data to the data playback task via a message queue. The data playback task then establishes a connection to the destination database instance and writes the converted target data to the destination database.

[0124] In one embodiment, combining Figure 1 ,like Figure 5 As shown, when the source database instance and the destination database instance are deployed on different cloud servers, the relay device can be connected to cloud server 1 through an internal network channel, and can also be connected to cloud server 2 through an internal network channel. Cloud server 1 can be the cloud server to which the source database instance belongs, and cloud server 2 can be the cloud server to which the destination database instance belongs. In this case, the source database instance and the destination database instance are deployed on different cloud servers. The relay device can call the proxy service of cloud server 1 and send the configuration information of the data extraction task to the DTS control module (i.e. Figure 5 The DTS control module of the cloud server 1 can create a data extraction task and perform task scheduling to select a DTS execution module (i.e. Figure 5Execution module in .

[0125] The DTS execution module then performs a data extraction task, obtaining the target data from the source database. It then sends the target data to the message queue of Cloud Server 2 via a TCP encrypted connection. Accordingly, Cloud Server 2's proxy service, through the DTS control module, selects the execution module to perform the data extraction and data playback tasks. Consequently, after the data extraction and data playback tasks are completed, the target data can be written to the destination database.

[0126] In one embodiment, combining Figure 5 ,like Figure 6 As shown, when the source and destination database instances are deployed on different cloud servers, after receiving the migration request message from the terminal, the relay device can initiate task scheduling and allocation through the configured task management module. The task management module can then call the distributed control system module to match the proxy configuration information. The distributed control system module can then determine the first and second proxy configuration information and return the first and second proxy configuration information to the task management module.

[0127] The relay device can then call the proxy service of cloud server 2 through the configured task management module to create a data playback task and a data conversion task. The proxy service of cloud server 2 can send task information of the data playback task and the data conversion task to the control module, so that the control module selects the corresponding execution module to sequentially create the data playback task and the data conversion task. The proxy service of cloud server 2 can then return metadata information of the data playback task and the data conversion task to the task management module of the relay device.

[0128] The relay device can then call the proxy service of cloud server 1 through the configured task management module to create a data extraction task. The proxy service of cloud server 1 can send the task information of the data extraction task to the control module, so that the control module selects the corresponding execution module to create the data extraction task. The proxy service of cloud server 1 can then return the metadata information of the data extraction task to the task management module of the relay device.

[0129] Then, the relay device can store metadata information of the data extraction task, data playback task, and data conversion task in the configured distributed control system module to store the metadata information. After successful storage, the relay device can return a response message indicating successful configuration to the terminal.

[0130] In one embodiment, combining Figure 1 ,like Figure 7 As shown, when the source database instance and the destination database instance are deployed on the same cloud server, that is, Figure 7 When the cloud server 3 is in the middle, the relay device can be connected to the cloud server 3 through the internal network channel. In this case, the relay device can call the proxy service of the cloud server 3 and send the configuration information of the data extraction task, data conversion task and data playback task to the DTS control module of the cloud server 3 (i.e. Figure 7 The DTS control module of the cloud server 3 can select the corresponding DTS execution module (i.e. Figure 7 The DTS execution module then executes the data extraction, conversion, and playback tasks in sequence to migrate the target data from the source database to the destination database.

[0131] In one embodiment, combining Figure 7 ,like Figure 8 As shown, when the source database instance and the destination database instance are deployed on the same cloud server, that is, Figure 8 When the relay device receives the migration request message from the terminal, it can initiate task scheduling and allocation through the configured task management module. The task management module can then call the distributed control system module to match the proxy configuration information. The distributed control system module can then determine the proxy configuration information for cloud server 3 and return it to the task management module.

[0132] The relay device can then call the proxy service of cloud server 3 through the configured task management module to create data playback tasks, data conversion tasks, and data extraction tasks. The proxy service of cloud server 3 can send task information for the data playback tasks, data conversion tasks, and data extraction tasks to the control module, so that the control module selects the corresponding execution modules to sequentially create the data playback tasks, data conversion tasks, and data extraction tasks. The proxy service of cloud server 3 can then return metadata information for the data playback tasks, data conversion tasks, and data extraction tasks to the task management module of the relay device.

[0133] Then, the relay device can store metadata information of the data extraction task, data playback task, and data conversion task in the configured distributed control system module to store the metadata information. After successful storage, the relay device can return a response message indicating successful configuration to the terminal.

[0134] In the embodiment of the present application, since the cloud server in the task configuration system is configured with a proxy service for providing data transmission services, and the relay device in the task configuration system stores a plurality of proxy configuration information corresponding one-to-one to the proxy services of a plurality of cloud servers. Therefore, after receiving the migration request message sent by the terminal, the relay device can, when the source database instance and the destination database instance are deployed on different cloud servers, configure the data playback task and the data conversion task in the cloud server to which the destination database instance belongs based on the proxy configuration information set, and configure the data extraction task in the cloud server to which the source database instance belongs, so that the target data in the source database instance can be migrated to the destination database instance based on the execution of each task. Therefore, the present application can be used to solve the technical problem that general technology is difficult to effectively support data migration between different cloud pools.

[0135] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0136] In the embodiment of the present application, the server can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. In actual implementation, other division methods can be used.

[0137] like Figure 9 FIG. 1 is a schematic diagram of a task configuration device provided in an embodiment of the present application. The task configuration device can be applied to a relay device to perform the following operations: Figure 3 and Figure 4 The task configuration method shown in FIG. The relay device belongs to a task configuration system including a terminal, a relay device, and multiple cloud servers. The task configuration device includes: a receiving unit 701 and a configuration unit 702.

[0138] Receiving unit 701 is used to receive a migration request message sent by a terminal; the migration request message is used to request to migrate the target data in the source database instance to the destination database instance; for example, in combination with Figure 3 , the receiving unit 701 can be used to execute S301.

[0139] Configuration unit 702 is used to configure the data playback task and data conversion task in the cloud server to which the destination database instance belongs, and to configure the data extraction task in the cloud server to which the source database instance belongs, based on the proxy configuration information set, when the source database instance and the destination database instance are deployed in different cloud servers. Figure 3 , the configuration unit 702 can be used to execute S302.

[0140] Optionally, the migration request message includes instance configuration information of the source database instance and instance configuration information of the destination database instance; the configuration unit 702 is specifically configured to:

[0141] Based on the proxy configuration information set, the instance configuration information of the source database instance and the instance configuration information of the destination database instance, the first proxy configuration information and the second proxy configuration information are determined; the first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs; the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs; for example, in combination with Figure 4 , the configuration unit 702 can be used to execute S401.

[0142] Call the proxy service corresponding to the second proxy configuration information, and configure the data playback task and the data conversion task in the cloud server to which the target database instance belongs; for example, Figure 4 , the configuration unit 702 can be used to execute S402.

[0143] Call the proxy service corresponding to the first proxy configuration information and configure the data extraction task in the cloud server to which the source database instance belongs. Figure 4 , the configuration unit 702 can be used to execute S403.

[0144] Optionally, the task configuration device further includes: a verification unit 703;

[0145] Verification unit 703 is configured to perform a validity check based on preset verification rules to obtain a verification result. The preset verification rules include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether a terminal has data migration permissions, and a compatibility verification rule for verifying compatibility between database instances. For example, verification unit 703 can be configured to execute S501.

[0146] Optionally, the configuration unit 702 is further configured to configure the data playback task, the data conversion task, and the data extraction task in a cloud server where both the source database instance and the target database instance are deployed. For example, the configuration unit 702 can be configured to execute S601.

[0147] Those skilled in the art will appreciate that, in one or more of the examples above, the functions described herein can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0148] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0150] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A task configuration method, characterized in that: Applied to a relay device; the relay device belongs to a task configuration system including a terminal, the relay device and a plurality of cloud servers; the cloud server is configured with a proxy service for providing a data transmission service; the relay device stores a set of proxy configuration information; The proxy configuration information set includes a plurality of proxy configuration information corresponding one-to-one to the proxy services of the plurality of cloud servers; The task configuration method includes: Receiving a migration request message sent by a terminal; the migration request message is used to request migration of target data in a source database instance to a destination database instance; the migration request message includes instance configuration information of the source database instance and instance configuration information of the destination database instance; Performing a validity check based on preset verification rules to obtain a verification result; the preset verification rules include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether the terminal has data migration permission, and a compatibility verification rule for verifying whether the database instances are compatible; When the source database instance and the destination database instance are deployed on different cloud servers, first proxy configuration information and second proxy configuration information are determined based on the proxy configuration information set, the instance configuration information of the source database instance, and the instance configuration information of the destination database instance; the first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs; the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs; Invoke the proxy service corresponding to the second proxy configuration information, and configure a data playback task and a data conversion task in the cloud server to which the destination database instance belongs; Invoke the proxy service corresponding to the first proxy configuration information and configure a data extraction task in the cloud server to which the source database instance belongs; When the source database instance and the destination database instance are deployed on the same cloud server, the data playback task, the data conversion task and the data extraction task are configured in the cloud server where the source database instance and the destination database instance are deployed at the same time.

2. A task configuration device, characterized in that: Applied to a relay device; the relay device belongs to a task configuration system including a terminal, the relay device and a plurality of cloud servers; the cloud server is configured with a proxy service for providing a data transmission service; the relay device stores a set of proxy configuration information; The proxy configuration information set includes a plurality of proxy configuration information corresponding one-to-one to the proxy services of the plurality of cloud servers; The task configuration device includes: a receiving unit, a verification unit and a configuration unit; The receiving unit is configured to receive a migration request message sent by a terminal; the migration request message is used to request migration of target data in a source database instance to a destination database instance; the migration request message includes instance configuration information of the source database instance and instance configuration information of the destination database instance; The verification unit is configured to perform a validity verification based on preset verification rules to obtain a verification result; the preset verification rules include a connection verification rule for verifying whether a connection can be established between database instances, a permission verification rule for verifying whether the terminal has data migration permission, and a compatibility verification rule for verifying whether the database instances are compatible; The configuration unit is configured to determine, when the source database instance and the destination database instance are deployed on different cloud servers, first proxy configuration information and second proxy configuration information based on the proxy configuration information set, the instance configuration information of the source database instance, and the instance configuration information of the destination database instance; the first proxy configuration information is the proxy configuration information of the cloud server to which the source database instance belongs; the second proxy configuration information is the proxy configuration information of the cloud server to which the destination database instance belongs; calling the proxy service corresponding to the second proxy configuration information to configure a data playback task and a data conversion task in the cloud server to which the destination database instance belongs; and calling the proxy service corresponding to the first proxy configuration information to configure a data extraction task in the cloud server to which the source database instance belongs; The configuration unit is further configured to configure the data playback task, the data conversion task, and the data extraction task in the cloud server where the source database instance and the destination database instance are deployed.

3. A task configuration device, characterized in that: It includes a memory and a processor; the memory is used to store computer-executable instructions, and the processor is connected to the memory via a bus; when the task configuration device is running, the processor executes the computer-executable instructions stored in the memory, so that the task configuration device performs the task configuration method as described in claim 1.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium includes computer-executable instructions. When the computer-executable instructions are executed on a computer, the computer is caused to execute the task configuration method according to claim 1 .

Images