A data export method, system and computer program product

Abstract

The present disclosure provides a data export method, system and computer program product, and relates to the field of data processing technology. The data export method involved in the present disclosure includes: generating task information for querying private cloud data in response to a user instruction, the task information includes an execution information set, the execution information set includes an execution order and an execution instruction based on the execution order; the task information is sent from a public cloud to a target private cloud storing private cloud data; the execution instruction is executed based on the execution order in the task information to query the local database of the target private cloud and obtain data results; the data results are returned to the public cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data export method, system and computer program product. Background Art

[0002] The hybrid cloud model is a cloud computing deployment method that combines public cloud and private cloud resources. The private cloud in the hybrid cloud can provide computing, storage and other services for applications, while the public cloud can provide data management services for one or more private clouds. Field personnel often need to export various data from the public cloud management system, and these data that need to be exported or the corresponding export functions may not be in the original function planning of the public cloud management system. Some data that need to be exported may only be stored in the private cloud, but not in the public cloud. Therefore, the database administrator needs to access the remote or local database one by one to export the required data, and cannot directly access or export through the public cloud management system.

[0003] To this end, one or more embodiments of the present specification provide a data export method, system, and computer program product, which are intended to export data in any private cloud database required by field personnel in the public cloud without modifying the public cloud management system. Summary of the invention

[0004] One or more embodiments of the present specification provide a data export method for remotely querying private cloud data through a public cloud, the method comprising: generating task information for querying the private cloud data in response to a user instruction, the task information comprising an execution information set, the execution information set comprising an execution order and execution instructions based on the execution order; the task information is sent from the public cloud to a target private cloud storing the private cloud data; the execution instructions are executed based on the execution order in the task information to query the local database of the target private cloud and obtain data results; the data results are returned to the public cloud.

[0005] According to the data export method provided in some embodiments of the present specification, the execution information set includes one or more execution instructions based on the execution order; executing the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtaining data results includes: executing one or more of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining one or more temporary data results corresponding to one or more of the execution instructions; generating the data result based on one or more of the temporary data results.

[0006] According to the data export method provided in some embodiments of the present specification, the execution instructions include query instructions and table creation instructions; executing one or more of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining one or more temporary data results corresponding to one or more of the execution instructions includes: executing the query instructions in the execution instructions based on the execution order to obtain temporary data results; storing the temporary data results in a temporary table established by the table creation instructions corresponding to the query instructions.

[0007] According to the data export method provided in some embodiments of the present specification, the execution information set includes one or more execution instructions based on the execution order; executing the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtain data results includes: executing one of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining a temporary data result corresponding to the execution instruction; updating the execution data parameters contained in the task information, and synchronizing the execution data parameters to the public cloud; the public cloud determines the next execution instruction to be executed by the target private cloud based on the execution data parameters, and sends it to the target private cloud to trigger the next execution instruction to obtain the corresponding temporary data result; generating the data result based on one or more of the temporary data results.

[0008] According to the data export method provided in some embodiments of the present specification, the execution instruction includes a query instruction and a table creation instruction; executing one of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining a temporary data result corresponding to the execution instruction includes: executing the query instruction in the execution instruction based on the execution order to obtain a temporary data result; storing the temporary data result in a temporary table established by the table creation instruction corresponding to the query instruction.

[0009] According to the data export method provided in some embodiments of the present specification, generating the data result based on one or more of the temporary data results includes: after executing all of the execution instructions in the execution information set, merging all of the temporary tables storing the temporary data results and generating the data result.

[0010] According to the data export method provided in some embodiments of the present specification, the generating of the data result based on one or more of the temporary data results includes: each time an execution instruction in the execution information set is executed, the temporary table storing the temporary data result established by the execution instruction is merged with the temporary table storing the temporary data result established by the previous execution instruction in the execution information set to generate a new temporary table, and after all the execution instructions in the execution information set are executed, the data result is generated based on the new temporary table generated by the final merger.

[0011] According to the data export method provided in some embodiments of the present specification, before executing the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtaining the data results, it also includes: deleting temporary data and / or temporary tables generated during the execution of the previous task information.

[0012] According to the data export method provided in some embodiments of the present specification, the task information also includes one or more of the following: a task UUID corresponding to the task information, a project ID corresponding to the target private cloud of the task information, notes, and execution data parameters; the execution data parameters include one or more of the following: the total number of temporary data results, task batches, operation types, execution status, execution information, creation time, last execution time, end time, and file download path.

[0013] According to some embodiments of the present specification, the data export method further includes: generating and storing a configuration file based on the execution information set included in the task information, so that the execution information set can be repeatedly executed.

[0014] One or more embodiments of the present specification also provide a data export system, comprising: a configuration module, which generates task information for querying the private cloud data in response to user instructions, the task information comprising an execution information set, the execution information set comprising an execution order and execution instructions based on the execution order; wherein the task information is sent from a public cloud to a target private cloud storing the private cloud data; a query module, which executes the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtain data results; wherein the data results are returned to the public cloud.

[0015] One or more embodiments of the present specification further provide a computer program product, including computer codes, which, when at least part of the computer codes are executed by a processor, can implement any of the above-mentioned data synchronization methods.

[0016] One or more embodiments of the present specification also provide a data export method for remotely querying private cloud data through a public cloud, the method comprising: generating task information for querying the private cloud data in response to a user instruction, the task information comprising an execution information set, the execution information set comprising an execution order and execution instructions based on the execution order; sending the task information to a target private cloud storing the private cloud data; receiving returned data results from the target private cloud, the data results being acquired by the target private cloud by querying a local database of the target private cloud by executing the execution instructions based on the execution order in the task information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] This specification will be further explained by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. The same numbers in the drawings represent the same structures or steps.

[0018] Figure 1 It is a schematic diagram of an application scenario of the data export method shown in some embodiments of this specification.

[0019] Figure 2 It is a flowchart of a data export method shown in some embodiments of this specification.

[0020] Figure 3 It is a schematic diagram of a process of obtaining data results according to some embodiments of this specification.

[0021] Figure 4 It is a schematic diagram of another process of obtaining data results according to some embodiments of this specification.

[0022] Figure 5 It is a flowchart of another data export method shown in some embodiments of this specification.

[0023] Figure 6 It is a flowchart of another data export method shown in some embodiments of this specification.

[0024] Figure 7 It is a system structure diagram of a data export system according to some embodiments of this specification. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the contents described below are some examples or embodiments of this specification. For ordinary technicians in this field, without paying creative work, the technical solutions or means disclosed in this specification can also be applied to other scenarios based on these technical contents.

[0026] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0027] Unless otherwise specified, technical terms used in this specification to describe components, elements, etc. do not refer to the singular, but may also include the plural. Generally speaking, terms such as "include", "comprise", etc. only indicate that the steps, elements, or components that have been clearly identified are included, and these steps, elements, and components do not constitute an exclusive list, such as the method or device described may also include other steps or components.

[0028] Flowcharts are used in this specification to illustrate the operation steps performed by the device or system of the relevant embodiments, but unless otherwise specified, the order used to describe these steps should not be understood as a limitation on the order in which the steps are performed. A person of ordinary skill in the art can adjust the order in which these steps are performed based on the knowledge and information conveyed by the embodiments of this specification, and the adjustment includes but is not limited to swapping the order, merging multiple steps, and splitting a certain step.

[0029] A hybrid cloud is a cloud computing model that interconnects and integrates different types of cloud computing resources, such as public clouds and private clouds. In some embodiments, the public cloud and private cloud in the hybrid cloud can jointly provide computing, storage and other services for applications, forming a unified, highly controllable computing platform. In other embodiments, the private cloud in the hybrid cloud can provide computing, storage and other services for applications, while the public cloud can provide data management services to one or more private clouds.

[0030] A private cloud can be a cloud service deployed in the data center of an enterprise or organization. In some embodiments, a private cloud can be built and managed by the enterprise or organization itself. In other embodiments, a private cloud can also be provided to a specific enterprise or organization by a dedicated cloud service provider. A public cloud can be a cloud computing platform owned and managed by a cloud service provider, providing services to users through a public network (such as the Internet). In a hybrid cloud scenario, public and private clouds can interact through a public network to achieve data synchronization, seamless migration and sharing of applications or services.

[0031] In some related embodiments, field personnel may need to export data from the public cloud management system. In some cases, the data to be exported may not be in the original functional planning of the public cloud management system. For example, the current design of the public cloud management system may not support the export of this data. Since the public cloud management system does not provide a direct export function for this data, field personnel may need to find other methods or tools to meet these export requirements, resulting in increased work complexity and time costs.

[0032] In other cases, some data that needs to be exported may only be stored in the private cloud, but not in the public cloud, so it cannot be directly accessed or exported through the public cloud management system. In order to obtain this data, it may be necessary to obtain it from each private cloud separately, such as by remotely connecting to the local server of each private cloud and using SQL statements to query and export each data one by one. This requires staff to understand SQL statements, convert the required data into several corresponding SQL statements, and then execute the query and storage commands one by one, which is costly.

[0033] To this end, one or more embodiments of the present specification provide a data export method, a data export system, and a computer program product, which can package user instructions into task information on the public cloud and send them to the private cloud for execution, obtain the data stored in the private cloud and transmit them back to the public cloud. The data export method can export the data required by field personnel in any private cloud database on the public cloud without modifying the current code of the public cloud management system. For example, any required data can be exported without being restricted by the export function currently designed by the public cloud management system. For another example, data can be obtained and exported from the database of the private cloud without being restricted by the data content stored in the public cloud management system.

[0034] Figure 1 It is a schematic diagram of an application scenario of the data export method shown in some embodiments of this specification. Figure 1 The illustrated scenario 100 includes a public cloud device 110 and one or more private cloud devices 120 .

[0035] In some embodiments, the private cloud device 120 is used to provide internal, relatively independent computing resources to each enterprise or organization that are not shared with other enterprises or organizations. In some embodiments, the private cloud device 120 may include a private cloud storage medium for storing data of the enterprise or organization. In some embodiments, the cloud service provider performs data storage and data management on each private cloud device 120 based on the public cloud device 110. In some embodiments, the public cloud device 110 may include a public cloud storage medium for storing data provided by the private cloud device 120.

[0036] In some embodiments, Figure 1 Data can be transmitted between devices in the data synchronization system 100 shown in the figure through a wired or wireless network, and can be specifically connected through one or more of the following networks: cable network, fiber optic network, telecommunication network, Internet, local area network (LAN), wide area network (WAN), wireless local area network (WLAN), metropolitan area network (MAN), public switched telephone network (PSTN), Bluetooth network, ZigBee network, near field communication (NFC), intra-device bus, intra-device line, cable connection, etc.

[0037] Figure 2 It is a flowchart of a data export method shown in some embodiments of this specification. Figure 2 The process 200 shown can be executed by the public cloud device and the private cloud device interacting with each other, for example, Figure 1 The public cloud device 110 and / or private cloud device 120 shown are implemented. In some embodiments, Figure 2 The process 200 shown can be performed by Figure 7 The data export system 700 shown is implemented. Figure 2 As shown, in some embodiments, process 200 may include the following steps.

[0038] Step 210, in response to the user instruction, task information for querying the private cloud data is generated, the task information includes an execution information set, the execution information set includes an execution order and an execution instruction based on the execution order; the task information is sent from the public cloud to the target private cloud storing the private cloud data. In some embodiments, step 210 can be implemented by the configuration module 1100 and the first data transmission module 1300 of the public cloud.

[0039] In one or more embodiments of the specification, a user, such as an outfield personnel, can provide relevant information of the data required by the user as a user instruction, generate task information based on the user instruction provided by the user, and send the task information to the target private cloud. In some embodiments, the relevant information of the data required by the user can be input through a graphical user interface provided by the public cloud management platform.

[0040] In some embodiments, the task information may include an execution information set, and the execution information set may include an execution order and execution instructions based on the execution order. In some embodiments, the tasks are executed in a predetermined order, and the execution instructions specify the specific operations that should be performed in each step. In some embodiments, the task information may further include one or more of a task UUID, a project ID, notes, and execution data parameters. In some embodiments, the task UUID corresponds to the task information. In some embodiments, the project ID corresponds to the target private cloud involved in the task. In some embodiments, the notes may provide information that the user wants to additionally mark or record. In some embodiments, the execution data parameters may be variables used to feedback the execution status and execution progress of the target private cloud in executing the task.

[0041] In some embodiments, the execution data parameters may include one or more of the following: the total number of temporary data results (for example, the total number of temporary data results generated by the target private cloud in the process of executing the current task), task batch, operation type (for example, the type of operation currently performed by the target private cloud, for example, the target private cloud is currently performing a query operation or other operation), execution status (for example, the target private cloud is currently in a state of executing a task or in a waiting state), execution information (for example, relevant information about the execution instruction currently being executed by the target private cloud), creation time, last execution time, end time, and file download path.

[0042] In some embodiments, the task information is sent from the public cloud to a target private cloud storing private cloud data. In some embodiments, the target private cloud stores the task information in a database of the target private cloud to call the task information and execute it, or to facilitate updating execution data parameters.

[0043] Step 220, execute the execution instruction based on the execution order in the task information to query the local database of the target private cloud and obtain the data result; the data result is returned to the public cloud. In some embodiments, step 220 can be implemented by the query module 2100 and the second data transmission module 2200 of the private cloud.

[0044] In one or more embodiments of this specification, the execution information set may include one or more execution instructions, and each execution instruction has a corresponding execution order. Exemplarily, the execution information set may include execution instruction 1, execution instruction 2, execution instruction 3, and so on. In some embodiments, the target private cloud executes one or more execution instructions in sequence according to the execution order, for example, executing execution instruction 1 first, then executing execution instruction 2, etc.

[0045] In some embodiments, the data result may be a table including all data obtained by querying the local database of the target private cloud, or a file generated based on the table. In some embodiments, the data result is returned to the public cloud, for example, by the second data transmission module 2200.

[0046] Figure 3 is a schematic diagram of a process for obtaining data results according to some embodiments of this specification. In some embodiments, Figure 2 Step 220 in the above example can be performed as follows: Figure 3 The process 300 shown is implemented. In some embodiments, Figure 3 The process 300 shown can be implemented by a processing device in a private cloud, for example, Figure 1 The private cloud device 120 shown is implemented. In some embodiments, Figure 3 The process 300 shown can be performed by Figure 7 The private cloud device 2000 in the data export system 700 shown is implemented. Figure 3 As shown, in some embodiments, process 300 may include the following steps.

[0047] Step 310 , execute one or more execution instructions based on the execution order to query the local database of the target private cloud, and obtain one or more temporary data results corresponding to the one or more execution instructions.

[0048] In some embodiments, the temporary data result may be temporary data obtained by the execution instruction in the execution instruction set in the target private cloud execution task information. Exemplarily, the target private cloud may execute execution instruction 1 and obtain the corresponding temporary data result td1, execute execution instruction 2 and obtain the corresponding temporary data result td2, and so on. In some embodiments, the temporary data may be stored in a data structure such as a temporary table for subsequent processing. In some embodiments, the target private cloud may execute an execution instruction to obtain a temporary data result, or execute multiple execution instructions in sequence to obtain multiple temporary data results.

[0049] In some embodiments, step 310 may include: executing a query instruction in the execution instruction based on the execution order to obtain a temporary data result; and storing the temporary data result in a temporary table created by a table creation instruction corresponding to the query instruction.

[0050] In one or more embodiments of the present specification, the execution instruction may include a query instruction and a table creation instruction.

[0051] In some embodiments, the query instruction may be a query statement, such as a SELECT statement, for extracting data from a database. In some embodiments, the syntax of the query statement may include: "SELECT column name FROM data table name WHERE query condition GROUP BY grouping field HAVING constraint condition after grouping ORDER BY sorting field LIMIT limit output row number; ". In some embodiments, data such as data table name, query condition, grouping field, etc. may be input by a user through a graphical user interface provided by a public cloud management platform.

[0052] In some embodiments, the table creation instruction may be a table creation statement, such as a CREATE TABLE statement, used to create a new table in a database. In some embodiments, the syntax of the table creation statement may include: "CREATE TABLE table name (column name 1 data type constraint, column name 2 data type constraint, ... column name N data type constraint);". In some embodiments, the table name created by the table creation statement may be created according to specific rules, such as a temporary table temp_task_*. In some embodiments, the column name, data type, constraint, etc. may be determined by the relevant information of the data input by the user through the graphical user interface provided by the public cloud management platform.

[0053] Exemplarily, execution instruction 1 may include query instruction 1 and table creation instruction 1, and execution instruction 2 may include query instruction 2 and table creation instruction 2, etc. In some embodiments, the target private cloud may execute query instruction 1 to obtain the corresponding temporary data result temp_data_1, execute table creation instruction 1 to create a temporary table temp_task_1, and store the temporary data result temp_data_1 in the temporary table temp_task_1, and continue to execute query instruction 2 to obtain the corresponding temporary data result temp_data_2, execute table creation instruction 2 to create a temporary table temp_task_2, and store the temporary data result temp_data_2 in the temporary table temp_task_2, and so on, until all execution instructions are executed.

[0054] Step 320 , generating a data result based on the one or more temporary data results.

[0055] In one or more embodiments of this specification, the target private cloud can generate a data result based on a temporary data result, or merge multiple temporary data results to generate a data result. In some embodiments, the target private cloud can generate a corresponding file based on one or more temporary data results and return the file to the public cloud.

[0056] In some embodiments, step 320 may include: after executing all execution instructions in the execution information set, merging all temporary tables storing temporary data results and generating data results.

[0057] Exemplarily, the target private cloud executes execution instructions 1 to execution instructions 10 in sequence, for example, executes query instruction 1, table creation instruction 1, query instruction 2, table creation instruction 2... in sequence, and correspondingly obtains temporary tables temp_task_1 to temporary tables temp_task_10 storing temporary data results temp_data_1 to temporary data results temp_data_10, then merges temporary tables temp_task_1 to temporary tables temp_task_10, and generates a file (i.e., data results) based on the merged tables.

[0058] In other embodiments, step 320 may include: each time an execution instruction in an execution information set is executed, a temporary table storing temporary data results established by the execution instruction is merged with a temporary table storing temporary data results established by a previous execution instruction in the execution information set to generate a new temporary table, and after all execution instructions in the execution information set are executed, data results are generated based on the new temporary table generated by the final merger.

[0059] Exemplarily, the target private cloud executes execution instructions 1 to 10 in sequence. In some embodiments, the target private cloud executes query instruction 1 and table creation instruction 1, and correspondingly obtains temporary table temp_task_1 storing temporary data result temp_data_1. The target private cloud further executes query instruction 2 and table creation instruction 2, and correspondingly obtains temporary table temp_task_2 storing temporary data result temp_data_2, and merges temporary table temp_task_2 with temporary table temp_task_1 to obtain temporary table temp_task_2 storing temporary data result temp_data_1 and temporary data result temp_data_2, and so on, finally obtaining temporary table temp_task_10 storing temporary data results temp_data_1 to temporary data results temp_data_10, and generating a file (i.e., data result) based on temporary table temp_task_10.

[0060] In some embodiments, the target private cloud may also merge the temporary table temp_task_2 storing the temporary data result temp_task_2 and the temporary table temp_task_1 storing the temporary data result temp_task_1 into the temporary table temp_task_1, and successively merge the subsequent temporary tables temp_task_3 to temporary tables temp_task_10 storing the corresponding temporary data results acquired in sequence into the temporary table temp_task_1, and generate a file (i.e., data result) based on the temporary table temp_task_1.

[0061] In some embodiments, the target private cloud may also create a new temporary table such as temporary table temp_task_x, and successively merge the temporary tables temp_task_1 to temp_task_10 storing corresponding temporary data results acquired in sequence into the temporary table temp_task_x, and generate a file (ie, data result) based on the temporary table temp_task_x.

[0062] In one or more embodiments of the present specification, in step 310 and step 320, the target private cloud continuously executes the execution instructions in sequence until all the execution instructions are executed, and then returns the data results to the public cloud.

[0063] Figure 4 is another flow chart of obtaining data results according to some embodiments of this specification. In some embodiments, Figure 2 Step 220 in can also be implemented according to process 400. Figure 3 The process 300 shown, Figure 4 The process 400 shown can be implemented by the interaction of processing devices in the public cloud and the private cloud, for example, Figure 1 The public cloud device 110 is shown to be implemented in interaction with the private cloud device 120. In some embodiments, the process 400 may include the following steps.

[0064] Step 410, execute an execution instruction based on the execution order to query the local database of the target private cloud, and obtain a temporary data result corresponding to the execution instruction.

[0065] In some embodiments, the temporary data result may be temporary data obtained by executing the execution instruction in the execution instruction set in the target private cloud execution task information. Exemplarily, the target private cloud may execute the execution instruction 1 and obtain the corresponding temporary data result td1, and perform subsequent steps (such as step 420 and step 430) after obtaining the temporary data result td1.

[0066] In some embodiments, step 410 may include: executing the query instruction in the execution instruction based on the execution order to obtain the temporary data result, and storing the temporary data result in a temporary table created by the table creation instruction corresponding to the query instruction.

[0067] In some embodiments, the target private cloud can execute query instruction 1 to obtain the corresponding temporary data result temp_data_1, execute table creation instruction 1 to create a temporary table temp_task_1, and store the temporary data result temp_data_1 in the temporary table temp_task_1. The target private cloud does not continuously execute the next execution instruction 2 (such as query instruction 2 and table creation instruction 2) but executes subsequent steps and then executes the next execution instruction 2 based on the decision of the subsequent steps.

[0068] Step 420: Update the execution data parameters included in the task information, and synchronize the execution data parameters to the public cloud.

[0069] In some embodiments, the target private cloud may update the execution data parameters at regular intervals or in real time, and synchronize the execution data parameters to the public cloud so that users in the public cloud can understand the status of the execution instructions executed by the target private cloud.

[0070] In some embodiments, the database or log of the target private cloud can be queried to synchronize the execution data parameters. Exemplarily, the target private cloud updates the values ​​of the execution data parameters (such as the total number of temporary data results, task batches, operation types, execution status, execution information, creation time, last execution time, end time, file download path, etc.) to the database. For example, the target private cloud stores the task information including the execution data parameters to the database of the target private cloud, and updates the values ​​of these execution data parameters in the database of the target private cloud. In some embodiments, the database of the target private cloud is synchronized with the database of the public cloud to synchronize the execution data parameters in the database of the target private cloud to the public cloud.

[0071] In some embodiments, the target private cloud is configured with an incremental subscription and consumption component (e.g., a canal component) to obtain incremental data of the target private cloud's database (e.g., obtaining incremental data through the log of the target private cloud's database), and send the incremental data to the public cloud through a message queue of a messaging system (e.g., a RabbitMQ component) to achieve database synchronization.

[0072] In step 430 , the public cloud determines the next execution instruction that the target private cloud should execute based on the execution data parameter, and sends it to the target private cloud to trigger the next execution instruction to obtain the corresponding temporary data result.

[0073] In some embodiments, the public cloud may determine whether the target private cloud starts executing the next execution instruction based on the execution data parameter. Exemplarily, the user may monitor the execution data parameter and make a decision through a graphical user interface provided by the public cloud management system.

[0074] For example, the user can determine whether sufficient data has been obtained based on the total number of temporary data results fed back in real time. When the user has obtained sufficient data, the user can send a command to return data results to the target private cloud through the public cloud management system. The target private cloud ends its currently executed command based on the command to return data results, and generates and returns data results based on the currently obtained temporary data results.

[0075] Step 440 , generating a data result based on the one or more temporary data results.

[0076] In some embodiments, step 440 is similar to step 320 , and the details may refer to the above description of step 320 , which will not be repeated here.

[0077] Figure 5 It is a flowchart of another data export method shown in some embodiments of this specification. Figure 5 The process 500 shown can be executed by the public cloud device and the private cloud device interacting with each other, for example, Figure 1 The public cloud device 110 and / or private cloud device 120 shown are implemented. In some embodiments, Figure 5 The process 500 shown can be performed by Figure 7 The data export system 700 shown is implemented. Figure 5 As shown, in some embodiments, process 500 may include the following steps.

[0078] Step 510, in response to the user instruction, task information for querying the private cloud data is generated, the task information includes an execution information set, the execution information set includes an execution order and an execution instruction based on the execution order; the task information is sent from the public cloud to the target private cloud storing the private cloud data. In some embodiments, step 510 can be implemented by the configuration module 1100 and the first data transmission module 1300 of the public cloud.

[0079] In some embodiments, step 510 is similar to step 210. For details, please refer to the above description of step 210, which will not be repeated here.

[0080] Step 520, delete the temporary data and / or temporary table generated during the execution of the previous task information. In some embodiments, step 520 is an optional step. In some embodiments, step 520 can be implemented by the temporary data deletion module 1200 of the shared cloud.

[0081] In some embodiments, the public cloud may send the same or similar task information to the same target private cloud, and based on the task information, the target private cloud may generate a temporary table with a similar naming rule. In some embodiments, in order to prevent the existing temporary table from affecting the data acquisition in subsequent tasks, before the target private cloud starts a new task, the temporary data generated during the execution of the previous task information, such as the temporary table, is deleted.

[0082] For example, when a new task may involve establishing a temporary table with a prefix of "temp_task_", all temporary tables with the prefix of "temp_task_" may be deleted before the target private cloud starts the new task.

[0083] Step 530, execute the execution instruction based on the execution order in the task information to query the local database of the target private cloud and obtain the data result; the data result is returned to the public cloud. In some embodiments, step 530 can be implemented by the query module 2100 and the second data transmission module 2200 of the private cloud.

[0084] In some embodiments, step 530 is similar to step 220 , and the details may refer to the above description of step 220 , which will not be repeated here.

[0085] Step 540, based on the execution information set included in the task information, a configuration file is generated and stored so that the execution information set can be repeatedly executed. In some embodiments, step 540 is an optional step. In some embodiments, step 540 can be implemented by the configuration file generation module 1400 of the public cloud.

[0086] In some implementations, the public cloud may store the execution information set set by the user, such as generating and storing a configuration file from the execution information set, and specifically generating and storing a configuration file from the task information, so that the execution information set or the task information can be repeatedly executed.

[0087] In some embodiments, the user may provide a user instruction, and the private cloud generates task information A based on the user instruction, and sends the task information A to the target private cloud A to obtain the corresponding data results in the database of the private cloud A. In some embodiments, the task information A or the execution information set A in the task information A is stored as a configuration file A. In some embodiments, the user directly calls the configuration file A, and sends the task information A involved in the configuration file A to the target private cloud B to obtain the same type of data results in the database of the private cloud B.

[0088] In some embodiments, multiple configuration files can be saved for user selection. In some embodiments, once the configuration file is generated, it can be repeatedly called without the need for field personnel to provide user instructions, so that field personnel can query required data without understanding database instructions (such as SQL statements).

[0089] In some embodiments, the configuration file may be provided and stored in advance by the developer and called by the field personnel. In some embodiments, the public cloud may store a preset configuration file. In some embodiments, since the task information involved in the configuration file or the execution instructions in the execution information set can be customized according to the user's needs, the field personnel can obtain data results from the database of the target private cloud when the current version of the public cloud management system does not support the export of certain data, without changing the code of the public cloud.

[0090] In some embodiments, the configuration file can be stored in or exported from the public cloud. In some related scenarios, when an outsider who does not understand database statements needs to obtain some data that the current version of the public cloud does not support, the outsider can request the R&D personnel to write, generate, and provide a configuration file, and store the configuration file in the public cloud, and obtain the required data by calling the configuration file.

[0091] Figure 6 It is a flowchart of a data export method according to some embodiments of this specification. Figure 6 The process shown can be executed by a processing device, for example, Figure 1 The public cloud device 110 shown is implemented. In some embodiments, Figure 6 The process 600 shown can be implemented by a data export system 1000 deployed on a public cloud processing device. Figure 6 As shown, in some embodiments, process 600 may include the following steps.

[0092] Step 610 , generating task information for querying private cloud data in response to a user instruction, the task information including an execution information set, the execution information set including an execution sequence and execution instructions based on the execution sequence.

[0093] In some embodiments, step 610 is similar to step 210 , and the details may refer to the description of step 210 above, which will not be repeated here.

[0094] Step 620, sending the task information to the target private cloud storing the private cloud data.

[0095] Step 630, receiving the returned data results from the target private cloud, the data results are obtained by the target private cloud by querying the local database of the target private cloud based on the execution order in the task information.

[0096] In some embodiments, the method by which the target private cloud executes the execution instruction based on the execution order in the task information to query the local database of the target private cloud is similar to step 220. For details, please refer to the descriptions of multiple relevant paragraphs in the previous step 220, which will not be repeated here.

[0097] Figure 7 is a schematic diagram of the system structure of a data export system according to some embodiments of this specification. Figure 7 As shown, the data export system 700 may include a public cloud device 1000 and a private cloud device 2000 .

[0098] In some embodiments, the public cloud device 1000 may include a configuration module 1100 and a first data transmission module 1300 .

[0099] The configuration module 1100 is used to generate task information for querying private cloud data in response to a user instruction, where the task information includes an execution information set, and the execution information set includes an execution order and execution instructions based on the execution order.

[0100] The first data transmission module 1300 is used to send task information from the public cloud to a target private cloud storing private cloud data, or to receive data results returned to the public cloud.

[0101] In some optional embodiments, the public cloud device 1000 may further include a temporary data deletion module 1200 and / or a configuration file generation module 1400 .

[0102] The temporary data deletion module 1200 is used to delete temporary data and / or temporary tables generated during the execution of the previous task information.

[0103] The configuration file generating module 1400 is used to generate and store a configuration file based on the execution information set included in the task information, so that the execution information set can be repeatedly executed.

[0104] In some embodiments, the private cloud device 2000 may include a query module 2100 and a second data transmission module 2200 .

[0105] The query module 2100 is used to execute the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtain data results.

[0106] The second data transmission module 2200 is used to receive task information sent from the public cloud to a target private cloud storing private cloud data, or to return data results to the public cloud.

[0107] For more information about each module, see Figures 1 to 6 The relevant description of will not be repeated here. It should be understood that Figure 7 The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or a dedicated design hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer executable instructions and / or control codes contained in a processor, such as a disk, CD or DVD-ROM, etc., and such codes are provided in a carrier medium or a memory of a programmable device. The system and its modules of this specification can not only be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software such as executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (for example, firmware). It should be noted that the above description of the system and its modules is only for the convenience of description and cannot limit this specification to the scope of the embodiments cited. It is understandable that, after understanding the principle of the system, those skilled in the art may arbitrarily combine the modules to form a subsystem connected to other modules without deviating from the principle. Or some modules may be split to obtain more modules or multiple units under the module. Such variations are all within the scope of the disclosure of this specification.

[0108] In one or more embodiments of this specification, a computer program product is also provided, including computer instructions or computer codes, when at least part of the computer instructions or computer codes are executed by a processor, the above-mentioned data export method can be implemented. In some embodiments, the computer program product may only involve computer instructions or computer codes, which may be carried by a storage medium or a processing device. In other embodiments, the computer program product may be a storage medium or a processing device containing the aforementioned computer instructions or computer codes. The processing device may include one or more processors, and a storage medium.

[0109] In some embodiments, the processor may be a combination of one or more of the following processors: a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a programmable logic controller (PLC), a reduced instruction set computer (RISC), and a microprocessor.

[0110] In some embodiments, the storage medium may include a combination of one or more of the following: a mass storage, a removable storage, a volatile read-write memory, and a read-only memory (ROM). Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state hard disk, and the like. Exemplary removable storage may include a flash disk, a floppy disk, an optical disk, a memory card, a compressed hard disk, a magnetic tape, and the like. Exemplary volatile read-write memory may include a random access memory (RAM). Exemplary random access memory may include a dynamic random access memory (DRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), a static random access memory (SRAM), a thyristor random access memory (T-RAM), and a zero capacitance memory (Z-RAM), and the like. Exemplary read-only memory may include a masked read-only memory (MROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM), and a digital multifunction hard disk read-only memory, and the like.

[0111] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) packaging user instructions into task information and sending it to the target private cloud, the target private cloud executes the execution information set in the task information to obtain data results and returns it to the public cloud, thereby allowing field personnel to obtain data not stored in the public cloud from the database of the private cloud; (2) the private cloud sequentially and continuously executes the execution instructions in the execution information set based on the execution book sequence, thereby realizing fast and batch export of data; (3) the private cloud feeds back execution data parameters to the public cloud so that the public cloud can monitor the task execution status of the private cloud, allowing the user to instruct the private cloud to execute the next instruction or instruct the private cloud to return data results according to needs; (4) before the target private cloud starts to execute the task, the previous temporary data is deleted to avoid data interference. (5) the execution information set is stored as a configuration file so that field personnel can request similar data from different private clouds. (6) a preset configuration file is provided so that field personnel can request data from the private cloud without understanding the database statement. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0112] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are taught in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

Claims

1. A data export method, characterized in that: Remotely querying private cloud data through a public cloud, the method comprising: generating task information for querying the private cloud data in response to a user instruction, the task information comprising an execution information set, the execution information set comprising an execution sequence and an execution instruction based on the execution sequence; the task information is sent from the public cloud to a target private cloud storing the private cloud data; The execution instructions are executed based on the execution order in the task information to query the local database of the target private cloud and obtain data results; the data results are returned to the public cloud.

2. The data export method according to claim 1, characterized in that: The execution information set includes one or more execution instructions based on the execution order; executing the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtain data results includes: Executing one or more of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining one or more temporary data results corresponding to the one or more execution instructions; The data result is generated based on one or more of the temporary data results.

3. The data export method according to claim 2, characterized in that: The execution instructions include query instructions and table creation instructions; The executing one or more of the execution instructions based on the execution order to query the local database of the target private cloud and obtaining one or more temporary data results corresponding to the one or more execution instructions includes: Execute the query instruction in the execution instruction based on the execution order to obtain a temporary data result; The temporary data result is stored in a temporary table created by the table creation instruction corresponding to the query instruction.

4. The data export method according to claim 1, characterized in that: The execution information set includes one or more execution instructions based on the execution order; executing the execution instructions based on the execution order in the task information to query the local database of the target private cloud and obtain data results includes: Executing one of the execution instructions based on the execution order to query the local database of the target private cloud, and obtaining a temporary data result corresponding to the execution instruction; Updating the execution data parameters contained in the task information, and synchronizing the execution data parameters to the public cloud; The public cloud determines the next execution instruction to be executed by the target private cloud based on the execution data parameter, and sends it to the target private cloud to trigger the next execution instruction to obtain the corresponding temporary data result; The data result is generated based on one or more of the temporary data results.

5. The data export method according to claim 4, characterized in that: The execution instructions include query instructions and table creation instructions; The executing one of the execution instructions based on the execution order to query the local database of the target private cloud and obtaining a temporary data result corresponding to the execution instruction includes: Execute the query instruction in the execution instruction based on the execution order to obtain a temporary data result; The temporary data result is stored in a temporary table created by the table creation instruction corresponding to the query instruction.

6. The data export method according to any one of claims 2 to 5, characterized in that: Generating the data result based on one or more of the temporary data results comprises: After all the execution instructions in the execution information set are executed, all the temporary tables storing the temporary data results are merged to generate the data results.

7. The data export method according to any one of claims 2 to 5, characterized in that: Generating the data result based on one or more of the temporary data results comprises: Each time an execution instruction in the execution information set is executed, the temporary table storing the temporary data result established by the execution instruction is merged with the temporary table storing the temporary data result established by the previous execution instruction in the execution information set to generate a new temporary table, and after all the execution instructions in the execution information set are executed, the data result is generated based on the new temporary table generated by the final merger.

8. The data export method according to claim 1, characterized in that: Before executing the execution instruction based on the execution order in the task information to query the local database of the target private cloud and obtain the data result, the method further includes: Delete the temporary data and / or temporary table generated during the execution of the previous task information.

9. The data export method according to claim 1, characterized in that: The task information also includes one or more of the following: a task UUID corresponding to the task information, a project ID corresponding to a target private cloud of the task information, remarks, and execution data parameters; The execution data parameters include one or more of the following: total number of temporary data results, task batch, operation type, execution status, execution information, creation time, last execution time, end time, and file download path.

10. The data export method according to claim 1 or 9, characterized in that: Also includes: A configuration file is generated based on the execution information set included in the task information and stored, so that the execution information set can be repeatedly executed.

11. A data export system, characterized in that: include: a configuration module, in response to a user instruction, generating task information for querying the private cloud data, the task information comprising an execution information set, the execution information set comprising an execution sequence and an execution instruction based on the execution sequence; wherein the task information is sent from the public cloud to a target private cloud storing the private cloud data; A query module executes the execution instruction based on the execution order in the task information to query the local database of the target private cloud and obtain data results; wherein the data results are returned to the public cloud.

12. A computer program product, comprising computer codes, which, when at least part of the computer codes is executed by a processor, can implement the data synchronization method according to any one of claims 1 to 10.

13. A data export method, characterized in that: Remotely querying private cloud data through a public cloud, the method comprising: generating task information for querying the private cloud data in response to a user instruction, the task information comprising an execution information set, the execution information set comprising an execution sequence and an execution instruction based on the execution sequence; Sending the task information to a target private cloud storing the private cloud data; The returned data result is received from the target private cloud, and the data result is acquired by the target private cloud by querying the local database of the target private cloud based on the execution order in the task information by executing the execution instruction.

Images