A data transmission method, system, device and storage medium

Abstract

The application discloses a data transmission method, system, device and storage medium, and applies to a data management technical field and a preset movable storage device, and comprises the following steps: after the movable storage device is connected with a source device, in a command line mode, database data to be exported of the source device is specified through a command line; the database data to be exported is read and converted into a readable and executable text file through the command line, and the text file is stored in the preset movable storage device; after the movable storage device is connected with a target device, the readable and executable text file stored in the movable storage device is converted into database data through the command line, and the database data is imported into a specified database of the target device. According to the scheme, data transmission can be effectively realized in a special scene, and the reliability and flexibility of data transmission are ensured.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a data transmission method, system, device and storage medium. Background Technology

[0002] In certain special scenarios, such as those with high security and confidentiality requirements, data cannot be transmitted via conventional online networks; that is, communication between different networks is limited or even impossible. Furthermore, in some special scenarios, the lack of a visual desktop operating system also adds complexity to data transmission.

[0003] In conclusion, how to effectively achieve data transmission in certain special scenarios is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this invention is to provide a data transmission method, system, device, and storage medium to effectively realize data transmission in some special scenarios.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A data transmission method, applied in a pre-defined removable storage device, includes:

[0007] After the removable storage device is connected to the source device, the database data to be exported from the source device is specified via the command line in command line mode.

[0008] The database data to be exported is read and converted into a readable and executable text file via the command line, and stored in a preset removable storage device.

[0009] After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted into database data via command line and imported into the specified database of the target device.

[0010] In one implementation, the database data to be exported is read and converted into a readable and executable text file via a command line, and stored in a preset removable storage device, including:

[0011] Using the command line, the different contents of the database data to be exported are read and converted into readable and executable text files in the order set for export, and the converted readable and executable text files are stored in a preset removable storage device.

[0012] In one implementation, via command line, different contents of the database data to be exported are read sequentially and converted into readable and executable text files according to a set export order, and the converted readable and executable text files are stored in a preset removable storage device, including:

[0013] The database data to be exported is read and converted into a readable and executable text file via command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0014] The table structure of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0015] The table data of the database to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0016] The table sequence index of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0017] The database view to be exported is read and converted into a readable and executable text file via command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0018] The stored procedure function of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device.

[0019] In one implementation, the process of reading the schema, table structure, table sequence index, view, or stored procedure function of the database data to be exported and converting it into a readable and executable text file, and storing the converted readable and executable text file in a preset removable storage device, is performed in parallel by K concurrent threads.

[0020] Where K is a positive integer not less than 2.

[0021] In one embodiment, the table data of the database data to be exported is read and converted into a readable and executable text file, and the converted readable and executable text file is stored in a preset removable storage device, including:

[0022] When the table data export configuration is determined to be in multi-table mode, M concurrent threads are used via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of multi-table mode. The converted readable and executable text file is then stored in a preset removable storage device.

[0023] When the table data export configuration is set to large table mode, M concurrent threads are used via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of large table mode. The converted readable and executable text file is then stored in a preset removable storage device.

[0024] The rules of the multi-table mode include: each time x tables are read from the table data of the database data to be exported, and only a single query statement is generated for each table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device;

[0025] The rules of the large table mode include: each time, x tables are read from the table data of the database data to be exported, and for the first type of table, only a single query statement is generated for a single table, and for the second type of table, multiple query statements are generated for a single table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device; M is a positive integer not less than 2, and x is a positive integer.

[0026] In one implementation, the table data of the database data to be exported is read and converted into a readable and executable text file via a command line, including:

[0027] The command line reads the table data of the database to be exported and converts it into a readable and executable Structured Query Language text file and a comma-separated value text file, and determines whether there are any errors.

[0028] If not, then the Structured Query Language text file and the comma-separated value text file will be used as readable and executable text files of the converted table data corresponding to the database data;

[0029] If so, the table data of the database data to be exported is read and converted into a readable and executable Structured Query Language (SCL) text file, and the SCL text file is used as the readable and executable text file corresponding to the converted table data of the database data.

[0030] In one implementation, after the removable storage device is connected to the target device, readable and executable text files stored in the removable storage device are converted into database data via command line and imported into a specified database on the target device, including:

[0031] After the removable storage device is connected to the target device, the different contents of the readable and executable text files stored in the removable storage device are converted into database data in the order set by the command line, and the converted database data is imported into the specified database of the target device.

[0032] In one implementation, after the removable storage device is connected to the target device, different contents of readable and executable text files stored in the removable storage device are converted into database data sequentially according to a set import order via command line, and the converted database data is imported into a specified database on the target device, including:

[0033] After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted via command line to obtain the database data schema, and the converted database data schema is imported into the specified database of the target device.

[0034] The command line is used to convert the readable and executable text file stored in the removable storage device to obtain the table structure of the database data, and then the converted table structure of the database data is imported into the specified database of the target terminal device.

[0035] The command line is used to convert readable and executable text files stored in the removable storage device to obtain table data of the database data, and then import the converted table data of the database data into the specified database of the target terminal device.

[0036] The readable and executable text files stored in the removable storage device are converted using the command line to obtain the table sequence index of the database data, and the converted table sequence index of the database data is imported into the specified database of the target terminal device.

[0037] The command line is used to convert readable and executable text files stored in the removable storage device to obtain a view of database data, and then the converted view of database data is imported into a specified database of the target device.

[0038] The command line is used to convert readable and executable text files stored in the removable storage device to obtain stored procedure functions for database data, and then imports the converted stored procedure functions into a specified database on the target device.

[0039] A data transmission system, applied in a pre-defined removable storage device, includes:

[0040] The data export determination module is used to specify the database data to be exported from the source device via command line in command line mode after the removable storage device is connected to the source device.

[0041] The export module is used to read the database data to be exported and convert it into a readable and executable text file via command line, and store it in a preset removable storage device;

[0042] The import module is used to convert readable and executable text files stored in the removable storage device into database data and import them into a specified database on the target device via command line after the removable storage device is connected to the target device.

[0043] A data transmission device, used in a pre-set removable storage device, includes:

[0044] Memory, used to store computer programs;

[0045] A processor for executing the computer program to implement the steps of the data transmission method as described above.

[0046] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data transmission method described above.

[0047] Applying the technical solution provided by the embodiments of this invention, considering that there is no visual desktop operating system in some special scenarios, all operations of the solution in this application can be implemented based on the command line in command-line mode, thereby effectively dealing with the situation of no visual desktop operating system and ensuring the implementation of the solution in this application. Specifically, this application is implemented based on a preset removable storage device. After connecting the removable storage device to the source device, in command-line mode, the database data to be exported from the source device is specified via the command line. Then, the database data to be exported is read and converted into a readable and executable text file via the command line and stored in the preset removable storage device. Subsequently, after connecting the removable storage device to the target device, the readable and executable text file stored in the removable storage device is converted into database data via the command line and imported into the specified database on the target device, thus completing the process of importing the specified database data to be exported from the source device into the specified database on the target device. It can be seen that the above data transmission process does not rely on a network connection, effectively dealing with scenarios where network communication is limited. Furthermore, since direct copying may lead to errors, the solution in this application reads the database data to be exported through the command line and converts it into a readable and executable text file. Then, the text file is converted back into database data and imported into the specified database on the target device. This effectively avoids data errors, thus ensuring the reliability of the data transmission solution in this application. In addition, converting it into a readable and executable text file makes it easy to modify, ensuring flexibility.

[0048] In summary, for some special scenarios, the solution proposed in this application can effectively realize data transmission while ensuring the reliability and flexibility of data transmission. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating an implementation of a data transmission method according to the present invention.

[0051] Figure 2 This is a schematic diagram of the structure of a data transmission system according to the present invention;

[0052] Figure 3 This is a schematic diagram of the structure of a data transmission device according to the present invention;

[0053] Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium according to the present invention;

[0054] Figure 5 This is a schematic diagram illustrating the process of reading and converting database data to be exported into a readable and executable text file in a specific embodiment of the present invention. Detailed Implementation

[0055] The core of this invention is to provide a data transmission method. For some special scenarios, the solution of this application can effectively realize data transmission and ensure the reliability and flexibility of data transmission.

[0056] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an implementation of a data transmission method according to the present invention. This data transmission method can be applied to a pre-defined removable storage device and includes the following steps:

[0058] Step S101: After the removable storage device is connected to the source device, in command line mode, specify the database data to be exported from the source device via the command line.

[0059] The specific type of removable storage device can be set and adjusted according to actual needs, as long as it can meet the functional requirements of this application. Of course, in practical applications, the source and target devices usually have dedicated interfaces for the special scenarios faced by this application. The removable storage device also needs to have such a dedicated interface in order to connect with the source and target devices.

[0060] After the removable storage device is connected to the source device, since it is necessary to deal with the scenario of no visual desktop operating system, the solution of this application is implemented in command line mode. When executing step S101, the database data to be exported from the source device is specified through the command line. That is, the database data under which path needs to be transferred to the specified database of the target device is specified through the command line.

[0061] The specific editing method of the command line can be set according to the actual situation. For example, in some cases, --source is used to specify the source device, --target is used to specify the target device, --path is used to specify the directory path, etc.

[0062] Furthermore, it should be noted that since the solution in this application requires command-line implementation, repeatedly reading user-inputted command lines to execute corresponding steps would be cumbersome and increase the probability of errors. Therefore, in practical applications, a semi-automatic or fully automatic implementation method can be adopted. For example, in a fully automatic implementation, the command lines required for each step can be pre-written and packaged into a script or tool. This script or tool can then automatically input the corresponding command lines and execute the steps, effectively improving the implementation efficiency of the solution. Similarly, the semi-automatic implementation described here also allows for pre-writing the command lines required for each step. During execution, the user can then select which command line to execute, offering greater flexibility.

[0063] Step S102: Read the database data to be exported and convert it into a readable and executable text file via the command line, and store it in a preset removable storage device.

[0064] Since direct copying may result in errors, the solution in this application reads the database data to be exported through the command line and converts it into a readable and executable text file. Then, the text file is converted into database data and imported into the specified database on the target device. This effectively avoids data errors and ensures the reliability of the data transmission solution in this application.

[0065] Furthermore, after reading the database data to be exported and converting it into a readable and executable text file, any modifications to the data can be made at this stage. That is, as needed, operations such as modification and deletion can be performed on the readable and executable text file.

[0066] The specific format of the readable and executable text file can be set and adjusted as needed. For example, in one scenario, the database data to be exported from the source device is specifically the database data of the Postgres SQL database, and the readable and executable text file obtained in step S102 can be, for example, specifically an SQL (Structured Query Language) text file.

[0067] Step S103: After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted into database data via command line and imported into the specified database of the target device.

[0068] After reading the database data to be exported and converting it into readable and executable text files, these text files need to be stored on a pre-defined removable storage device. Then, the user needs to unplug the removable storage device from the source device and connect it to the target device. Next comes the import process, the reverse of the export process. This involves using the command line to convert the readable and executable text files stored on the removable storage device into database data, which is then imported into the specified database on the target device.

[0069] In one specific embodiment of the present invention, step S102 may specifically include:

[0070] Using the command line, the different contents of the database data to be exported are read and converted into readable and executable text files in the order set for export, and the converted readable and executable text files are stored in a preset removable storage device.

[0071] This implementation further considers that, during the process of reading the database data to be exported and converting it into a readable and executable text file, since the database data to be exported has different aspects of content, errors will also occur if it is not exported in a certain order. Therefore, in this implementation, when executing step S102, the different contents of the database data to be exported are read in sequence according to the set export order and converted into a readable and executable text file.

[0072] Furthermore, it is understandable that the specific export order is usually determined by the database type of the source device.

[0073] For example, in one specific embodiment of the present invention, step S102 may specifically include:

[0074] Step 1: Using the command line, read the schema of the database data to be exported and convert it into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device;

[0075] Step 2: Using the command line, read the table structure of the database data to be exported and convert it into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device;

[0076] Step 3: Using the command line, read the table data of the database to be exported and convert it into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device;

[0077] Step 4: Using the command line, read the table sequence index of the database data to be exported and convert it into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device;

[0078] Step 5: Using the command line, read the view of the database data to be exported and convert it into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device;

[0079] Step Six: Using the command line, read the stored procedure functions of the database data to be exported and convert them into a readable and executable text file, then store the converted readable and executable text file in a preset removable storage device.

[0080] This implementation method is a commonly used way to set the export order. For example, this implementation method can be used to execute step S102 for the PostgresSql database described above. Of course, in other cases, the export order that needs to be set when executing step S102 can be adjusted based on the specific type of the database.

[0081] In this implementation, the schema of the database data to be exported is first read and converted into readable and executable text files via command line. These converted text files, corresponding to the schemas, are then stored in a pre-set removable storage device. For example, in one scenario, a schema directory can be created first, then the schema of the database data to be exported is read and converted, thereby generating specific SQL text files for each schema in the schema directory. For example, the naming convention could be = incrementing number + schema + schema name. See also... Figure 5 This is a schematic diagram illustrating, in one specific implementation, reading database data to be exported and converting it into a readable and executable text file. Figure 5 In the process, the first step is to read the schema of the database data to be exported and convert it into a readable and executable text file via the command line. Figure 5 In this paper, Mode 1 and Mode 2 are used to represent different modes of the database data to be exported.

[0082] Then, the table structure of the database data to be exported is read and converted into readable and executable text files via the command line, and these converted text files corresponding to the table structure are stored in a preset removable storage device. For example, in one scenario, a table structure directory can be created first, and then the table structure of the database data to be exported can be read and converted, thereby generating SQL text files for each specific table structure in the table structure directory. For example, the naming rule could be = incrementing number + tableStructure + table structure name. Figure 5 In the process, after reading the schema of the database data to be exported and converting it into a readable and executable text file, the next step is to use the command line to read the table structure of the database data to be exported and convert it into a readable and executable text file. Figure 5 The table structure is represented by Table Structure 1, Table Structure 2, and Table Structure 3 to indicate the different table structures of the database data to be exported.

[0083] Then, the table data of the database to be exported is read and converted into readable and executable text files via the command line, and these text files corresponding to the table data are stored in a preset removable storage device.

[0084] For example, in one scenario, a master table directory can be created first. Then, within this master directory, separate table directories can be created for each individual table. The naming convention for these table directories could be: =incrementing number + tableData + master table directory name. For each table directory, the specific table data related to that table can be read from the database data to be exported, converted, and used to generate an SQL text file containing that table's data. Again, the naming convention for this SQL text file within each table directory could be: =incrementing number + tableData + table name.

[0085] Figure 5 In this process, the table data of the database to be exported is read and converted into a readable and executable text file, using Table Data 1, Table Data 2, and Table Data 3 to represent the different tables of the database to be exported. Furthermore, Figure 5 Specifically, the implementation uses SQL+CSV, which will be explained in detail in the following implementation method. This involves using the command line to read the table data of the database to be exported and converting it into a readable and executable Structured Query Language text file and a comma-separated value text file. Figure 5 In this context, for Table 1, the specific components include the SQL file and the CSV file for Table 1. Figure 5Taking Table 1 as an example, the first SQL file of Table 1 is represented as Table 1.sql, the first CSV file of Table 1 is represented as Table 1.csv, and correspondingly, the second SQL file of Table 1 is represented as Table 2.sql, the second CSV file of Table 1 is represented as Table 2.csv, the third SQL file of Table 1 is represented as Table 3.sql, and the third CSV file of Table 1 is represented as Table 3.csv.

[0086] Then, the sequence indexes of the database data to be exported are read and converted into readable and executable text files via the command line, and these text files corresponding to the sequence indexes are stored in a preset removable storage device. For example, in one scenario, a sequence index directory can be created first, and then the sequence indexes of the database data to be exported can be read and converted, thereby generating a series of SQL text files related to the sequence indexes in the sequence index directory. Figure 5 In this process, the sequence indexes of the database data to be exported are read and converted into readable and executable text files. Sequence index one, sequence index two, and sequence index three are used to represent the different sequence indexes of the database data to be exported.

[0087] Then, the view of the database data to be exported is read and converted into a readable and executable text file via the command line, and these converted text files corresponding to the view are stored in a preset removable storage device. For example, in one scenario, a view directory can be created first, and then the view of the database data to be exported can be read and converted, thereby generating a series of SQL text files about the view in the view directory. Figure 5 In this process, the view containing the database data to be exported is read and converted into a readable and executable text file. Figure 1 ,See Figure 2 and vision Figure 3 To represent different views of the database data to be exported.

[0088] Finally, the stored procedure functions of the database data to be exported are read and converted into readable and executable text files via the command line, and these text files corresponding to the stored procedure functions are stored in a preset removable storage device. For example, in one scenario, a directory of stored procedure functions can be created first, and then the stored procedure functions of the database data to be exported can be read and converted, thereby generating a series of SQL text files related to the stored procedure functions in the directory. Figure 5In this process, the stored procedure functions of the database data to be exported are read and converted into a readable and executable text file. The different stored procedure functions of the database data to be exported are represented by stored procedure function one, stored procedure function two, and stored procedure function three.

[0089] Furthermore, in one specific embodiment of the present invention, in the process of reading the schema, table structure, table sequence index, view, or stored procedure function of the database data to be exported and converting it into a readable and executable text file, and storing the converted readable and executable text file in a preset removable storage device, parallel processing can be performed by K concurrent threads; where K is a positive integer not less than 2.

[0090] This implementation takes into account that, during the execution of steps one, two, four, five, and six above, a multi-threaded concurrent control approach can be adopted. That is, these steps can be processed in parallel by K concurrent threads, thereby effectively improving the execution efficiency of the solution in this application. For example, in a specific case, K is set to 8.

[0091] For each thread, the starting concurrency threshold can also be set as needed. For example, it can usually be set to 1000, which means that for a single thread, when reading the corresponding data content in the database, it will read a maximum of 1000 rows and convert the read data content into a readable and executable text file.

[0092] In one specific embodiment of the present invention, step three above may specifically include:

[0093] When the table data export configuration is set to multi-table mode, the table data of the database to be exported is read and converted into a readable and executable text file by M concurrent threads via the command line, according to the rules of multi-table mode, and the converted readable and executable text file is stored in a preset removable storage device.

[0094] When the table data export configuration is set to large table mode, M concurrent threads are used via command line to read the table data of the database to be exported and convert it into a readable and executable text file according to the rules of large table mode. The converted readable and executable text file is then stored in a preset removable storage device.

[0095] For table data in the database, since the amount of data in the table data is very large and the amount of data in different tables may vary greatly, two rules are set in this implementation method.

[0096] The rules for the multi-table mode include: each time, x tables are read from the table data of the database data to be exported, and only a single query statement is generated for each table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device.

[0097] In multi-table mode, each table is treated as a relatively evenly distributed table without an extremely large amount of data. For example, in one scenario, there are 10,000 tables. Since it is impossible to read so many tables directly, only x tables are read from the table data of the database to be exported at a time, where x is a positive integer, for example, x is 100. That is, 100 tables are read from 10,000 tables at a time. After these 100 tables are completely processed, the next 100 tables can be processed until all 10,000 tables have been processed.

[0098] After reading 100 tables each time, only a single query statement is generated for each table. Whenever a query statement is generated and an idle thread exists, that idle thread can execute the query statement, thereby reading the table data corresponding to the query statement and converting it into a readable and executable text file. M is a positive integer not less than 2, for example, M=8, and assuming all 8 threads are currently idle, after the first 8 tables of the 100 tables each generate one query statement, they can be processed by these 8 idle threads respectively. It is understandable that when a thread finishes processing, it becomes an idle thread again, and then that thread can process the query statement corresponding to the 9th table, thereby reading the table data corresponding to the query statement and converting it into a readable and executable text file.

[0099] The rules for the large table mode include: each time, x tables are read from the table data of the database data to be exported. For the first type of table, only a single query statement is generated for a single table, and for the second type of table, multiple query statements are generated for a single table. Whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file. The converted readable and executable text file is then stored in a preset removable storage device.

[0100] As can be seen, in the large table mode, it is assumed that some tables have extremely large amounts of data. Therefore, in the large table mode, a fuzzy query can be performed on the number of rows in the table to determine whether it is a large table, i.e., whether it is a second type of table. If it is determined to be a first type of table, then, similar to the previous mode, only a single query statement is generated for a single table. For a second type of table, it indicates that it is a large table, therefore, multiple query statements can be generated for a single table. The specific number of query statements generated can be set according to the actual situation, for example, set to a fixed value, or set based on the determination result, without affecting the implementation of the present invention. It can be seen that in the large table mode, multiple threads will process the data of the same table, avoiding the situation where the processing time of large tables is too high, thus effectively ensuring the execution efficiency of the solution of this application.

[0101] In one specific embodiment of the present invention, step three, which describes reading the table data of the database to be exported and converting it into a readable and executable text file via command line, may specifically include:

[0102] The command line reads the table data of the database to be exported and converts it into a readable and executable Structured Query Language text file and a comma-separated value text file, and checks for errors.

[0103] If not, the Structured Query Language text file and the comma-separated value text file will be used as readable and executable text files of the converted table data corresponding to the database data;

[0104] If so, the table data of the database to be exported will be read and converted into a readable and executable Structured Query Language (SCL) text file, and the SCL text file will be used as the readable and executable text file corresponding to the converted table data of the database.

[0105] This implementation takes into account that, apart from table data, database data content such as schemas, table structures, views, sequence indexes, and stored procedure functions occupy relatively little space. Therefore, in practical applications, these aspects of data can usually be directly read and converted into readable and executable Structured Query Language text files, i.e., converted into SQL files.

[0106] Since table data occupies a lot of space, this implementation method takes into account that SQL+CSV (Comma-Separated Values) can be used. That is, the table data of the database to be exported is read through the command line and converted into a readable and executable Structured Query Language text file and a comma-separated value text file.

[0107] When using the SQL+CSV implementation method, only the SQL text file needs to be used in the header of the converted readable and executable text file, and the rest can use the CSV text file. The execution speed is very high when converting to a CSV text file, which ensures the implementation efficiency of the solution in this application.

[0108] Of course, this implementation also takes into account that since CSV text files require delimiters to separate content, although invisible special characters are used by default, there are still rare cases where the database data contains such delimiters, leading to errors in data splitting. Therefore, when executing this implementation, after obtaining the SQL+CSV text file, it will determine whether there is an error. If not, subsequent operations can continue. If there is an error, only the SQL conversion method can be used, that is, step three is re-executed. During the re-execution, only the table data of the database data to be exported is read and converted into a readable and executable Structured Query Language (SCL) text file, and the SCL text file is used as the readable and executable text file corresponding to the table data of the database data.

[0109] It can be seen that this implementation method ensures the reliability of the solution while also taking into account the need to improve execution efficiency.

[0110] In one specific embodiment of the present invention, step S103 may specifically include:

[0111] After the removable storage device is connected to the target device, the different contents of the readable and executable text files stored in the removable storage device are converted into database data in the order set by the command line, and the converted database data is imported into the specified database on the target device.

[0112] Similar to the export process described above, in this embodiment, during the process of converting readable and executable text files stored in the removable storage device into database data and importing them into the specified database of the target device, errors will occur if the database data to be imported does not contain different aspects of content and is not imported in a certain order. Therefore, in this embodiment, when executing step S103, the different contents of the readable and executable text files stored in the removable storage device are read and converted into database data in the order set in the import order, and then the converted database data is imported into the specified database of the target device.

[0113] Similarly, it is understandable that the specific import order is usually determined by the type of database specified on the target device.

[0114] In one specific embodiment of the present invention, step S103 may specifically include:

[0115] After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted via command line to obtain the database data schema, and the converted database data schema is imported into the specified database on the target device.

[0116] The command line is used to convert readable and executable text files stored in a removable storage device to obtain the table structure of the database data, and then import the converted database table structure into the specified database on the target device.

[0117] The command line is used to convert readable and executable text files stored in a removable storage device to obtain table data of the database, and then import the converted table data of the database into a specified database on the target device.

[0118] The command line is used to convert readable and executable text files stored in a removable storage device to obtain the table sequence index of the database data, and then import the converted table sequence index of the database data into the specified database of the target device.

[0119] The command line is used to convert readable and executable text files stored in a removable storage device to obtain a view of the database data, and then import the converted database data view into a specified database on the target device.

[0120] The command line is used to convert readable and executable text files stored in a removable storage device to obtain stored procedure functions for database data, and then import the converted stored procedure functions into a specified database on the target device.

[0121] This implementation method is a commonly used way to set the import order, and it can be seen that it is similar to the export order setting method described in the previous embodiment. That is, first, the text file is imported as the schema, then the text file is imported as the table structure, then the text file is imported as the table data, and then the text file is imported as the table sequence index, view, and stored procedure function in that order. Since the principle is the same as above, it will not be repeated here. It can also be understood that the import process can usually be based on multi-threaded concurrent execution to ensure efficiency.

[0122] Applying the technical solution provided by the embodiments of this invention, considering that there is no visual desktop operating system in some special scenarios, all operations of the solution in this application can be implemented based on the command line in command-line mode, thereby effectively dealing with the situation of no visual desktop operating system and ensuring the implementation of the solution in this application. Specifically, this application is implemented based on a preset removable storage device. After connecting the removable storage device to the source device, in command-line mode, the database data to be exported from the source device is specified via the command line. Then, the database data to be exported is read and converted into a readable and executable text file via the command line and stored in the preset removable storage device. Subsequently, after connecting the removable storage device to the target device, the readable and executable text file stored in the removable storage device is converted into database data via the command line and imported into the specified database on the target device, thus completing the process of importing the specified database data to be exported from the source device into the specified database on the target device. It can be seen that the above data transmission process does not rely on a network connection, effectively dealing with scenarios where network communication is limited. Furthermore, since direct copying may lead to errors, the solution in this application reads the database data to be exported through the command line and converts it into a readable and executable text file. Then, the text file is converted back into database data and imported into the specified database on the target device. This effectively avoids data errors, thus ensuring the reliability of the data transmission solution in this application. In addition, converting it into a readable and executable text file makes it easy to modify, ensuring flexibility.

[0123] In summary, for some special scenarios, the solution proposed in this application can effectively realize data transmission while ensuring the reliability and flexibility of data transmission.

[0124] Corresponding to the above method embodiments, this invention also provides a data transmission system, which can be referred to in conjunction with the above description.

[0125] See Figure 2 The diagram shown is a structural schematic of a data transmission system according to the present invention, applied in a pre-defined removable storage device, comprising:

[0126] The data export determination module 201 is used to specify the database data to be exported from the source device via command line in command line mode after the removable storage device is connected to the source device.

[0127] The export module 202 is used to read the database data to be exported and convert it into a readable and executable text file via the command line, and store it in a preset removable storage device;

[0128] Import module 203 is used to convert readable and executable text files stored in the removable storage device into database data via command line after the removable storage device is connected to the target device, and then import them into a specified database on the target device.

[0129] In one specific embodiment of the present invention, the export module 202 is specifically used for:

[0130] Using the command line, the different contents of the database data to be exported are read and converted into readable and executable text files in the order set for export, and the converted readable and executable text files are stored in a preset removable storage device.

[0131] In one specific embodiment of the present invention, the export module 202 includes:

[0132] The schema export unit is used to: read the schema of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device;

[0133] The table structure export unit is used to: read the table structure of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device;

[0134] The table data export unit is used to: read the table data of the database to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device;

[0135] The table sequence index export unit is used to: read the table sequence index of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device;

[0136] The view export unit is used to: read the view of the database data to be exported via command line and convert it into a readable and executable text file, and store the converted readable and executable text file in a preset removable storage device;

[0137] The stored procedure function export unit is used to: read the stored procedure functions of the database data to be exported via the command line, convert them into a readable and executable text file, and store the converted readable and executable text file in a preset removable storage device.

[0138] In one specific embodiment of the present invention, in the process of reading the schema, table structure, table sequence index, view, or stored procedure function of the database data to be exported and converting it into a readable and executable text file, and storing the converted readable and executable text file in a preset removable storage device, parallel processing is performed by K concurrent threads.

[0139] Where K is a positive integer not less than 2.

[0140] In one specific embodiment of the present invention, the table data export unit includes:

[0141] The multi-table mode subunit is used to: when the table data export configuration is determined to be multi-table mode, use M concurrent threads via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of multi-table mode, and store the converted readable and executable text file in a preset removable storage device.

[0142] The large table mode subunit is used to: when the table data export configuration is determined to be in large table mode, use M concurrent threads via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of large table mode, and store the converted readable and executable text file in a preset removable storage device.

[0143] The rules for the multi-table mode include: each time, x tables are read from the table data of the database data to be exported, and only a single query statement is generated for each table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device.

[0144] The rules for the large table mode include: each time, x tables are read from the table data of the database data to be exported. For the first type of table, only a single query statement is generated for a single table, and for the second type of table, multiple query statements are generated for a single table. Whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file. The converted readable and executable text file is then stored in a preset removable storage device. M is a positive integer not less than 2, and x is a positive integer.

[0145] In one specific embodiment of the present invention, the table data export unit includes:

[0146] The judgment sub-unit is used to read the table data of the database to be exported via the command line, convert it into a readable and executable Structured Query Language text file and a comma-separated value text file, and determine whether there is an error.

[0147] If not, the first execution unit is triggered to convert the structured query language text file and the comma-separated value text file into a readable and executable text file corresponding to the table data in the database, and to store the converted readable and executable text file in a preset removable storage device.

[0148] If so, the second execution unit is triggered to read the table data of the database data to be exported and convert it into a readable and executable Structured Query Language (SCL) text file, and use the SCL text file as the readable and executable text file corresponding to the converted table data of the database data.

[0149] In one specific embodiment of the present invention, the import module 203 is specifically used for:

[0150] After the removable storage device is connected to the target device, the different contents of the readable and executable text files stored in the removable storage device are converted into database data in the order set by the command line, and the converted database data is imported into the specified database on the target device.

[0151] In one specific embodiment of the present invention, the import module 203 includes:

[0152] The schema import unit is used to: after the removable storage device is connected to the target device, convert the readable and executable text file stored in the removable storage device through the command line to obtain the schema of the database data, and import the converted schema of the database data into the specified database of the target device;

[0153] The table structure import unit is used to: convert a readable and executable text file stored in a removable storage device through a command line to obtain the table structure of the database data, and import the converted table structure of the database data into a specified database on the target device.

[0154] The table data import unit is used to: convert readable and executable text files stored in a removable storage device through command line to obtain table data of database data, and import the converted table data of database data into a specified database of the target device;

[0155] The table sequence index import unit is used to: convert a readable and executable text file stored in a removable storage device through a command line to obtain a table sequence index of the database data, and import the converted table sequence index of the database data into a specified database on the target device;

[0156] The view import unit is used to: convert a readable and executable text file stored in a removable storage device via command line to obtain a view of the database data, and import the converted view of the database data into a specified database on the target device;

[0157] The stored procedure function import unit is used to: convert readable and executable text files stored in a removable storage device via command line to obtain stored procedure functions for database data, and import the converted stored procedure functions for database data into a specified database on the target device.

[0158] Corresponding to the above methods and system embodiments, this invention also provides a data transmission device and a computer-readable storage medium, which can be referred to in conjunction with the above.

[0159] See also Figure 3 This data transmission device can be applied to pre-defined removable storage devices, including:

[0160] Memory 301 is used to store computer programs;

[0161] Processor 302 is configured to execute a computer program to implement the steps of the data transmission method in any of the above embodiments.

[0162] See also Figure 4 The computer-readable storage medium 40 stores a computer program 41, which, when executed by a processor, implements the steps of the data transmission method in any of the above embodiments. The computer-readable storage medium 40 referred to herein includes random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0163] It should also be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0164] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0165] This application uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data transmission method, characterized in that, Applied to pre-defined removable storage devices, including: After the removable storage device is connected to the source device, the database data to be exported from the source device is specified via the command line in command line mode. The database data to be exported is read and converted into a readable and executable text file via the command line, and stored in a preset removable storage device. After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted into database data via command line and imported into the specified database of the target device; wherein the source device and the target device cannot communicate, and neither the source device nor the target device has a visual desktop operating system; The database data to be exported is read and converted into a readable and executable text file via command line, and stored in a preset removable storage device. This includes: reading and converting different contents of the database data to be exported into a readable and executable text file in a set export order via command line, and storing the converted readable and executable text file in a preset removable storage device. Specifically, this includes: reading the schema of the database data to be exported via command line and converting it into a readable and executable text file, and storing the converted readable and executable text file in a preset removable storage device; The table structure of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device. The table data of the database to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device. The table sequence index of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device. The database view to be exported is read and converted into a readable and executable text file via command line, and the converted readable and executable text file is stored in a preset removable storage device. The stored procedure function of the database data to be exported is read and converted into a readable and executable text file via the command line, and the converted readable and executable text file is stored in a preset removable storage device. The process of reading the table data of the database data to be exported and converting it into a readable and executable text file includes: reading the table data of the database data to be exported and converting it into a readable and executable Structured Query Language text file and a comma-separated value text file via a command line, and determining whether there are any errors; If not, then the Structured Query Language text file and the comma-separated value text file will be used as readable and executable text files of the converted table data corresponding to the database data; If so, the table data of the database data to be exported is read and converted into a readable and executable Structured Query Language (SCL) text file, and the SCL text file is used as the readable and executable text file corresponding to the converted table data of the database data.

2. The data transmission method according to claim 1, characterized in that, The process of reading the schema, table structure, table sequence index, view, or stored procedure function of the database data to be exported and converting it into a readable and executable text file, and storing the converted readable and executable text file in a preset removable storage device, is performed in parallel by K concurrent threads. Where K is a positive integer not less than 2.

3. The data transmission method according to claim 1, characterized in that, The system reads the table data of the database to be exported via command line, converts it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device, including: When the table data export configuration is determined to be in multi-table mode, M concurrent threads are used via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of multi-table mode. The converted readable and executable text file is then stored in a preset removable storage device. When the table data export configuration is set to large table mode, M concurrent threads are used via command line to read the table data of the database data to be exported and convert it into a readable and executable text file according to the rules of large table mode. The converted readable and executable text file is then stored in a preset removable storage device. The rules of the multi-table mode include: each time x tables are read from the table data of the database data to be exported, and only a single query statement is generated for each table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device; The rules of the large table mode include: each time, x tables are read from the table data of the database data to be exported, and for the first type of table, only a single query statement is generated for a single table, and for the second type of table, multiple query statements are generated for a single table; whenever a query statement is generated and there is an idle thread, the idle thread executes the query statement to read the table data corresponding to the query statement and convert it into a readable and executable text file, and stores the converted readable and executable text file in a preset removable storage device; M is a positive integer not less than 2, and x is a positive integer.

4. The data transmission method according to claim 1, characterized in that, After the removable storage device is connected to the target device, the readable and executable text files stored on the removable storage device are converted into database data via command line and imported into a specified database on the target device, including: After the removable storage device is connected to the target device, the different contents of the readable and executable text files stored in the removable storage device are converted into database data in the order set by the command line, and the converted database data is imported into the specified database of the target device.

5. The data transmission method according to claim 4, characterized in that, After the removable storage device is connected to the target device, the different contents of the readable and executable text files stored in the removable storage device are converted into database data sequentially according to the set import order via command line. The converted database data is then imported into a specified database on the target device, including: After the removable storage device is connected to the target device, the readable and executable text files stored in the removable storage device are converted via command line to obtain the database data schema, and the converted database data schema is imported into the specified database of the target device. The command line is used to convert the readable and executable text file stored in the removable storage device to obtain the table structure of the database data, and then the converted table structure of the database data is imported into the specified database of the target terminal device. The command line is used to convert readable and executable text files stored in the removable storage device to obtain table data of the database data, and then import the converted table data of the database data into the specified database of the target terminal device. The readable and executable text files stored in the removable storage device are converted using the command line to obtain the table sequence index of the database data, and the converted table sequence index of the database data is imported into the specified database of the target terminal device. The command line is used to convert readable and executable text files stored in the removable storage device to obtain a view of database data, and then the converted view of database data is imported into a specified database of the target device. The command line is used to convert readable and executable text files stored in the removable storage device to obtain stored procedure functions for database data, and then imports the converted stored procedure functions into a specified database on the target device.

6. A data transmission system, characterized in that, Applied to pre-defined removable storage devices, including: The data export determination module is used to specify the database data to be exported from the source device via command line in command line mode after the removable storage device is connected to the source device. The export module is used to read the database data to be exported and convert it into a readable and executable text file via command line, and store it in a preset removable storage device; The import module is used to convert readable and executable text files stored in the removable storage device into database data via command line after the removable storage device is connected to the target device, and then import them into a specified database on the target device; wherein the source device and the target device cannot communicate, and neither the source device nor the target device has a visual desktop operating system; The export module is specifically used to: read and convert different contents of the database data to be exported into readable and executable text files in the order set by the command line, and store the converted readable and executable text files in a preset removable storage device. The export module includes: a schema export unit, used to: read the schema of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device; The table structure export unit is used to: read the table structure of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device; The table data export unit is used to: read the table data of the database to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device; The table sequence index export unit is used to: read the table sequence index of the database data to be exported and convert it into a readable and executable text file via command line, and store the converted readable and executable text file in a preset removable storage device; The view export unit is used to: read the view of the database data to be exported via the command line and convert it into a readable and executable text file, and store the converted readable and executable text file in a preset removable storage device; The stored procedure function export unit is used to: read the stored procedure functions of the database data to be exported via the command line, convert them into a readable and executable text file, and store the converted readable and executable text file in a preset removable storage device; The table data export unit includes: The judgment sub-unit is used to read the table data of the database to be exported via the command line, convert it into a readable and executable Structured Query Language text file and a comma-separated value text file, and determine whether there is an error. If not, the first execution unit is triggered to convert the structured query language text file and the comma-separated value text file into a readable and executable text file corresponding to the table data in the database, and to store the converted readable and executable text file in a preset removable storage device. If so, the second execution unit is triggered to read the table data of the database data to be exported and convert it into a readable and executable Structured Query Language (SCL) text file, and use the SCL text file as the readable and executable text file corresponding to the converted table data of the database data.

7. A data transmission device, characterized in that, Applied to pre-defined removable storage devices, including: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data transmission method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data transmission method as described in any one of claims 1 to 5.

Images