A memory-based dumping method and system for earthquake database

Abstract

The present invention discloses a memory-based dump method for a seismic database, comprising: obtaining database information of a source database and a target database; parsing an SQL model file in the source database to obtain database table dump information; creating a source database memory object and a target database memory object based on the database information and initializing the memory objects; reading memory data from the source database memory object based on the database table dump information, writing the memory data to a cache in a target database, and cyclically reading and writing to perform a database memory dump; writing the cache of the target database to a disk and closing the target database and the source database. The present invention also discloses a memory-based dump system for a seismic database. The present invention realizes fast memory dumping between different operating systems, helping database administrators complete data backup and recovery between databases on different platforms, which is not only efficient but also saves disk resources.

Description

Technical Field

[0001] The present invention relates to the field of geophysical exploration technology, and in particular to a seismic database dumping method and system based on memory. Background Art

[0002] Seismic exploration is a crucial step in the oil exploration and development process, and seismic data is essential for geologists to understand underground structures. Seismic data is divided into two-dimensional and three-dimensional data. The three-dimensional seismic data volume for a seismic work area is typically around 10GB. GeoEast is a commonly used integrated software system for seismic data processing and interpretation. As GeoEast software gradually shifts to support the Windows platform, data transfer between different operating system platforms also faces new challenges. Traditional data transfer mainly involves transferring data between databases of the same type, such as the PG database, which has built-in database backup and recovery capabilities. First, the database is transferred to disk using SQL statements or internal formats. The transferred disk file then needs to be copied to the target node. Finally, the target database loads the dump file to complete the data migration between databases on different nodes.

[0003] However, while backing up and restoring data using a dump is feasible on Linux systems, it's not feasible between Linux and Windows. Windows data platforms use SQLite as their database, while Linux uses PostgreSQL. Different databases differ in language and functionality, and SQLite doesn't support many of the SQL features of PostgreSQL. Migrating PostgreSQL data to SQLite requires manual editing of SQL statements, requiring them to be SQL-supported. This approach is both tedious and inefficient.

[0004] Therefore, there is a need in the prior art for improving the seismic database dumping method and system. Summary of the Invention

[0005] In view of this, the purpose of the embodiments of the present invention is to propose a memory-based dump method and system for a seismic database. The present invention realizes fast memory dump between different operating systems, helping database administrators to complete data backup and recovery between databases on different platforms, which is not only efficient but also saves disk resources.

[0006] Based on the above purpose, an embodiment of the present invention provides a method for dumping a seismic database based on memory, comprising the following steps:

[0007] S1. Obtain database information of the source database and the target database;

[0008] S2. Parse the SQL model file in the source database to obtain database table dump information;

[0009] S3. Create a source database memory object and a target database memory object based on the database information and initialize the memory objects;

[0010] S4. Based on the database table dump information, read the memory data from the source database memory object, write the memory data to the target database cache, and cyclically read and write to perform a database memory dump;

[0011] S5. Write the cache of the target database to disk and close the target database and the source database.

[0012] In some embodiments, the method further comprises:

[0013] In response to an error in parsing the SQL model file, an error in loop reading, and / or an error in writing, an error message is generated and stored in a log module.

[0014] In some implementations, the SQL model file error includes one or more of the following: the SQL model file does not exist, an SQL statement parsing error, and an incorrect keyword.

[0015] In some embodiments, the method further includes: obtaining characteristics of the target database, determining whether the target database supports transactional operations, and enabling transaction processing during the memory dump process in response to the target database supporting transactional operations.

[0016] In some implementations, in S1 , the source database and the target database are network databases, and obtaining database information includes obtaining one or more of a host address, an IP address, a port number, a database name, a user name, and a password.

[0017] In some implementations, in S1 , the source database and the target database are file databases, and obtaining database information includes: obtaining and specifying a database file path.

[0018] In some implementations, in S2, parsing the SQL model file in the source database includes:

[0019] Read the SQL model file and read it into memory line by line;

[0020] Determine whether each line is a SQL statement to create a database table. If so, parse the line; otherwise, skip the line without parsing.

[0021] In some implementations, in S4, cyclically reading and writing to perform a database memory dump includes cyclically executing the following steps:

[0022] Each time the information of an input database table is retrieved, an SQL statement is executed on the source database, all data from the current table is read and returned, the data of the database table is accessed row by row in a loop, and organized into multiple insert SQL statements, which are stored in the first container;

[0023] Transaction processing is enabled based on the characteristics of the target database, and insert statements in the first container are executed one by one to insert data into the target database to complete the memory dump.

[0024] In another aspect, the present invention provides a memory-based dump system for a seismic database, comprising:

[0025] An information acquisition module, the information acquisition module is configured to acquire database information of a source database and a target database;

[0026] Parsing module, the parsing module is configured to parse the SQL model file in the source database to obtain database table dump information;

[0027] The object creation module is configured to create a source database memory object and a target database memory object based on database information and perform initialization operations on the memory objects;

[0028] The memory dump module is configured to dump information based on the database table, read memory data from the memory object of the source database, write the memory data to the cache of the target database, cyclically read and write to perform database memory dump, write the cache of the target database to the disk, and close the target database and the source database.

[0029] In some embodiments, further comprising:

[0030] The log module is configured to generate error information in response to an error in parsing the SQL model file, an error in loop reading, and / or an error in writing, and store the error information in the log module.

[0031] The present invention has at least the following beneficial technical effects:

[0032] 1. This invention solves the problem of data dump between different SQL databases under different operating systems and is compatible with mainstream databases on the market;

[0033] 2. The present invention solves the shortcomings of traditional dumping based on external memory. Traditional dumping not only takes up a large amount of disk, but also has low dumping efficiency. The performance of dumping based on internal memory is several times that of dumping based on external memory.

[0034] 3. The present invention provides a customized dump solution that can dump specified data tables;

[0035] 4. The present invention provides log management, which improves the robustness of database dump. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of an embodiment of a method for dumping a seismic database based on memory provided by the present invention;

[0038] Figure 2 A schematic diagram of an embodiment of memory dump based on a PG system and an SQLITE3 system provided by the present invention;

[0039] Figure 3 This is a schematic diagram of an embodiment of a memory-based dump system for a seismic database provided by the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0041] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. Subsequent embodiments will not explain this one by one.

[0042] Based on the above objectives, a first aspect of the embodiments of the present invention provides an embodiment of a method for dumping a seismic database based on memory. Figure 1 FIG. 1 is a schematic diagram of an embodiment of a method for dumping a seismic database based on memory provided by the present invention. Figure 1 As shown, a method for dumping a seismic database based on memory according to an embodiment of the present invention includes the following steps:

[0043] S1. Obtain database information of the source database and the target database;

[0044] S2. Parse the SQL model file in the source database to obtain database table dump information;

[0045] S3. Create a source database memory object and a target database memory object based on the database information and initialize the memory objects;

[0046] S4. Based on the database table dump information, read the memory data from the source database memory object, write the memory data to the target database cache, and cyclically read and write to perform a database memory dump;

[0047] S5. Write the cache of the target database to disk and close the target database and the source database.

[0048] In this embodiment, the method of the present invention further includes generating error information in response to errors in parsing the SQL model file, loop reading errors, and / or write errors, and storing the error information in a log module. During the specific execution process, the log module needs to be initialized. If various types of errors occur during the subsequent memory dump step, these error messages are written to the file in the form of logs. When subsequently tracking problems during the database dump process, this log information can be used to determine the cause of the error, thereby resolving the problem more quickly.

[0049] In some examples, log initialization includes:

[0050] Specify a file path, which will be used as the file path for storing logs.

[0051] Initialize the log management object.

[0052] Furthermore, in S1, the database type includes network databases and file databases. Network databases are such as Oracle and Postgre (abbreviated as PG), and file databases are such as SQLITE3. If the source database and the target database are network databases, then obtaining the database information in S1 specifically includes obtaining the host address (or IP address) and port number of the source or target database, obtaining the database name of the source or target database, and obtaining one or more of the user name and password of the source or target database. If the source database and the target database are file databases, the full path of the file needs to be specified. The purpose of S1 in the method of the present invention is to obtain the specific information of the source database and the target database, so as to establish connections between the source database and the target database respectively, for subsequent transmission and access of database data.

[0053] Furthermore, in S2, the SQL model file in the source database is parsed, that is, the original model file for creating the database table. The purpose of this step is to determine which database tables need to be dumped and which database tables do not need to be dumped. The specific steps of parsing include:

[0054] S-2.1. Read the SQL model file and read it into memory line by line.

[0055] S-2.2. Determine whether the row is a SQL statement for creating a table. If so, parse the row; otherwise, skip the row.

[0056] If an error occurs in the above two steps, such as the SQL model file does not exist, SQL statement parsing error, incorrect keyword, etc., an error message will be generated and written to the error log.

[0057] Furthermore, in S3, a database memory object is created. The purpose of this step is to create and initialize the corresponding memory object based on the database information, providing the basic conditions for the next step of dump execution.

[0058] The specific steps include:

[0059] S-3.1. Create corresponding memory database objects according to database information, such as Figure 2 As shown in the following example, the source database uses a PG database as an example. A pg in-memory database variable is created to reference the actual PG database. The target database uses an SQLITE3 database as an example. A sq in-memory database variable is created. If this step fails, an error message is written to the log file and execution is terminated.

[0060] S-3.2. Perform initialization operations according to the corresponding database objects.

[0061] S-3.3. Obtain the characteristics of the target database and determine whether the database supports transactional operations. If so, this option will be enabled during the dump process. This option can effectively improve dump performance. If the database does not support transactional operations, this option will not be used during the dump process.

[0062] Furthermore, in S4, based on the database table dump information, the database table dump information is input cyclically, and the information of one database table is taken out each time, and the SQL statement is executed on the source database, all the data is read from the current table and returned, the data of the database table is accessed row by row in a loop, and organized into multiple insert SQL statements, which are stored in the first container, and transaction processing is started based on the characteristics of the target database, and the insert statements in the first container are executed one by one, and the data is inserted into the target database to complete the memory dump.

[0063] This operation is executed in a loop. The operation object references pg and sq in S3. The following description uses pg as the source database object and sq as the target database object.

[0064] S-4.1. Input the model information of the database table parsed in step 3 in a loop, taking out one database table information at a time. Here, mark the current database table name as table_i.

[0065] S-4.2. Execute the SQL statement "SELECT * from table-i;" on the PG database to read all table data from the current table. The returned data is recorded as table_i_data. If the execution fails, an error message is written to the log, and the program jumps to S-4.1 to continue executing the next database table. Otherwise, the program continues to S-4.3.

[0066] S-4.3. Loop through the table data table_i_data row by row, organize it into multiple insert SQL statements, and store them in a container, denoted as insert_vector_i.

[0067] S-4.4. Based on the characteristics of the target database, if transaction processing is supported, enable transactions and execute the insert statements in insert_vector_i one by one to insert the data into the sq database. If an error occurs during execution, write it to the log and continue execution. After completing all insert statements, close transaction processing.

[0068] Furthermore, in S5, the cache of the target database is written to the disk and the target database and the source database are closed. If a log module is used in the process, the log module is closed at the same time as the database is closed.

[0069] The specific implementation of the present invention is further described below based on specific examples.

[0070] It should be pointed out in particular that the various steps in the various embodiments of the above-mentioned memory-based dumping method for a seismic database can be cross-linked, replaced, added, and deleted with each other. Therefore, these reasonable permutations, combinations, and transformations of the memory-based dumping method for a seismic database should also fall within the scope of protection of the present invention, and the scope of protection of the present invention should not be limited to the embodiments.

[0071] Based on the above objectives, a second aspect of an embodiment of the present invention provides a memory-based dump system for a seismic database. Figure 3 FIG. 1 is a schematic diagram of an embodiment of a memory-based dump system for earthquake databases provided by the present invention. Figure 3 As shown, a memory-based dump system for a seismic database according to an embodiment of the present invention includes the following modules:

[0072] Information acquisition module 011, the information acquisition module 011 is configured to obtain database information of the source database and the target database;

[0073] An information parsing module 012 is configured to parse the SQL model file in the source database to obtain database table dump information;

[0074] The object creation module 013 is configured to create a source database memory object and a target database memory object based on database information and perform initialization operations on the memory objects;

[0075] Memory dump module 014, memory dump module 014 is configured to read memory data from the source database memory object based on database table dump information, write the memory data to the target database cache, loop read and write to perform database memory dump, write the target database cache to disk and close the target database and source database.

[0076] In some embodiments, a log module is further included, where the log module is configured to generate error information in response to an error in parsing the SQL model file, an error in loop reading, and / or an error in writing, and store the error information in the log module.

[0077] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. A program for a memory-based dump method for a seismic database can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. The storage medium for the program can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). The above-described computer program embodiments can achieve the same or similar effects as any of the corresponding aforementioned method embodiments.

[0078] In addition, the method disclosed in the embodiment of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above functions defined in the method disclosed in the embodiment of the present invention are performed.

[0079] In addition, the above method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.

[0080] It will also be appreciated by those skilled in the art that the various exemplary logic blocks, modules, circuits and algorithmic steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given of the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0081] In one or more exemplary designs, the function can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, and the communication media include any media that helps to transmit a computer program from one location to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. As an example and not limitation, the computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of an instruction or data structure and can be accessed by a general or special-purpose computer or a general or special-purpose processor. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DOL) or wireless technologies such as infrared, radio and microwaves are used to send software from a website, server or other remote source, the above-mentioned coaxial cable, optical fiber cable, twisted pair, DOL or wireless technologies such as infrared, radio and microwaves are all included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0082] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope of the embodiments disclosed in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless expressly limited to the singular.

[0083] It should be understood that, as used herein, the singular forms "a" and "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" is intended to include any and all possible combinations of one or more of the associated listed items.

[0084] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0085] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program instructing the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0086] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples. Within the spirit of the embodiments of the present invention, the technical features of the above embodiments or different embodiments may be combined, and there are many other variations of different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of the embodiments of the present invention.

Claims

1. A method for dumping a seismic database based on memory, characterized in that: include: S1. Obtain database information of the source database and the target database; S2. Parse the SQL model file in the source database to obtain database table dump information; S3. Create a source database memory object and a target database memory object based on the database information and initialize the memory objects; S4. Based on the database table dump information, read memory data from the source database memory object, write the memory data into the cache of the target database, and cyclically read and write to perform database memory dump; S5. Write the cache of the target database to disk and close the target database and the source database.

2. The memory-based dumping method for earthquake database according to claim 1, characterized in that: Also includes: In response to an error in parsing a SQL model file, an error in loop reading, and / or an error in writing, error information is generated and stored in a log module.

3. The memory-based dumping method for earthquake database according to claim 2, characterized in that: The SQL model file error includes one or more of the following: SQL model file does not exist, SQL statement parsing error, and incorrect keyword.

4. The memory-based dumping method for earthquake database according to claim 2, characterized in that: Also includes: Obtaining characteristics of the target database and determining whether the target database supports transactional operations; In response to the target database supporting transactional operations, transaction processing is enabled during the memory dump process.

5. The memory-based dumping method of the earthquake database according to claim 2, characterized in that: In S1, the source database and the target database are network databases, and obtaining the database information includes: obtaining one or more of a host address, an IP address, a port number, a database name, a user name, and a password.

6. The memory-based dumping method of earthquake database according to claim 2, characterized in that: In S1, the source database and the target database are file databases, and obtaining the database information includes: obtaining and specifying a database file path.

7. The memory-based dumping method for earthquake database according to claim 2, characterized in that: In S2, parsing the SQL model file in the source database includes: Read the SQL model file and read it line by line into memory; Determine whether each line is a SQL statement to create a database table. If so, parse the line; otherwise, skip the line without parsing.

8. The memory-based dumping method for a seismic database according to any one of claims 2 or 4, characterized in that: In S4, looping through reads and writes to dump a database memory dump involves looping through the following steps: Each time, the information of the database table is retrieved, an SQL statement is executed on the source database, all data is read from the current table and returned, the data of the database table is accessed row by row in a loop, and the data is organized into multiple insert SQL statements, which are stored in the first container; Transaction processing is enabled based on the characteristics of the target database, insert statements in the first container are executed one by one, and data is inserted into the target database to complete the memory dump.

9. A memory-based dump system for earthquake database, characterized in that: include: An information acquisition module configured to acquire database information of a source database and a target database; A parsing module configured to parse the SQL model file in the source database to obtain database table dump information; An object creation module, the object creation module is configured to create a source database memory object and a target database memory object based on the database information and perform initialization operations on the memory objects; A memory dump module is configured to read memory data from the source database memory object based on the database table dump information, write the memory data to the cache of the target database, cyclically read and write to perform a database memory dump, write the cache of the target database to disk, and close the target database and the source database.

10. The memory-based dump system for earthquake database according to claim 9, characterized in that: Also includes: A log module is configured to generate error information in response to an error in parsing a SQL model file, an error in loop reading, and / or an error in writing, and store the error information in the log module.

Images