Data access method and apparatus

Abstract

The application provides a data access method and device. The method comprises the following steps: a database instance receives a data write request; the database instance generates a plurality of transaction records based on the data write request; and the database instance writes the plurality of transaction records into a plurality of transaction log files of a file system in parallel. Thus, the data write delay can be reduced, and the performance of the database is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data storage, and in particular to a data access method and device. BACKGROUND

[0002] Databases are very sensitive to latency, in order to improve performance, the database has very high requirements for the write latency of the file system / storage system, which often becomes the bottleneck of the database performance.

[0003] The existing log writing process of the relational database is as follows: the client sends three sql statements to the database at the same time, the database receives the request of the three sql statements, and simultaneously starts three parallel threads, each thread executes: first, parses the sql statement to obtain transaction information, the transaction information includes table space identifier (Identifier, ID), page ID and data to be written, then generates a log sequence number (LSN) according to the overall running state information of the database, wherein the LSN is an incremental integer, which represents the total amount of bytes of the transaction written to the transaction log, finally the database organizes the transaction information and the LSN into transaction records in a certain format, and finally obtains transaction record 1, transaction record 2 and transaction record 3. Due to the write of the transaction log (redo log) of the relational database, the log must follow the principle of log first (that is, all data is written to the log file / system in a certain format according to the transaction execution order before being written), and in order to ensure the modification order of the data, the transaction log must be sequentially written, generally a file is written full, and then the next file is written. Therefore, the existing relational database will use a global transaction cache, and the transaction records are sequentially written into the global transaction cache in the order of LSN from small to large. Since the memory space of the computer system is generally limited, the global transaction cache space is generally set to be very small, and the transaction record 1 and the transaction record 2 may completely occupy the global transaction cache space, at this time the global transaction cache is full, the database needs to write the data of the transaction record 1 and the transaction record 2 into the transaction log file (persistently write to the disk or storage system), after the writing is completed, the global transaction cache is emptied, and then the transaction record 3 is written into the global transaction cache, finally the transaction record 3 in the global transaction cache is written into the transaction log file, and after the writing is completed, a response message is sent to the client.

[0004] When the data cached by the global transaction is written into the transaction log file, it is not written all at once, but in batches. For example, the global transaction cache has 1G, and when writing, it is written once every 1ms, 1MB per time, and 1ms is consumed for each Input / Output (I / O) operation, and it takes 1s to write 1G of data. Therefore, the delay of each I / O operation is high, the data write delay is high, and the performance of the database is poor. SUMMARY

[0005] The present application provides a data access method and device, which can reduce the data write delay and improve the performance of the database.

[0006] In a first aspect, the present application provides a data access method, comprising: receiving, by a database instance, a data write request; generating, by the database instance, a plurality of transaction records based on the data write request; and writing, by the database instance, the plurality of transaction records into a plurality of transaction log files of a file system in parallel.

[0007] The data access method provided by the embodiment can reduce the data write delay and improve the performance of the database, and can also reduce the performance requirements on the file system.

[0008] In a possible design, the method of the embodiment further comprises: writing, by the database instance, the generated plurality of transaction records into a global transaction cache, so as to write the plurality of transaction records into the plurality of transaction log files of the file system in parallel through the global transaction cache.

[0009] In a possible design, the writing, by the database instance, of the plurality of transaction records into the plurality of transaction log files of the file system in parallel comprises:

[0010] The database instance writes the plurality of transaction records into the plurality of transaction log files in parallel based on log sequence numbers (LSNs) of the plurality of transaction records and identifiers (IDs) of the plurality of transaction log files, and each transaction record in the plurality of transaction records has a unique LSN.

[0011] The data access method provided by the embodiment can reduce the data write delay and improve the performance of the database, and can also reduce the performance requirements on the file system.

[0012] In a possible design, the database instance writes the plurality of transaction records into a plurality of transaction log files of the file system in parallel, including that the database instance records a log sequence number (LSN) of a previous transaction record in each of the plurality of transaction records.

[0013] In a possible design, the method of the embodiment further includes that the database instance writes IDs of the plurality of transaction log files and information used for indicating the sequence among the plurality of transaction log files into a metadata file of the file system.

[0014] In a possible design, the information used for indicating the sequence among the plurality of transaction log files includes an ID of a previous transaction log file of a current transaction log file and an ID of a next transaction log file, where the ID of the previous transaction log file included in a first transaction log file is a first ID set in advance, and the ID of the next transaction log file included in a last transaction log file is a second ID set in advance.

[0015] In a possible design, the method of the embodiment further includes that the database instance creates the plurality of transaction log files and the metadata file.

[0016] In a second aspect, the application provides a data access method, including:

[0017] The database instance receives a data read request, reads IDs of a plurality of transaction log files and information used for indicating a sequence among the plurality of transaction log files stored in a metadata file of a file system, and reads and concatenates a plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information used for indicating the sequence among the plurality of transaction log files.

[0018] The data access method provided by the embodiment includes that the database instance receives a data read request, reads IDs of a plurality of transaction log files and information used for indicating a sequence among the plurality of transaction log files stored in a metadata file of a file system, and reads and concatenates a plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information used for indicating the sequence among the plurality of transaction log files, so that continuous transaction records can be restored in sequence after being stored in a plurality of transaction log files, the performance of a database is improved, and the performance requirement of the file system is reduced.

[0019] In a possible design, the information for indicating the sequence among the plurality of transaction log files includes an ID of a previous transaction log file of a current transaction log file and an ID of a next transaction log file of the current transaction log file, where the ID of the previous transaction log file included in a first transaction log file is a first preset ID, and the ID of the next transaction log file included in a last transaction log file is a second preset ID.

[0020] In a possible design, the preset field of each transaction record in the plurality of transaction log files further carries an LSN of a previous transaction record, and the database instance reads and concatenates the plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information for indicating the sequence among the plurality of transaction log files, including:

[0021] The database instance determines the sequence of the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information for indicating the sequence among the plurality of transaction log files.

[0022] The database instance first finds out a last transaction record stored in each of the plurality of transaction log files, determines a transaction record carrying an LSN of a previous transaction record that is the largest among the LSNs of the previous transaction records carried by the plurality of transaction records according to the LSN of the previous transaction record carried by each transaction record, reads the transaction record, and determines the read transaction record as a first transaction record.

[0023] The database instance sequentially performs the following operations until all transaction records in the plurality of transaction log files are read out:

[0024] The database instance finds and reads a previous transaction record of an M-1th transaction record according to an LSN of a previous transaction record carried by the M-1th transaction record, determines a read Mth transaction record, and stores the previous transaction record of the M-1th transaction record in a transaction log file where the M-1th transaction record is located, or in the first N transaction log files before the transaction log file where the M-1th transaction record is located, or in the last N-1 transaction log files after the transaction log file where the M-1th transaction record is located, where M is greater than or equal to 2.

[0025] The database instance concatenates the first transaction record, the second transaction record, and the Mth transaction record read out according to the sizes of the LSNs of the previous transaction records carried by the first transaction record, the second transaction record, and the Mth transaction record, to obtain all transaction records in the plurality of transaction log files.

[0026] In a possible design, the preset field of each transaction record in the plurality of transaction log files further carries an LSN of a previous transaction record, and the database instance reads and concatenates the plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information for indicating the sequence among the plurality of transaction log files, including:

[0027] The database instance determines the order of the plurality of transaction log files according to IDs of the plurality of transaction log files and information used to indicate the order among the plurality of transaction log files;

[0028] The database instance reads all transaction records in each of the plurality of transaction log files in sequence according to the order of the plurality of transaction log files;

[0029] The database instance concatenates all the read transaction records according to the size of the LSN of the previous transaction record carried by each transaction record, to obtain all transaction records in the plurality of transaction log files.

[0030] In a third aspect, the present application provides a data access device, comprising:

[0031] The receiving module is configured to receive a data write request;

[0032] The generating module is configured to generate a plurality of transaction records based on the data write request;

[0033] The writing module is configured to write the plurality of transaction records into a plurality of transaction log files of a file system in parallel.

[0034] In a possible design, the writing module is further configured to:

[0035] write the generated plurality of transaction records into a global transaction cache, so as to write the plurality of transaction records into the plurality of transaction log files of the file system in parallel through the global transaction cache.

[0036] In a possible design, the writing module is configured to:

[0037] write the plurality of transaction records into the plurality of transaction log files in parallel based on log sequence numbers (LSNs) of the plurality of transaction records and IDs of the plurality of transaction log files, each of the plurality of transaction records having a unique LSN.

[0038] In a possible design, the writing module is configured to:

[0039] record a log sequence number (LSN) of a previous transaction record in each of the plurality of transaction records.

[0040] In a possible design, the writing module is further configured to:

[0041] write the IDs of the plurality of transaction log files and information used to indicate the order among the plurality of transaction log files into a metadata file of the file system.

[0042] In a possible design, the information for indicating the sequence among the plurality of transaction log files includes an ID of a previous transaction log file of a current transaction log file and an ID of a next transaction log file, where the ID of the previous transaction log file included in the first transaction log file is a first preset ID, and the ID of the next transaction log file included in the last transaction log file is a second preset ID.

[0043] In a possible design, the apparatus provided in the embodiment further includes:

[0044] The creating module is configured to create the plurality of transaction log files and the metadata file.

[0045] The data access apparatus provided in the third aspect and the possible designs of the third aspect has the advantages of the first aspect and the possible designs of the first aspect, which will not be repeated here.

[0046] In a fourth aspect, the present application provides a data access apparatus, including:

[0047] The receiving module is configured to receive a data read request.

[0048] The reading module is configured to read, from a metadata file of a file system, an identification ID of a plurality of transaction log files and information for indicating a sequence among the plurality of transaction log files.

[0049] The processing module is configured to read and splice a plurality of transaction records in the plurality of transaction log files according to the ID of the plurality of transaction log files and the information for indicating the sequence among the plurality of transaction log files.

[0050] In a possible design, the information for indicating the sequence among the plurality of transaction log files includes an ID of a previous transaction log file of a current transaction log file and an ID of a next transaction log file, where the ID of the previous transaction log file included in the first transaction log file is a first preset ID, and the ID of the next transaction log file included in the last transaction log file is a second preset ID.

[0051] In a possible design, the preset field of each transaction record in the plurality of transaction log files further carries an LSN of a previous transaction record, and the processing module is configured to:

[0052] determine the sequence of the plurality of transaction log files according to the ID of the plurality of transaction log files and the information for indicating the sequence among the plurality of transaction log files.

[0053] find out the last transaction record stored in each of the plurality of transaction log files, determine the LSN of the previous transaction record carried by each transaction record, determine the transaction record with the maximum LSN of the previous transaction record carried by the plurality of transaction records, read the transaction record, and determine the first transaction record read out;

[0054] sequentially perform the following operations until all transaction records in the plurality of transaction log files are read out:

[0055] find the previous transaction record of the M-1th transaction record according to the LSN of the previous transaction record carried by the M-1th transaction record read out, read the previous transaction record, determine the Mth transaction record read out, the previous transaction record of the M-1th transaction record is stored in the transaction log file where the M-1th transaction record is located, or in the first N transaction log files of the transaction log file where the M-1th transaction record is located, or in the last N-1 transaction log files of the transaction log file where the M-1th transaction record is located, M is greater than or equal to 2;

[0056] splice the first transaction record, the second transaction record, …, and the Mth transaction record read out according to the size order of the LSN of the previous transaction record carried by each transaction record, to obtain all transaction records in the plurality of transaction log files.

[0057] In a possible design, the preset field of each transaction record in the plurality of transaction log files further carries the LSN of the previous transaction record, and the processing module is configured to:

[0058] determine the order of the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information used to indicate the order between the plurality of transaction log files;

[0059] read all transaction records in each transaction log file in the order of the plurality of transaction log files;

[0060] splice all transaction records read out according to the size order of the LSN of the previous transaction record carried by each transaction record, to obtain all transaction records in the plurality of transaction log files.

[0061] The data access apparatus provided in the fourth aspect and the possible designs of the fourth aspect has the beneficial effects of the second aspect and the possible implementations of the second aspect, which will not be described herein again.

[0062] In a fifth aspect, the present application provides a computing device, comprising a processor and a memory, the memory storing computer instructions; the processor executes the computer instructions stored in the memory, so that the computing device executes the method provided by the first aspect or various possible implementations of the first aspect or the method provided by the second aspect or various possible implementations of the second aspect, so that the computing device deploys the data access device provided by the third aspect or various possible implementations of the third aspect or the data access device provided by the fourth aspect or various possible implementations of the fourth aspect.

[0063] In a sixth aspect, the present application provides a computer readable storage medium, which stores computer instructions, the computer instructions instructing the computing device to execute the method provided by the first aspect or various possible implementations of the first aspect or the method provided by the second aspect or various possible implementations of the second aspect, or the computer instructions instructing the computing device to deploy the data access device provided by the third aspect or various possible implementations of the third aspect or the data access device provided by the fourth aspect or various possible implementations of the fourth aspect.

[0064] In a seventh aspect, the present application provides a computer program product, which comprises computer instructions stored in a computer readable storage medium. The processor of the computing device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computing device executes the method provided by the first aspect or various possible implementations of the first aspect or the method provided by the second aspect or various possible implementations of the second aspect, so that the computing device deploys the data access device provided by the third aspect or various possible implementations of the third aspect or the data access device provided by the fourth aspect or various possible implementations of the fourth aspect. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 A log writing process diagram of a relational database provided by the present application;

[0066] Figure 2 A flowchart of an embodiment of a data access method provided by the present application;

[0067] Figure 3 A flowchart of an embodiment of a data access method provided by the present application;

[0068] Figure 4 A flowchart of an embodiment of a data access method provided by the present application;

[0069] Figure 5 A flowchart of an embodiment of a data access method provided by the present application;

[0070] Figure 6A structural schematic diagram of an embodiment of a data access device provided in the present application is shown in FIG. 1.

[0071] Figure 7 A structural schematic diagram of an embodiment of a data access device provided in the present application is shown in FIG. 1.

[0072] Figure 8 A structural schematic diagram of an embodiment of a data access device provided in the present application is shown in FIG. 1.

[0073] Figure 9 A possible basic hardware architecture of the computing device described in the present application is schematically provided. DETAILED DESCRIPTION

[0074] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, and should not be necessarily construed as being preferred or advantageous over other embodiments or suitable for exclusive use. Rather, the use of "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0075] In the related art, in the log writing process of a relational database, after the database receives a request of multiple SQL statements, the multiple SQL statements are parsed to obtain multiple transaction information, then an LSN is generated according to overall running state information of the database, finally the database organizes the multiple transaction information and the corresponding LSN into transaction records in a certain format to obtain multiple transaction records, writes the multiple transaction records into a global transaction cache in order of the LSN from small to large, and then writes data (i.e., the multiple transaction records) of the global transaction cache into a transaction log file. When the data of the global transaction cache is written into the transaction log file, it is not written down all at once, but is written down in batches, and the time delay of one I / O operation is high, so the data writing time delay is high, which leads to poor database performance. To solve this problem, the present application provides a data access method and device. When the database writes multiple transaction records in the global transaction cache into a transaction log file, the multiple transaction records in the global transaction cache are written into multiple transaction log files in parallel according to the size of the LSN of the transaction record and the ID of the transaction log file. Since the multiple transaction records are written into the multiple transaction log files in parallel, the data writing time delay can be reduced, the performance of the database can be improved, and the performance requirement of the file system can be reduced. In addition, the ID of the transaction log file and information indicating the order between the multiple transaction log files are saved in the metadata file of the file system, and the transaction record written into the multiple transaction log files carries the LSN of the previous transaction record. When the database reads the transaction record written into the multiple transaction log files, the multiple transaction records in the multiple transaction log files can be read and spliced according to the ID of the multiple transaction log files, the information indicating the order between the multiple transaction log files, and the LSN of the previous transaction record carried by each transaction record in the multiple transaction log files. Thus, after the continuous transaction records are stored in the multiple transaction log files, they can be restored in order when read. The specific process of the data access method provided by the present application will be described in detail below with reference to the drawings.

[0076] First, the following explains some terms in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0077] 1. Transaction: a unit for maintaining the consistency and integrity of database data, which has the feature of "all success or all failure". Before a transaction is committed, the database is not responsible for the consistency and integrity of its data. But once the transaction is committed, the database guarantees the consistency and integrity of the data, which simply means that the data cannot be lost after the transaction is committed. In order to improve performance, a relational database generally commits after writing transaction data into a transaction log and a buffer pool, rather than writing it into a page file.

[0078] 2. Transaction Log (Redo Log): Transactions are committed once they are written to the transaction log and the cache pool. However, the "dirty page flushing" process is asynchronous. If a failure occurs while some dirty pages have not yet been written to disk, the data in the cache pool will be lost. In this case, the transaction log is needed to restore the data. The transaction log is a set of files stored directly on disk, but its data structure is simpler than that of page files, and it is stored sequentially, ensuring high write efficiency.

[0079] 3. Tablespace: A tablespace is a logical division in a relational database. A tablespace can only belong to one database. All database objects are stored in a designated tablespace, but the primary objects are tables, hence the name tablespace. Each tablespace has a unique tablespace identifier (ID).

[0080] 4. Log Sequence Number (LSN): This is an incrementing integer representing the total number of bytes written to the transaction log. Because it continuously increments, the LSN can serve as a unique ID to identify transactions written at different times. The LSN is primarily used for data recovery in case of database failures. The LSN is stored in both the transaction log and the cache pool.

[0081] This application can be used in relational databases. Figure 1 This is a schematic diagram of the log writing process of a relational database according to this application, as shown below. Figure 1 As shown, the client sends a data write request to the database, which can be a request for one or more SQL statements. The database instance generates multiple transaction records based on the data write request. Specifically, this could involve the database parsing one or more SQL statements to obtain multiple transaction information, generating LSNs based on the overall database operating status information, and organizing each transaction information and its corresponding LSN into transaction records according to a specific format, resulting in transaction record 1, transaction record 2, ..., transaction record n. Then, the database sequentially writes these multiple transaction records into the global transaction cache in ascending order of LSN. Finally, the database writes the multiple transaction records from the global transaction cache in parallel into multiple transaction log files in the file system.

[0082] Figure 2 A flowchart of an embodiment of a data access method provided in this application is shown below. Figure 2 As shown, the execution entity in this embodiment can be a database instance, also known as a database master node, which is a writable node of a database. A database instance can also be simply referred to as a database. The method in this embodiment may include:

[0083] Step S101: The database instance receives a data write request.

[0084] Specifically, the client sends a data write request to the database, which can be a request for one SQL statement or a request for multiple SQL statements.

[0085] In step S102, the database instance generates multiple transaction records based on the data write request.

[0086] Specifically, if the data write request is a request for one SQL statement, the database instance parses the request for one SQL statement to obtain one or more transaction information, generates an LSN according to the overall running state information of the database, organizes each transaction information and the corresponding LSN into a transaction record in a certain format to obtain one or more transaction records. If the data write request is a request for multiple SQL statements, the database instance parses the request for multiple SQL statements to obtain corresponding multiple transaction information, generates an LSN according to the overall running state information of the database, organizes each transaction information and the corresponding LSN into a transaction record in a certain format to obtain multiple transaction records. Optionally, when a certain transaction information is large, the transaction information can be split into multiple transaction information, and the split multiple transaction information is respectively assigned a corresponding LSN, and each split transaction information and the corresponding LSN are combined into a transaction record to obtain multiple transaction records.

[0087] Optionally, after step S102, the method of the embodiment can further include: the database instance writes the generated multiple transaction records into a global transaction cache, so as to write the multiple transaction records into multiple transaction log files of a file system in parallel through the global transaction cache.

[0088] In step S103, the database instance writes the multiple transaction records into the multiple transaction log files of the file system in parallel.

[0089] As an implemented manner, step S103 can specifically be: the database instance writes the multiple transaction records into the multiple transaction log files in parallel based on the LSNs of the multiple transaction records and the IDs of the multiple transaction log files, each of the multiple transaction records having a unique LSN. The LSN is an incremental integer number representing the total amount of bytes of the transaction written into the transaction log, and since it is always increasing, the LSN can be used as a unique ID to represent transaction records written at different times. For example, there are 2 transaction log files, which are transaction log file 1 and transaction log file 2, and 5 transaction records, which are transaction record 1, transaction record 2, transaction record 3, transaction record 4 and transaction record 5. The database instance writes the 5 transaction records into the 2 transaction log files in parallel according to the LSNs of the transaction records and the IDs of the transaction log files in order, and after the writing is completed, the transaction log file 1 stores transaction record 1, transaction record 3 and transaction record 5, and the transaction log file 2 stores transaction record 2 and transaction record 4.

[0090] Optionally, when the database instance writes the plurality of transaction records in parallel into the plurality of transaction log files of the file system, the database instance records the LSN of the previous transaction record in each of the plurality of transaction records. For example, the LSN of the previous transaction record is recorded in each of the plurality of transaction records before the plurality of transaction records are written in parallel into the plurality of transaction log files of the file system. For example, the LSN of the previous transaction record is recorded in each of the plurality of transaction records while the plurality of transaction records are written in parallel into the plurality of transaction log files of the file system, such as recording the LSN of transaction record 1 in transaction record 2 while writing transaction record 1 into transaction log file 1. Optionally, the database instance records the LSN of the previous transaction record in each of the plurality of transaction records in a preset field of the transaction record, such as a header field or a tail field of the transaction record, for quick reading. It is to be noted that the LSN of the previous transaction record of the first transaction record written in the first transaction log file can be 0 or a preset value.

[0091] Optionally, after step S102, the method of the present embodiment can further comprise: writing, by the database instance, the generated plurality of transaction records into a global transaction cache, so as to write the plurality of transaction records in parallel into the plurality of transaction log files of the file system through the global transaction cache.

[0092] Further, the method of the present embodiment can further comprise: writing, by the database instance, the IDs of the plurality of transaction log files and information for indicating the sequence among the plurality of transaction log files into a metadata file of the file system. Optionally, the information for indicating the sequence among the plurality of transaction log files comprises the ID of the previous transaction log file and the ID of the next transaction log file of the current transaction log file, wherein the ID of the previous transaction log file included in the first transaction log file is a preset first ID, and the ID of the next transaction log file included in the last transaction log file is a preset second ID. The first ID and the second ID can be two preset special IDs, such as, for transaction log file 1, the ID of the previous transaction log file is "0" and the ID of the next transaction log file is "2", and for transaction log file 2, the ID of the previous transaction log file is "1" and the ID of the next transaction log file is "3".

[0093] Optionally, the method of the embodiment can further include that the database instance creates a plurality of transaction log files and a metadata file. The metadata file can be created when the database is started. The plurality of transaction log files can be created when the database is started, and the plurality of transaction log files can also be created continuously when the created plurality of transaction log files are full. It should be noted that the number of the created transaction log files is the same when the transaction log files are created when the database is started or when the created plurality of transaction log files are full, for example, N.

[0094] In the embodiment, the ID of the transaction log file and the information for indicating the sequence between the plurality of transaction log files are stored in the metadata file of the file system, and the transaction record written into the plurality of transaction log files carries the LSN of the previous transaction record. When the database reads the transaction record written into the plurality of transaction log files, the plurality of transaction records in the plurality of transaction log files can be read and spliced according to the ID of the plurality of transaction log files and the information for indicating the sequence between the plurality of transaction log files and the LSN of the previous transaction record carried by each transaction record in the plurality of transaction log files. Thus, the continuous transaction records can be restored in sequence after being stored in the plurality of transaction log files.

[0095] The data access method provided by the embodiment receives a data write request based on the database instance, generates a plurality of transaction records based on the data write request, and writes the plurality of transaction records into a plurality of transaction log files of a file system in parallel. Since the plurality of transaction records are written into the plurality of transaction log files in parallel, the data write delay can be reduced, the performance of the database can be improved, and the performance requirement of the file system can be reduced.

[0096] Figure 3 The flowchart of the data access method provided by the embodiment is shown in Figure 3 The embodiment is a data reading process corresponding to the data writing process shown in Figure 2 The execution subject of the embodiment can be a database instance, and the method of the embodiment can include the following steps.

[0097] In step S201, the database instance receives a data reading request.

[0098] In step S202, the database instance reads the ID of the plurality of transaction log files and the information for indicating the sequence between the plurality of transaction log files stored in the metadata file of the file system.

[0099] Specifically, the metadata file of the file system stores the IDs of the plurality of transaction log files and information indicating the order among the plurality of transaction log files, and the database instance reads the IDs of the plurality of transaction log files and the information indicating the order among the plurality of transaction log files from the metadata file after receiving the data read request. Optionally, the information indicating the order among the plurality of transaction log files includes the ID of the previous transaction log file of the current transaction log file and the ID of the next transaction log file, wherein the ID of the previous transaction log file included in the first transaction log file is a first ID set in advance, and the ID of the next transaction log file included in the last transaction log file is a second ID set in advance. The first ID and the second ID can be two special IDs preset in advance, for example, for the transaction log file 1, the ID of the previous transaction log file is "0", and the ID of the next transaction log file is "2"; for the transaction log file 2, the ID of the previous transaction log file is "1", and the ID of the next transaction log file is "3".

[0100] Step S203, the database instance reads and splices the plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information indicating the order among the plurality of transaction log files.

[0101] As an implementable manner, the preset field of each transaction record in the plurality of transaction log files also carries the LSN of the previous transaction record, for example, in the header field or the tail field of the transaction record, facilitating quick reading. It should be noted that the LSN of the previous transaction record of the first transaction record written in the first transaction log file can be 0 or a preset value.

[0102] Step S203 can specifically include:

[0103] Step S2031, the database instance determines the order of the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information indicating the order among the plurality of transaction log files.

[0104] Specifically, for example, there are four transaction log files in total, the IDs are 1, 2, 3, and 4 respectively, and the information indicating the order among the transaction log files includes the ID of the previous transaction log file of the current transaction log file and the ID of the next transaction log file, so the order of the four transaction log files can be determined as: transaction log file 1, transaction log file 2, transaction log file 3, and transaction log file 4.

[0105] Step S2032, the database instance first finds out the last transaction record stored in each of the plurality of transaction log files, determines the transaction record carrying the LSN of the previous transaction record with the maximum LSN from each transaction record, reads it, and determines the first transaction record read.

[0106] Specifically, for example, there are 4 transaction log files, the number of transaction log files written in parallel at a time is 2, transaction log file 1 writes transaction record 1 and transaction record 3, transaction log file 2 writes transaction record 2, transaction log file 3 writes transaction record 4 and transaction record 5, and transaction log file 4 writes transaction record 6. Transaction log file 2 only writes transaction record 2 because transaction log file 2 is not enough to store transaction record 4, so two transaction log files 3 and 4 need to be created. First, find out the last transaction record stored in each of the four transaction log files, which are transaction record 3, transaction record 2, transaction record 5 and transaction record 6, respectively. Then, according to the LSN of the previous transaction record carried by each transaction record, it is determined that the transaction record with the maximum LSN of the previous transaction record carried is transaction record 6, and transaction record 6 is determined as the first transaction record read. It should be noted that the transaction record with the maximum LSN can also be determined according to the unique LSN of each transaction record itself, but using the unique LSN itself requires parsing the field, which takes a long time. Using the LSN of the previous transaction record carried, since the LSN of the previous transaction record is in a preset field of the transaction record, such as a header field or a tail field, it can be easily and quickly obtained.

[0107] Step S2033, the database instance sequentially performs the following operations until all transaction records in the plurality of transaction log files are read out:

[0108] The database instance finds the previous transaction record of the M-1th transaction record according to the LSN of the previous transaction record carried by the M-1th transaction record read, reads it, and determines the Mth transaction record read. The previous transaction record of the M-1th transaction record is stored in the transaction log file where the M-1th transaction record is located, or in the first N transaction log files of the transaction log file where the M-1th transaction record is located, or in the last N-1 files of the transaction log file where the M-1th transaction record is located. M is greater than or equal to 2.

[0109] For example, the database instance finds and reads the previous transaction record of the first transaction record according to the LSN of the previous transaction record carried by the first transaction record, determines that the read second transaction record is the previous transaction record of the first transaction record, and the previous transaction record of the first transaction record is stored in the transaction log file where the first transaction record is located, or in the first N transaction log files of the transaction log file where the first transaction record is located, or in the last N-1 files of the transaction log file where the first transaction record is located, N being the number of transaction log files written in parallel;

[0110] The database instance finds and reads the previous transaction record of the second transaction record according to the LSN of the previous transaction record carried by the second transaction record, determines that the read third transaction record is the previous transaction record of the second transaction record, and the previous transaction record of the second transaction record is stored in the transaction log file where the second transaction record is located, or in the first N transaction log files of the transaction log file where the second transaction record is located, or in the last N-1 files of the transaction log file where the second transaction record is located.

[0111] Step S2034, the database instance splices the read first transaction record, second transaction record, …, and Mth transaction record in order of the size of the LSN of the previous transaction record carried, to obtain all transaction records in the plurality of transaction log files.

[0112] Specifically, it can be in the order from small to large, or in the order from large to small.

[0113] In this embodiment, when splicing the read data, it is not necessary to traverse all the read historical transaction log files, but only to find in order from back to front, traverse part of the historical transaction log files.

[0114] As another implementable manner, the preset field of each transaction record in the plurality of transaction log files further carries the LSN of the previous transaction record, and step S203 can specifically include:

[0115] Step S2031', the database instance determines the order of the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information for indicating the order between the plurality of transaction log files.

[0116] Step S2032', the database instance reads all transaction records in each transaction log file in order.

[0117] Specifically, for example, there are two transaction log files 1 and transaction log file 2, all transaction records in the transaction log file 1 and the transaction log file 2 are read in order.

[0118] Step S2033': The database instance concatenates all the transaction records it reads according to the LSN size of the previous transaction record, to obtain all the transaction records in multiple transaction log files.

[0119] Specifically, for example, given two transaction log files, Transaction Log 1 and Transaction Log 2, all transaction records in Transaction Log 1 and Transaction Log 2 are read sequentially. For example, if transaction record 1, transaction record 3, transaction record 5, transaction record 2, and transaction record 4 are read, and the LSN of the preceding transaction record carried by transaction record 1 is 0, then the records are concatenated according to the LSN of the preceding transaction record carried by each transaction record. This concatenation can be in ascending order or descending order. For example, concatenating them in ascending order would result in the following transaction records in the two transaction log files: Transaction Record 1, Transaction Record 2, Transaction Record 3, Transaction Record 4, and Transaction Record 5.

[0120] In this embodiment, the IDs of the transaction log files and information indicating the order among multiple transaction log files are stored in the file system's metadata file. Transaction records written to multiple transaction log files carry the LSN of the preceding transaction record. When the database reads transaction records written to multiple transaction log files, it can read and concatenate multiple transaction records from the multiple transaction log files based on the IDs of the multiple transaction log files, the information indicating the order among the multiple transaction log files, and the LSN of the preceding transaction record carried by each transaction record in the multiple transaction log files. This achieves the goal of breaking down consecutive transaction records and storing them in multiple transaction log files, allowing for sequential restoration during retrieval.

[0121] The data access method provided in this embodiment receives data read requests through a database instance, reads the IDs of multiple transaction log files stored in the file system's metadata file, and reads and concatenates multiple transaction records from the multiple transaction log files based on their IDs and the information indicating their order. This allows for the sequential restoration of consecutive transaction records after they have been scattered and stored across multiple transaction log files. This improves database performance and reduces the performance requirements of the file system.

[0122] The following is a specific embodiment to illustrate... Figure 2 and Figure 3 The technical solutions of the illustrated method embodiments will be described in detail.

[0123] Figure 4 A flowchart of an embodiment of a data access method provided in this application is shown below. Figure 4As shown, the execution subject in this embodiment can be a database instance, and this embodiment is a data writing process. The method in this embodiment may include:

[0124] Step S301: When the database instance starts, it creates multiple transaction log files and metadata files, and writes the IDs of the multiple transaction log files and information indicating the order between the multiple transaction log files into the metadata file.

[0125] Specifically, the file system's metadata file stores the IDs of multiple transaction log files and information indicating the order among them. In this embodiment, the file system is an atomically writable, append-only storage system. The file system provides data storage services to the database instance; neither the primary nor backup nodes of the database store actual data, which is stored within this file system. When writing data to the database, an append-only interface / method is used to write data from the database instance to the file system.

[0126] It should be noted that in modern distributed storage systems, data is ultimately written to a fixed-size file, and each file is assigned a unique ID. Therefore, if you want to store the ever-growing data in multiple fixed-size files, you need to use some specific data to manage these files. The data that manages these files is called metadata (there may be multiple files). Metadata is the data that manages all transactional data files.

[0127] Step S302: The database instance receives a data write request.

[0128] Step S303: The database instance generates multiple transaction records based on the data write request.

[0129] Specifically, for example, a data write request is a request for multiple SQL statements. The database instance parses the request for multiple SQL statements to obtain the corresponding multiple transaction information, and generates LSNs based on the overall operating status information of the database. Each transaction information and its corresponding LSN are organized into transaction records according to a certain format to obtain multiple transaction records.

[0130] Step S304: The database instance writes the generated transaction records into the global transaction cache.

[0131] Step S305: Based on the LSNs of multiple transaction records and the IDs of multiple transaction log files, the database instance writes multiple transaction records in the global transaction cache to multiple transaction log files in the file system in parallel, and records the LSN of the previous transaction record in each of the multiple transaction records. Each of the multiple transaction records has a unique LSN.

[0132] Specifically, the LSN of the previous transaction record can be recorded in each transaction record before the multiple transaction records are written into the multiple transaction log files of the file system in parallel, or the LSN of the previous transaction record can be recorded in each transaction record while the multiple transaction records are written into the multiple transaction log files of the file system in parallel, for example, when transaction record 1 is written into transaction log file 1, the LSN of transaction record 1 is recorded in transaction record 2 at the same time. The LSN of the previous transaction record recorded in each transaction record can be recorded in a preset field of the transaction record, for example, in the head field or the tail field of the transaction record, to facilitate quick reading. It should be noted that the LSN of the previous transaction record of the first transaction record written in the first transaction log file can be 0 or a preset value.

[0133] If the database instance receives a data write request again, steps S303-S305 are continued.

[0134] If the N transaction log files created in step S301 are full, N transaction log files are created again, and the IDs of the multiple transaction log files and the information for indicating the order between the multiple transaction log files are written into the metadata file. This is to process subsequent data write.

[0135] Figure 5 The flowchart of an embodiment of a data access method provided in the present application is shown in FIG. 4, the execution subject of the present embodiment can be a database instance, the present embodiment is a data read process, and the method of the present embodiment can include: Figure 5

[0136] Step S401, the database instance receives a data read request.

[0137] Step S402, the database instance reads the IDs of the multiple transaction log files and the information for indicating the order between the multiple transaction log files stored in the metadata file of the file system.

[0138] Step S403, the database instance reads and splices the multiple transaction records in the multiple transaction log files according to the IDs of the multiple transaction log files and the information for indicating the order between the multiple transaction log files.

[0139] Figure 5 ​Taking two transaction log files as an example, step S403 can specifically be: the database instance determines the order of the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information used to indicate the order between the plurality of transaction log files. The database instance first finds the last transaction record stored in each transaction log file in the plurality of transaction log files, determines the transaction record carrying the LSN of the previous transaction record that is the largest among the plurality of transaction records according to the LSN of the previous transaction record carried by each transaction record, reads the transaction record, and determines the first transaction record read. The database instance sequentially performs the following operations until all transaction records in the plurality of transaction log files are read:

[0140] The database instance finds the previous transaction record of the M-1th transaction record according to the LSN of the previous transaction record carried by the M-1th transaction record read, reads the previous transaction record, determines the Mth transaction record read, and the previous transaction record of the M-1th transaction record is stored in the transaction log file where the M-1th transaction record is located, or in the first N transaction log files before the transaction log file where the M-1th transaction record is located, or in the last N-1 transaction log files after the transaction log file where the M-1th transaction record is located, M is greater than or equal to 2.

[0141] Finally, the database instance splices the first transaction record, the second transaction record, …, and the Mth transaction record read according to the size order of the LSN of the previous transaction record carried, to obtain all transaction records in the plurality of transaction log files.

[0142] In an implementable manner, the data read request carries the ID of the transaction log file, and after the database instance receives the data read request, only the transaction log files before the transaction log file corresponding to the ID of the transaction log file are read and spliced, without reading and splicing all transaction log files.

[0143] The data access method provided in this embodiment stores the IDs of the transaction log files and the information used to indicate the order between the plurality of transaction log files in the metadata file of the file system, and the transaction record written into the plurality of transaction log files carries the LSN of the previous transaction record, when the database reads the transaction record written into the plurality of transaction log files, the plurality of transaction records in the plurality of transaction log files can be read and spliced according to the IDs of the plurality of transaction log files and the information used to indicate the order between the plurality of transaction log files and the LSN of the previous transaction record carried by each transaction record in the plurality of transaction log files. Thus, after continuous transaction records are stored in the plurality of transaction log files, the transaction records can be restored in order when read.

[0144] Figure 6 The structure schematic diagram of an embodiment of a data access device provided in this application is as follows:Figure 6 The apparatus of the embodiment can include a receiving module 11, a generating module 12, and a writing module 13, wherein the receiving module 11 is configured to receive a data write request; the generating module 12 is configured to generate a plurality of transaction records based on the data write request; and the writing module 13 is configured to write the plurality of transaction records in parallel into a plurality of transaction log files of a file system.

[0145] The writing module 13 is configured to write the plurality of transaction records in parallel into the plurality of transaction log files of the file system.

[0146] Further, the writing module 13 is further configured to write the generated plurality of transaction records into a global transaction cache, so as to write the plurality of transaction records in parallel into the plurality of transaction log files of the file system through the global transaction cache.

[0147] Further, the writing module 13 is configured to write the plurality of transaction records in parallel into the plurality of transaction log files based on log sequence numbers (LSNs) of the plurality of transaction records and identification (ID) of the plurality of transaction log files, each of the plurality of transaction records having a unique LSN.

[0148] Further, the writing module 13 is configured to record a log sequence number (LSN) of a previous transaction record in each of the plurality of transaction records.

[0149] Further, the writing module 13 is further configured to write the ID of the plurality of transaction log files and information for indicating the order between the plurality of transaction log files into a metadata file of the file system.

[0150] Optionally, the information for indicating the order between the plurality of transaction log files includes an ID of a previous transaction log file of a current transaction log file and an ID of a next transaction log file, wherein the ID of the previous transaction log file included in a first transaction log file is a first ID set in advance, and the ID of the next transaction log file included in a last transaction log file is a second ID set in advance.

[0151] The apparatus of the embodiment can be used to execute the method of the embodiment. Figure 2 The technical solutions of the method embodiment are similar in principle, and thus will not be described herein.

[0152] The data access apparatus provided by the embodiment receives a data write request through the receiving module, generates a plurality of transaction records based on the data write request through the generating module, and writes the plurality of transaction records in parallel into a plurality of transaction log files of a file system through the writing module. Since the plurality of transaction records are written in parallel into the plurality of transaction log files, the data write latency can be reduced, the performance of the database can be improved, and the performance requirement for the file system can be reduced.

[0153] Figure 7 A structural schematic diagram of a data access apparatus embodiment provided by the present application is shown in FIG. 1.Figure 7 As shown in the figure, the device of the embodiment can be used for Figure 6 Further, based on the device structure shown, the device can further include a creating module 14 configured to create a plurality of transaction log files and a metadata file.

[0154] The device of the embodiment can be used for performing Figure 2 The technical solutions of the method embodiment have similar implementation principles and technical effects, and thus will not be described here.

[0155] Figure 8 A structural schematic diagram of a data access device embodiment provided in the application is shown in the figure. Figure 8 As shown in the figure, the device of the embodiment can include a receiving module 21, a reading module 22, and a processing module 23, wherein,

[0156] The receiving module 21 is configured to receive a data reading request.

[0157] The reading module 22 is configured to read the identification ID of the plurality of transaction log files and the information for indicating the sequence between the plurality of transaction log files stored in the metadata file of the file system.

[0158] The processing module 23 is configured to read and splice the plurality of transaction records in the plurality of transaction log files according to the ID of the plurality of transaction log files and the information for indicating the sequence between the plurality of transaction log files.

[0159] Further, the information for indicating the sequence between the plurality of transaction log files includes the ID of the previous transaction log file and the ID of the next transaction log file of the current transaction log file, wherein the ID of the previous transaction log file included in the first transaction log file is a first ID set in advance, and the ID of the next transaction log file included in the last transaction log file is a second ID set in advance.

[0160] Further, the preset field of each transaction record in the plurality of transaction log files also carries the LSN of the previous transaction record, and the processing module 23 is configured to:

[0161] determine the sequence of the plurality of transaction log files according to the ID of the plurality of transaction log files and the information for indicating the sequence between the plurality of transaction log files;

[0162] first find the last transaction record stored in each transaction log file in the plurality of transaction log files, determine the transaction record carrying the LSN of the previous transaction record that is the largest in the plurality of transaction records according to the LSN of the previous transaction record carried by each transaction record, read the transaction record, and determine the read transaction record as the first transaction record;

[0163] The following operations are sequentially performed until all transaction records in the plurality of transaction log files are read out:

[0164] The previous transaction record of the M-1th transaction record is found and read according to the LSN of the previous transaction record carried by the read M-1th transaction record, and is determined as the Mth transaction record read out, the previous transaction record of the M-1th transaction record is stored in the transaction log file where the M-1th transaction record is located, or in the first N transaction log files of the transaction log file where the M-1th transaction record is located, or in the last N-1 files of the transaction log file where the M-1th transaction record is located, M is greater than or equal to 2;

[0165] The first transaction record, the second transaction record, …, and the Mth transaction record read out are spliced in the order of the size of the LSN of the previous transaction record carried, to obtain all transaction records in the plurality of transaction log files.

[0166] Further, the preset field of each transaction record in the plurality of transaction log files also carries the LSN of the previous transaction record, and the processing module is used to:

[0167] The order of the plurality of transaction log files is determined according to the IDs of the plurality of transaction log files and the information for indicating the order between the plurality of transaction log files;

[0168] All transaction records in each transaction log file are sequentially read out in the order of the plurality of transaction log files;

[0169] All transaction records read out are spliced in the order of the size of the LSN of the previous transaction record carried, to obtain all transaction records in the plurality of transaction log files.

[0170] The device of the embodiment can be used to execute Figure 3 The technical solutions of the method embodiment are similar in implementation principle, and will not be described here.

[0171] The data access device provided in the embodiment receives a data read request through the receiving module, reads the IDs of the plurality of transaction log files and the information for indicating the order between the plurality of transaction log files stored in the metadata file of the file system through the reading module, and then reads and splices the plurality of transaction records in the plurality of transaction log files according to the IDs of the plurality of transaction log files and the information for indicating the order between the plurality of transaction log files, so that the continuous transaction records can be restored in order after being stored in the plurality of transaction log files, the performance of the database is improved, and the performance requirement for the file system is reduced.

[0172] The data access apparatus can be divided into functional modules according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division manner.

[0173] Optionally, Figure 9 A possible basic hardware architecture of the computing device described in the present application is schematically provided.

[0174] Referring to Figure 9 The computing device 300 includes a processor 301, a memory 302, a communication interface 303, and a bus 304.

[0175] In the computing device 300, the number of processors 301 can be one or more, Figure 9 Only one processor 301 is shown. Optionally, the processor 301 can be a central processing unit (CPU). If the computing device 300 has multiple processors 301, the types of the multiple processors 301 can be different or can be the same. Optionally, the multiple processors 301 of the computing device 300 can also be integrated into a multi-core processor.

[0176] The memory 302 stores computer instructions and data; the memory 302 can store computer instructions and data required to implement the data access method provided by the present application, for example, the memory 302 stores instructions for implementing the steps of the data access method. The memory 302 can be any one or any combination of the following storage media: non-volatile memory (such as read-only memory (ROM), solid state disk (SSD), hard disk (HDD), optical disk), volatile memory.

[0177] The communication interface 303 can be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and the like devices with network access functions.

[0178] The communication interface 303 is used for data communication between the computing device 300 and other computing devices or terminals.

[0179] The bus 304 can connect the processor 301 with the memory 302 and the communication interface 303. In this way, the processor 301 can access the memory 302 through the bus 304, and also can use the communication interface 303 to interact with other computing devices or terminals for data exchange.

[0180] In the present application, the computing device 300 executes computer instructions in the memory 302, so that the computing device 300 implements the data access method provided by the present application, or so that the computing device 300 deploys the data access apparatus.

[0181] The present application also provides a computer readable storage medium, which stores computer instructions. The computer instructions instruct the computing device to execute the data access method in the method embodiments, or the computer instructions instruct the computing device to deploy the data access apparatus in the apparatus embodiments.

[0182] The present application also provides a computer program product chip, which includes computer instructions stored in a computer readable storage medium. The processor of the computing device can read the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computing device executes the data access method in the method embodiments, or the computer instructions instruct the computing device to deploy the data access apparatus in the apparatus embodiments.

[0183] Those skilled in the art can understand that, in the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A data access method, characterized in that, include: The database instance creates multiple transaction log files and metadata files; The database instance receives a data write request; The database instance generates multiple transaction records based on the data write request, and each of the multiple transaction records has a unique LSN; The database instance writes the multiple transaction records into the multiple transaction log files in parallel based on the log sequence number (LSN) of each transaction record and the identifier (ID) of each transaction log file. Each transaction record written into the multiple transaction log files records the log sequence number (LSN) of the previous transaction record. The database instance writes the ID of each transaction log file and information indicating the order among the multiple transaction log files into the metadata file. The information indicating the order among the multiple transaction log files includes the ID of the previous transaction log file and the ID of the next transaction log file for the current transaction log file.

2. The method according to claim 1, characterized in that, The method further includes: The database instance writes the generated multiple transaction records into a global transaction cache, so that the multiple transaction records can be written in parallel into multiple transaction log files of the file system through the global transaction cache.

3. The method according to claim 1, characterized in that, The ID of the previous transaction log file included in the first transaction log file is a pre-set first ID, and the ID of the next transaction log file included in the last transaction log file is a pre-set second ID.

4. A data access method, characterized in that, include: The database instance receives a data read request; The database instance reads the identifier ID of each transaction log file and information indicating the order among the multiple transaction log files from the metadata file of the file system. The database instance reads and concatenates multiple transaction records in the multiple transaction log files based on the ID of each transaction log file and information indicating the order among the multiple transaction log files. The information indicating the order among the multiple transaction log files includes the ID of the previous transaction log file and the ID of the next transaction log file for the current transaction log file.

5. The method according to claim 4, characterized in that, The ID of the previous transaction log file included in the first transaction log file is a pre-set first ID, and the ID of the next transaction log file included in the last transaction log file is a pre-set second ID.

6. The method according to claim 4 or 5, characterized in that, Each transaction record in the multiple transaction log files also carries a preset field containing the LSN of the previous transaction record. The database instance reads and concatenates multiple transaction records from the multiple transaction log files based on the IDs of the multiple transaction log files and information indicating the order among the multiple transaction log files, including: The database instance determines the order of the multiple transaction log files based on their IDs and information indicating the order among them. The database instance first finds the last transaction record stored in each of the multiple transaction log files. Based on the LSN of the previous transaction record carried by each transaction record, it determines the transaction record with the largest LSN of the previous transaction record among the multiple transaction records and reads it, and determines it as the first transaction record read. The database instance performs the following operations sequentially until all transaction records in the multiple transaction log files are read: The database instance searches for and reads the preceding transaction record of the (M-1)th transaction record based on the LSN of the preceding transaction record carried by the (M-1)th transaction record, and determines it as the read Mth transaction record. The preceding transaction record of the (M-1)th transaction record is stored in the transaction log file where the (M-1)th transaction record is located, or in the first N transaction log files of the transaction log file where the (M-1)th transaction record is located, or in the last N-1 files of the transaction log file where the (M-1)th transaction record is located, where M is greater than or equal to 2. The database instance concatenates the first transaction record, the second transaction record, ..., the Mth transaction record according to the LSN size of the preceding transaction record, to obtain all transaction records in the multiple transaction log files.

7. The method according to claim 4 or 5, characterized in that, Each transaction record in the multiple transaction log files also carries a preset field containing the LSN of the previous transaction record. The database instance reads and concatenates multiple transaction records from the multiple transaction log files based on the IDs of the multiple transaction log files and information indicating the order among the multiple transaction log files, including: The database instance determines the order of the multiple transaction log files based on their IDs and information indicating the order among them. The database instance reads all transaction records from each of the multiple transaction log files sequentially according to their order. The database instance concatenates all the transaction records it reads according to the LSN size of the previous transaction record to obtain all the transaction records in the multiple transaction log files.

8. A data access device, characterized in that, include: The creation module is used to create multiple transaction log files and metadata files; The receiving module is used to receive data write requests; The generation module is used to generate multiple transaction records based on the data write request, wherein each of the multiple transaction records has a unique LSN; The writing module is used to: write the multiple transaction records in parallel to the multiple transaction log files based on the log sequence number LSN of each transaction record and the identifier ID of each transaction log file, wherein the log sequence number LSN of the previous transaction record is recorded in each transaction record written to the multiple transaction log files; The ID of each transaction log file and information indicating the order among the multiple transaction log files are written into the metadata file. The information indicating the order among the multiple transaction log files includes the ID of the previous transaction log file and the ID of the next transaction log file.

9. The apparatus according to claim 8, characterized in that, The writing module is also used for: The generated multiple transaction records are written to a global transaction cache so that the multiple transaction records can be written in parallel to multiple transaction log files of the file system through the global transaction cache.

10. The apparatus according to claim 8, characterized in that, The ID of the previous transaction log file included in the first transaction log file is a pre-set first ID, and the ID of the next transaction log file included in the last transaction log file is a pre-set second ID.

11. A data access device, characterized in that, include: The receiving module is used to receive data read requests; The reading module is used to read the identifier ID of each transaction log file and the information indicating the order among the multiple transaction log files stored in the metadata file of the file system; The processing module is used to read and concatenate multiple transaction records in the multiple transaction log files according to the ID of each transaction log file and information indicating the order among the multiple transaction log files. The information indicating the order among the multiple transaction log files includes the ID of the previous transaction log file and the ID of the next transaction log file of the current transaction log file.

12. The apparatus according to claim 11, characterized in that, The ID of the previous transaction log file included in the first transaction log file is a pre-set first ID, and the ID of the next transaction log file included in the last transaction log file is a pre-set second ID.

13. The apparatus according to claim 11 or 12, characterized in that, Each transaction record in the plurality of transaction log files also carries a preset field containing the LSN of the previous transaction record, and the processing module is used to: The order of the multiple transaction log files is determined based on the ID of each transaction log file and information used to indicate the order among the multiple transaction log files; First, find the last transaction record stored in each of the multiple transaction log files. Based on the LSN of the previous transaction record carried by each transaction record, determine the transaction record with the largest LSN of the previous transaction record among the multiple transaction records and read it, and determine it as the first transaction record read. Perform the following operations sequentially until all transaction records in the multiple transaction log files are read: The preceding transaction record of the (M-1)th transaction record is found and read based on the LSN of the preceding transaction record carried by the (M-1)th transaction record. This preceding transaction record is determined to be the read Mth transaction record. The preceding transaction record of the (M-1)th transaction record is stored in the transaction log file where the (M-1)th transaction record is located, or in the first N transaction log files of the transaction log file where the (M-1)th transaction record is located, or in the last N-1 files of the transaction log file where the (M-1)th transaction record is located, where M is greater than or equal to 2. The first transaction record, the second transaction record, ..., the Mth transaction record are concatenated in order of the LSN of the preceding transaction record to obtain all the transaction records in the multiple transaction log files.

14. The apparatus according to claim 11 or 12, characterized in that, Each transaction record in the plurality of transaction log files also carries a preset field containing the LSN of the previous transaction record, and the processing module is used to: The order of the multiple transaction log files is determined based on the ID of each transaction log file and information used to indicate the order among the multiple transaction log files; Read all transaction records from each of the multiple transaction log files sequentially according to their order. All the read transaction records are concatenated in order of the LSN of the previous transaction record to obtain all the transaction records in the multiple transaction log files.

15. A computing device, characterized in that, The computing device includes a processor and memory. The memory stores computer instructions; The processor executes computer instructions stored in the memory, causing the computing device to perform the data access method according to any one of claims 1-3 or 4-7, and causing the computing device to deploy the data access apparatus according to any one of claims 8-10 or 11-14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that instruct the computing device to perform the data access method according to any one of claims 1-3 or 4-7, or that instruct the computing device to deploy the data access apparatus according to any one of claims 8-10 or 11-14.

Images