Data writing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a data writing method and device, computer equipment and a storage medium, and belongs to the technical field of databases. In the embodiment of the invention, when any thread in the computer equipment does not obtain the available memory page in the plurality of idle memory pages corresponding to the target data table, N idle data pages can be expanded for the thread, the idle memory pages are expanded based on the N idle data pages, and then data are written in the expanded idle memory pages. Therefore, any thread which cannot obtain the available memory page can automatically trigger the data table expansion process to timely expand N data pages for the thread for use. Moreover, data pages do not need to be expanded for other threads, so that the time required for expanding the data pages for each thread is short, the thread blocking time can be shortened, and the data writing performance is improved.

Description

Technical Field

[0001] The present application relates to the field of databases, and in particular to a data writing method, device, computer equipment and storage medium. Background Art

[0002] At present, in databases, files are usually stored in the form of data tables. A data table includes multiple data pages, which are persistently stored in the disk, and each data page can correspond to a memory page. When a computer device updates a data page, the computer device can load the data page into the corresponding memory page for updating.

[0003] In the related art, the free memory pages corresponding to the free data pages in a data table can be managed by a page binary tree. When multiple threads in a computer device want to write data in the same data table, the multiple threads can respectively search for available memory pages with sufficient space through the page binary tree according to the amount of data to be written. In this way, multiple threads may find the same available memory page. In this case, multiple threads can queue up to add write locks to the available memory page in turn, and write data to the available memory page after adding the write lock successfully. Based on this, if the available memory page is the last free memory page corresponding to the data table, and after a thread finishes writing data, the remaining capacity of the available memory page is not enough for the subsequent multiple threads to write data, then the subsequent multiple threads will apply for the expansion of the data table at the same time. In this case, one of the threads will expand multiple free data pages for the multiple threads at one time according to the number of threads applying for the expansion of the data table, while the other threads will all wait, which will cause the data table to be expanded in an untimely manner. Summary of the invention

[0004] The present application provides a data writing method, apparatus, computer equipment and storage medium for improving data writing performance when multiple threads concurrently write data to a data table.

[0005] In order to achieve the above purpose, this application adopts the following technical solutions:

[0006] In a first aspect, a data writing method is provided, which can be applied to a first thread in a computer device, wherein the first thread is any one of a plurality of threads to be written to data running in the computer device, and the method comprises: if no available memory page is obtained from a plurality of free memory pages corresponding to a target data table, then N free data pages are expanded for the first thread in the target data table, where N is a preset integer greater than 0; free memory pages are expanded for the target data table based on the N free data pages; and data is written in the expanded free memory pages.

[0007] In the present application, when any thread in the computer device fails to obtain an available memory page from the multiple free memory pages corresponding to the target data table, it can expand N free data pages for itself, and expand the free memory pages based on the N free data pages, and then write data in the expanded free memory pages. It can be seen that any thread that fails to obtain an available memory page can trigger the data table expansion process by itself to promptly expand N data pages for itself for its own use. In addition, since there is no need to expand data pages for other threads, the time required for each thread to expand data pages is shorter. Compared with a solution in which one thread centrally expands data pages for all threads that have not obtained available memory pages, while other threads are waiting, the thread blocking time can be shortened, thereby improving data writing performance.

[0008] Optionally, the implementation process of expanding N free data pages for the first thread in the target data table may include: adding a write lock to the target data table; if adding a write lock to the target data table is successful, applying for N extended storage areas for the target data table in a persistent storage device; writing 0 in the N extended storage areas to obtain the N free data pages; and releasing the write lock of the target data table.

[0009] In the present application, the first thread first locks the target data table when expanding the free data page to prevent other threads from accessing the target data table during the expansion process and causing a read-write conflict. In addition, the process of the first thread expanding the data table includes two sub-processes: expanding the free data page in the persistent storage device and allocating memory pages to the free data page and updating the page binary tree. Based on this, when the first thread expands the free data page in the persistent storage device, it adds a write lock to the target data table, but does not add a write lock to the page binary tree corresponding to the target data table. In this way, in the process of the first thread expanding the free data page, other threads that have completed the data page expansion can update the page binary tree. In this way, different threads can execute different sub-processes in the data table expansion process in time-sharing, so that the expansion efficiency of the data table can be improved.

[0010] Optionally, the implementation process of expanding the free memory pages for the target data table based on the N free data pages may include: allocating corresponding memory pages to the N free data pages; based on the memory pages corresponding to the N free data pages, updating the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

[0011] Optionally, the implementation process of updating the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages may include: adding a write lock to the page binary tree corresponding to the target data table; if adding a write lock to the page binary tree corresponding to the target data table is successful, adding page nodes corresponding to the memory pages corresponding to the N free data pages in the page binary tree corresponding to the target data table; and releasing the write lock of the page binary tree corresponding to the target data table.

[0012] In the present application, the first thread adds a write lock to the page binary tree before updating the page binary tree to prevent multiple threads from updating the page binary tree of the target data table at the same time and causing conflicts. In addition, the process of the first thread expanding the data table includes two sub-processes: expanding the data page in the persistent storage device and allocating memory pages for the expanded data page and updating the page binary tree. Based on this, when the first thread updates the page binary tree, it adds a write lock to the page binary tree, but does not add a write lock to the target data table. In this way, during the process of the first thread updating the page binary tree, other threads that trigger the data table expansion process can expand the data page. In this way, different threads can execute different sub-processes in the table expansion process in time-sharing, thereby improving the expansion efficiency of the data table.

[0013] Optionally, the method also includes: searching for an available memory page in multiple free memory pages corresponding to the target data table; if adding a write lock to the currently found available memory page fails, counting the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is greater than a reference threshold, determining that no available memory page is obtained from the multiple free memory pages corresponding to the target data table.

[0014] In the present application, the first thread does not need to wait until it is determined that the last free memory page is also used up before expanding the data table. Instead, when the write lock fails to be added to the available memory page found multiple times and the number of failures exceeds the threshold, the data table expansion process can be directly triggered. That is, the first thread can trigger the data table expansion in advance when there are still a small number of free memory pages in the data table. In this way, the problem of a long waiting time caused by one thread performing expansion after the last free memory page is used up and other threads waiting together can be improved.

[0015] In a second aspect, a data writing method is provided, which can be applied to a first thread of a computer device, where the first thread is any one of multiple threads to be written in the computer device. The first thread can search for available memory pages in multiple free memory pages corresponding to a target data table; if adding a write lock to the currently found available memory page fails, then searching for other available memory pages in the multiple free memory pages; and writing data to other available memory pages to which the write lock is successfully added.

[0016] The first thread is a transaction thread of a database application running on a computer device, and the transaction thread is used to write data in a target data table. Based on this, the first thread can search for available memory pages in multiple free memory pages corresponding to the target data table. If the write lock for the currently found available memory page fails, it means that other threads are accessing the available memory page. In this case, the first thread does not wait for other threads to release the lock of the available memory page, but directly searches for other available memory pages in multiple free memory pages, and writes data in other available memory pages that are successfully locked, so that the waiting time of the first thread can be shortened, thereby improving data writing performance.

[0017] Optionally, when the first thread searches for other available memory pages among the multiple free memory pages, it can start searching for the target page node from the next page node of the first page node in the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table, the first page node is used to indicate the available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; the free memory page indicated by the target page node is used as other available memory pages.

[0018] That is to say, in the present application, after the first thread fails to add a write lock to the currently found available memory page, it can continue to search for other available memory pages from the next node of the page node corresponding to the available memory page that failed to add a write lock, instead of re-searching from the root node of the page binary tree. In this way, the search time can be shortened.

[0019] Optionally, if adding a write lock to the currently found available memory page fails, the first thread may first count the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is not greater than a reference threshold, then execute the step of searching for other available memory pages from the multiple free memory pages.

[0020] If the cumulative number of available memory pages that failed to add write locks is not greater than the reference threshold, it means that the first thread has failed to add write locks a small number of times, and there may be other available memory pages with sufficient space for the first thread to write data in the multiple free memory pages corresponding to the target data table. In this case, the first thread can continue to search for other available memory pages in the multiple free memory pages.

[0021] Optionally, if adding a write lock to the currently found available memory page fails, the first thread may first count the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is greater than the reference threshold, N free data pages are expanded for the first thread in the target data table, where N is a preset integer greater than 0; based on the N free data pages, the free memory pages are expanded for the target data table, and data is written in the expanded free memory pages.

[0022] In the present application, the first thread does not need to wait until it is determined that the last free memory page is also used up before expanding the data table. Instead, when the write lock fails to be added to the found available memory page multiple times and the number of failures exceeds the threshold, the data table expansion process can be directly triggered. That is, the first thread can trigger the data table expansion in advance when there are still a small number of free memory pages in the data table. In this way, the problem of a long waiting time caused by one thread performing the expansion after the last free memory page is used up and other threads waiting intensively can be improved. In addition, since each thread can directly trigger the data table expansion process under the above conditions, the efficiency of data table expansion can be improved, thereby improving the data writing speed.

[0023] Optionally, the implementation process of expanding the free memory pages for the target data table based on the N free data pages may include: allocating corresponding memory pages to the N free data pages, and updating the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages, so that each thread can find the expanded free memory page to write data through the updated page binary tree. The page binary tree includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

[0024] Optionally, when the first thread expands N free data pages for itself in the target data table, it can first add a write lock to the target data table. If the write lock for the target data table is successfully added, N extended storage areas are applied for the target data table in the persistent storage device; 0 is written in the N extended storage areas to obtain the N free data pages; and then, the write lock of the target data table is released.

[0025] In the present application, when the first thread expands the data page, it first adds a write lock to the target data table to prevent other threads from accessing the target data table during the table expansion process and causing a read-write conflict. In addition, the process of the first thread expanding the data table includes two sub-processes: expanding the data page in the persistent storage device and allocating memory pages for the expanded data page and updating the page binary tree. Based on this, when the first thread expands the data page in the persistent storage device, it adds a write lock to the target data table, but does not add a write lock to the page binary tree corresponding to the target data table. In this way, in the process of the first thread expanding the data page, other threads that have completed the data page expansion can update the page binary tree. In this way, different threads can execute different sub-processes in the table expansion process in time-sharing, so that the expansion efficiency of the data table can be improved.

[0026] Optionally, the implementation process of the first thread updating the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages may include: adding a write lock to the page binary tree corresponding to the target data table; if adding a write lock to the page binary tree corresponding to the target data table is successful, then adding page nodes corresponding to the memory pages corresponding to the N free data pages in the page binary tree corresponding to the target data table; and releasing the write lock of the page binary tree corresponding to the target data table.

[0027] In the present application, the first thread adds a write lock to the page binary tree before updating the page binary tree to prevent multiple threads from updating the page binary tree of the target data table at the same time and causing conflicts. In addition, the process of the first thread expanding the data table includes two sub-processes: expanding the data page in the persistent storage device and allocating memory pages for the expanded data page and updating the page binary tree. Based on this, when the first thread updates the page binary tree, it adds a write lock to the page binary tree, but does not add a write lock to the target data table. In this way, during the process of the first thread updating the page binary tree, other threads that trigger the data table expansion process can expand the data page. In this way, different threads can execute different sub-processes in the table expansion process in time-sharing, thereby improving the expansion efficiency of the data table.

[0028] In a third aspect, a data writing device is provided, the data writing device comprising at least one module, the at least one module being used to execute the data writing method described in the first aspect through a first thread in a computer device. Exemplarily, the at least one module may include a first expansion module, a second expansion module, and a writing module.

[0029] Among them, the first expansion module is used to expand N free data pages for the first thread in the target data table through the first thread if no available memory page is obtained from the multiple free memory pages corresponding to the target data table, wherein N is a preset integer greater than 0, and the first thread is any one of the multiple threads to be written to data running in the computer device; the second expansion module is used to expand the free memory pages for the target data table based on the N free data pages through the first thread; the writing module is used to write data in the expanded free memory pages through the first thread.

[0030] Optionally, the first expansion module is specifically used to: add a write lock to the target data table; if the write lock for the target data table is successfully added, apply for N extended storage areas for the target data table in the persistent storage device; write 0 in the N extended storage areas to obtain the N free data pages; and release the write lock of the target data table.

[0031] Optionally, the second expansion module is specifically used to: allocate corresponding memory pages to the N free data pages; based on the memory pages corresponding to the N free data pages, update the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

[0032] Optionally, the second expansion module is specifically used to: add a write lock to the page binary tree corresponding to the target data table; if the write lock is successfully added to the page binary tree corresponding to the target data table, then add a page node corresponding to the memory page corresponding to the N free data pages in the page binary tree corresponding to the target data table; and release the write lock of the page binary tree corresponding to the target data table.

[0033] Optionally, the device also includes: a search module, used to search for available memory pages in multiple free memory pages corresponding to the target data table through the first thread; a statistical module, used to count the cumulative number of available memory pages that failed to add a write lock to the currently found available memory page through the first thread if the cumulative number of available memory pages that failed to add a write lock is greater than a reference threshold, and a determination module, used to determine that the first thread has not obtained an available memory page in the multiple free memory pages corresponding to the target data table.

[0034] In a fourth aspect, a data writing device is provided, the data writing device comprising at least one module, the at least one module being used to execute the data writing method described in the second aspect through a first thread in a computer device. Exemplarily, the at least one module may include a search module and a write module.

[0035] Among them, the search module is used to search for available memory pages in multiple free memory pages corresponding to the target data table through the first thread; if the write lock fails to be added to the currently found available memory page, then search for other available memory pages in the multiple free memory pages, and the first thread is any one of the multiple threads to be written to the data running in the computer device; the writing module is used to write data in other available memory pages to which the write lock is successfully added through the first thread.

[0036] Optionally, the search module is specifically used to: search for the target page node starting from the next page node of the first page node in the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table, the first page node is used to indicate the available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; use the free memory page indicated by the target page node as other available memory pages.

[0037] Optionally, the search module is specifically used to: if adding a write lock to the currently found available memory page fails, then counting the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is not greater than a reference threshold, then searching for other available memory pages from the multiple free memory pages.

[0038] Optionally, the device also includes: a first expansion module, used to expand N free data pages for the first thread in the target data table through the first thread if the cumulative number of available memory pages that fail to add write locks is greater than the reference threshold; a second expansion module, used to allocate corresponding memory pages to the N free data pages through the first thread; based on the memory pages corresponding to the N free data pages, updating the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

[0039] Optionally, the first expansion module is specifically used to: add a write lock to the target data table; if the write lock for the target data table is successfully added, apply for N extended storage areas for the target data table in the persistent storage device; write 0 in the N extended storage areas to obtain the N free data pages; and release the write lock of the target data table.

[0040] Optionally, the second expansion module is specifically used to: add a write lock to the page binary tree corresponding to the target data table; if the write lock is successfully added to the page binary tree corresponding to the target data table, then add a page node corresponding to the memory page corresponding to the N free data pages in the page binary tree corresponding to the target data table; and release the write lock of the page binary tree corresponding to the target data table.

[0041] In a fifth aspect, a computer device is provided, the computer device comprising a processor and a memory, the processor being used to execute at least one program instruction or code stored in the memory to implement the data writing method described in the first aspect or the second aspect. Optionally, the memory is a memory, and the computer device may further comprise a persistent storage device, which may be a hard disk.

[0042] In a sixth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer device, the computer device executes the data writing method described in the first aspect or the second aspect.

[0043] In a seventh aspect, a computer program product comprising instructions is provided. When the computer program product is run on a computer device, the computer device executes the data writing method described in the first aspect or the second aspect.

[0044] The technical effects obtained in the above-mentioned third to seventh aspects are similar to the technical effects obtained by the corresponding technical means in the first or second aspect, and will not be repeated here.

[0045] In an embodiment of the present application, when any thread in the computer device fails to obtain an available memory page from the multiple free memory pages corresponding to the target data table, it can expand N free data pages for itself, and expand the free memory pages based on the N free data pages, and then write data in the expanded free memory pages. It can be seen that any thread that fails to obtain an available memory page can trigger the data table expansion process by itself to promptly expand N data pages for itself for its own use. In addition, since there is no need to expand data pages for other threads, the time required for each thread to expand data pages is shorter. Compared with a solution in which one thread centrally expands data pages for all threads that have not obtained available memory pages, while other threads are waiting, the thread blocking time can be shortened, thereby improving data writing performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A storage system architecture diagram involved in the data writing method provided in an embodiment of the present application;

[0047] Figure 2A diagram of the operation architecture of a database application provided in an embodiment of the present application;

[0048] Figure 3 Another storage system architecture diagram involved in the data writing method provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of a data writing method provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a lock state variable corresponding to a memory page provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of a lock state variable corresponding to another memory page provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of a multiple thread extended data table provided in an embodiment of the present application;

[0053] Figure 8 A flowchart of another data writing method provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of a page binary tree provided in an embodiment of the present application;

[0055] Fig.10 A flowchart of another data writing method provided in an embodiment of the present application;

[0056] Fig.11 A data writing performance comparison curve diagram provided in an embodiment of the present application;

[0057] Fig.12 A schematic diagram of the structure of a data writing device provided in an embodiment of the present application;

[0058] Fig.13 A schematic diagram of the structure of another data writing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0060] Before explaining the embodiments of the present application in detail, the application scenarios involved in the embodiments of the present application are first introduced.

[0061] In a database, files are usually stored in the form of data tables. A data table includes multiple data pages, which are persistently stored in a persistent storage device, such as a disk of a computer device. The size of each data page can be equal, for example, 8KB, that is, each data page can include 8KB of data. In addition, each data page can correspond to a memory page in a computer device, where a memory page can be understood as a storage area in the memory of a computer device. When a computer device accesses a data page, the computer device can load the data page from the persistent storage device to the corresponding memory page. For example, when a computer device wants to write data to a data page in a data table, it can load the data page into the corresponding memory page and write data in the memory page. After the data is written, the computer device can store the data in the memory page in the persistent storage device.

[0062] At present, computer devices usually access databases through transactions. In addition, the free memory pages corresponding to the data pages with free space (i.e., data pages that are not fully written with data) in a data table in the database can be managed through a page binary tree. Based on this, when a transaction thread in a computer device wants to write data in a data table, the transaction thread can use the page binary tree to find available memory pages with sufficient space. After finding the available memory page, the transaction thread adds a write lock to the available memory page. If the write lock is successfully added, the data is written to the available memory page. If the write lock fails, it means that other transaction threads have currently locked the available memory page. In this case, the transaction thread waits for other transaction threads to unlock the available memory page. After other transaction threads unlock the available memory page, the transaction thread determines whether there is enough space to write data in the available memory page. If there is enough space to write data, the available memory page is locked again, and data is written to the available memory page after the lock is successfully added.

[0063] In some scenarios, multiple transaction threads running in a computer device may write data in the same data table at the same time. The "simultaneous" in the embodiment of the present application may refer to the same time or a short time interval. In this scenario, when multiple transaction threads search for available memory pages through the page binary tree, there is a high probability that the same available memory page will be found. In this case, after a transaction thread among the multiple transaction threads successfully locks the available memory page, the other transaction threads all need to wait for the transaction thread to write data before continuing to write data in the available memory page. In other words, the multiple transaction threads need to write data in the available memory page in sequence in a serial manner, and the writing efficiency is low. Not only that, in the process of serial writing, there may be a situation where the free space of the available memory page is insufficient after a transaction thread writes data, so that other transaction threads waiting to write data later will no longer be able to write data in the available memory page. In this case, other transaction threads waiting to write data later need to start from the root node of the page binary tree to find the available memory page again, which will further increase the writing time of these transaction threads.

[0064] Furthermore, if the available memory page is the last available memory page with sufficient space among the free memory pages managed by the page binary tree, then other transaction threads waiting to write data will find that they cannot find the available memory page after re-searching the available memory page from the root node of the page binary tree. In this case, these transaction threads will simultaneously apply to expand the data table, that is, to expand the free data page in the data table. When a transaction thread successfully applies to expand the data table, the other transaction threads will all enter the waiting state, and the transaction thread will determine the number of expanded pages according to the number of transaction threads currently applying to expand the data table, and expand the corresponding number of data pages in the data table and allocate corresponding memory pages for each data page. In this way, other transaction threads that subsequently apply to expand the data table can find the free memory page corresponding to the expanded free data page through the page binary tree and write data therein. It can be seen from this that when multiple transaction threads apply to expand the data table at the same time, it is actually one of the transaction threads that expands the required data pages for the multiple transaction threads at one time, while the other transaction threads are all waiting, and the data table is not expanded in time. In this case, the more transaction threads that apply for expanding the data table at the same time, the more data pages that need to be expanded at one time. In this way, the waiting time of other transaction threads is also longer, resulting in a longer thread blocking time.

[0065] Based on the above problems, an embodiment of the present application provides a data writing method, which can be applied to a scenario where multiple transaction threads concurrently write a large amount of data in the same data table, so as to improve the data writing speed in this scenario and improve the problem of untimely data table expansion, thereby improving the single table writing performance.

[0066] Next, the system architecture involved in the data writing method provided in the embodiment of the present application is introduced.

[0067] Figure 1 is an architecture diagram of a storage system involved in a data writing method provided in an embodiment of the present application. Figure 1 As shown, the storage system includes one or more servers 10 ( Figure 1 The three servers 10 are shown in FIG. 1 , but are not limited to the three servers 10 . The servers 10 can communicate with each other. The server 10 is a device that has both computing and storage capabilities, such as a server, a desktop computer, etc. At the hardware level, Figure 1 As shown, the server 10 includes at least a processor 101, a memory 102, a network card 103, and a hard disk 104. The processor 101, the memory 102, the network card 103, and the hard disk 104 are connected via a bus. At the software level, an application program, such as a database application, can be deployed in the server 10 to process the read and write operations of data pages in the database. It should be noted that the above-mentioned application program can be stored in the memory 102 of the server 10 and executed by the processor 101 to implement the corresponding functions.

[0068] Specifically, the processor 101 is a central processing unit (CPU), which is used to process data access requests from outside the server 10 (application server or other server 10), and is also used to process requests generated inside the server 10. Exemplarily, when the processor 101 receives a write data request, it temporarily stores the data in the write data request in the memory 102. When the total amount of data in the memory 102 reaches a certain threshold, the processor 101 sends the data stored in the memory 102 to the hard disk 104 for persistent storage. In addition, the processor 101 is also used for data calculation or processing, such as metadata management, deduplication, data compression, data verification, virtualized storage space, and address conversion. Figure 1 Only one CPU 101 is shown in the figure. In actual applications, there are often multiple CPUs 101, wherein one CPU 101 has one or more CPU cores. The embodiment of the present application does not limit the number of CPUs and the number of CPU cores.

[0069] The memory 102 refers to an internal memory that directly exchanges data with the processor. The storage space of the memory 102 can be divided into multiple memory pages. It can read and write data at any time, and the speed is very fast. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memory. For example, the memory can be either a random access memory or a read-only memory (ROM). For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard disks, but the access speed is slower than DRAM, and the cost is cheaper than DRAM. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memory (SRAM). As for the read-only memory, for example, it can be a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), etc. In addition, the memory 102 can also be a dual in-line memory module or a dual-line memory module (DIMM), that is, a module composed of DRAM, or a solid state disk (SSD). In actual applications, multiple memories 102 and different types of memories 102 can be configured in the server 10. This embodiment does not limit the number and type of memory 102. In addition, the memory 102 can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 102 will not be lost. The memory with a power-saving function is called a non-volatile memory.

[0070] The hard disk 104 is used to provide storage resources, for example, to store data in a database in the form of data pages, where each data page can be mapped to a memory page in the memory 102. The hard disk 104 can be a disk or other types of storage media, such as a solid state drive or a shingled magnetic recording hard disk. The network card 103 is used to communicate with other application servers 10.

[0071] For example, Figure 2 1 is a schematic diagram of running a database application on a server 10 provided in an embodiment of the present application. Figure 2 As shown, a database main process may be run in the processor, and the database main process includes a main thread group, a data disk thread group, and a redo log (redolog) thread group. Among them, when a database update request or a data insertion request is received, the transaction thread in the main thread group can load the data page stored in the hard disk into the memory page of the memory, and then write the data to be written into the memory page. The redo log thread group includes a write log (write log) thread and a redo log thread. The write log thread can generate a transaction log that records the change information of the data in the memory page, and the redo log thread can restore the database by reading the transaction log of the persistent storage. The data disk thread group includes a checkpoint (checkpoint) thread and a write page (page write) thread. Among them, the checkpoint thread can trigger the write page thread to write the data of the memory page to the hard disk for persistent storage.

[0072] The above is an exemplary system architecture of a storage system used by the data writing method in the embodiment of the present application. In addition, the data writing method can also be applied to other types of storage systems, for example, a storage system with separated storage and computing. The so-called storage and computing separation refers to separating the computing function and the storage function and deploying them on different devices. Among them, the device mainly used to implement the computing function is called a computing node, and the device mainly providing data storage function is called a storage node. For example, Figure 3 This is an architecture diagram of a storage system with separated storage and computing. The data writing method provided in the embodiment of the present application can also be applied to the storage system. Figure 3 As shown, the storage system may include multiple computing nodes 30 and multiple storage nodes 31. The computing nodes 30 may communicate with each other through a network, and each computing node 30 may access any storage node 31 through the network.

[0073] The computing node 30 is a computing device, such as a server, a desktop computer, etc. At the hardware level, Figure 3As shown, the computing node 30 includes at least a processor 301, a memory 302, and a network card 303. At the software level, an application program may be deployed in the computing node 30. For example, a client of a database application may be deployed in the computing node 30 to process the read and write operations of data pages in the database. It should be noted that the above application program may be stored in the memory 302 of the computing node 30 and executed by the processor 301 to implement the corresponding function.

[0074] Specifically, the processor 301 is a central processing unit (CPU) for processing data access requests from outside the computing node 30 or requests generated inside the computing node 30. Exemplarily, when the processor 301 receives a write data request sent by a user, the data in the write data request will be temporarily stored in the memory 302. When the total amount of data in the memory 302 reaches a certain threshold, the processor 301 sends the data stored in the memory 302 to the storage node 31 for persistent storage. For example, taking the processor 301 running a database application as an example, the processor 301 can run a main thread, a data disk thread, and a log recording thread. Among them, when a database update request or a data insertion request is received, the main thread can load the data page stored in the storage node 31 into the memory page of the memory 302, and then write the data to be written into the memory page. The log recording thread can record the change information of the data in the memory page. When the total amount of data in the memory page reaches a certain threshold, the data disk thread can send the data of the memory page to the storage node 31 for persistent storage.

[0075] In addition, the processor 301 is also used for computing or processing data, such as metadata management, data deduplication, data compression, virtualized storage space, and address conversion. Figure 3 Only one CPU 301 is shown in the figure. In actual applications, there are often multiple CPUs 301, wherein one CPU 301 has one or more CPU cores. This embodiment does not limit the number of CPUs and the number of CPU cores.

[0076] The memory 302 refers to the internal memory that directly exchanges data with the processor 301. The storage space of the memory 302 can be divided into multiple memory pages. It can read and write data at any time, and the speed is very fast. It serves as a temporary data storage for the operating system or other running programs. The detailed implementation of the memory 302 can be referred to in the above Figure 1 The relevant introduction of the memory in the server shown will not be repeated here in the embodiment of the present application.

[0077] The network card 303 is used to communicate with the storage node 31. For example, when the total amount of data in the memory 302 reaches a certain threshold, the computing node 30 can send a request to the storage node 31 through the network card 303 to store the data persistently. In addition, the computing node 30 can also include a bus for communication between the components inside the computing node 30. Figure 3 The main function of the computing node 30 in the example is computing services. When storing data, remote storage can be used to achieve persistent storage. Therefore, it has less local storage than a conventional server, thereby saving costs and space. However, this does not mean that the computing node 30 cannot have local storage. In actual implementation, the computing node 30 can also have a small number of hard disks built in or externally connected.

[0078] Any computing node 30 can access any storage node 31 through the network. The storage system in the embodiment of the present application includes multiple storage nodes 31 ( Figure 3 Three storage nodes 31 are shown in the figure, but are not limited to three storage nodes 31). The storage node 31 may include one or more controllers 311, a network card 312 and multiple hard disks 313. The network card 312 is used to communicate with the computing node 30. The hard disk 313 is used to store data, for example, to store data in a database in the form of data pages, wherein each data page can be mapped to a memory page in the memory 102. The hard disk 313 can be a disk or other types of storage media, such as a solid-state hard disk or a shingled magnetic recording hard disk. The controller 311 is used to write data to the hard disk 313 or read data from the hard disk 313 according to the read / write data request sent by the computing node 30. In the process of reading and writing data, the controller 311 needs to convert the address carried in the read / write data request into an address that the hard disk can recognize. It can be seen that the controller 311 also has some simple computing functions.

[0079] In practical applications, the controller 311 may have various forms. In one embodiment, the controller 311 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 313, or read from the hard disk 313 to be sent to the computing node 30.

[0080] In another embodiment, the functions of the controller 311 can be offloaded to the network card 312. In other words, the storage node 31 does not have a controller 311 inside, but the network card 312 completes data reading and writing, address conversion and other computing functions. In this case, the network card 312 is an intelligent network card. It can include a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store data to be written to the hard disk 313, or read from the hard disk 313 to be sent to the computing node 30.

[0081] In addition, in the embodiment of the present application, at the software level, a storage program may be deployed in the storage node 31. For example, a server of a database application may be deployed to provide functions such as data page access and access permission management in the database. It should be noted that the storage program may be stored in the memory of the controller 311, and the CPU of the controller 311 may execute the storage program to implement the functions of the server of the database application. Alternatively, the storage program may be stored in the memory of the network card 312, and the CPU in the network card 312 may execute the storage program to implement the functions of the server of the database application. It should be noted that when in the above Figure 3 When a database application is run in the storage system shown in FIG. 1 , the operation architecture diagram of the database application can refer to the aforementioned Figure 2 In this case, Figure 2 The memory in may be the memory 302 in the computing node 30 , and the hard disk may be the hard disk 313 in the storage node 31 .

[0082] Next, the data writing method provided in the embodiment of the present application is explained in detail.

[0083] The data writing method provided in the embodiment of the present application can be executed by a computer device. Figure 1 In the storage system shown in the figure, the computer device may refer to any server in the storage system. Figure 3 In the storage system shown in FIG. 1 , the computer device may refer to a computing node in the storage system. The computer device may run multiple threads to write data, and the method may be executed by any one of the multiple threads. For example, a database application may be deployed in the computer device, and the running architecture of the database application may be as follows: Figure 2 As shown, the multiple threads may be multiple transaction threads included in the main thread group in the database main process. For the convenience of description, the first thread in the multiple threads is used as the execution subject for introduction in the following embodiments.

[0084] Figure 4 is a flow chart of a data writing method provided in an embodiment of the present application, see Figure 4 , the method comprises the following steps:

[0085] Step 401: If no available memory page is obtained from the multiple free memory pages corresponding to the target data table, N free data pages are expanded for the first thread in the target data table, where N is a preset integer greater than 0.

[0086] In an embodiment of the present application, the data in the database can be stored in a persistent storage device in the form of a data table, wherein the persistent storage device can be a hard disk included in the computer device itself, or a hard disk in another device other than the computer device.

[0087] In addition, a data table may include multiple data pages, and the size of each data page is used to indicate the amount of data included in the corresponding data page. The sizes of the data pages may be equal, for example, all 8KB. And each data page corresponds to a memory page. The memory page corresponding to the free data page with free space in the data table will also have free space, so it can be called a free memory page.

[0088] When a first thread in a computer device wants to write data in a target data table, the first thread can first obtain an available memory page from a plurality of free memory pages corresponding to the target data table. If an available memory page is obtained, the first thread can write data in the available memory page. If an available memory page is not obtained, the first thread can trigger a data table expansion process to expand N free data pages for itself in the target data table. The target data table is any data table stored in a database.

[0089] In one possible implementation, the first thread may search for an available memory page in the free memory pages corresponding to the target data table; if adding a write lock to the currently found available memory page fails, the cumulative number of available memory pages that failed to add a write lock is counted; if the cumulative number of available memory pages that failed to add a write lock is greater than a reference threshold, it is determined that no available memory page is obtained in the multiple free memory pages corresponding to the target data table.

[0090] Wherein, the free memory pages corresponding to the free data pages of a data table can be managed by a page binary tree. Wherein, the page binary tree may include multiple page nodes, each page node is used to indicate a free memory page. Exemplarily, a page identifier and remaining capacity of a free memory page may be recorded in each page node. Based on this, the first thread can search for available memory pages in multiple free memory pages corresponding to multiple free data pages in the target data table through the page binary tree corresponding to the target data table.

[0091] Exemplarily, the first thread can start from the root node of the page binary tree or the next page node of the first page node corresponding to the available memory page that failed to add a write lock most recently, and search for page nodes whose recorded remaining capacity is not less than the amount of data to be written, and use the free memory page indicated by the first page node found that meets the above conditions as the currently found available memory page.

[0092] Optionally, the free memory pages corresponding to the free data pages in the data table can also be managed in other ways. For example, the computer device can store the page information of each free memory page in a certain order, wherein the page information includes the page identifier and the remaining capacity of the memory page. On this basis, the first thread can start from the first page information or the page information of the available memory page that failed to add a write lock in the most recent storage order, and sequentially search for page information including a remaining capacity that is not less than the amount of data to be written, and use the free memory page identified by the page identifier in the first page information that meets the above conditions as the currently found available memory page.

[0093] After finding the available memory page, the first thread may add a write lock to the currently found available memory page.

[0094] It should be noted that each memory page may correspond to a lock state variable. The lock state variable may include a write lock state bit and a read lock count bit. When the value on the write lock state bit is a first value, it indicates that a certain thread currently holds the write lock of the memory page corresponding to the lock state variable. A write lock may also be referred to as an exclusive lock. When a thread holds the write lock of the memory page, it indicates that the thread is writing data in the memory page. In this case, in order to avoid read-write conflicts, other threads will not be able to add a write lock to the memory page, nor will they be able to add a read lock to the memory page, that is, other threads will not be able to hold the write lock and read lock of the memory page. When the value on the write lock state bit is a second value, it indicates that no thread currently holds the write lock of the memory page. Among them, the first value may be 1, and the second value may be 0. Alternatively, the first value is 0, and the second value may be 1.

[0095] In addition, when the count value indicated by the read lock count bit of the lock state variable is 0, it means that the memory page corresponding to the lock state variable is not currently read locked. When the count value indicated by the read lock count bit is not 0, it means that the memory page corresponding to the lock state variable is currently read locked. At this time, the number of threads holding the read lock of the memory page is equal to the above count value. In the case where one or more threads hold the read lock of the memory page, in order to avoid read-write conflicts, other threads cannot write lock the memory page.

[0096] For example, see Figure 5 and Figure 6 , the lock state variable can be an 8-bit variable, in which the highest bit is the write lock state bit and the remaining bits are the read lock count bits. Figure 5 As shown in the figure, when the write lock status bit is 1, it means that a thread currently holds the write lock of the memory page corresponding to the lock status variable. At this time, the count value of the read lock count bit is 0, that is, no thread holds the read lock of the memory page corresponding to the lock status variable. In this case, other threads will no longer be able to add write locks and read locks to the memory page. Figure 6 As shown in the figure, the write lock status bit is 0, indicating that no thread currently holds the write lock of the memory page corresponding to the lock status variable, and the count value of the read lock count bit is 3, indicating that there are currently 3 threads holding the read lock of the memory page corresponding to the lock status variable. In this case, other threads can continue to add read locks to the memory page, but cannot add write locks to the memory page.

[0097] Based on the above introduction, in an embodiment of the present application, the first thread can determine whether the write lock state bit in the lock state variable corresponding to the currently found available memory page is the second value, and whether the count value indicated by the read lock count bit is 0.

[0098] If the write lock status bit in the lock status variable corresponding to the currently found available memory page is the second value, and the count value indicated by the read lock count bit is 0, it means that currently no thread holds the write lock of the available memory page, and no thread holds the read lock of the available memory page. In this case, the first thread can set the write lock status bit of the lock status variable corresponding to the available memory page to the first value to implement write lock on the available memory page. After the first thread successfully sets the write lock status bit to the first value, it means that the first thread successfully adds a write lock to the available memory page. In this case, it can be considered that the first thread has obtained the available memory page, and at this time, the first thread can write data in the available memory page.

[0099] If the write lock status bit in the lock status variable corresponding to the available memory page currently found is not the second value, or the count value indicated by the read lock count bit is not equal to 0, it means that other threads currently hold the write lock or read lock of the available memory page. In this case, the first thread will not be able to add a write lock to the available memory page. Therefore, the first thread can determine that the write lock for the available memory page has failed.

[0100] After failing to add a write lock to the currently found available memory page, the first thread can count the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is greater than the reference threshold, it means that the number of available memory pages that failed to add a write lock by the first thread is already relatively large, and there may be very few available memory pages with enough space for the first thread to write data in the multiple free memory pages corresponding to the target data table, that is, there are very few free data pages in the target data table that can be used by the first thread to write data. In this case, the first thread can determine that no available memory page has been obtained, thereby triggering the data table expansion process, that is, the first thread can expand N free data pages for itself in the target data table. Among them, the reference threshold can be a preset threshold, and the reference threshold can be set according to the size of N. For example, the reference threshold can be a positive integer not less than N.

[0101] Optionally, if the cumulative number of available memory pages that fail to add a write lock is not greater than a reference threshold, the first thread may continue to search for other available memory pages using the method described above.

[0102] In another possible implementation, the first thread can search for an available memory page in the free memory page corresponding to the target data table; if the write lock for the currently found available memory page fails, wait; when the write lock of the currently found available memory page is released and the first thread successfully adds a write lock to the available memory page again, the first thread can determine whether the remaining capacity of the available memory page is less than the amount of data to be written. If the available memory page is less than the amount of data to be written, and the available memory page is the last available memory page of the first thread in the target data table, the first thread can determine that the available memory page is not obtained in the multiple free memory pages corresponding to the target data table. In this case, the first thread can trigger the extended data table process to expand N free data pages for itself. That is, when the first thread determines that the remaining capacity of the last available memory page after other threads have written data is not enough for itself to write data, it can trigger the extended data table process.

[0103] When expanding the target data table, the first thread may add a write lock to the target data table; if adding the write lock to the target data table is successful, N extended storage areas are applied for the target data table in the persistent storage device; 0 is written into the N extended storage areas to obtain N free data pages; and then, the write lock of the target data table is released.

[0104] Among them, each data table in the database can correspond to a lock state variable. The first thread can refer to the aforementioned method of locking the memory page and set the write lock state bit of the lock state variable corresponding to the target data table to write the target data table. If the write lock is successfully added to the target data table, other threads will not be able to load the data page in the target data table from the hard disk to the memory page for reading and writing, and the data page in the memory page corresponding to the target data table cannot be persistently stored in the hard disk. In this way, it can prevent other threads from accessing the target data table in the process of expanding the data table and causing read-write conflicts.

[0105] If the write lock is successfully added to the target data table, the first thread may apply for N extended storage areas for the data table from the free storage area of ​​the persistent storage device according to the size of the data page in the target data table, wherein the size of each extended storage area is equal to the size of the data page.

[0106] It should be noted that the above-mentioned N is the number of data pages required to be expanded preset for each thread, that is, the N refers to the number of data pages required to be expanded for a thread. In other words, any thread among the multiple threads running by the computer device that triggers the data table expansion process by failing to obtain available memory pages can expand N data pages, and the N will not increase with the increase in the number of threads that want to expand the data table. Among them, N can be set according to the maximum amount of data that the computer device can write in the persistent storage device within a reference time. For example, the reference time can be the operation time required for the computer device to write a data page in the persistent storage device.

[0107] Since random data may be stored in the extended storage area, after applying for N extended storage areas, the computer device can write 0 in each extended storage area, that is, clear the data in each extended storage area. At this time, each extended storage area stores a free data page, and the data in the free data page is all 0, that is, the free data page is an empty data page. The N free data pages stored in the N extended storage areas are the data pages obtained by the first thread for its own expansion.

[0108] For example, assuming that the size of the data page in the target data table is 8KB, the operation time required for the computer device to write an 8KB data page and write 128KB data in the persistent storage device is equal, so N can be set to 16. On this basis, the first thread can apply for 16 8KB extended storage areas from the free storage area of ​​the persistent storage device, and then write 0 in these 16 extended storage areas, thereby obtaining 16 free data pages. Since the operation time required for the computer device to write an 8KB data page and write 128KB data is equal, the time required for the first thread to concentrate on writing 0 in 16 8KB extended storage areas is equal to the time required for writing data in an 8KB storage area. It can be seen that when N is set according to the method described above, the thread applies for a corresponding number of extended storage areas at one time, and then concentrates on writing data in the applied extended storage areas to obtain multiple free data pages, which can reduce the number of input and output IOs and reduce IO latency.

[0109] Optionally, in a possible situation, the first thread may also apply for an extended storage area each time, that is, write 0 into the extended storage area, thereby obtaining a free data page.

[0110] After the first thread expands N free data pages for itself, it can release the write lock of the target data table by modifying the write lock state bit of the lock state variable corresponding to the target data table, so that other threads can access the data table later.

[0111] Step 402: Expand free memory pages for the target data table based on the N free data pages.

[0112] After the first thread expands N free data pages for itself, it can allocate corresponding memory pages for the N free data pages; then, based on the memory pages corresponding to the N free data pages, it updates the page binary tree corresponding to the target data table, and the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

[0113] The first thread may allocate corresponding memory pages for the N data pages from the free storage area of ​​the memory, so that when the free data pages are subsequently accessed, the free data pages may be loaded into the corresponding free memory pages for operation.

[0114] Since the corresponding memory pages allocated to the N free data pages are free memory pages, when the free memory pages of the target data table are managed by a page binary tree, after the first thread allocates corresponding memory pages to the N free data pages, it can also update the information of these free memory pages to the page binary tree, so that other subsequent threads can find these free memory pages through the page binary tree.

[0115] Exemplarily, the first thread may first add a write lock to the page binary tree corresponding to the target data table. If the write lock is successfully added to the page binary tree, a page node corresponding to the memory page corresponding to the N free data pages is added to the page binary tree; then, the write lock of the page binary tree is released.

[0116] The first thread may add a write lock to the page binary tree corresponding to the target data table with reference to the aforementioned method of adding a write lock to the available memory page. This prevents multiple threads from updating the page binary tree of the target data table at the same time and causing conflicts.

[0117] If the first thread successfully adds a write lock to the page binary tree, the first thread can add corresponding page nodes in the page binary tree according to the generation rules of the page binary tree and the page information of the memory page corresponding to each extended free data page. For example, the first thread can add page nodes for indicating the memory page corresponding to the free data page starting from the bottom leaf node in the page binary tree. Among them, each page node can record the page information of the indicated memory page, such as the page identifier and the remaining capacity. Since the memory page corresponding to the free data page at this time is a newly added free memory page that has not yet been written with data, the remaining capacity recorded in the newly added page node for indicating the memory page corresponding to the free data page can be equal to the capacity of the memory page.

[0118] After updating the page binary tree of the target data table, the first thread may release the write lock of the page binary tree.

[0119] Optionally, when managing the page information of the free memory pages corresponding to the target data table by other means, after allocating corresponding memory pages to the extended N free data pages, the first thread can also update the page information of multiple free memory pages corresponding to the managed target data table according to the page information of the memory pages corresponding to the N free data pages. The embodiments of the present application will not be repeated here.

[0120] In summary, in the embodiment of the present application, when the first thread expands the data table, it first locks the target data table so as to expand the data page in the persistent storage device. After the data page expansion is completed, the write lock of the target data table is released, and then the page binary tree is written to update the page binary tree. That is, the embodiment of the present application uses two write locks to control the two sub-processes of page expansion and binary tree update in the data table expansion process. In this way, Figure 7 As shown, when the first thread expands the data page, it does not affect other threads that have completed the data page expansion before (such as Figure 7 When the first thread updates the page binary tree after expanding the data page, it does not affect other subsequent threads (such as Figure 7 In other words, the first thread and other threads can execute different sub-processes in the data table expansion process in parallel, so that the expansion efficiency of the data table can be improved.

[0121] Optionally, in some possible implementations, as can be seen from the above description, the first thread can apply for an extended area each time, that is, write 0 in the extended storage area, thereby obtaining a free data page. Based on this, the first thread can also obtain a free data page in each extension, that is, allocate a corresponding memory page to the free data page, and update the page binary tree based on the memory page corresponding to the free data page. Afterwards, the first thread extends the next free data page.

[0122] Step 403: Write data into the extended free memory page.

[0123] After the target data table is expanded, the first thread may write data into the memory pages corresponding to the N free data pages expanded by the first thread itself.

[0124] Among them, the first thread can randomly determine an idle memory page from the memory pages corresponding to the N idle data pages, and the idle memory page is the available memory page. Afterwards, the first thread can add a write lock to the available memory page with reference to the method described above, and write data to the available memory page after the write lock is successfully added.

[0125] Optionally, the first thread may also continue to search for available memory pages from the next page node of the page node corresponding to the memory page on which the write lock failed most recently in the page binary tree corresponding to the target data table, or may start to search for available memory pages from the root node of the page binary tree. Since a page node for indicating the memory page corresponding to the N extended free data pages has been newly added to the page binary tree, the first thread will find a free memory page of the extension as an available memory page, and then write lock the available memory page using the method described above, and write data to the available memory page after the write lock is successfully added.

[0126] In an embodiment of the present application, when any thread in the computer device fails to obtain an available memory page from the multiple free memory pages corresponding to the target data table, it can expand N free data pages for itself, and expand the free memory pages based on the N free data pages, and then write data in the expanded free memory pages. It can be seen that any thread that fails to obtain an available memory page can trigger the data table expansion process by itself to promptly expand N data pages for itself for its own use. In addition, since there is no need to expand data pages for other threads, the time required for each thread to expand data pages is shorter, which can shorten the thread blocking time compared to a solution in which one thread centrally expands data pages for all threads that have not obtained available memory pages, while other threads are waiting.

[0127] In addition, in the embodiment of the present application, there is no need to wait until the last free memory page is used up before expanding the data table. Instead, when the first thread fails to add a write lock to the found available memory page multiple times and the number of failures exceeds the threshold, the data table expansion process can be directly triggered. That is, the embodiment of the present application can trigger the data table expansion in advance when there are still a small number of free memory pages in the data table. On this basis, the data table expansion will be carried out by different threads in a time-sharing manner, rather than by one thread to expand, and other threads will wait together. In this way, the efficiency of data table expansion can be improved, the thread waiting time can be shortened, and the data writing speed can be increased.

[0128] Figure 8 is a flow chart of another data writing method provided by an embodiment of the present application. Figure 8 , the method comprises the following steps:

[0129] Step 801: Search for available memory pages in a plurality of free memory pages corresponding to a target data table.

[0130] In an embodiment of the present application, the data in the database can be stored in a persistent storage device in the form of a data table, wherein the persistent storage device can be a hard disk included in the computer device itself, or a hard disk in another device other than the computer device.

[0131] In addition, a data table may include multiple data pages, and the size of each data page is used to indicate the amount of data included in the corresponding data page. The sizes of the data pages may be equal, for example, all 8KB. And each data page corresponds to a memory page. The memory page corresponding to the free data page with free space in the data table will also have free space, so it can be called a free memory page.

[0132] In an embodiment of the present application, a page binary tree can be used to manage the free memory pages corresponding to the free data pages of a data table. The page binary tree may include multiple page nodes, each of which is used to indicate a free memory page. Exemplarily, each page node may record a page identifier and remaining capacity of a free memory page.

[0133] Based on this, when the first thread in the computer device wants to write data in the target data table, the first thread can first search for available memory pages in multiple free memory pages corresponding to multiple free data pages in the target data table through the page binary tree corresponding to the target data table.

[0134] Exemplarily, the first thread can search for page nodes whose recorded remaining capacity is not less than the amount of data to be written, starting from the root node of the page binary tree, based on the amount of data to be written, and use the free memory page indicated by the first page node found that meets the above conditions as the currently found available memory page.

[0135] Optionally, the free memory pages corresponding to the free data pages in the data table can also be managed in other ways. For example, the computer device can store the page information of each free memory page in a certain order, wherein the page information includes the page identifier and the remaining capacity of the memory page. On this basis, the first thread can start from the first page information in the storage order, and sequentially search for the page information including the remaining capacity not less than the amount of data to be written, and use the free memory page identified by the page identifier in the first page information found that meets the above conditions as the currently found available memory page.

[0136] Step 802: If adding a write lock to the currently found available memory page fails, then search for other available memory pages in multiple free memories.

[0137] After finding the available memory page, the first thread may add a write lock to the currently found available memory page. The implementation method of adding a write lock to the currently found available memory page by the first thread may refer to the relevant implementation method introduced in the aforementioned step 401, and the embodiment of the present application will not be repeated here.

[0138] If the first thread successfully adds a write lock to the currently found available memory page, the first thread can write data in the available memory page.

[0139] If the first thread fails to add a write lock to the currently found available memory page, the first thread may search again for other available memory pages in multiple free memory pages corresponding to multiple free data pages in the target data table.

[0140] Exemplarily, taking the case of managing free memory pages through a page binary tree as an example, the first thread can start searching for the target page node from the next page node of the first page node in the page binary tree corresponding to the target data table, wherein the first page node is used to indicate the available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; the free memory page indicated by the target page node is used as another available memory page.

[0141] That is to say, the first thread may not start searching again from the root node of the binary tree of pages, but may start from the next page node of the first page node corresponding to the available memory page that failed to add the write lock, and search one by one for page nodes whose remaining capacity is not less than the amount of data to be written, and use the first page node that meets the above conditions as the target page node, and use the free memory page indicated by the target page node as the available memory page found again. At this time, the available memory page found again will be different from the available memory page found last time.

[0142] After re-finding other available memory pages, the first thread can refer to the method described above to add a write lock to the available memory page. If adding the write lock fails, the first thread can again search for other available memory pages from multiple free memory pages until the write lock is successfully added to the found available memory page. The first thread then executes the following step 803.

[0143] For example, Fig. 9 A schematic diagram of searching for available memory pages through a page binary tree corresponding to a target data table is shown, such as Fig. 9 As shown by the dotted arrow in the middle, assuming that the first thread starts from the root node of the page binary tree and finds the page node n2 used to indicate the available memory page P2, after failing to add a write lock to the available memory page P2, the first thread can start from the next node n4 of n2 to search for other available memory pages again. Assuming that the first thread re-finds the page node n8 used to indicate the available memory page P8, and adding a write lock to the available memory page P8 still fails, the first thread can continue to search for other available memory pages from the next node of n8, and so on, until successfully adding a write lock to the found available memory page.

[0144] Optionally, after the first thread fails to add a write lock to the currently found available memory page, it can also count the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is not greater than a reference threshold, then perform the step of searching for other available memory pages from multiple free memory pages. If the cumulative number of available memory pages that failed to add a write lock is greater than the reference threshold, then expand N free data pages for itself in the target data table; then, expand the free memory pages for the target data table based on the N free data pages, and then write data in the expanded free memory pages. Wherein, N is a preset integer greater than 0.

[0145] As can be seen from the above introduction, whenever the first thread finds an available memory page, it can add a write lock to the available memory page. If adding the write lock fails, the first thread can count the cumulative number of available memory pages that failed to add the write lock, and the cumulative number is the total number of times the first thread failed to add the write lock. If the cumulative number is not greater than the reference threshold, it means that the number of available memory pages that failed to add the write lock of the first thread is small, and there may be other available memory pages with sufficient space for the first thread to write data in the multiple free memory pages corresponding to the target data table. In this case, the first thread can continue to search for other available memory pages in the multiple free memory pages.

[0146] If the accumulated number is greater than the reference threshold, it means that the number of available memory pages for which the first thread failed to add a write lock is relatively large, and there may be few available memory pages with enough space for the first thread to write data in the multiple free memory pages corresponding to the target data table, that is, there are few free data pages in the target data table for the first thread to write data. In this case, the first thread can expand the target data table to increase the free data pages.

[0147] Among them, when expanding the target data table, the implementation process of the first thread expanding N free data pages for itself in the target data table can refer to the relevant introduction in the aforementioned step 401. Afterwards, the implementation process of the first thread expanding the free memory pages for the target data table based on the N free data pages can refer to the relevant introduction in the aforementioned embodiment of step 402, which will not be repeated in the embodiment of the present application.

[0148] After the target data table is expanded, the first thread may write data into the expanded free memory pages.

[0149] Among them, the first thread can directly randomly obtain a free memory page from the free memory pages corresponding to the expanded N free data pages, use the free memory page as an available memory page, add a write lock to the available memory page, and execute step 403 after the write lock is successfully added.

[0150] Optionally, the first thread may continue to search for available memory pages from the next page node of the page node corresponding to the memory page that failed to add a write lock the most recently, or may start to search for available memory pages from the root node of the page binary tree. Since a page node for indicating the memory page corresponding to the extended data page has been added to the page binary tree, the first thread will find the memory page corresponding to the extended data page as an available memory page, and then add a write lock to the available memory page using the method described above, until step 403 is executed after the write lock is successfully added.

[0151] Optionally, in some possible situations, after the first thread fails to add a write lock to the currently found available memory page, it may also perform different operations according to different reasons for the failure to add the write lock.

[0152] For example, if the first thread fails to add a write lock because the currently found available memory page has been read-locked, the first thread can decide whether to wait or continue to search for other available memory pages based on the number of threads currently holding the read lock of the available memory page. For example, if the number of threads holding the read lock of the available memory page is not greater than the first threshold, the first thread can wait for all threads holding the read lock of the available memory page to release the read lock before adding a write lock to the available memory page to write data. If the number of threads holding the read lock of the available memory page is greater than the first threshold, the first thread can continue to search for other available memory pages with reference to the method described above.

[0153] If the first thread fails to add a write lock because the currently found available memory page has been write-locked, the first thread can determine whether there are other threads waiting in line to add a write lock to the available memory page. If there are waiting threads, the number of waiting threads is counted. If the number of waiting threads is not greater than the second threshold, the first thread can wait in line for other threads to write data and release the write lock before adding a write lock to the available memory page to write data. If the number of waiting threads is greater than the second threshold, the first thread can continue to search for other available memory pages with reference to the method described above.

[0154] Step 803: Write data into other available memory pages to which write locks are successfully added.

[0155] After the first thread successfully adds a write lock to the other found available memory page, the first thread can write data in the available memory page.

[0156] In an embodiment of the present application, when the first thread in the computer device writes data to the target data table, it can search for available memory pages in multiple free memory pages corresponding to the target data table. If the write lock fails for the currently found available memory page, it means that other threads are accessing the available memory page. In this case, the first thread does not wait for other threads to release the lock of the available memory page, but directly searches for other available memory pages in multiple free memory pages, and writes data in other available memory pages that successfully add the write lock, so that the waiting time of the first thread can be shortened, thereby improving data writing performance. Not only that, in an embodiment of the present application, when the first thread searches for other available memory pages, it directly searches from the next memory page of the available memory page that failed to lock the current page according to the page binary tree, so that the search time is shortened compared to re-searching from the root node of the page binary tree.

[0157] In addition, in the embodiment of the present application, there is no need to wait until the last free memory page is used up before expanding the data table. Instead, when the first thread fails to add a write lock to the found available memory page multiple times and the number of failures exceeds the reference threshold, the data table expansion process can be directly triggered. That is, the embodiment of the present application can trigger the data table expansion in advance when there are still a small number of free memory pages in the data table. On this basis, the data table expansion will be carried out by different threads in a time-sharing manner, rather than by one thread to expand, and other threads will wait together. In this way, the efficiency of data table expansion can be improved, the thread waiting time can be shortened, and the data writing speed can be increased.

[0158] Combined with the above Figure 4 and Figure 8 The data writing method described in the embodiment shown in the figure, the embodiment of the present application also provides a detailed example of the data writing method. Fig.10 As shown, the method comprises the following steps:

[0159] Step 1001: Search for available memory pages starting from the root node of the page binary tree corresponding to the target data table.

[0160] The implementation of this step may refer to the implementation of step 801 in the aforementioned embodiment.

[0161] Step 1002: add a write lock to the currently found available memory page. If adding the write lock succeeds, execute step 1003; if adding the write lock fails, execute step 1004.

[0162] The implementation of this step may refer to the relevant introduction in step 401 in the above embodiment.

[0163] Step 1003: Write data into the available memory page that has been successfully write-locked.

[0164] Step 1004: Count the cumulative number of available memory pages for which the first thread fails to acquire a write lock.

[0165] Step 1005: Determine whether the cumulative number of available memory pages that failed to add write locks is greater than a reference threshold. If not, execute step 1006; if greater than the reference threshold, execute step 1007.

[0166] Step 1006: Start searching for other available memory pages from the next page node of the page node indicating the available memory page that failed to add a write lock in the page binary tree corresponding to the target data table, and then return to execute step 1002.

[0167] Step 1007: N free data pages are expanded for the target data table itself, and free memory pages are expanded for the target data table based on the N free data pages.

[0168] The implementation of this step may refer to the relevant implementations of step 401 and step 402 in the aforementioned embodiment.

[0169] Step 1008: Write data into the extended free memory page.

[0170] The implementation of this step may refer to the relevant implementation of step 403 in the aforementioned embodiment.

[0171] Fig.11 This is a curve comparison chart of data writing performance tested in a certain data writing scenario. In this data writing scenario, the size of the data to be written by each transaction thread is 0.75KB, the amount of data written per second in the same data table is 300MB, and the CPU of the computer device includes 48 physical cores, which can concurrently process up to 96 transaction threads. Based on this, when writing data using the method in the related art, the relationship between the number of concurrent threads and the number of transactions per second (TPS) is as follows: Fig.11 As shown by the dotted line in FIG, when the data writing method provided in the embodiment of the present application is used to write data, the relationship between the number of concurrent threads and TPS is as follows: Fig.11 As shown by the solid line in , TPS is the number of transactions that can be processed per second, where one transaction is used to write a piece of data, so TPS can indicate the amount of data written per second. It can be seen that in the scenario where a large amount of data is concurrently written into a single table, the data writing performance is significantly improved by using the data writing method provided by the embodiment of the present application compared to the data writing method in the related art.

[0172] Next, the data writing device provided in the embodiment of the present application is introduced.

[0173] Fig.12is a schematic diagram of a data writing device 1200 provided in an embodiment of the present application. The device can be deployed in a computer device, for example, in the aforementioned Figure 1 In each server in the storage system shown, or, it can be deployed in Figure 3 In each computing node in the storage system shown in FIG. Fig.12 As shown, the data writing device 1200 may include a first expansion module 1201 , a second expansion module 1202 and a writing module 1203 .

[0174] A first expansion module 1201, configured to execute step 401 in the above embodiment through a first thread;

[0175] A second expansion module 1202, configured to execute step 402 in the above embodiment through a first thread;

[0176] The writing module 1203 is used to execute step 403 in the above embodiment through the first thread.

[0177] Optionally, the first expansion module 1201 is specifically used to: add a write lock to the target data table; if the write lock is successfully added to the target data table, apply for N extended storage areas for the target data table in the persistent storage device; write 0 in the N extended storage areas to obtain N free data pages; and release the write lock of the target data table.

[0178] Optionally, the second expansion module 1202 is specifically used to: allocate corresponding memory pages to N free data pages; based on the memory pages corresponding to the N free data pages, update the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, and each page node is used to indicate a free memory page corresponding to the target data table.

[0179] Optionally, the second expansion module 1202 is specifically used to: add a write lock to the page binary tree corresponding to the target data table; if the write lock is successfully added to the page binary tree corresponding to the target data table, then add page nodes corresponding to the memory pages corresponding to N free data pages in the page binary tree corresponding to the target data table; and release the write lock of the page binary tree corresponding to the target data table.

[0180] Optionally, the device 1200 also includes: a search module, used to search for available memory pages in multiple free memory pages corresponding to the target data table through the first thread; a statistical module, used to count the cumulative number of available memory pages that failed to add a write lock to the currently found available memory page through the first thread if the write lock fails; and a determination module, used to determine that the first thread has not obtained an available memory page in the multiple free memory pages corresponding to the target data table if the cumulative number of available memory pages that failed to add a write lock is greater than a reference threshold.

[0181] In an embodiment of the present application, when any thread in the computer device fails to obtain an available memory page from the multiple free memory pages corresponding to the target data table, it can expand N free data pages for itself, and expand the free memory pages based on the N free data pages, and then write data in the expanded free memory pages. It can be seen that any thread that fails to obtain an available memory page can trigger the data table expansion process by itself to promptly expand N data pages for itself for its own use. In addition, since there is no need to expand data pages for other threads, the time required for each thread to expand data pages is shorter, which can shorten the thread blocking time compared to a solution in which one thread centrally expands data pages for all threads that have not obtained available memory pages, while other threads are waiting.

[0182] Fig.13 is a schematic diagram of a data writing device 1300 provided in an embodiment of the present application. The device can be deployed in a computer device, for example, in the aforementioned Figure 1 In each server in the storage system shown, or, it can be deployed in Figure 3 In each computing node in the storage system shown in FIG. Fig.13 As shown, the data writing device includes a searching module 1301 and a writing module 1302 .

[0183] A search module 1301, configured to execute steps 801 and 802 in the above embodiment through a first thread;

[0184] The writing module 1302 is used to execute step 803 in the above embodiment through the first thread.

[0185] Optionally, the search module 1301 is specifically used to: search for the target page node starting from the next page node of the first page node in the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table, the first page node is used to indicate the available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; the free memory page indicated by the target page node is used as another available memory page.

[0186] Optionally, the search module 1301 is specifically used to: if adding a write lock to the currently found available memory page fails, then count the cumulative number of available memory pages that failed to add a write lock; if the cumulative number of available memory pages that failed to add a write lock is not greater than a reference threshold, then search for other available memory pages from multiple free memory pages.

[0187] Alternatively, see Fig.13The device 1300 may also include: a first expansion module 1303, which is used to expand N free data pages for the first thread in the target data table through the first thread if the cumulative number of available memory pages that fail to add write locks is greater than a reference threshold; a second expansion module 1304, which is used to allocate corresponding memory pages to the N free data pages through the first thread, and update the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages, where the page binary tree corresponding to the target data table includes multiple page nodes, and each page node is used to indicate a free memory page corresponding to the target data table.

[0188] Optionally, the first expansion module 1303 is specifically used to: add a write lock to the target data table; if the write lock is successfully added to the target data table, apply for N extended storage areas for the target data table in the persistent storage device; write 0 in the N extended storage areas to obtain N free data pages; and release the write lock of the target data table.

[0189] Optionally, the second expansion module 1304 is specifically used to: add a write lock to the page binary tree corresponding to the target data table; if the write lock is successfully added to the page binary tree corresponding to the target data table, then add page nodes corresponding to the memory pages corresponding to N free data pages to the page binary tree corresponding to the target data table; and release the write lock of the page binary tree corresponding to the target data table.

[0190] In an embodiment of the present application, the computer device can search for an available memory page in multiple free memory pages corresponding to the target data table through the first thread. If adding a write lock to the currently found available memory page fails, it means that other threads are accessing the available memory page. In this case, the first thread does not wait for other threads to release the lock of the available memory page, but directly searches for other available memory pages in multiple free memory pages, and writes data to other available memory pages that have successfully added a write lock, so that the waiting time of the first thread can be shortened, thereby improving data writing performance.

[0191] It should be noted that the division of modules in the data writing device provided in the above embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0192] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a network device (which may be a router or switch, etc.) or a processor (processor) to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0193] In addition, the data writing device and the data writing method provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0194] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

Claims

1. A data writing method, characterized in that: Applied to a first thread in a computer device, the first thread being any one of a plurality of threads to be written to data running in the computer device, the method comprising: If no available memory page is obtained from the multiple free memory pages corresponding to the target data table, N free data pages are expanded for the first thread in the target data table, where N is a preset integer greater than 0; Expanding free memory pages for the target data table based on the N free data pages; Write data in the extended free memory page.

2. The method according to claim 1, characterized in that The step of extending N idle data pages for the first thread in the target data table includes: Add a write lock to the target data table; If adding a write lock to the target data table is successful, applying for N extended storage areas for the target data table in the persistent storage device; Writing 0 into the N extended storage areas to obtain the N free data pages; Release the write lock of the target data table.

3. The method according to claim 1 or 2, characterized in that: The step of extending the free memory pages for the target data table based on the N free data pages includes: Allocate corresponding memory pages to the N idle data pages; Based on the memory pages corresponding to the N free data pages, the page binary tree corresponding to the target data table is updated, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

4. The method according to claim 3, characterized in that The updating of the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages includes: Add a write lock to the page binary tree corresponding to the target data table; If the write lock is successfully added to the page binary tree corresponding to the target data table, then the page nodes corresponding to the memory pages corresponding to the N free data pages are added to the page binary tree corresponding to the target data table; Release the write lock of the page binary tree corresponding to the target data table.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Searching for an available memory page among a plurality of free memory pages corresponding to the target data table; If adding a write lock to the currently found available memory page fails, the cumulative number of available memory pages that failed to add a write lock is counted; If the accumulated number of available memory pages that failed to add a write lock is greater than a reference threshold, it is determined that no available memory page is acquired from the plurality of free memory pages corresponding to the target data table.

6. A data writing method, characterized in that: Applied to a first thread in a computer device, the first thread being any one of a plurality of threads to be written to data running in the computer device, the method comprising: Search for available memory pages in multiple free memory pages corresponding to the target data table; If adding a write lock to the currently found available memory page fails, searching for other available memory pages in the plurality of free memory pages; Write data to other available memory pages that have successfully added write locks.

7. The method according to claim 6, characterized in that The searching for other available memory pages in the plurality of free memory pages includes: Searching for a target page node from the next page node of the first page node in the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes a plurality of page nodes, each page node is used to indicate a free memory page corresponding to the target data table, the first page node is used to indicate an available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; The free memory page indicated by the target page node is used as another available memory page.

8. The method according to claim 6 or 7, characterized in that: If adding a write lock to the currently found available memory page fails, searching for other available memory pages in the plurality of free memory pages includes: If adding a write lock to the currently found available memory page fails, the cumulative number of available memory pages that failed to add a write lock is counted; If the accumulated number of available memory pages that fail to add a write lock is not greater than a reference threshold, other available memory pages are searched from the plurality of free memory pages.

9. The method according to claim 8, characterized in that The method further comprises: If the accumulated number of available memory pages that failed to add a write lock is greater than the reference threshold, N free data pages are expanded for the first thread in the target data table; Allocate corresponding memory pages to the N idle data pages; Based on the memory pages corresponding to the N free data pages, the page binary tree corresponding to the target data table is updated, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

10. The method according to claim 9, characterized in that The step of extending N idle data pages for the first thread in the target data table includes: Add a write lock to the target data table; If adding a write lock to the target data table is successful, applying for N extended storage areas for the target data table in the persistent storage device; Writing 0 into the N extended storage areas to obtain the N free data pages; Release the write lock of the target data table.

11. The method according to claim 9 or 10, characterized in that: The updating of the page binary tree corresponding to the target data table based on the memory pages corresponding to the N free data pages includes: Add a write lock to the page binary tree corresponding to the target data table; If the write lock is successfully added to the page binary tree corresponding to the target data table, then the page nodes corresponding to the memory pages corresponding to the N free data pages are added to the page binary tree corresponding to the target data table; Release the write lock of the page binary tree corresponding to the target data table.

12. A data writing device, characterized in that: The device comprises: A first expansion module is used for expanding N free data pages for the first thread in the target data table through a first thread if no available memory page is obtained from the multiple free memory pages corresponding to the target data table, wherein N is a preset integer greater than 0, and the first thread is any one of the multiple threads to be written to the data running in the computer device; A second expansion module, configured to expand free memory pages for the target data table based on the N free data pages through the first thread; A writing module is used to write data in the extended free memory page through the first thread.

13. The device according to claim 12, characterized in that The first expansion module is specifically used for: Add a write lock to the target data table; If adding a write lock to the target data table is successful, applying for N extended storage areas for the target data table in the persistent storage device; Writing 0 into the N extended storage areas to obtain the N free data pages; Release the write lock of the target data table.

14. The device according to claim 12 or 13, characterized in that The second expansion module is specifically used for: Allocate corresponding memory pages to the N idle data pages; Based on the memory pages corresponding to the N free data pages, the page binary tree corresponding to the target data table is updated, the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

15. The device according to claim 14, characterized in that The second expansion module is specifically used for: Add a write lock to the page binary tree corresponding to the target data table; If the write lock is successfully added to the page binary tree corresponding to the target data table, then the page nodes corresponding to the memory pages corresponding to the N free data pages are added to the page binary tree corresponding to the target data table; Release the write lock of the page binary tree corresponding to the target data table.

16. The device according to any one of claims 12 to 15, characterized in that The device also includes: A search module, configured to search for an available memory page in a plurality of free memory pages corresponding to the target data table through the first thread; A statistics module, configured to count the cumulative number of available memory pages that failed to add a write lock if adding a write lock to the currently found available memory page fails through the first thread; The determination module is configured to determine that the first thread has not obtained any available memory page from the plurality of free memory pages corresponding to the target data table if the accumulated number of available memory pages that have failed to add a write lock is greater than a reference threshold.

17. A data writing device, characterized in that: The device comprises: A search module, configured to search for an available memory page in a plurality of free memory pages corresponding to a target data table through a first thread; if adding a write lock to a currently found available memory page fails, searching for other available memory pages in the plurality of free memory pages, wherein the first thread is any one of a plurality of threads to be written to data running in a computer device; A writing module is used to write data in other available memory pages to which the write lock is successfully added through the first thread.

18. The device according to claim 17, characterized in that The search module is specifically used for: Searching for a target page node from the next page node of the first page node in the page binary tree corresponding to the target data table, the page binary tree corresponding to the target data table includes a plurality of page nodes, each page node is used to indicate a free memory page corresponding to the target data table, the first page node is used to indicate an available memory page that currently fails to be locked, and the remaining capacity of the free memory page indicated by the target page node is not less than the amount of data to be written; The free memory page indicated by the target page node is used as another available memory page.

19. The device according to claim 17 or 18, characterized in that The search module is specifically used for: If adding a write lock to the currently found available memory page fails, the cumulative number of available memory pages that failed to add a write lock is counted; If the accumulated number of available memory pages that fail to add a write lock is not greater than a reference threshold, other available memory pages are searched from the plurality of free memory pages.

20. The device according to claim 19, characterized in that The device also includes: A first expansion module, configured to expand N free data pages for the first thread in the target data table through the first thread if the accumulated number of available memory pages that fail to add a write lock is greater than the reference threshold; The second expansion module is used to allocate corresponding memory pages to the N free data pages through the first thread; based on the memory pages corresponding to the N free data pages, the page binary tree corresponding to the target data table is updated, and the page binary tree corresponding to the target data table includes multiple page nodes, each page node is used to indicate a free memory page corresponding to the target data table.

21. The device according to claim 20, characterized in that The first expansion module is specifically used for: Add a write lock to the target data table; If adding a write lock to the target data table is successful, applying for N extended storage areas for the target data table in the persistent storage device; Writing 0 into the N extended storage areas to obtain the N free data pages; Release the write lock of the target data table.

22. The device according to claim 20 or 21, characterized in that The second expansion module is specifically used for: Add a write lock to the page binary tree corresponding to the target data table; If the write lock is successfully added to the page binary tree corresponding to the target data table, then the page nodes corresponding to the memory pages corresponding to the N free data pages are added to the page binary tree corresponding to the target data table; Release the write lock of the page binary tree corresponding to the target data table.

23. A computer device, characterized in that: The computer device includes a processor and a memory, the memory stores at least one program instruction or code, and the processor is used to execute the at least one program instruction or code to implement the data writing method according to any one of claims 1 to 5 or claims 6 to 11.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer device, the computer device executes the data writing method according to any one of claims 1 to 5 or claims 6 to 11.

Images