A method for reading data, a method for writing data, a device and a system

Abstract

The embodiment of the present application discloses a method for reading data, a method for writing data, a device and a system for reducing the competition between different threads for the lock resources of the log buffer, so as to improve the performance of the database. In the embodiment of the present application, the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller for controlling the buffer, and the method includes: obtaining a write request for target data; in response to the write request, determining a target page for storing the target data in the storage device; determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes; writing the target data to a page of the target buffer by controlling the target controller of the target buffer; writing the data of the page of the target buffer to the target page in the storage device.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a method for reading data, a method for writing data, a device and a system. Background Art

[0002] In the database, data is often read from or written to the database disk. However, in some scenarios, the data in the disk will be read multiple times, or the same page in the disk will be written multiple times. Therefore, if the disk is interacted with each time to read or write data, a large number of input / output I / O operations will inevitably increase, thereby increasing the I / O overhead and reducing the performance of the database.

[0003] To this end, a buffer is usually used for data reading and writing operations. That is, the data is first read from the disk into the buffer, and then the data only needs to be read from the buffer instead of from the disk. Alternatively, the data is first written to the buffer, and when the data in the buffer reaches a certain amount, the data in the buffer is written to the disk in pages.

[0004] The buffer is generally managed by the controller. When a business thread needs to read data from or write data to the buffer, the controller will lock the buffer. At this time, other business threads cannot access the buffer. Only after the controller releases the lock can other business threads access the buffer.

[0005] This shows that business threads will compete for buffer lock resources, thus affecting database performance. Summary of the invention

[0006] The embodiments of the present application provide a method for reading data, a method for writing data, a device and a system. The method can reduce the competition between different threads for the lock resources of the log buffer, thereby improving the performance of the database.

[0007] In a first aspect, the present application provides a method for writing data, which is applied to a computer device, wherein the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture (NUMA) system, each node having a buffer and a controller for controlling the buffer, and the method includes: obtaining a write request for target data; in response to the write request, determining a target page for storing the target data in the storage device; determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes; writing the target data to a page of the target buffer by controlling a target controller of the target buffer; and writing data of the page of the target buffer to a target page in the storage device.

[0008] Since multiple nodes have buffers and buffer controllers, even if the buffer on one node is locked, it will not affect the writing of target data through the buffers on other nodes. Therefore, the competition for buffer lock resources among different threads can be reduced, thereby improving the performance of the database.

[0009] In some implementations, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data.

[0010] In response to the write request, determining a target page in the storage device for storing the target data includes: determining a target page in the storage device for storing the target data based at least on an identifier of the target data and a storage space required for each piece of data of a preset type.

[0011] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0012] In some implementations, the preset type data indicates a status of a database transaction.

[0013] In some implementations, the identifier of the target data is the serial number of the target data.

[0014] In some implementations, determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes includes: determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes according to the page number of the target page.

[0015] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0016] In some implementations, determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes based on the page number of the target page includes: using a hash algorithm to obtain a hash value corresponding to the page number of the target page; determining the target buffer corresponding to the target page based on the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0017] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0018] In some implementations, writing target data to a page of a target buffer by controlling a target controller of the target buffer includes: determining at least two uninitialized disk pages corresponding to the target buffer by controlling the target controller of the target buffer based on the page number of the target page being greater than a first page number, the at least two disk pages including the target disk page; initializing at least two disk pages in the target buffer by controlling the target controller of the target buffer; wherein the first page number is a maximum page number of initialized pages in the storage device.

[0019] Initializing at least two uninitialized disk pages can merge initialization actions and reduce the number of initialization operations, thereby reducing lock contention and frequent memory operations.

[0020] A second aspect of the present application provides a method for reading data, which is applied to a computer device, wherein the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and the method includes: obtaining a read request for target data; in response to the read request, determining a target page for storing the target data in the storage device; determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes; reading the target data in the target page in the storage device into a page of the target buffer; and reading the target data in the page of the target buffer by controlling the target controller of the target buffer.

[0021] Since multiple nodes have buffers and buffer controllers, even if the buffer on one node is locked, it will not affect the reading of target data through the buffers on other nodes. Therefore, the competition for buffer lock resources among different threads can be reduced, thereby improving the performance of the database.

[0022] In some implementations, a storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data; in response to a read request, determining a target page in the storage device for storing the target data includes: determining a target page in the storage device for storing the target data based at least on the identifier of the target data and the storage space required for each preset type data.

[0023] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0024] In some implementations, the preset type data indicates a status of a database transaction.

[0025] In some implementations, the identifier of the target data is the serial number of the target data.

[0026] In some implementations, determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes includes: determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes according to the page number of the target page.

[0027] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0028] In some implementations, determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes based on the page number of the target page includes: using a hash algorithm to obtain a hash value corresponding to the page number of the target page; determining the target buffer corresponding to the target page based on the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0029] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0030] In a third aspect, the present application provides a device for writing data, which is applied to a computer device, wherein the computer device includes multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller for controlling the buffer, and the buffer in each node is used to read or write data to multiple disk pages in a disk; the device includes:

[0031] An acquisition unit, used for acquiring a write request for target data;

[0032] a determination unit, configured to determine, in response to a write request, a target page in a storage device for storing target data;

[0033] The determination unit is further used to determine a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes;

[0034] A first writing unit, used for writing target data into a page of a target buffer by controlling a target controller of the target buffer;

[0035] The second writing unit is further used to write the data of the page of the target buffer into the target page in the storage device.

[0036] In some implementations, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data.

[0037] The determination unit is used to determine a target page for storing the target data in the storage device based at least on an identifier of the target data and a storage space required for each piece of preset type data.

[0038] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0039] In some implementations, the preset type data indicates a status of a database transaction.

[0040] In some implementations, the identifier of the target data is the serial number of the target data.

[0041] In some implementations, the determination unit is configured to determine, according to a page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0042] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0043] In some implementations, the determination unit is used to use a hash algorithm to obtain a hash value corresponding to the page number of the target page; determine a target buffer corresponding to the target page according to the hash value, and among multiple buffers corresponding to multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0044] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0045] In some implementations, the first write unit is used to determine at least two uninitialized disk pages corresponding to the target buffer by controlling the target controller of the target buffer based on the page number of the target page being greater than the first page number, the at least two disk pages including the target disk page; initialize the at least two disk pages in the target buffer by controlling the target controller of the target buffer; wherein the first page number is the maximum page number of the initialized pages in the storage device.

[0046] Initializing at least two uninitialized disk pages can merge initialization actions and reduce the number of initialization operations, thereby reducing lock contention and frequent memory operations.

[0047] A fourth aspect of the present application provides a device for reading data, which is applied to a computer device, wherein the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and the device includes:

[0048] An acquisition unit, used for acquiring a read request for target data;

[0049] a determination unit, configured to determine, in response to a read request, a target page in a storage device for storing target data;

[0050] The determination unit is further used to determine a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes;

[0051] A first reading unit, configured to read target data in a target page in a storage device into a page in a target buffer;

[0052] The second reading unit is used to read the target data in the page of the target buffer by controlling the target controller of the target buffer.

[0053] In some implementations, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data; and the determination unit is used to determine the target page for storing the target data in the storage device based at least on the identifier of the target data and the storage space required for each preset type data.

[0054] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0055] In some implementations, the preset type data indicates a status of a database transaction.

[0056] In some implementations, the identifier of the target data is the serial number of the target data.

[0057] In some implementations, the determination unit is configured to determine, according to a page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0058] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0059] In some implementations, the determination unit is used to use a hash algorithm to obtain a hash value corresponding to the page number of the target page; determine a target buffer corresponding to the target page according to the hash value, and among multiple buffers corresponding to multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0060] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0061] In a fifth aspect, the present application provides a NUMA system, including: multiple nodes, each node having a buffer and a controller for controlling the buffer, and at least one of the multiple nodes is used to execute the method described in any one of the first aspects above.

[0062] The sixth aspect of the present application provides a computer device, which includes multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller for controlling the buffer, and at least one of the multiple nodes is used to execute a method as described in any one of the above-mentioned first aspects.

[0063] The seventh aspect of the present application provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and at least one processor are interconnected by lines, and the at least one processor is used to run computer programs or instructions to execute the method described in any one of the first aspects above.

[0064] In some implementations, at least one processor is a non-uniform memory access architecture NUMA system comprising multiple nodes, and one or more of the multiple nodes jointly execute the method described in any one of the first aspects above.

[0065] In an eighth aspect, the present application provides a computer storage medium, the computer storage medium is used to store computer software instructions used for the above-mentioned terminal device, including a program designed for executing the computer device;

[0066] The computer device may be the computer device described in the sixth aspect above.

[0067] A ninth aspect of the present application provides a computer program product, which includes computer software instructions, and the computer software instructions can be loaded by a processor to implement the method described in any one of the first aspects above.

[0068] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0069] The embodiment of the present application is applied to a computer device, which includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and the method includes: obtaining a write request for target data; in response to the write request, determining a target page for storing target data in the storage device; determining a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes; writing the target data to a page of the target buffer by controlling a target controller of the target buffer; writing the data of the page of the target buffer to the target page in the storage device; in this way, even if the buffer on one node is locked, it will not affect the reading and writing of the buffers on other nodes. Therefore, the embodiment of the present application can reduce the competition for lock resources of the buffer among different threads, thereby improving the performance of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1This is a schematic diagram of the architecture of the NUMA system in an embodiment of the present application;

[0071] Figure 2 This is a schematic diagram of an embodiment of a node accessing memory in a NUMA system in an embodiment of the present application;

[0072] Figure 3 This is a first working schematic diagram of the buffer zone in an embodiment of the present application;

[0073] Figure 4 This is a schematic diagram of the application of the LRU algorithm in the embodiment of the present application;

[0074] Figure 5 It is a working schematic diagram of a buffer zone in the prior art;

[0075] Figure 6 A schematic diagram of an embodiment of a method for writing data in an embodiment of the present application;

[0076] Figure 7 A schematic diagram of an application example of the method for writing data in an embodiment of the present application;

[0077] Figure 8 A schematic diagram of another application example of the method for writing data in an embodiment of the present application;

[0078] Fig. 9 A schematic diagram of another embodiment of the method for writing data in an embodiment of the present application;

[0079] Fig.10 A schematic diagram showing the relationship between the first page number and the page number of the target disk page;

[0080] Fig.11 A schematic diagram of an embodiment of a method for reading data in an embodiment of the present application;

[0081] Fig.12 A schematic diagram of an embodiment of a device for writing data in an embodiment of the present application;

[0082] Fig.13 This is a schematic diagram of an embodiment of a device for reading data in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The embodiment of the present application provides a method for reading data, a method for writing data, a device and a system. The method is used to reduce the competition between different threads for the lock resources of the log buffer, thereby improving the performance of the database.

[0084] The embodiments of the present application can be applied to a NUMA system. The NUMA system is introduced below.

[0085] like Figure 1As shown, the NUMA system 100 may include one or more slots, and one or more nodes may be installed in each slot. Each node may include multiple central processing units (CPUs) and memories. In each node, the CPU manages the memory through a memory controller (MC).

[0086] exist Figure 1 In the example, the NUMA system 100 includes a socket 101a and a socket 101b. The socket 101a includes two nodes, and the socket 101b also includes two nodes. Each node includes four CPUs. It should be noted that Figure 1 The number of sockets, the number of nodes, and the number of CPUs in each node are merely examples and do not constitute a specific limitation on the NUMA system 100. The NUMA system 100 may also include other numbers of sockets and nodes, and other numbers of CPUs in each node.

[0087] In a NUMA system, any two nodes can be communicatively coupled to each other via an interconnect 104. Figure 2 As shown, taking two nodes as an example, one node can access the memory within the node (also called proximal memory), and one node can also access the memory within another node (also called remote memory).

[0088] It should be understood that logs have an important impact on the performance of database systems. For example, when a database system fails, the database system can restore the database through log playback to ensure the consistency and integrity of the database. Logs are stored on disk. However, if the reading and writing of each log directly interacts with the disk, it will inevitably increase a large number of input / output I / O operations, thereby reducing the performance of the database system. Since the log files on the disk are in pages, and the storage space required for each log is much smaller than a page, in order to reduce the I / O overhead caused by reading and writing logs, a separate area will be allocated in the memory. This area is called the log buffer, which is a piece of memory in the physical sense. In this way, the log buffer can interact with the disk to realize the reading and writing of logs.

[0089] Specifically, Figure 3As shown in the figure, the log buffer includes multiple memory pages, where the memory page has the same size as the disk page in the disk. When the read / write thread needs to write logs to the disk, it can first write the logs to the memory page in the log buffer. When the number of logs in the log buffer reaches a certain number, the logs are batch-written to the disk pages of the disk in units of pages. When the read / write thread needs to read logs from the disk, it can first read the logs from the disk page to the memory page of the log buffer, and then read the logs from the memory page of the log buffer.

[0090] It should be understood that memory resources are limited, so the size of the log buffer is limited. Therefore, in order to achieve reuse of memory pages in the log buffer, a least recently used (LRU) algorithm is usually used to manage the log buffer.

[0091] The management of the log buffer is generally the responsibility of the controller. The controller uses the LRU algorithm to manage the memory pages in the log buffer. Specifically, the controller records the life cycle of the data in each memory page. Figure 4 As shown in the figure, assume that there are four memory pages in the log buffer, and elements A, B, C, and D are written into these four memory pages in turn. Since element D is the most recently stored and element A is the earliest written, the life cycles of elements A, B, C, and D are 0, 1, 2, and 3 respectively. Figure 4 In the example, taking element B as an example, B(1) indicates that the life cycle of element B is 1. The other elements are similar and will not be described here. If element E is to be written into the log buffer at this time, since element A has the shortest life cycle, element E will be written into the memory page where element A is located, that is, element A will be replaced by element E. Similarly, if element F is to be written, element B will be replaced by element F.

[0092] In addition to controlling the reading and writing of logs in the log buffer, the controller is also used to manage the state of the log buffer. The state of the log buffer may include the size of the log buffer, the number of memory pages in the log buffer, the state of the memory pages in the log buffer, the maximum write page, the pointer of the log buffer, and the lock of the log buffer. It can be understood that data can be written to disk pages in the disk through the buffer, and the maximum write page refers to the maximum number of disk pages that have written data through the log buffer. The lock of the log buffer is globally unique and is used to lock the log buffer. For example, if Figure 5 As shown, when read / write thread 1 needs to write a log on a memory page of the log buffer, the buffer controller will lock the log buffer. The locked log buffer can only be operated by the current read / write thread, and read / write thread 2 and read / write thread 3 cannot read or write to the log buffer.

[0093] It can be seen that different threads need to compete for the lock resources of the log buffer, and this competition relationship limits the improvement of the database system performance. To this end, the embodiment of the present application partitions the log buffer, and then allocates the partitioned buffers to each node in the NUMA system. In this way, if the log buffer on a node is locked, it will not affect the reading and writing of the log buffer on other nodes. Therefore, the embodiment of the present application can reduce the competition for the lock resources of the log buffer among different threads, thereby improving the performance of the database.

[0094] Since each node in a NUMA system is usually the same, the log buffer can be evenly distributed to each node in the NUMA system. For example, if the user expects the log buffer size to be 4MB and the NUMA system includes 4 nodes, a 1MB log buffer can be allocated to each node. Accordingly, a buffer controller for controlling the log buffer is also configured on each node. For any node, the CPU of the node can access the log buffer in the near-end memory or the log buffer in the far-end memory.

[0095] It should be noted that the above text describes the method for reading / writing data provided by the embodiment of the present application with a log buffer. In addition, the method for reading / writing data provided by the embodiment of the present application can also be applied to other similar buffers. For example, a piece of memory can be allocated as a buffer for reading / writing data such as a courier number and a product number. The method for writing data and the method for reading data will be introduced below.

[0096] First, the method for writing data provided by the embodiment of the present application is introduced. Figure 6 , a schematic diagram of an embodiment of a method for writing data in an embodiment of the present application. Figure 6 As shown, an embodiment of the present application provides a method for writing data, which is applied to a computer device, wherein the computer device includes multiple nodes of a non-uniform memory access architecture NUMA system, and each node has a buffer and a controller for controlling the buffer.

[0097] The computer device may be a terminal device or a server. The method for writing data provided in the embodiment of the present application is more preferably applied to a server.

[0098] The buffer and the controller that controls the buffer have been described in the previous text, so please refer to the aforementioned related descriptions for understanding.

[0099] The method for writing data provided in the embodiment of the present application includes:

[0100] Step 101: Obtain a write request for target data.

[0101] It should be noted that the write request may not include the target data, but only include information related to the target data. For example, the write request may include the identifier of the target data, and may include the storage space required by the target data.

[0102] There may be multiple types of target data, which are not limited in the embodiments of the present application. For example, in the embodiments of the present application, the target data may indicate the status of a database transaction, or may indicate an express delivery number or a product number.

[0103] Step 102: In response to a write request, determine a target page in a storage device for storing target data.

[0104] The storage device may be any type of non-transitory storage medium including a magnetic disk, a hard disk, a storage array, etc.

[0105] When the storage device is a disk, the data in the disk is composed of multiple disk files, and each disk file is composed of 8KB disk pages.

[0106] After receiving a write request, you need to first determine the target page, which is used to store the target data.

[0107] There are many situations of target pages, which are explained below using a disk as an example, where the target page is identified by a target disk page. The first situation is that if a write request indicates that the target data needs to be written, there is a disk page in the disk, and the disk page stores historical data corresponding to the target data, then in the process of writing the target data, the historical data in the disk page will be replaced with the target data. In this scenario, the target page is a disk page in the disk that stores historical data corresponding to the target data. If a read / write request indicates that the target data needs to be read, it means that the target disk page is also a disk page in the disk that stores historical data corresponding to the target data. The second situation is that if a read / write request indicates that the target data needs to be written, but there is no disk page in the disk that stores historical data corresponding to the target data, in this case, a blank disk page needs to be used as the target disk page.

[0108] It should be noted that there are many methods for determining the target page, which are not limited in the embodiments of the present application.

[0109] Exemplarily, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed.

[0110] The preset type data may indicate the status of a database transaction. The status of a database transaction includes processing, commit, abort, and subtransaction commit. The data indicating the status of a database transaction is also called a transaction commit log. The storage space required for each transaction commit log is 2 bits.

[0111] It should be understood that data is usually stored continuously in a storage device. Specifically, taking a disk as an example, during the writing process of a transaction commit log, the transaction commit log will be written sequentially to the current disk page. If the current disk page is full, the transaction commit log will continue to be written to the next disk page. Therefore, if the disk page is only used to store preset type data, and the storage space required for each preset type of data is fixed, the disk page to which each preset type of data belongs is determined. It can be seen that the page in the storage device to which each preset type of data belongs is also determined.

[0112] Based on this, if the target data belongs to a preset type of data and the write request includes an identifier of the target data, then in response to the write request, determining a target page in the storage device for storing the target data includes:

[0113] A target page for storing the target data in the storage device is determined based at least on the identifier of the target data and the storage space required for each piece of data of a preset type.

[0114] The identifier of the target data may be the number of the target data; when the target data indicates the state of a database transaction, the number of the target data may be the number of the database transaction.

[0115] For example, if the storage device is a disk, the preset type data indicates the status of the database transaction, and the target data number is the database transaction number, then you can first obtain the product of the data transaction number and the storage space required for each preset type of data (here is 2 bits), and then divide the product by the size of the disk page (that is, 8k), then you can get the target disk page number, and thus determine the target disk page.

[0116] Step 103 : determining a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0117] After determining the target page, the page allocator will allocate the target page to a buffer of one of the buffers in the multiple nodes. Specifically, the page allocator needs to select a buffer of one of the buffers in the multiple nodes as the target buffer, and control the target buffer to interact with the storage device through the target controller to read or write the target data on the target page.

[0118] It should be noted that there are multiple methods for determining the target buffer. Exemplarily, a buffer of a node may be randomly selected as the target buffer.

[0119] In addition, corresponding rules may also be preset so that the target buffer is selected according to the rules.

[0120] Exemplarily, the method for determining a target buffer includes: determining, according to a page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0121] As an implementable manner, according to the page number of the target page, determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes includes:

[0122] A hash algorithm is used to obtain a hash value corresponding to the page number of the target page, wherein the hash value can be set according to actual needs.

[0123] The target buffer corresponding to the target page is determined according to the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer. As an implementable method, the buffer having the same encoding and the hash value is the target buffer.

[0124] For example, a hash algorithm may be used to obtain the remainder of dividing the page number of the target disk page by the number of buffers, and then based on the remainder, a target buffer corresponding to the target page is determined from multiple buffers corresponding to multiple nodes.

[0125] For example, Figure 1 Taking the NUMA system shown as an example, the NUMA system includes 4 nodes, and these 4 nodes can be pre-numbered as 0, 1, 2 and 3. When the remainder is 0, the buffer in the node numbered 0 is used as the target buffer; when the remainder is 2, the buffer in the node numbered 2 is used as the target buffer.

[0126] This method can ensure that disk pages are evenly distributed to the buffers in each node as the page numbers increase.

[0127] Exemplarily, determining the target buffer according to the page number of the target disk page may also include: distributing the disk pages in the disk into multiple groups according to the page numbers, each group of disk pages belonging to a buffer in a node; after acquiring the target disk page, first determining the group to which the target disk page belongs, and then determining the target buffer according to the group to which the target disk page belongs.

[0128] For example, suppose all disk pages on the disk are numbered from 1 to 400. Figure 1 As an example, in the NUMA system shown in Figure 1 In the example, the page numbers of a group of disk pages corresponding to the four nodes 102a, 102b, 103a and 103b are 1 to 100, 101 to 200, 201 to 300 and 301 to 400 respectively. If the page number of the target disk page is 150, the buffer in the node 102b can be determined to be the target buffer. If the page number of the target disk page is 301, the buffer in the node 103b can be determined to be the target buffer.

[0129] Step 104 , writing the target data into the page of the target buffer by controlling the target controller of the target buffer.

[0130] It should be noted that writing target data into a page of a target buffer is a relatively mature technology, so it will not be described in detail here.

[0131] Step 105: write the data of the page in the target buffer into the target page in the storage device.

[0132] Likewise, writing the data of a page of the target buffer into a target page in the storage device is a relatively mature technology, so it will not be described in detail here.

[0133] In the embodiment of the present application, multiple nodes are provided with a buffer and a controller for controlling the buffer, so even if the buffer on one node is locked, it will not affect the reading and writing of the buffer on other nodes. Therefore, the embodiment of the present application can reduce the competition for the lock resource of the buffer by different threads, thereby improving the performance of the database.

[0134] For ease of understanding, the following Figure 7 and Figure 8 The method of the embodiment of the present application is further described.

[0135] like Figure 7 As shown, the number of buffers is three. The read / write thread sends a write request for the target data to the page allocator, and the page allocator determines the target page for storing the target data according to the write request. Then, one buffer is selected from the three buffers as the target buffer, and then the target data is written into the target buffer through the controller of the target buffer.

[0136] like Figure 8 As shown, the storage device is a disk, and the number of read / write threads, buffers, and disk files in the disk is three. By using the method provided in the embodiment of the present application, the same read / write thread can write target data through different buffers. Figure 5 , three read-write threads can only write target data through one buffer. The embodiment of the present application can reduce the competition among different threads for the lock resources of the buffer, thereby improving the performance of the database.

[0137] Based on the above embodiments, Fig. 9 As shown, in another embodiment of the method for writing data provided by an embodiment of the present application, writing target data into a page of a target buffer by controlling a target controller of the target buffer includes:

[0138] Step 201 : based on the page number of the target page being greater than the first page number, at least two uninitialized pages in a storage device corresponding to the target buffer are determined by controlling a target controller of the target buffer, the at least two pages including the target page.

[0139] The first page number is the maximum page number of the initialized pages in the storage device. If the page number of the target page is greater than the first page number, it means that the target page has not been initialized.

[0140] Initializing the target page refers to associating a blank memory page in the target buffer with the target page, so that target data can be written into the target page through the blank memory page.

[0141] It should be noted that in some scenarios, the target buffer may correspond to other uninitialized pages, so at least two uninitialized pages corresponding to the target buffer can be determined, and then the at least two uninitialized pages corresponding to the target buffer can be initialized.

[0142] For example, Fig.10 As shown, taking the disk page as an example, the target page is the target disk page. Fig.10 The disk pages in are all disk pages corresponding to the target buffer. Fig.10 It can be seen that there are three uninitialized disk pages between the first page number and the page number of the target disk page.

[0143] Step 202 : Initialize at least two pages in the target buffer by controlling the target controller of the target buffer.

[0144] It should be noted that, in the process of initializing at least two pages, it is first necessary to ensure that the target buffer contains at least two blank memory pages. It should be noted that, in a specific scenario, the target buffer has at least two blank memory pages; in another specific scenario, the target buffer does not have a blank memory page. At this time, the data in some memory pages can be cleared based on the aforementioned LRU algorithm to obtain at least two blank memory pages.

[0145] In the embodiment of the present application, at least two uninitialized pages are initialized, and the initialization actions can be merged to reduce the number of initialization operations, thereby reducing lock contention and frequent memory operations.

[0146] The above describes a method for writing data. The following describes a method for reading data provided in an embodiment of the present application.

[0147] like Fig.11As shown, an embodiment of the present application provides a method for reading data, which is applied to a computer device, wherein the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and the method includes:

[0148] Step 301: Obtain a read request for target data.

[0149] Step 302: In response to a read request, determine a target page in a storage device for storing target data.

[0150] Step 303: determine a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0151] Step 304: read the target data in the target page in the storage device into a page in the target buffer.

[0152] Step 305 , reading target data in a page of the target buffer by controlling a target controller of the target buffer.

[0153] It should be noted that the method for reading data is similar to the method for writing data, the only difference being that, in the process of reading data, the target data in the target page in the storage device needs to be read to the page of the target buffer first, and then the target controller for controlling the target buffer reads the target data in the page of the target buffer. For other information, please refer to the relevant description of the method for writing data in the aforementioned embodiment.

[0154] In another embodiment of the method for reading data provided in the embodiment of the present application, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request contains an identifier of the target data.

[0155] In response to the read request, determining a target page in the storage device for storing target data includes:

[0156] A target page for storing the target data in the storage device is determined based at least on the identifier of the target data and the storage space required for each piece of data of a preset type.

[0157] It should be noted that the process of determining the target page can be understood by referring to the relevant description of the method of writing data in the above embodiment.

[0158] In another embodiment of the method for reading data provided by an embodiment of the present application, the preset type data indicates the status of the database transaction.

[0159] In another embodiment of the method for reading data provided by the embodiment of the present application, the identifier of the target data is the serial number of the target data.

[0160] In another embodiment of the method for reading data provided by an embodiment of the present application, determining a target buffer corresponding to a target page from a plurality of buffers corresponding to a plurality of nodes includes:

[0161] According to the page number of the target page, a target buffer corresponding to the target page is determined from a plurality of buffers corresponding to a plurality of nodes.

[0162] It should be noted that the process of determining the target buffer can be understood by referring to the relevant description of the method of writing data in the above embodiment.

[0163] In another embodiment of the method for reading data provided by the embodiment of the present application, determining, according to the page number of the target page, a target buffer corresponding to the target page from multiple buffers corresponding to multiple nodes includes:

[0164] Use the hash algorithm to obtain the hash value corresponding to the page number of the target page;

[0165] The target buffer corresponding to the target page is determined according to the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0166] It should be noted that the process of determining the target buffer can be understood by referring to the relevant description of the method of writing data in the above embodiment.

[0167] See also Fig.12 The embodiment of the present application provides a device for writing data, which is applied to a computer device, wherein the computer device includes multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller for controlling the buffer, and the buffer in each node is used to read or write data to multiple disk pages in a disk; the device includes:

[0168] An acquisition unit 401 is used to acquire a write request for target data;

[0169] A determination unit 402, configured to determine, in response to a write request, a target page in a storage device for storing target data;

[0170] The determining unit 402 is further configured to determine a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes;

[0171] A first writing unit 403, used to write target data into a page of a target buffer by controlling a target controller of the target buffer;

[0172] The second writing unit 404 is further configured to write the data of the page in the target buffer into the target page in the storage device.

[0173] In some implementations, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data.

[0174] The determination unit 402 is configured to determine a target page in the storage device for storing the target data based at least on an identifier of the target data and a storage space required for each piece of preset type data.

[0175] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0176] In some implementations, the preset type data indicates a status of a database transaction.

[0177] In some implementations, the identifier of the target data is the serial number of the target data.

[0178] In some implementations, the determination unit 402 is configured to determine, according to a page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0179] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0180] In some implementations, the determination unit 402 is used to use a hash algorithm to obtain a hash value corresponding to the page number of the target page; determine a target buffer corresponding to the target page according to the hash value, and among multiple buffers corresponding to multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0181] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0182] In some implementations, the first write unit 403 is used to determine at least two uninitialized disk pages corresponding to the target buffer by controlling the target controller of the target buffer based on the page number of the target page being greater than the first page number, and the at least two disk pages include the target disk page; by controlling the target controller of the target buffer, the at least two disk pages are initialized in the target buffer; wherein the first page number is the maximum page number of the initialized pages in the storage device.

[0183] Initializing at least two uninitialized disk pages can merge initialization actions and reduce the number of initialization operations, thereby reducing lock contention and frequent memory operations.

[0184] See also Fig.13The embodiment of the present application provides a device for reading data, which is applied to a computer device, wherein the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and the device includes:

[0185] An acquisition unit 501 is used to acquire a read request for target data;

[0186] A determination unit 502, configured to determine, in response to a read request, a target page in a storage device for storing target data;

[0187] The determining unit 502 is further configured to determine a target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes;

[0188] A first reading unit 503, configured to read target data in a target page in a storage device into a page in a target buffer;

[0189] The second reading unit 504 is configured to read target data in a page of the target buffer by controlling a target controller of the target buffer.

[0190] In some implementations, the storage device is used to store preset type data, and the storage space required for each preset type data is fixed; the target data belongs to the preset type data, and the write request includes an identifier of the target data; the determination unit 502 is used to determine the target page for storing the target data in the storage device based at least on the identifier of the target data and the storage space required for each preset type data.

[0191] This implementation provides a feasible solution for determining a target page in a scenario where a storage device is used to store preset type data and the storage space required for each preset type data is fixed.

[0192] In some implementations, the preset type data indicates a status of a database transaction.

[0193] In some implementations, the identifier of the target data is the serial number of the target data.

[0194] In some implementations, the determining unit 502 is configured to determine, according to a page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to a plurality of nodes.

[0195] In this implementation, the target buffer is determined by the page number of the target page, thereby providing a feasible solution for determining the target buffer.

[0196] In some implementations, the determination unit 502 is used to use a hash algorithm to obtain a hash value corresponding to the page number of the target page; determine the target buffer corresponding to the target page according to the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer corresponding to the encoding and the hash value is the target buffer.

[0197] The hash algorithm makes the process of determining the target buffer more accurate and efficient.

[0198] It should be noted that, in one embodiment, the device for reading and writing data and the device for writing data are computer programs running on a computer device, and the computer program can be executed by a processor of the computer device to realize its functions.

[0199] An embodiment of the present application also provides a NUMA system, including: multiple nodes, each node having a buffer and a controller for controlling the buffer, at least one of the multiple nodes is used to execute the method for reading data or the method for writing data as described in any one of the aforementioned embodiments.

[0200] An embodiment of the present application also provides a computer device, which includes multiple nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer, and at least one of the multiple nodes is used to execute the method for reading data or the method for writing data as described in any one of the aforementioned embodiments.

[0201] The embodiment of the present application also provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Fig.12 The operation performed by the device for writing data in the embodiment shown in the figure, or the ... Fig.13 The operations performed by the device for reading data in the illustrated embodiment will not be described in detail herein.

[0202] The communication interface in the chip may be an input / output interface, a pin or a circuit.

[0203] In some implementations, at least one processor is a non-uniform memory access architecture NUMA system including multiple nodes, and one or more of the multiple nodes jointly execute the method for reading data or the method for writing data as described in any one of the aforementioned embodiments.

[0204] The embodiment of the present application also provides a computer storage medium, which is used to store computer software instructions used for the above-mentioned control device, including a program designed for executing a computer device.

[0205] The computer device can be as described above Fig.12The device for writing data described above Fig.13 Described is the device for reading data.

[0206] An embodiment of the present application also provides a computer program product, which includes computer software instructions. The computer software instructions can be loaded by a processor to implement the process of the method for reading data or the method for writing data described in any one of the aforementioned embodiments.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

Claims

1. A method for writing data, characterized in that: Applied to a computer device, the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture (NUMA) system, each node has a buffer and a controller for controlling the buffer, the method includes: Get the write request of the target data; In response to the write request, determining a target page in the storage device for storing the target data; Determine a target buffer corresponding to the target page from a plurality of buffers corresponding to the plurality of nodes; Writing the target data into a page of the target buffer by controlling a target controller of the target buffer; Writing data of a page of the target buffer into the target page in the storage device; The storage device is used to store preset type data, and the storage space required for each piece of the preset type data is fixed; The target data belongs to the preset type of data, and the write request includes an identifier of the target data; In response to the write request, determining a target page in the storage device for storing the target data comprises: A target page in the storage device for storing the target data is determined based at least on the identifier of the target data and the storage space required for each piece of the preset type of data.

2. The method according to claim 1, characterized in that The preset type data indicates the status of a database transaction.

3. The method according to claim 1 or 2, characterized in that: The identifier of the target data is the serial number of the target data.

4. The method according to any one of claims 1 to 2, characterized in that: Determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes includes: According to the page number of the target page, a target buffer corresponding to the target page is determined from a plurality of buffers corresponding to the plurality of nodes.

5. The method according to claim 4, characterized in that Determining, according to the page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to the plurality of nodes comprises: Using a hash algorithm to obtain a hash value corresponding to the page number of the target page; A target buffer corresponding to the target page is determined according to the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer whose code corresponds to the hash value is the target buffer.

6. The method according to any one of claims 1 to 2, characterized in that: Writing the target data into a page of the target buffer by controlling the target controller of the target buffer comprises: Based on the page number of the target page being greater than the first page number, determining at least two uninitialized pages in the storage device corresponding to the target buffer by controlling the target controller of the target buffer, the at least two pages including the target page; Initializing the at least two pages in the target buffer by controlling a target controller of the target buffer; The first page number is the maximum page number of initialized pages in the storage device.

7. A method for reading data, characterized in that: Applied to a computer device, the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture (NUMA) system, each node has a buffer and a controller for controlling the buffer, the method includes: Get a read request for the target data; In response to the read request, determining a target page in the storage device for storing the target data; Determine a target buffer corresponding to the target page from a plurality of buffers corresponding to the plurality of nodes; Reading target data in the target page in the storage device to a page in the target buffer; Reading the target data in a page of the target buffer by controlling a target controller of the target buffer; The storage device is used to store preset type data, and the storage space required for each piece of the preset type data is fixed; The target data belongs to the preset type of data, and the read request includes an identifier of the target data; In response to the read request, determining a target page in the storage device for storing the target data comprises: A target page in the storage device for storing the target data is determined based at least on the identifier of the target data and the storage space required for each piece of the preset type of data.

8. The method according to claim 7, characterized in that The preset type data indicates the status of a database transaction.

9. The method according to claim 7 or 8, characterized in that: The identifier of the target data is the serial number of the target data.

10. The method according to any one of claims 7 to 8, characterized in that Determining the target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes includes: According to the page number of the target page, a target buffer corresponding to the target page is determined from a plurality of buffers corresponding to the plurality of nodes.

11. The method according to claim 10, characterized in that Determining, according to the page number of the target page, a target buffer corresponding to the target page from a plurality of buffers corresponding to the plurality of nodes comprises: Using a hash algorithm to obtain a hash value corresponding to the page number of the target page; A target buffer corresponding to the target page is determined according to the hash value, and among the multiple buffers corresponding to the multiple nodes, the buffer whose code corresponds to the hash value is the target buffer.

12. A device for writing data, characterized in that: Applied to a computer device, the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller to control the buffer, and the buffer in each node is used to read or write data to multiple disk pages in a disk; the device includes: An acquisition unit, used for acquiring a write request for target data; a determining unit, configured to determine, in response to the write request, a target page in the storage device for storing the target data; The determining unit is further used to determine a target buffer corresponding to the target page from a plurality of buffers corresponding to the plurality of nodes; A first writing unit, configured to write the target data into a page of the target buffer by controlling a target controller of the target buffer; A second writing unit is further used to write the data of the page of the target buffer into the target page in the storage device; The storage device is used to store preset type data, and the storage space required for each piece of the preset type data is fixed; The target data belongs to the preset type of data, and the write request includes an identifier of the target data; The determining unit is used to determine, in response to the write request, a target page in the storage device for storing the target data, including: A target page in the storage device for storing the target data is determined based at least on the identifier of the target data and the storage space required for each piece of the preset type of data.

13. A device for reading data, characterized in that: Applied to a computer device, the computer device includes a storage device and multiple nodes of a non-uniform memory access architecture NUMA system, each node has a buffer and a controller for controlling the buffer, the device includes: An acquisition unit, used for acquiring a read request for target data; a determining unit, configured to determine, in response to the read request, a target page in the storage device for storing the target data; The determining unit is further configured to determine a target buffer corresponding to the target page from the multiple buffers corresponding to the multiple nodes; A first reading unit, configured to read target data in the target page in the storage device into a page in the target buffer; A second reading unit, configured to read the target data in a page of the target buffer by controlling a target controller of the target buffer; The storage device is used to store preset type data, and the storage space required for each piece of the preset type data is fixed; The target data belongs to the preset type of data, and the read request includes an identifier of the target data; The determining unit is used to determine, in response to the read request, a target page in the storage device for storing the target data, including: A target page in the storage device for storing the target data is determined based at least on the identifier of the target data and the storage space required for each piece of the preset type of data.

14. A NUMA system, characterized in that: include: A plurality of nodes, each node having a buffer and a controller for controlling the buffer; At least one node among the multiple nodes is used to execute the method according to any one of claims 1 to 11.

15. A computer device, characterized in that: The computer device includes a plurality of nodes of a non-uniform memory access architecture NUMA system, each node having a buffer and a controller for controlling the buffer; At least one node among the multiple nodes is used to execute the method according to any one of claims 1 to 11.

16. A computer-readable storage medium storing one or more computer-executable instructions, characterized in that: When the computer-executable instructions are executed by a processor, the processor performs the method as described in any one of claims 1 to 11 above.

Images