Read-write request processing method and device, equipment, storage medium and program product
By identifying the NUMA node corresponding to the target disk in the distributed storage system and utilizing the target resource pool to handle IO read and write requests, the performance degradation caused by cross-NUMA node access is resolved, achieving more efficient read and write request processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DAWNING INFORMATION IND (BEIJING) CO LTD
- Filing Date
- 2025-12-19
- Publication Date
- 2026-04-28
AI Technical Summary
In distributed storage systems, IO read and write requests are slow when processing data across NUMA nodes, resulting in performance degradation.
The NUMA node corresponding to the target disk is determined by a preset correspondence, and read and write requests are processed based on the target resources of the target resource pool, avoiding cross-NUMA node access to the target disk and improving processing performance.
It reduces the overhead of accessing across NUMA nodes and improves the performance of IO read and write request processing.
Smart Images
Figure CN121934764A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a read / write request processing method, apparatus, device, storage medium, and program product. Background Technology
[0002] In a distributed storage system, the CPU (Central Processing Unit), network interface card (NIC), and NVMe (Non-Volatile Memory Express) disks are distributed across different NUMA (Non-Uniform Memory Access) nodes. If the resource utilization of NUMA nodes is not properly planned during the processing of I / O (Input / Output) read / write requests, these requests may need to process data across NUMA nodes. Since processing data across NUMA nodes is slower than processing data within a single NUMA node, this can lead to a decrease in I / O read / write request performance.
[0003] Therefore, improving the processing performance of IO read and write requests has become an urgent technical problem to be solved. Summary of the Invention
[0004] Therefore, it is necessary to provide a read / write request processing method, apparatus, device, storage medium, and program product that can improve the performance of read / write request processing in response to the above-mentioned technical problems.
[0005] Firstly, this application provides a read / write request processing method, which includes:
[0006] Receive read / write requests sent by the client; the read / write requests carry the identification information of the target disk to be accessed;
[0007] Based on the identification information and the preset correspondence, the target resource pool for processing read and write requests is determined. The correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk.
[0008] Read and write requests are processed based on the target resources in the target resource pool.
[0009] In the above embodiment, firstly, a read / write request carrying the identification information of the target disk to be accessed is received from the client. Then, based on the identification information and a preset correspondence, the target resource pool for processing the read / write request is determined. This correspondence is the relationship between each disk in the storage system and its corresponding Non-Unified Memory Access Architecture (NUMA) node. Finally, the read / write request is processed based on the target resources in the target resource pool. In this way, when processing the client's read / write request, the preset correspondence determines the NUMA node corresponding to the target disk, and the read / write request is processed based on the target resources of the target resource pool of the NUMA node. This avoids the additional overhead of accessing the target disk across NUMA nodes and improves the processing performance of read / write requests.
[0010] In one embodiment, the target resource pool for processing read / write requests is determined based on the identification information and a preset correspondence, including:
[0011] Based on the identification information and the corresponding relationship, determine the target NUMA node corresponding to the target disk;
[0012] The resource pool corresponding to the target NUMA node is identified as the target resource pool.
[0013] In the above embodiments, the target NUMA node corresponding to the target disk is first determined, and then the target resource pool is determined based on the resource pool corresponding to the target NUMA node to process the read and write requests, thereby avoiding access to the target disk across NUMA nodes, reducing the overhead of accessing the disk across NUMA nodes, and achieving efficient scheduling.
[0014] In one embodiment, the target resources include target page resources and target processing resources. Processing read / write requests based on target resources in the target resource pool includes:
[0015] Utilize the target page resources to store the data information corresponding to read and write requests;
[0016] The target processing resource stores the control information corresponding to read and write requests.
[0017] In the above embodiments, read and write requests are processed through target page resources and target processing resources in the target resources to avoid accessing the target disk across NUMA nodes.
[0018] In one embodiment, the method further includes:
[0019] For each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node;
[0020] Based on the allocable resources, construct the resource pool corresponding to the NUMA node.
[0021] In the above embodiments, a resource pool corresponding to each NUMA node can be constructed based on the allocable resources corresponding to each NUAM node. The read and write requests of the disks corresponding to each NUMA node are processed according to the resource pool, and it is no longer necessary to access the disks across NUMA nodes.
[0022] In one embodiment, a resource pool corresponding to a NUMA node is constructed based on allocable resources, including:
[0023] Based on allocable resources, construct page resources in the resource pool;
[0024] Based on the allocatable resources and / or the allocatable resources corresponding to other NUMA nodes, construct the processing resources in the resource pool.
[0025] In the above embodiments, page resources, due to their large data volume, need to consider the overhead of accessing across NUMA nodes and are constructed using the allocatable resources of the target NUMA node. Processing resources, due to their small data volume, do not consider the overhead of accessing across NUMA nodes and can be allocated using the allocatable resources of any NUMA node. Therefore, the construction method of the resource pool is more flexible.
[0026] In one embodiment, determining the allocatable resources corresponding to the NUMA node includes:
[0027] The allocable resources corresponding to a NUMA node are determined based on the resource allocation weight of that NUMA node.
[0028] In the above embodiments, the allocable resources are determined according to the resource allocation weight corresponding to each NUAM node, thereby realizing the determination of allocable resources based on the actual usage of each NUAM node and avoiding the allocation of the resource pool exceeding the space size of the NUAM node.
[0029] In one embodiment, processing read / write requests based on the target resource includes:
[0030] Determine the target I / O thread corresponding to the target resource pool;
[0031] Read and write requests are handled based on the target resource pool and the target IO thread.
[0032] In this embodiment, the resource pool and the data I / O thread form a one-to-one correspondence, avoiding competition for multi-threaded resources.
[0033] In one embodiment, the method further includes:
[0034] If the storage system detects the addition of a first disk, the corresponding relationship is updated based on the first disk and the NUMA node corresponding to the first disk.
[0035] Alternatively, if the storage system detects that the second disk has been deleted, the corresponding relationship is updated based on the second disk and the NUMA node corresponding to the second disk.
[0036] In this embodiment, by dynamically updating the correspondence, the accuracy of determining the target resource pool for processing read and write requests is ensured, the overhead of accessing the disk across NUMA nodes is reduced, and efficient scheduling is achieved.
[0037] Secondly, this application also provides a read / write request processing apparatus, which includes:
[0038] The receiving module is used to receive read and write requests sent by the client; the read and write requests carry the identification information of the target disk to be accessed;
[0039] The determination module is used to determine the target resource pool for processing read and write requests based on the identification information and the preset correspondence. The correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk.
[0040] The processing module is used to process read and write requests based on the target resources in the target resource pool.
[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the first aspect above.
[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps: the steps of any of the methods described in the first aspect above.
[0043] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: the steps of any of the methods described in the first aspect above.
[0044] The aforementioned read / write request processing method, apparatus, device, storage medium, and program product first receive a read / write request sent by a client, carrying identification information of the target disk to be accessed. Then, based on the identification information and a preset correspondence, a target resource pool for processing the read / write request is determined. This correspondence refers to the mapping between each disk in the storage system and its corresponding Non-Unified Memory Access Architecture (NUMA) node. Finally, the read / write request is processed based on the target resources in the target resource pool. In this way, when processing a client's read / write request, the preset correspondence determines the NUMA node corresponding to the target disk, and the read / write request is processed based on the target resources of the NUMA node's target resource pool. This avoids the additional overhead of accessing the target disk across NUMA nodes and improves the processing performance of read / write requests. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 This is an application environment diagram of a read / write request processing method in one embodiment;
[0047] Figure 2 This is a flowchart illustrating a read / write request processing method in one embodiment;
[0048] Figure 3 This is a flowchart illustrating the steps for determining the target resource pool in one embodiment;
[0049] Figure 4 This is a flowchart illustrating the steps of the target resource pool in processing read / write requests in another embodiment;
[0050] Figure 5 This is a flowchart illustrating the steps involved in building a resource pool for a NUMA node in one embodiment.
[0051] Figure 6 This is a flowchart illustrating the resource pool construction steps in one embodiment;
[0052] Figure 7 This is a flowchart illustrating the read / write request processing method in another embodiment;
[0053] Figure 8 This is a flowchart illustrating the read / write request processing method in another embodiment;
[0054] Figure 9This is a schematic diagram of the storage system architecture in another embodiment;
[0055] Figure 10 This is a structural block diagram of a read / write request processing device in one embodiment;
[0056] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0058] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0059] NUMA nodes are the basic building blocks in a non-uniform memory access architecture. Each NUMA node includes computing cores, memory, and I / O resources. Disks are connected to the CPUs of different NUMA nodes via the PCIe bus. For example, disk 0 can be directly connected to the CPU of NUMA node 0 via the PCIe bus, and disk 1 can be directly connected to the CPU of NUMA node 1 via the PCIe bus. This physical connection method may have affinity issues. If the utilization of NUMA node resources is not planned properly during the processing of I / O read / write requests, I / O read / write requests may need to process data across NUMA nodes. Since processing data across NUMA nodes is slower than processing data within a single NUMA node, this may lead to a decrease in I / O read / write request performance.
[0060] For example, when a process running on NUMA node 0 needs to access data on disk 0, the data flow is completed directly within NUMA node 0, without needing to cross NUMA nodes. This results in the shortest request processing path, lowest latency, and best performance. However, if a process running on NUMA node 1 needs to access data on disk 0, the data flow will be: NUMA node 1 CPU -> node interconnect link -> NUMA node 0 CPU -> NUMA node 0 PCIe -> disk 0. This request processing path requires crossing NUMA nodes, from NUMA node 1 to NUMA node 0, thus resulting in higher latency and lower bandwidth. Therefore, improving the processing performance of I / O read / write requests has become a pressing technical problem.
[0061] In view of this, this application proposes a read / write request processing method. When processing read / write requests from clients, the method determines the NUMA node corresponding to the target disk through a preset correspondence, and processes the read / write requests according to the target resources of the target resource pool of the NUMA node. This avoids the additional overhead caused by accessing the target disk across NUMA nodes and improves the processing performance of read / write requests.
[0062] The read / write request processing method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, client 102 communicates with storage system 104 via a network. Client 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted displays, etc. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Storage system 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0063] In one exemplary embodiment, such as Figure 2 As shown, a read / write request processing method is provided, which is applied to... Figure 1 Taking the storage system in the example, the explanation includes the following steps 201 to 203. Wherein:
[0064] Step 201: Receive read / write requests sent by the client.
[0065] The read / write request carries the identification information of the target disk to be accessed; the target disk is the disk that the read / write request intends to access. The read / write request may also include the operation type and operation data for accessing the target disk. The operation type includes a read operation or a write operation, and the operation data includes the data to be written to the target disk or the location of the target disk to be read.
[0066] Optionally, upon receiving a read / write request from the client, a security check can be performed on the read / write request, such as verifying whether the client is an authorized client and whether the format of the read / write request conforms to the protocol requirements. If the read / write request passes the security check, the next step of processing can be continued; if the read / write request fails the security check, the client's access can be prohibited.
[0067] Step 202: Determine the target resource pool for processing read and write requests based on the identification information and the preset correspondence.
[0068] The mapping relationship refers to the correspondence between each disk in the storage system and its corresponding Non-Unified Memory Access Architecture (NUMA) node. This mapping relationship can be determined based on the physical architecture of the storage system. It is generated and saved by analyzing the physical slot location of each disk and the corresponding NUMA node number. The NUMA node corresponding to each disk is also the NUMA node to which the disk is connected via the PCIe bus. Therefore, the NUMA node to which the disk is connected can be determined based on the disk's identification information and the mapping relationship.
[0069] For example, the correspondence can be dynamically updated based on changes in the storage system's topology. If a first disk is added to the storage system, the correspondence is updated based on the first disk and its corresponding NUMA node; or, if a second disk is deleted from the storage system, the correspondence is updated based on the second disk and its corresponding NUMA node.
[0070] For example, disks on the storage system can be scanned periodically, and the results compared with the previous scan to identify disks that were not present in the previous scan—these are the newly added first disks. Alternatively, disks that existed in the previous scan but are not present in the current scan are the deleted second disks. Based on the configuration attributes of the first and / or second disks, the corresponding NUMA nodes for the first and / or second disks are determined, and the first disk and its corresponding NUMA node are added to the mapping relationship, while the corresponding NUMA nodes for the second disk and the second disk are deleted from the mapping relationship.
[0071] Alternatively, changes to disks on the storage system can be determined through events. For example, an event indicating the addition of a disk can confirm the addition of a first disk, and an event indicating the deletion of a second disk can confirm the deletion of a second disk. The corresponding relationships are then updated. This dynamic updating of the relationships ensures the accuracy of identifying the target resource pool for handling read and write requests, reduces the overhead of accessing disks across NUMA nodes, and achieves efficient scheduling.
[0072] As described above, in a NUMA architecture, disks are connected to NUMA nodes via a PCIe bus. When accessing a target disk, read / write requests are processed using the resources of the NUMA node connected to the target disk. This reduces the need to access the target disk across NUMA nodes, thus minimizing the overhead associated with such access. Therefore, the target NUMA node connected to the target disk can be determined based on the target disk's identification information and its corresponding relationship. The target resource pool is the resource pool provided by the target NUMA node. Accessing the target disk through the target resource pool of the target NUMA node achieves intra-node access to the target disk, minimizing the path for read / write request processing. Optionally, one resource pool can handle one read / write request.
[0073] Step 203: Process read and write requests based on the target resources in the target resource pool.
[0074] Optionally, the target resource pool includes target resources, which may include CPU resources, memory resources, and I / O resources. Read and write requests are processed through these target resources. Taking memory resources as an example, when a client's read request is for data, the data read from the disk is first cached in the memory resource and then returned to the client. If the read / write request is for data, the data to be written can be first written to the memory resource, and then the data in the memory resource is written to the disk, completing the read / write request processing. Target resources enable caching, scheduling, and control of read and write requests.
[0075] In the above embodiment, firstly, a read / write request carrying the identification information of the target disk to be accessed is received from the client. Then, based on the identification information and a preset correspondence, the target resource pool for processing the read / write request is determined. This correspondence is the relationship between each disk in the storage system and its corresponding Non-Unified Memory Access Architecture (NUMA) node. Finally, the read / write request is processed based on the target resources in the target resource pool. In this way, when processing the client's read / write request, the preset correspondence determines the NUMA node corresponding to the target disk, and the read / write request is processed based on the target resources of the target resource pool of the NUMA node. This avoids the additional overhead of accessing the target disk across NUMA nodes and improves the processing performance of read / write requests.
[0076] In the embodiments of this application, the target resource pool for processing read and write requests is determined based on the identification information and the preset correspondence, such as... Figure 3 As shown, it includes:
[0077] Step 301: Determine the target NUMA node corresponding to the target disk based on the identification information and the corresponding relationship.
[0078] The identification information can be the serial number of the target disk, and the identification information of the target disk is unique. A query is performed in the corresponding relationship based on the identification information of the target disk, and the target NUMA node corresponding to the target disk is determined based on the query result; that is, the NUMA node that the target disk directly connects to via PCIe.
[0079] Step 302: Determine the resource pool corresponding to the target NUMA node as the target resource pool.
[0080] The resource pool corresponding to the target NUMA node is a set of resources including all physical CPU cores, memory resources, and related I / O device bandwidth under the target NUMA node. This resource pool is obtained by logically partitioning the resource set of the target NUMA node. The resource pool corresponding to the target NUMA node provides the memory access capability of this NUMA node and also includes associated cache hierarchy structures, processor execution units, and other hardware resources. This ensures that read and write requests are preferentially utilized by the resources of this node, reducing latency caused by cross-node memory access, thereby improving the overall performance and resource utilization of the storage system. Optionally, the target NUMA node can be divided into multiple resource pools or into a single resource pool. When the target NUMA node is divided into multiple resource pools, an unused resource pool can be selected as the target resource pool. When the target NUMA node is divided into a single resource pool, if that resource pool is unused, it is used as the target resource pool to handle read and write requests to the target disk.
[0081] In this embodiment, the target NUMA node corresponding to the target disk is first determined, and then the target resource pool is determined based on the resource pool corresponding to the target NUMA node to process the read and write requests, thereby avoiding cross-NUMA node access to the target disk, reducing the overhead of cross-NUMA node disk access, and achieving efficient scheduling.
[0082] In one embodiment, the target resources include target page resources and target processing resources. Read and write requests are processed based on the target resources in the target resource pool, such as... Figure 4 As shown, it includes:
[0083] Step 401: Utilize the target page resource storage to store the data information corresponding to the read / write request.
[0084] The target page resource is a memory page allocated based on the memory of the NUMA node. When processing read and write requests, the target page resource is used to store the data information corresponding to the read and write requests, such as the data being read or the data being written.
[0085] Step 402: Use the target processing resource to store the control information corresponding to the read / write request.
[0086] The control information may include address information, scheduling information, and context information during the processing of read and write requests. The target processing resource may be a data transfer resource. By recording the control information during the processing of read and write requests through the target processing resource, the normal processing of read and write requests can be ensured.
[0087] In the above embodiments, read and write requests are processed through target page resources and target processing resources in the target resources to avoid accessing the target disk across NUMA nodes.
[0088] In one embodiment, such as Figure 5 As shown, the method also includes:
[0089] Step 501: For each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node.
[0090] The allocatable resources corresponding to a NUMA node can include CPU resources, memory resources, and I / O resources. Taking the memory resources of a NUMA node as an example, the allocatable resources can be the available memory resources of the NUMA node. Optionally, the allocatable resources of a NUMA node can be determined according to its resource allocation weight. The resource allocation weight can be the memory allocation ratio, determined by multiplying the memory resources of the NUMA node by its memory allocation ratio. The resource allocation weight of each NUMA node can be different, determined based on the resource situation of each NUMA node.
[0091] Optionally, the available resources of a NUMA node can be determined by monitoring its resource usage. By obtaining the resource usage of NUMA nodes and combining it with the overall load requirements of the storage system, a reasonable resource allocation weight can be set for each NUMA node. Meanwhile, to reserve some buffer space, the memory usage of a single NUMA node can be limited to no more than 80% of its total capacity.
[0092] In the above embodiments, the allocable resources are determined according to the resource allocation weight corresponding to each NUAM node, thereby realizing the determination of allocable resources based on the actual usage of each NUAM node and avoiding the allocation of the resource pool exceeding the space size of the NUAM node.
[0093] Step 502: Based on the allocable resources, construct the resource pool corresponding to the NUMA node.
[0094] The resource pool corresponding to a NUMA node can include page resources and processing resources. Optionally, such as... Figure 6 As shown, the process of building a resource pool includes:
[0095] Step 601: Based on the allocable resources, construct the page resources in the resource pool.
[0096] After determining the allocatable resources of the NMUA node, these resources can be divided into multiple page resources according to a preset page space size, resulting in multiple page resource channels. These page resource channels can be used to cache data information for read and write requests.
[0097] Step 602: Based on the allocable resources and / or the allocable resources corresponding to other NUMA nodes, construct the processing resources in the resource pool.
[0098] Since the control information for read / write requests in the resource processing records is relatively small, there's no need to consider the resource overhead of cross-node access. Therefore, the processing resources can be obtained from the allocable resources of the target NUMA node, or from the allocable resources of other NUMA nodes. Multiple processing resources are obtained based on the allocable resources of the NUMA nodes, resulting in multiple processing resource channels. Optionally, a page resource channel and a processing resource channel can form a resource pool for processing a single read / write request. Based on the allocable resources of the target NUMA node, N resource pools can be constructed, each containing N page resource channels and N processing resource channels, where N is a positive integer.
[0099] In the above embodiments, resource pools corresponding to each NUMA node can be constructed based on the allocatable resources corresponding to each NUAM node. Disk read / write requests for each NUMA node are then processed using these resource pools, eliminating the need for cross-NUMA node disk access. Furthermore, page resources, due to their large data volume, require consideration of the overhead of cross-NUAM node access and are constructed using the allocatable resources of the target NUMA node. Processing resources, due to their small data volume, do not require consideration of cross-NUMA node access overhead and can be allocated using the allocatable resources of any NUMA node. Therefore, the resource pool construction method is more flexible.
[0100] In one embodiment of this application, read / write requests are processed based on the target resource, such as... Figure 7 As shown, it includes:
[0101] Step 701: Determine the target IO thread corresponding to the target resource pool.
[0102] To avoid resource contention among multiple threads during read / write request processing, multiple data I / O threads are created, with a one-to-one correspondence between each resource pool and each data I / O thread. Each data I / O thread can only access resources within its own resource pool, reducing thread lock contention. Optionally, during the initialization phase, a thread mapping table is generated and saved based on the resource pool's identifier and the data I / O thread's identifier. After determining the target resource pool for the read / write request, the thread mapping table is queried based on the target resource pool's identifier to determine the target I / O thread corresponding to that resource pool.
[0103] Step 702: Process read and write requests based on the target resource pool and the target IO thread.
[0104] Based on the target resource pool and the corresponding target IO thread, read and write requests are processed. In this way, a read or write request is processed through the target IO thread and the resources in the target resource pool. The target IO thread corresponds to its own page resources and processing resources, avoiding contention between thread locks.
[0105] In this embodiment, the resource pool and the data I / O thread form a one-to-one correspondence, avoiding competition for multi-threaded resources.
[0106] In the embodiments of this application, such as Figure 8 As shown, a read / write request processing method is provided, including:
[0107] Step 801: For each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node.
[0108] Step 802: Based on the allocable resources, construct the resource pool corresponding to the NUMA node.
[0109] Step 803: Receive read / write requests sent by the client.
[0110] Step 804: Determine the target NUMA node corresponding to the target disk based on the identification information and the corresponding relationship.
[0111] Step 805: Determine the resource pool corresponding to the target NUMA node as the target resource pool.
[0112] Step 806: Determine the target IO thread corresponding to the target resource pool.
[0113] Step 807: Utilize the target IO thread and target page resource storage to store the data information corresponding to the read / write requests.
[0114] Step 808: Utilize the target IO thread and target processing resource to store the control information corresponding to the read / write requests.
[0115] In one embodiment, such as Figure 9The diagram shows the architecture of the storage system to which the read / write request processing method in this embodiment is applied. The storage system includes a management module, a network processing module, a resource pool management module, a read / write disk management module, and a daemon process. Each NUMA node may include at least one disk. A preset mapping relationship can be determined by the management module based on the storage system architecture and sent to the client. When the client sends a read / write request to the data service module of the storage system through the network processing module, it carries the identification information of the target disk and the preset mapping relationship. The data service module determines the target resource pool based on the identification information carried in the read / write request and the preset mapping relationship. One read / write request corresponds to one target resource pool. The read / write request process is processed through the target resource pool and the target IO thread. Based on the data in the read / write request, the data of the target disk within the target NUMA node is read and written through the page resources in the target resource pool.
[0116] The daemon process can be a monitoring process. During the initialization phase, it specifies the resource allocation weight of each NUMA node by input parameters. Based on the resource allocation weight, it determines the allocatable resources of each NUMA node, thereby dividing each NUMA node into a resource pool that includes page resources and processing resources.
[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0118] Based on the same inventive concept, this application also provides a read / write request processing apparatus for implementing the read / write request processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more read / write request processing apparatus embodiments provided below can be found in the limitations of the read / write request processing method described above, and will not be repeated here.
[0119] In one exemplary embodiment, such as Figure 10As shown, a read / write request processing device 1000 is provided, including: a receiving module 1001, a determining module 1002, and a processing module 1003, wherein:
[0120] The receiving module 1001 is used to receive read / write requests sent by the client; the read / write request carries the identification information of the target disk to be accessed;
[0121] The determination module 1002 is used to determine the target resource pool for processing read and write requests based on the identification information and the preset correspondence. The correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk.
[0122] Processing module 1003 is used to process read and write requests based on target resources in the target resource pool.
[0123] In one embodiment, the determining module 1002 is specifically used to determine the target NUMA node corresponding to the target disk based on the identification information and the correspondence; and to determine the resource pool corresponding to the target NUMA node as the target resource pool.
[0124] In one embodiment, the target resource includes a target page resource and a target processing resource. The processing module 1003 is specifically used to store data information corresponding to read / write requests using the target page resource and to store control information corresponding to read / write requests using the target processing resource.
[0125] In one embodiment, the apparatus further includes a construction module for determining the allocatable resources corresponding to each NUMA node in the storage system; and constructing a resource pool corresponding to the NUMA node based on the allocatable resources.
[0126] In one embodiment, the building module is specifically used to build page resources in the resource pool based on allocable resources; and to build processing resources in the resource pool based on allocable resources and / or allocable resources corresponding to other NUMA nodes.
[0127] In one embodiment, the construction module is specifically used to determine the allocatable resources corresponding to the NUMA node based on the resource allocation weight corresponding to the NUMA node.
[0128] In one embodiment, the processing module 1003 is specifically used to determine the target IO thread corresponding to the target resource pool; and to process read and write requests based on the target resource pool and the target IO thread.
[0129] In one embodiment, the device further includes an update module, configured to update the correspondence between the first disk and the corresponding NUMA node if the storage system detects the addition of a first disk; or, if the storage system detects the deletion of a second disk, update the correspondence between the second disk and the corresponding NUMA node.
[0130] Each module in the aforementioned read / write request processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.
[0131] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores read / write request data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a read / write request processing method.
[0132] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0133] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: receiving a read / write request sent by a client; the read / write request carries identification information of the target disk to be accessed; determining a target resource pool for processing the read / write request based on the identification information and a preset correspondence, wherein the correspondence is the correspondence between each disk in the storage system and the corresponding Non-Unified Memory Access Architecture (NUMA) node; and processing the read / write request based on the target resources in the target resource pool.
[0134] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the target NUMA node corresponding to the target disk based on the identification information and the correspondence; and determining the resource pool corresponding to the target NUMA node as the target resource pool.
[0135] In one embodiment, the target resource includes a target page resource and a target processing resource. When the processor executes the computer program, it further performs the following steps: using the target page resource to store data information corresponding to the read / write request; and using the target processing resource to store control information corresponding to the read / write request.
[0136] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node; and construct a resource pool corresponding to the NUMA node based on the allocatable resources.
[0137] In one embodiment, when the processor executes the computer program, it further performs the following steps: constructing page resources in the resource pool based on the allocable resources; and constructing processing resources in the resource pool based on the allocable resources and / or the allocable resources corresponding to other NUMA nodes.
[0138] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the allocatable resources corresponding to the NUMA node based on the resource allocation weight corresponding to the NUMA node.
[0139] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the target I / O thread corresponding to the target resource pool; and processing read / write requests based on the target resource pool and the target I / O thread.
[0140] In one embodiment, when the processor executes the computer program, it further performs the following steps: if a first disk is detected to be added to the storage system, the correspondence is updated according to the first disk and the NUMA node corresponding to the first disk; or, if a second disk is detected to be deleted from the storage system, the correspondence is updated according to the second disk and the NUMA node corresponding to the second disk.
[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: receiving a read / write request sent by a client; the read / write request carries identification information of the target disk to be accessed; determining the target resource pool for processing the read / write request based on the identification information and a preset correspondence, wherein the correspondence is the correspondence between each disk in the storage system and the corresponding Non-Unified Memory Access Architecture (NUMA) node; and processing the read / write request based on the target resources in the target resource pool.
[0142] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target NUMA node corresponding to the target disk based on the identification information and the correspondence; and determining the resource pool corresponding to the target NUMA node as the target resource pool.
[0143] In one embodiment, the target resource includes a target page resource and a target processing resource. When the computer program is executed by the processor, it further performs the following steps: using the target page resource to store data information corresponding to the read / write request; and using the target processing resource to store control information corresponding to the read / write request.
[0144] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node; and construct a resource pool corresponding to the NUMA node based on the allocatable resources.
[0145] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing page resources in the resource pool based on the allocable resources; and constructing processing resources in the resource pool based on the allocable resources and / or the allocable resources corresponding to other NUMA nodes.
[0146] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the allocatable resources corresponding to the NUMA node based on the resource allocation weight corresponding to the NUMA node.
[0147] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target I / O thread corresponding to the target resource pool; and processing read / write requests based on the target resource pool and the target I / O thread.
[0148] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if a first disk is detected to be added to the storage system, the correspondence is updated according to the first disk and the NUMA node corresponding to the first disk; or, if a second disk is detected to be deleted from the storage system, the correspondence is updated according to the second disk and the NUMA node corresponding to the second disk.
[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: receiving a read / write request sent by a client; the read / write request carrying identification information of the target disk to be accessed; determining a target resource pool for processing the read / write request based on the identification information and a preset correspondence, wherein the correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk; and processing the read / write request based on the target resources in the target resource pool.
[0150] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target NUMA node corresponding to the target disk based on the identification information and the correspondence; and determining the resource pool corresponding to the target NUMA node as the target resource pool.
[0151] In one embodiment, the target resource includes a target page resource and a target processing resource. When the computer program is executed by the processor, it further performs the following steps: using the target page resource to store data information corresponding to the read / write request; and using the target processing resource to store control information corresponding to the read / write request.
[0152] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: for each NUMA node in the storage system, determine the allocatable resources corresponding to the NUMA node; and construct a resource pool corresponding to the NUMA node based on the allocatable resources.
[0153] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing page resources in the resource pool based on the allocable resources; and constructing processing resources in the resource pool based on the allocable resources and / or the allocable resources corresponding to other NUMA nodes.
[0154] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the allocatable resources corresponding to the NUMA node based on the resource allocation weight corresponding to the NUMA node.
[0155] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the target I / O thread corresponding to the target resource pool; and processing read / write requests based on the target resource pool and the target I / O thread.
[0156] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: if a first disk is detected to be added to the storage system, the correspondence is updated according to the first disk and the NUMA node corresponding to the first disk; or, if a second disk is detected to be deleted from the storage system, the correspondence is updated according to the second disk and the NUMA node corresponding to the second disk.
[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for processing read / write requests, characterized in that, The method includes: Receive read / write requests sent by the client; the read / write requests carry the identification information of the target disk to be accessed; Based on the identification information and the preset correspondence, the target resource pool for processing the read and write requests is determined. The correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk. The read / write request is processed based on the target resource in the target resource pool.
2. The method according to claim 1, characterized in that, The step of determining the target resource pool for processing the read / write request based on the identification information and the preset correspondence includes: Based on the identification information and the correspondence, the target NUMA node corresponding to the target disk is determined; The resource pool corresponding to the target NUMA node is determined as the target resource pool.
3. The method according to claim 1 or 2, characterized in that, The target resources include target page resources and target processing resources. Processing the read / write request based on the target resources in the target resource pool includes: The target page resources are used to store the data information corresponding to the read / write request; The target processing resource is used to store the control information corresponding to the read / write request.
4. The method according to claim 1, characterized in that, The method further includes: For each NUMA node in the storage system, determine the allocatable resources corresponding to that NUMA node; Based on the allocable resources, a resource pool corresponding to the NUMA node is constructed.
5. The method according to claim 4, characterized in that, The step of constructing a resource pool corresponding to the NUMA node based on the allocable resources includes: Based on the allocable resources, construct the page resources in the resource pool; Based on the allocable resources and / or the allocable resources corresponding to other NUMA nodes, the processing resources in the resource pool are constructed.
6. The method according to claim 4 or 5, characterized in that, Determining the allocatable resources corresponding to the NUMA node includes: The allocatable resources corresponding to the NUMA node are determined based on the resource allocation weight corresponding to the NUMA node.
7. The method according to claim 1, characterized in that, The process of processing the read / write request based on the target resource includes: Determine the target I / O thread corresponding to the target resource pool; The read / write requests are processed based on the target resource pool and the target IO thread.
8. The method according to claim 1, characterized in that, The method further includes: If the storage system detects the addition of a first disk, the correspondence is updated based on the first disk and the NUMA node corresponding to the first disk. Alternatively, if the storage system detects that the second disk has been deleted, the correspondence is updated based on the second disk and the NUMA node corresponding to the second disk.
9. A read / write request processing device, characterized in that, The device includes: The receiving module is used to receive read / write requests sent by the client; the read / write request carries the identification information of the target disk to be accessed; The determination module is used to determine the target resource pool for processing the read and write requests based on the identification information and the preset correspondence, wherein the correspondence is the correspondence between each disk in the storage system and the corresponding NUMA node of each disk. The processing module is used to process the read / write request based on the target resource in the target resource pool.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.