NFS file system I / O processing method based on NUMA architecture

Abstract

The invention discloses an NFS (Network File System) I / O (Input / Output) processing method based on an NUMA (Non Uniform Memory Access) architecture, which is characterized in that a corresponding configuration mechanism of a network port and a memory node is introduced under the NUMA architecture, so that an NFS server can identify a NODE to which a connection link belongs in a client mounting stage, and a working process is selected and awakened from a corresponding nfsd process pool based on the NODE attribute in an I / O request stage; therefore, the front-end network data and the rear-end processing thread are guaranteed to be located in the same NUMA node, localization of data processing and memory access is achieved, the problem of cross-node memory access caused by random scheduling of the nfsd process in the prior art is effectively avoided, high delay and bandwidth bottleneck caused by remote memory access are eliminated, and the service life of the NUMA node is prolonged. The I / O data is transmitted and copied in the same node, the I / O throughput rate and response speed of the NFS server in a high-concurrency scene are remarkably improved, and the CPU cache hit rate and the resource utilization rate are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of NFS file system I / O processing based on NUMA architecture, in particular to an NFS file system I / O processing method based on NUMA architecture. BACKGROUND

[0002] With the continuous growth of multi-processor server scale and storage bandwidth demand, non-uniform memory access (NUMA) architecture is widely used because it divides processors and memories into multiple nodes. The characteristics of low local memory access delay and high cross-node access delay of the architecture put higher requirements on the upper I / O path design. Network NFS file system (NFS) is a mainstream solution for sharing files and directories between different computer systems. Clients can access remote files by mounting the server. On the server side, requests are written to the backend storage or memory after being processed by the kernel protocol stack from the application buffer area. The data path involves at least two copies, and the nfsd worker process completes protocol analysis and data processing. This makes the I / O performance of NFS particularly sensitive to memory access locality and thread scheduling strategy.

[0003] Existing NFS has introduced some NUMA design, for example, providing GLOBAL, PER_NODE, PER_CPU three working modes through nfsd_thread_pool during initialization to constrain the CPU affinity of threads and the allocation node of memory pages at different granularities, thereby improving the locality to some extent. However, in specific request scheduling, the thread pool usually wakes up idle nfsd processes in sequence to enter the request, and does not match and select according to the "NUMA node where the front-end network interface / service IP is located". Specifically, in the PER_NODE working mode, the thread pool is initialized to multiple pool resources according to the node attribute of the CPU, each pool resource applies for memory on the specified node, and the processes in the pool can only run on the CPU set corresponding to the node. The thread pool wakes up an idle nfsd process pool through network card interrupt or active polling, and selects an idle nfsd process in the pool to work. This process does not consider the location attribute of the front-end network card. When processing read and write commands and copying data from the front-end network port to the cache page, if the CPU node where the awakened nfsd is located and the memory page corresponding to the data are not in the same node, cross-node memory access will occur, resulting in increased I / O latency, decreased bandwidth utilization, and further affecting the overall performance.

[0004] Therefore, in the NFS file service under the NUMA architecture server environment, the NUMA topology of the front-end network interface and the working thread is not perceived, cross-node memory access is prone to occur during the I / O data copying process, and the existing thread pool scheduling mode lacks intelligent matching based on connection links, which are problems that need to be solved. SUMMARY

[0005] The application provides a NUMA architecture-based NFS file system I / O processing method, aiming to solve the problems of the prior art in the NFS file service of the NUMA architecture server environment, such as the unperceived NUMA topology of the front-end network interface and the working thread, the easy cross-node memory access in the I / O data copying process, and the lack of intelligent matching based on the connection link in the existing thread pool scheduling mode.

[0006] In a first aspect, a NUMA architecture-based NFS file system I / O processing method, the method comprising:

[0007] Establishing, in a storage controller, a corresponding relationship configuration file of a record storage pool, a business network port, a virtual IP group, and a network card NODE, and saving and synchronizing the configuration file;

[0008] Loading the configuration file when the NFS service is initialized, initializing an independent nfsd working process pool for each NODE, limiting the nfsd working processes in the process pool to run within the CPU set range of the NODE, and applying memory pages from the system memory of the NODE;

[0009] When a client initiates a mounting request, the server determines the NODE to which the link enters based on the server IP in the mounting request, and sets the corresponding NODE attribute in the xprt structure used to identify the connection;

[0010] When a read-write I / O request is generated, an idle nfsd working process is selected and awakened from the nfsd working process pool of the corresponding NODE to execute I / O processing according to the NODE attribute in the xprt structure.

[0011] In the above solution, optionally, the establishing, in a storage controller, of a corresponding relationship configuration file of a record storage pool, a business network port, a virtual IP group, and a network card NODE, and the saving and synchronizing of the configuration file comprise:

[0012] Selecting a disk on any controller to create a storage pool;

[0013] Configuring at least one business network port and its virtual IP address group for each storage pool;

[0014] Aggregating multiple network ports to form a link and serving as the business network port of the storage pool when needed;

[0015] Writing the corresponding relationship of the storage pool, the business network port, the virtual IP group, and the network card NODE into the configuration file.

[0016] Optionally, the method further comprises:

[0017] A plurality of configuration records are generated for each storage pool, each record containing: storage pool identification, service network port identification, virtual IP address group, and NODE identification corresponding to the network port;

[0018] The configuration file is synchronously written into the local storage area of the dual controller;

[0019] The configuration file is read and parsed at system startup or configuration update, so that the nfsd work process pool retrieves NODE information according to the service end IP requested by the network, to wake up the idle NFS service process to retrieve NODE information according to the service end IP.

[0020] Optionally, the method further comprises:

[0021] An independent nfsd process pool identification and task queue are allocated for each NODE;

[0022] The nfsd work processes in the process pool are bound to the CPU set corresponding to the NODE;

[0023] The memory page of the nfsd work process is applied from the system memory space corresponding to the NODE.

[0024] Optionally, when receiving a client mounting request, an xprt structure body used to identify the connection is created, and the service end IP is parsed from the mounting request;

[0025] Based on the service end IP, a matching record is queried in the configuration file, the NODE to which the link entered by the connection belongs is determined, and the NODE identification is written into the NODE attribute field of the xprt structure body.

[0026] Optionally, when the connection generates a read-write I / O request, the NODE attribute in the xprt structure body includes:

[0027] The xprt structure body corresponding to the target connection is read to obtain the NODE attribute;

[0028] The nfsd process pool corresponding to the NODE attribute is determined as the target process pool;

[0029] Idle nfsd work processes in the target process pool are sequentially searched;

[0030] When an idle nfsd work process is detected, the process is woken up to perform I / O processing.

[0031] Optionally, the method further comprises:

[0032] A plurality of external communication ports are set on the storage controller to establish a communication link with the client;

[0033] The physical connection of the communication link is routed via an internal backplane of the storage device;

[0034] The mapping relationship between the network interface to which the communication port belongs and the corresponding NODE is recorded in a configuration file.

[0035] In the above scheme, optionally, the method further comprises:

[0036] One or more NFS file systems are created under the storage pool, one or more shared directories are created under each NFS file system and are taken as the root directory accessible by the client, configuration items such as quota, compression algorithm, deduplication strategy and block size are set for the NFS file system and the shared directory, the configuration items are written into a specific area of the storage pool and are loaded into the memory at the service startup.

[0037] In a second aspect, a computer readable storage medium stores a computer program, and the program is used to implement the method of the first aspect when executed by a processor.

[0038] In a third aspect, an NFS server comprises a processor and a memory, and the memory stores program instructions running on the processor, and the program instructions make the processor execute the method of the first aspect when executed.

[0039] Compared with the prior art, the present application has at least the following beneficial effects:

[0040] Based on further analysis and research of existing technical problems, this application recognizes that in NFS file services under a NUMA architecture server environment, the NUMA topology of the front-end network interface and worker threads is not perceived, cross-node memory access is prone to occur during I / O data copying, and the existing thread pool scheduling mode lacks intelligent matching based on connection links. By introducing a corresponding configuration mechanism of network ports and memory nodes under the NUMA architecture, the NFS server can identify the NODE to which the connection link belongs during the client mounting stage, and select and wake up the worker process from the corresponding nfsd process pool based on the NODE attribute during the I / O request stage. This ensures that the front-end network data and the back-end processing thread are located in the same NUMA node, realizing the localization of data processing and memory access. This solution effectively avoids the cross-node memory access problem caused by random scheduling of NFSD processes in existing technologies, eliminates the high latency and bandwidth bottleneck caused by remote memory access, and enables I / O data to be transmitted and copied within the same node. This significantly improves the I / O throughput and response speed of the NFS server in high-concurrency scenarios, improves CPU cache hit rate and resource utilization, and realizes intelligent scheduling and efficient execution of NFS file system I / O performance under NUMA architecture. It fundamentally solves the problem of "access latency and performance degradation caused by no topological association in process selection" in the background technology. Attached Figure Description

[0041] Figure 1 A schematic diagram illustrating the application environment of an NFS file system I / O processing method based on NUMA architecture provided in one embodiment of this application;

[0042] Figure 2 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] Explanation of key technologies in the embodiments of this application:

[0045] NUMA (Non-Uniform Memory Access) is a multiprocessor computer memory design architecture. Its core feature is dividing processors and memory into multiple nodes, solving the memory bandwidth bottleneck caused by the shared bus in traditional SMP (Symmetric Multiprocessor) architectures and supporting larger-scale processor expansion. Each NUMA node contains a set of processors, local memory, and I / O resources. Processors access local memory fastest, while accessing remote memory on other nodes has higher latency. Nodes communicate via a high-speed interconnect network, but cross-node access speed is still significantly slower than accessing local memory. Typical application scenarios include high-performance computing, database servers, and other applications requiring large memory bandwidth. For these applications, NUMA affinity needs to be optimized to avoid cross-node access, maximizing processor processing of memory within the same node to improve customer business performance.

[0046] NFS (Network File System) is a network-based file system, an application layer protocol that supports underlying network communication protocols such as TCP / IP and RoCE. It is used to share files and directories between computer systems. NFS consists of a client and a server; an NFS client can access files and directories on a remote computer by mounting an NFS server.

[0047] The NFS server receives data from the client application cache, encapsulates it via the kernel protocol stack, and sends it. The server receives and decodes the data before writing it to backend storage or memory. This process involves at least two data copies. Internal data processing is handled by the NFSD worker process, and the NFSD kernel can run on any CPU core.

[0048] NFS itself is designed with NUMA in mind. Its implementation prioritizes the allocation of memory resources from the same node for the core of the worker process (the nfsd process) and the memory resources required by the worker process itself.

[0049] NFS already has NUMA design, and all tasks are executed by the worker process pool nfsd_thread_pool created during NFS initialization. NFS has three default working modes, including GLOBAL, PER_NODE, and PER_CPU.

[0050] When initializing NFSD process pool resources:

[0051] GLOBAL: Memory page resources are allocated uniformly, and all NFSD processes have consistent CPU processor affinity.

[0052] PER_NODE: Initialized on a node-by-node basis. The process pool corresponding to each node requests the memory page with the best access speed for that node. The corresponding NFSD process can only run on the CPU set corresponding to the node.

[0053] PER_CPU: Initialized on a per-CPU core basis. Each CPU requests memory pages from its own node. Each NFSD process can only run on a single matched CPU.

[0054] In one embodiment, such as Figure 1 As shown, an NFS file system I / O processing method based on NUMA architecture is provided, including the following steps:

[0055] In the storage controller, establish a configuration file that maps the storage pool, service network interface, virtual IP group, and the node where the network card is located, and save and synchronize the configuration file.

[0056] The configuration file is loaded during NFS service initialization to initialize an independent NFSD worker process pool for each NODE. This restricts the NFSD worker processes in the process pool to run within the CPU set of that NODE and requests memory pages from the system memory of that NODE.

[0057] When the client initiates a mount request, the server queries the configuration file based on the server IP in the mount request to determine the NODE to which the current connection belongs, and sets the corresponding NODE attribute in the xprt structure used to identify the connection.

[0058] When a connection generates a read / write I / O request, based on the NODE attribute in the xprt structure, an idle NFSD worker process is selected and woken up from the NFSD worker process pool corresponding to the NODE to perform the I / O processing.

[0059] This embodiment provides an NFS file system I / O processing method based on NUMA architecture. In a Non-Uniform Memory Access (NUMA) server environment, by establishing a mapping relationship between network ports and memory nodes on the NFS server, the node identification and same-node scheduling execution of client requests are realized, thereby avoiding cross-node memory access during NFS file system I / O and improving the overall I / O processing performance of the system.

[0060] In its implementation, this method includes the following main steps:

[0061] Step S1: Create and save the configuration file: Create a configuration file in the storage controller that records the correspondence between "storage pool - service interface - virtual IP group - network card NODE". This configuration file is in text or structured data format (e.g., table, JSON, or key-value pairs). Each record corresponds to one service interface and includes: storage pool number, service interface identifier, network card NODE number, and bound virtual IP address group.

[0062] After generation, the configuration file is simultaneously written to the local storage areas of both controller nodes to ensure data consistency. During system initialization or configuration updates, the controllers synchronize by verifying version numbers, ensuring that any controller obtains consistent results when resolving the IP-to-NODE mapping. The creation of this configuration file is the fundamental data source for NUMA-aware scheduling.

[0063] Step S2: Initialize the nfsd process pool corresponding to the NUMA node:

[0064] When the system starts the NFS service, the NFS server loads the above configuration file and initializes the nfsd process pool corresponding to each NODE according to the NODE number identified in the configuration.

[0065] Each process pool has an independent task queue and process management structure. The NFSD worker processes in the process pool are only bound to the CPU set of that NODE, and the memory pages required for their operation are allocated from the system memory area of ​​the same NODE.

[0066] During initialization, the system assigns a unique identifier to each process pool and registers it in the "NODE—CPU Set—Process Pool" mapping table to enable subsequent process retrieval based on the NODE attribute. This step achieves static binding between NFS worker processes and NUMA node resources, ensuring that CPU and memory resources on different nodes do not interfere with each other.

[0067] Step S3: Identify the NODE attribute during mount request:

[0068] When a client initiates a mount request, it needs to provide the server's IP address, the shared directory path, and the local mount point information. After receiving the request, the server parses the server's IP address contained in the mount request and matches the corresponding Node number by querying the configuration file established in step S1.

[0069] Subsequently, the NFS server creates a unique identifier structure xprt for the connection at the kernel layer, which includes a server IP field, a client identifier field, a protocol type field, and a NODE attribute field.

[0070] The system writes the found NODE number into the NODE attribute field of the xprt structure. This attribute remains unchanged throughout the entire connection lifecycle and serves as the core basis for subsequent I / O scheduling.

[0071] In this way, the server can establish the mapping relationship between network links and memory nodes during the mounting phase, realizing NUMA awareness of I / O paths.

[0072] Step S4: Select and wake up the NFSD worker process based on the NODE attribute:

[0073] When a client initiates an I / O request (such as a file read / write command) through this mounted connection, the NFS server first receives the I / O notification and parses the corresponding xprt structure, from which it reads the NODE attribute value.

[0074] Based on the NODE attribute value, the system locates the corresponding NFSD process pool for that NODE, and then searches for an idle NFSD worker process within that pool. When an idle process is detected, it is immediately woken up to perform I / O request processing, including packet parsing, reading from the front-end data buffer, and writing to the back-end storage.

[0075] Since the selected nfsd process is on the same NUMA node as the front-end network interface, the process directly accesses the memory of the local node when handling I / O, without involving cross-node data copying, thus significantly reducing the memory access latency of the I / O path.

[0076] Once the entire process is complete, the I / O processing results are returned to the client via the same link, ensuring that the data flow path is consistent with the NUMA node topology.

[0077] Through the implementation of the above steps, this embodiment constructs a NUMA-aware I / O scheduling mechanism for the NFS server under a NUMA architecture server, which can effectively solve the problem of "high latency in cross-node access caused by random selection of nfsd processes" in the background technology. The technical effects are mainly reflected in the following aspects:

[0078] Eliminate cross-node access bottlenecks: In traditional NFS, the scheduling of worker processes is independent of the network link topology. This can lead to situations where the front-end network interface is handled by the nfsd thread of NODE1 while it is located in NODE0, causing I / O data to need to be transmitted across the NUMA bus, resulting in a significant increase in access latency.

[0079] This embodiment identifies the IP address corresponding to the node during the mounting phase and schedules threads according to the node during the I / O phase, thereby binding the NFSD worker process to the same node as the front-end network interface. Data copying and caching are completed within the same NUMA node, effectively avoiding cross-node access.

[0080] Improving I / O locality and CPU cache hit rate: Because the I / O data path is a closed loop within the same node, the locality of CPU access to memory pages is guaranteed. Each node's process pool only accesses its own memory region, reducing the number of remote memory accesses, improving CPU cache utilization, and significantly reducing inter-processor bus contention.

[0081] Improve overall throughput and stability of NFS service: Through NUMA-aware scheduling, the system can achieve load balancing and resource isolation between nodes in high-concurrency I / O request scenarios. Each node's nfsd process pool runs independently and does not preempt each other, thereby avoiding resource contention between different nodes and significantly improving the throughput performance and response stability of the NFS server under high load.

[0082] A general mechanism for building NUMA topology adaptation: This mechanism does not depend on a specific hardware architecture and can automatically adapt the number of nodes and CPU layout on multi-NUMA node servers through configuration files; at the same time, the NODE attribute of the mounted connection is used as a scheduling index, which can be flexibly extended to multi-NIC aggregation or multi-controller storage environments, providing a general framework for subsequent NFS protocol layer optimization.

[0083] In this embodiment, the step of establishing a configuration file in the storage controller to map the record storage pool, service network interface, virtual IP group, and the NIC's node, and then saving and synchronizing the configuration file, includes:

[0084] Select a disk on any controller to create a storage pool;

[0085] Configure at least one service network interface and its virtual IP address group for each storage pool;

[0086] When needed, multiple network ports can be aggregated to form a link and used as the service network port for the storage pool;

[0087] Write the mapping relationship between storage pool, service network interface, virtual IP group and the NODE where the network card is located into the configuration file.

[0088] In this embodiment, the method further includes:

[0089] Multiple configuration records are generated for each storage pool. Each record includes: storage pool identifier, service network interface identifier, virtual IP address group, and the NODE identifier corresponding to the network interface.

[0090] The configuration file is synchronously written to the local storage area of ​​the dual controllers;

[0091] The configuration file is read and parsed during system startup or configuration update so that the mounting process can retrieve NODE information by server IP.

[0092] In this embodiment, the method further includes:

[0093] Each NODE is assigned an independent NFSD process pool ID and task queue;

[0094] Bind the NFSD worker processes in the process pool to the CPU set corresponding to the NODE;

[0095] The memory pages of the nfsd worker process are requested from the system memory space corresponding to the NODE.

[0096] In this embodiment, an xprt structure is created to identify the connection when a client mount request is received, and the server IP is parsed from the mount request;

[0097] Based on the server IP, the system queries the configuration file to find the matching record, determines the NODE to which the connection belongs, and writes the NODE identifier into the NODE attribute field of the xprt structure.

[0098] In this embodiment, when a connection generates a read / write I / O request, the NODE attribute in the xprt structure is used as the basis for:

[0099] Read the xprt structure corresponding to the target connection to obtain the NODE property;

[0100] Identify the NFSD process pool corresponding to the NODE attribute as the target process pool;

[0101] Search for idle NFSD worker processes in the target process pool in turn;

[0102] When an idle NFSD worker process is detected, the process is woken up to perform I / O processing.

[0103] In this embodiment, the method further includes:

[0104] Multiple external communication ports are configured on the storage controller to establish communication links with clients;

[0105] The physical connection of the communication link is laid out via the internal backplane of the storage device;

[0106] The configuration file records the mapping relationship between the network interface to which the communication port belongs and its corresponding node.

[0107] In this embodiment, the method further includes: creating one or more NFS file systems under the storage pool, creating one or more shared directories under each NFS file system and using them as root directories accessible to clients, setting configuration items such as quotas, compression algorithms, deduplication strategies and block sizes for the NFS file systems and shared directories, writing the configuration items into a specific area of ​​the storage pool and loading them into memory when the service starts.

[0108] In one embodiment, the current NFS incorporates NUMA design considerations, but only ensures that the core and required memory pages of its NFS worker processes are identical to those of the node, and that accessing these NFS service protocols and their commonly used variables is identical to that of the DIE. Upon receiving a client request, a worker process is randomly awakened from the NFS thread pool to perform data processing, including parsing its own protocol and processing and copying incoming business data from the network frontend.

[0109] The NFS thread pool selects worker processes by sequentially searching for an idle NFSD and waking it up; it does not use an algorithm based on the location of the front-end network interface. When processing read / write commands, data from the front-end network interface needs to be copied to a cached memory page. Since the node of the memory page containing front-end network interface data is unknown, when the NFS thread pool randomly selects an NFSD worker process to execute the task, it cannot be guaranteed that the CPU of the selected NFSD worker process and the memory page input from the front-end network interface belong to the same node. If they are on different nodes, memory access during data copying will involve cross-node issues, impacting I / O performance.

[0110] This embodiment optimizes the selection of NFS process in NFS under NUMA architecture. When processing read / write IO commands that require copying data from front-end memory pages, it can automatically select the NFS process on the same node, avoiding cross-node memory access.

[0111] This embodiment creates a NAS service on the storage system and loads the NODE configuration:

[0112] Establish communication links between the controller and external clients: The controller is equipped with multiple communication ports to establish external communication links with clients. The physical connection of the communication links is established on the backplane inside the storage device.

[0113] Create storage resources: Select a disk on any controller to create a storage pool. The storage pool can be customized with a RAID level.

[0114] Each storage pool must be configured with a service network interface and a virtual IP group, and clients must access the storage pool through the configured service network interface and virtual IP group.

[0115] Each storage pool can be configured with multiple network ports as the service network ports of the pool, and supports using the links of multiple network ports aggregated as the service network ports of the storage pool.

[0116] The configuration file, which records the mapping between the storage pool, service network interface, virtual IP group, and the node where the network card is located, is synchronously written to both controllers. Example of configuration file format:

[0117] Storage Pool 1 - Network Port 1 - Network Port 1 NODE Information - IP1, IP2, IP3;

[0118] Storage Pool 1 - Network Port 2 - Network Port 2 NODE Information - IP4;

[0119] Create an NFS file system under the storage pool. The storage pool supports the creation of multiple NFS file systems. Under the NFS file system, multiple shared directories can be created. The shared directories are used as the root directories accessible to clients.

[0120] When creating an NFS file system and shared directory, you can set configuration items for the NFS file system and shared directory, such as quota, compression algorithm, deduplication, block size, etc. These configuration items are stored in a specific area of ​​the storage pool and become inherent attributes of the NFS file system / shared directory after configuration.

[0121] Configure NAS service: After the shared directory is created, configure the NAS service for the shared directory: a. Select NFS as the NAS protocol type for exporting the shared directory; further configure other configuration items according to the configured protocol type, such as: enabling read / write caching, client authentication information, user information, NAS protocol supported versions, etc.

[0122] NFS mount initiation I / O process:

[0123] NFS mounting process: Setting the node properties of the connection:

[0124] When an NFS client initiates a mount request to the server, it needs to input information such as the server IP, shared directory, and local mount point. The established connection is uniquely identified by the xprt structure, where the server IP identifies the IO command data link channel. After receiving the mount request command, the server uses the local service IP configuration table established in "Creating Storage Resources" above to identify the link node into which the current connection has entered and set special attributes for subsequent IO use.

[0125] NFS client read / write IO after mounting: After the client completes mounting, the IO process is as follows: Upon receiving the xprt connection notification that there is a data request command.

[0126] Based on the xprt node identifier attribute, the most suitable NFSD worker process is woken up from the corresponding node worker process pool to handle the request.

[0127] The awakened NFSD process begins processing I / O data, copying the front-end data memory pages to the back-end data memory pages.

[0128] In one embodiment, there are two communication links between the controller and the two clients: channel 1 corresponding to network interface card 1 and channel 2 corresponding to network interface card 2.

[0129] The two network interface cards (NICs) are inserted into the corresponding PCIe slots on node0 and node1 of the server backplane, respectively. Each NIC has two service ports: eth1, eth2, eth3, and eth4, and is configured with IP1, IP2, IP3, and IP4.

[0130] Create a storage pool on the controller, and create several NFS file systems and shared directories on the storage pool.

[0131] All shared directories have completed NAS service configuration;

[0132] The NAS resource configuration is as follows:

[0133] ETH1-NODE0-IP1-IP2;

[0134] ETH2-NODE1-IP3-IP4;

[0135] The NFS service is initialized, NAS resource configuration is loaded, and process pool 1 and process pool 2 are initialized separately. Processes in process pool 1 can only run on the CPU set corresponding to node0, and memory is allocated from the system memory on node0.

[0136] Processes in process pool 2 can only run on the CPU set corresponding to node1, and memory is also requested from the system memory on node1.

[0137] Scenario 1: Client A, whose communication link is channel 1, initiates the mounting process.

[0138] Client A initiates a shared directory mount using the server's IP address 1. After detecting the mount command, the server identifies that the parameter IP1 belongs to node0 by checking the background configuration items, and sets the node attribute of the connection structure xprt to node0.

[0139] When initiating I / O after mounting, the nfsd worker process is woken up by connecting the xprt property of the connection structure to the relevant node worker process pool 1.

[0140] Scenario 2: Client B, whose communication link is channel 2, initiates the mounting process.

[0141] Client B initiates a shared directory mount using the server's IP address 3. After detecting the mount command, the server identifies that the parameter IP3 belongs to node1 by checking the background configuration items, and sets the node attribute of the connection structure xprt to node1.

[0142] When initiating I / O after mounting, the nfsd worker process is woken up by connecting the xprt property of the connection structure to the relevant node worker process pool 2.

[0143] This embodiment uses the method of pre-identifying the network interface node attribute on the storage system and querying the configuration items when the client mounts the device to ensure that all NFSD worker threads match the front-end network interface node. This reduces cross-node access when worker threads copy data and improves the overall IO performance of the NFS service.

[0144] In one embodiment, an NFS server is provided, including a processor and a memory, wherein the memory stores program instructions that run on the processor, and the program instructions, when executed, enable the NFS file system I / O processing method based on a NUMA architecture.

[0145] In one embodiment, an NFS server is provided, the internal structure of which can be shown as follows: Figure 2 As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities, and the communication interface enables wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The computer device implements the aforementioned NUMA-based NFS file system I / O processing method by loading and running a computer program. The display screen can be an LCD screen or an e-ink screen, and the input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0146] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0147] In one embodiment, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, relate to all or part of the processes in the methods of the above embodiments.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for I / O processing of an NFS file system based on a NUMA architecture, characterized in that, The method includes: In the storage controller, establish a configuration file that maps the storage pool, service network interface, virtual IP group, and the node where the network card is located, and save and synchronize the configuration file. The configuration file is loaded during NFS service initialization to initialize an independent NFSD worker process pool for each NODE. This restricts the NFSD worker processes in the process pool to run within the CPU set of that NODE and requests memory pages from the system memory of that NODE. When the client initiates a mount request, the server queries the configuration file based on the server IP in the mount request to determine the NODE to which the current connection belongs, and sets the corresponding NODE attribute in the xprt structure used to identify the connection. When a connection generates a read / write I / O request, based on the NODE attribute in the xprt structure, an idle NFSD worker process is selected and woken up from the NFSD worker process pool corresponding to the NODE to perform the I / O processing.

2. The method according to claim 1, characterized in that, The step of establishing a configuration file in the storage controller to map the storage pool, service network interface, virtual IP group, and the node where the network card is located, and then saving and synchronizing the configuration file, includes: Select a disk on any controller to create a storage pool; Configure at least one service network interface and its virtual IP address group for each storage pool; When needed, multiple network ports can be aggregated to form a link and used as the service network port for the storage pool; Write the mapping relationship between storage pool, service network interface, virtual IP group and the NODE where the network card is located into the configuration file.

3. The method according to claim 1, characterized in that, The method further includes: Multiple configuration records are generated for each storage pool. Each record includes: storage pool identifier, service network interface identifier, virtual IP address group, and the NODE identifier corresponding to the network interface. The configuration file is synchronously written to the local storage area of ​​the dual controllers; The configuration file is read and parsed during system startup or configuration updates so that the NFSD worker process pool can retrieve NODE information based on the server IP of the network request, thereby waking up idle NFS service processes to retrieve NODE information based on the server IP.

4. The method according to claim 1, characterized in that, The method further includes: Each NODE is assigned an independent NFSD process pool ID and task queue; Bind the NFSD worker processes in the process pool to the CPU set corresponding to the NODE; The memory pages of the nfsd worker process are requested from the system memory space corresponding to the NODE.

5. The method according to claim 1, characterized in that, When receiving a client mount request, create an xprt structure to identify the connection and resolve the server IP from the mount request; Based on the server IP, the system queries the configuration file to find the matching record, determines the NODE to which the connection belongs, and writes the NODE identifier into the NODE attribute field of the xprt structure.

6. The method according to claim 1, characterized in that, When a connection generates a read / write I / O request, the NODE attribute in the xprt structure includes: Read the xprt structure corresponding to the target connection to obtain the NODE property; Identify the NFSD process pool corresponding to the NODE attribute as the target process pool; Search for idle NFSD worker processes in the target process pool in turn; When an idle NFSD worker process is detected, the process is woken up to perform I / O processing.

7. The method according to claim 1, characterized in that, The method further includes: Multiple external communication ports are configured on the storage controller to establish communication links with clients; The physical connection of the communication link is laid out via the internal backplane of the storage device; The configuration file records the mapping relationship between the network interface to which the communication port belongs and its corresponding node.

8. The method according to claim 1, characterized in that, The method further includes: One or more NFS file systems are created under the storage pool. One or more shared directories are created under each NFS file system and used as the root directory accessible to clients. Quotas, compression algorithms, deduplication strategies, block sizes, and other configuration items are set for the NFS file systems and shared directories. The configuration items are written to a specific area of ​​the storage pool and loaded into memory when the service starts.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, is used to implement the method as claimed in any one of claims 1 to 8.

10. An NFS server, comprising a processor and a memory, wherein the memory stores program instructions that run on the processor, the program instructions, when executed, cause the processor to perform the method according to any one of claims 1 to 8.

Images