Optimization method, device, computer equipment, storage medium and program product

Abstract

The application relates to an optimization method, device, computer equipment, storage medium and computer program product. The method comprises the following steps: obtaining a target optimization strategy, and performing an optimization process on a server based on the target optimization strategy. The optimization process comprises the following steps: determining a target NUMA node corresponding to a target network card in the server, binding the target network card and a target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node and a target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core responds to an interrupt request of the target network card by accessing the target storage device. The method can maximize the performance of a Ceph distributed block storage system.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an optimization method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Ceph is one of the most popular distributed storage technologies on the market. It can provide storage services with block, file and object storage interfaces. It also has high scalability and reliability. OpenStack uses Ceph as its native storage backend. 70% to 80% of cloud platforms on the market use Ceph as their underlying storage platform. Ceph's block storage module uses librbd to provide block storage interfaces to upper-layer physical hosts or virtual machines. The backend interfaces with librados and the underlying RADOS object storage system.

[0003] Currently, due to limitations in integrated circuit manufacturing processes, CPU architecture is evolving towards multi-NUMA architecture. Multi-NUMA architecture CPUs, represented by AMD, are becoming increasingly widespread in the market. In order to maximize the performance of Ceph block storage servers based on such CPU architecture platforms, hardware specifications are often optimized and upgraded, namely, using high-performance NVMe protocol SSDs as the underlying storage hardware devices.

[0004] However, this optimization method only targets the hardware level, which means that users cannot get the highest performance improvement on the basis of expensive hardware costs, that is, high cost and low return. Summary of the Invention

[0005] Therefore, it is necessary to provide an optimization method, apparatus, computer device, computer-readable storage medium, and computer program product that can maximize the performance of Ceph distributed block storage in response to the above-mentioned technical problems.

[0006] Firstly, this application provides an optimization method. The method includes:

[0007] Obtain the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0008] In one embodiment, the method further includes: configuring a target number of request queues for interrupt requests of the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0009] In one embodiment, configuring a target number for the interrupt request queue of the target network card based on the number of target CPU cores includes: configuring the target number to be the same as the number of target CPU cores; or, configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or, configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node.

[0010] In one embodiment, binding the target NUMA node to the target storage device based on the distance between the target NUMA node and each storage device in the server includes: binding the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device.

[0011] In one embodiment, binding the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each of the storage devices includes: determining the target storage device closest to the target NUMA node from among the storage devices based on the distance between the target NUMA node and each of the storage devices, and determining the storage device management process corresponding to the target storage device; binding the target NUMA node to the storage device management process corresponding to the target storage device.

[0012] In one embodiment, binding the target NUMA node to the storage device management process corresponding to the target storage device includes: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0013] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the NIC is bound to the target CPU core, including: binding the NIC to the target CPU core local to the target NUMA node based on the correspondence between the NIC and the target NUMA node; or binding the NIC to the target CPU core contained in the entire CPU where the target NUMA node is located based on the correspondence between the NIC and the target NUMA node.

[0014] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the NIC is bound to the target CPU core, including: determining a binding script according to the type of the NIC; and using the binding script to bind the NIC to the target CPU core.

[0015] In one embodiment, before obtaining the target optimization strategy, the method further includes: obtaining multiple candidate optimization strategies, performing the optimization process based on each candidate optimization strategy, and after performing the optimization process, determining performance improvement parameters, which are used to characterize the improvement of read and write performance of the storage server cluster where the server is located before and after the optimization process is performed; and determining the target optimization strategy from the multiple candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0016] Secondly, this application also provides an optimization device. The device includes:

[0017] The acquisition module is used to acquire the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0018] In one embodiment, the optimization device further includes a configuration module, which is configured to: configure a target number of request queues for interrupt requests of the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0019] In one embodiment, the configuration module is specifically configured to: configure the target quantity to be the same as the target CPU core quantity; or, configure the target quantity to be less than the target CPU core quantity and the same as the logical CPU core quantity of the target NUMA node; or, configure the target quantity to be less than the target CPU core quantity and the same as the physical CPU core quantity of the target NUMA node.

[0020] In one embodiment, the acquisition module is further configured to: bind the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each of the storage devices.

[0021] In one embodiment, the acquisition module is further configured to: determine the target storage device closest to the target NUMA node from among the storage devices based on the distance between the target NUMA node and each of the storage devices, and determine the storage device management process corresponding to the target storage device; and bind the target NUMA node to the storage device management process corresponding to the target storage device.

[0022] In one embodiment, the acquisition module is further configured to: directly bind the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0023] In one embodiment, the acquisition module is further configured to: bind the target network card to the target CPU core local to the target NUMA node based on the correspondence between the target network card and the target NUMA node; or, bind the target network card to the target CPU core contained in the entire CPU where the target NUMA node is located based on the correspondence between the target network card and the target NUMA node.

[0024] In one embodiment, the acquisition module is further specifically used to: determine a binding script based on the type of the target network card; and use the binding script to bind the target network card to the target CPU core.

[0025] In one embodiment, the optimization device further includes an execution module, which is configured to: acquire a plurality of candidate optimization strategies, execute the optimization process based on each candidate optimization strategy, and after executing the optimization process, determine performance improvement parameters, which are used to characterize the improvement of read and write performance of the storage server cluster where the server is located before and after the optimization process is executed; and determine the target optimization strategy from the plurality of candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0026] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform any of the steps described in the first aspect above.

[0027] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs any of the steps described in the first aspect above.

[0028] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements any of the steps described in the first aspect above.

[0029] The aforementioned optimization method, apparatus, computer equipment, storage medium, and computer program product first obtain a target optimization strategy, and then perform an optimization process on the server based on the target optimization strategy. This optimization process includes: determining the target NUMA node corresponding to the target network interface card (NIC) in the server; binding the target NIC to the target CPU core based on the correspondence between the target NIC and the target NUMA node; and binding the target NUMA node to the target storage device based on the distance between the target NUMA node and each storage device in the server, so that the target CPU core responds to the interrupt request of the target NIC by accessing the target storage device. The optimization method provided in this application binds the target network card to the target CPU core, enabling the target CPU core to respond to interrupt requests from the target network card. It also binds the target NUMA node to the target storage device, allowing the target CPU core to respond to interrupt requests from the target network card by accessing the target storage device. The optimization method provided in this application optimizes at the hardware level while also jointly optimizing at the operating system and Ceph software level. This effectively reduces communication overhead between Ceph distributed block storage systems, shortens network card interrupt response time, and reduces cross-NUMA memory access, thereby maximizing the performance of the Ceph distributed block storage system despite high costs. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the optimization method in one embodiment;

[0031] Figure 2 This is a flowchart illustrating the optimization method in another embodiment;

[0032] Figure 3 This is a flowchart illustrating the optimization method in another embodiment;

[0033] Figure 4 This is a flowchart illustrating the optimization method in another embodiment;

[0034] Figure 5 This is a flowchart illustrating the optimization method in another embodiment;

[0035] Figure 6 This is a structural block diagram of the optimization device in one embodiment;

[0036] Figure 7 This is a structural block diagram of the optimization device in another embodiment;

[0037] Figure 8 This is a structural block diagram of the optimization device in another embodiment;

[0038] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] Ceph is one of the most popular distributed storage technologies on the market. It can provide storage services with block, file and object storage interfaces. It also has high scalability and reliability. OpenStack uses Ceph as its native storage backend. 70% to 80% of cloud platforms on the market use Ceph as their underlying storage platform. Ceph's block storage module uses librbd to provide block storage interfaces to upper-layer physical hosts or virtual machines. The backend interfaces with librados and the underlying RADOS object storage system.

[0041] Currently, due to limitations in integrated circuit manufacturing processes, CPU (central processing unit) architecture is evolving towards multi-NUMA (Non-Uniform Memory Access) architecture. Multi-NUMA architecture CPUs, represented by AMD, are becoming increasingly widespread in the market. In order to maximize the performance of Ceph block storage servers based on such CPU architecture platforms, hardware specifications are often optimized and upgraded, namely, using high-performance NVMe protocol SSDs as the underlying storage hardware devices.

[0042] However, this optimization method only targets the hardware level, which means that users cannot get the highest performance improvement on the basis of expensive hardware costs, that is, high cost and low return, resulting in the low performance of Ceph distributed block storage.

[0043] In view of this, this application provides an optimization method that can maximize the performance of the Ceph distributed block storage system.

[0044] The optimization method provided in this application embodiment can be executed by a computer device, which can be a server.

[0045] In one embodiment, such as Figure 1 As shown, an optimization method is provided, including the following steps:

[0046] Step 101: Obtain the target optimization strategy and perform the optimization process on the server based on the target optimization strategy.

[0047] The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0048] Optionally, the optimization strategy can be developed by technical personnel and can be used to optimize the performance of the Ceph distributed block storage system.

[0049] Optionally, the target network card refers to the network card that sends interrupt requests to the CPU. Optionally, the network card can be a Ceph public network card or a Ceph cluster network card.

[0050] The target NUMA node refers to the NUMA node where the target network card is located, and the target CPU core can be a core in the target NUMA node.

[0051] The target storage device refers to the storage device that is closest to the target NUMA node.

[0052] In one possible implementation, the target NUMA node corresponding to the target network card in the server can be determined by processing the parameters of the target network card using an algorithm. Assuming that the target network card is a Ceph public network card with the parameter enp33s0f0, the target NUMA node of the target network card can be determined by processing the parameter using an algorithm.

[0053] In an optional embodiment, basic hardware and software environment preparation is required before performing the optimization process on the server based on the target optimization strategy.

[0054] In one possible implementation, the preparation of the basic hardware and software environment can be understood as preparing the hardware of the server under test and setting up at least 3 nodes, that is, configuring NVMe SSDs on the server as cache disks or storage hardware, and installing an appropriate operating system on the server under test as the basic software environment for Ceph deployment.

[0055] In an optional embodiment, Ceph needs to be deployed before the optimization process is performed on the server based on the target optimization strategy.

[0056] In one possible implementation, the Ceph deployment specifically refers to deploying the corresponding version of the Ceph distributed storage system on all nodes based on the installed operating system and the appropriate deployment tools according to business needs, forming a Ceph storage cluster of no less than 3 nodes, and creating a corresponding number of rbds using the rbd create command as block storage test interfaces.

[0057] In an optional embodiment, basic testing is required before performing the optimization process on the server based on the target optimization strategy.

[0058] In one possible implementation, the FIO testing tool (rbd engine) can be used on the client to test the RBD of the server cluster under test for four basic metrics: 4k random read / write and 128k sequential read / write, and the baseline performance can be recorded.

[0059] The aforementioned optimization method, apparatus, computer equipment, storage medium, and computer program product first obtain a target optimization strategy, and then perform an optimization process on the server based on the target optimization strategy. This optimization process includes: determining the target NUMA node corresponding to the target network interface card (NIC) in the server; binding the target NIC to the target CPU core based on the correspondence between the target NIC and the target NUMA node; and binding the target NUMA node to the target storage device based on the distance between the target NUMA node and each storage device in the server, so that the target CPU core responds to the interrupt request of the target NIC by accessing the target storage device. The optimization method provided in this application binds the target network card to the target CPU core, enabling the target CPU core to respond to interrupt requests from the target network card. It also binds the target NUMA node to the target storage device, allowing the target CPU core to respond to interrupt requests from the target network card by accessing the target storage device. The optimization method provided in this application optimizes at the hardware level while also jointly optimizing at the operating system and Ceph software level. This effectively reduces communication overhead between Ceph distributed block storage systems, shortens network card interrupt response time, and reduces cross-NUMA memory access, thereby maximizing the performance of the Ceph distributed block storage system despite high costs.

[0060] In one embodiment, such as Figure 2 As shown, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core, including the following steps:

[0061] Step 201: Determine the binding script based on the type of the target network card.

[0062] Optionally, the type of the target network card refers to the brand of the target network card, and the binding script refers to a program provided by the network card manufacturer that can be used for binding network card interrupts.

[0063] In one possible implementation, the target network card could be an Intel network card, in which case the corresponding network card interrupt binding script would be the set_irq_affinity script.

[0064] In another possible implementation, the target network interface card (NIC) can also be a Mellanox NIC, in which case the corresponding NIC interrupt binding script is the set_irq_affinity_cpulist.sh script in the mlnx_tuning_scripts directory.

[0065] Step 202: Use the binding script to bind the target network card to the target CPU core.

[0066] Optionally, binding the target network card to the target CPU core means binding the network card interrupt of the target network card to the target CPU core.

[0067] In an optional embodiment, the target network interface card (NIC) is bound to the target CPU core based on the correspondence between the target NIC and the target NUMA node, including: binding the target NIC to the target CPU core local to the target NUMA node based on the correspondence between the target NIC and the target NUMA node; or binding the target NIC to the target CPU core contained in the entire CPU where the target NUMA node is located based on the correspondence between the target NIC and the target NUMA node.

[0068] In one possible implementation, the NUMA node where the target network card is located can be queried first. This NUMA node is the target NUMA node. Then, the lscpu command is used to query the CPU corresponding to the target NUMA node. This CPU is the target CPU. Assuming the query result is 0-7 or 32-39, the network card interrupt binding script, network card parameters, and query results are then processed by an algorithm to bind the network card interrupt to the core of the target CPU.

[0069] In another possible implementation, the NUMA node where the target network card is located can be queried first. This NUMA node is the target NUMA node. Then, the lscpu command is used to query the CPU corresponding to the target NUMA node. This CPU is the target CPU. Assuming the query result is 0-7 or 32-39, the network card interrupt binding script, network card parameters, and query results are then processed by an algorithm to bind the network card interrupt to the cores contained in the entire CPU of the target NUMA node, that is, to the cores contained in the entire socket of the CPU of the target NUMA node.

[0070] The network card interrupt binding method described above allows for binding to either the local NUMA node of the network card or all cores of the CPU socket on the same node for Ceph public network cards. However, for Ceph clusternetwork network cards, the binding of network card interrupts must be done locally on the NUMA node to achieve the fastest response time and minimize the data synchronization overhead between nodes.

[0071] The method described above, which binds the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, can effectively shorten the network card interrupt response time, thereby improving performance.

[0072] In one embodiment, the method further includes: configuring a target number of request queues for interrupt requests of the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0073] In an optional embodiment, configuring a target number for the interrupt request queue of the target network interface card based on the number of target CPU cores includes: configuring the target number to be the same as the number of target CPU cores; or, configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or, configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node.

[0074] The target number refers to the number of interrupt request queues for the target network interface card.

[0075] In one possible implementation, the parameters of the target number of cores and the target network interface card can be processed using the ethtool method to configure a request queue that is the same as the number of target CPU cores, so that the target CPU cores can respond to network interface card interrupts.

[0076] In another possible implementation, the number of logical CPU cores of the target NUMA node and the parameters of the target network card can be processed by the ethtool method to configure a request queue with the same number of logical CPU cores as the target NUMA node, so that the target CPU cores can respond to network card interrupts.

[0077] In another possible implementation, the number of physical CPU cores of the target NUMA node and the parameters of the target network card can be processed by the ethtool method to configure a request queue with the same number of physical CPU cores as the target NUMA node, so that the target CPU cores can respond to network card interrupts.

[0078] The method described above, which configures the target number of interrupt request queues for the target network card based on the number of target CPU cores, can speed up the response of the target CPU cores to interrupt requests from the target network card.

[0079] In one embodiment, binding the target NUMA node to the target storage device based on the distance between the target NUMA node and each storage device in the server includes: binding the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device.

[0080] Optionally, the storage device management process can be an OSD, i.e., the process used by Ceph to manage storage devices.

[0081] In an optional embodiment, such as Figure 3 As shown, based on the distance between the target NUMA node and each of the storage devices, the target NUMA node is bound to the storage device management process corresponding to the target storage device, including the following steps:

[0082] Step 301: Based on the distance between the target NUMA node and each of the storage devices, determine the target storage device that is closest to the target NUMA node from among the storage devices, and determine the storage device management process corresponding to the target storage device.

[0083] Step 302: Bind the target NUMA node to the storage device management process corresponding to the target storage device.

[0084] In an optional embodiment, binding the target NUMA node to the storage device management process corresponding to the target storage device includes: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0085] In one possible implementation, an OSD can be identified first, and the following binding process can be performed on it. First, the storage device corresponding to the OSD is queried based on the lsblk method and algorithm. This storage device is the target storage device. Then, the NUMA node corresponding to the target storage device is determined. This NUMA node is the target NUMA node. It can be understood that the target storage device is the storage device that is closest to the target NUMA node among multiple storage devices. Then, the target NUMA node and the OSD are bound based on the numactl command, the service file corresponding to the OSD, and the algorithm.

[0086] In another possible implementation, an OSD can be identified first, and the following binding process can be performed on it. First, the storage device corresponding to the OSD is queried based on the lsblk method and algorithm. This storage device is the target storage device. Then, the NUMA node corresponding to the target storage device is determined. This NUMA node is the target NUMA node. It can be understood that the target storage device is the storage device that is closest to the target NUMA node among multiple storage devices. Then, the CPU core in the target NUMA node is bound to the OSD based on the numactl command, the service file corresponding to the OSD, and the algorithm.

[0087] In another possible implementation, an OSD can be identified first, and the following binding process can be performed on it. First, the storage device corresponding to the OSD is queried based on the lsblk method and algorithm. This storage device is the target storage device. Then, the NUMA node corresponding to the target storage device is determined. This NUMA node is the target NUMA node. It can be understood that the target storage device is the storage device that is closest to the target NUMA node among multiple storage devices. Then, based on the numactl command, the service file corresponding to the OSD, and the algorithm, the L3 cache in the target NUMA node is bound to the OSD. Here, the L3 cache is the L3 cache. The L3 cache can be understood as the CPU core / CPU core group corresponding to the L3 cache. That is, the binding is performed on the OSD in units of the CPU core / CPU core group corresponding to the L3 cache.

[0088] In an optional embodiment, after executing the above binding process, it is also necessary to reload the configuration file using the commands systemctl daemon-reload and systemctl restart ceph-osd@0, and restart the OSD, i.e., restart the storage device management process.

[0089] In an optional embodiment, the above binding process can be applied to all OSDs, and corresponding scripts can be written to improve efficiency.

[0090] The method described above, which binds the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device, can reduce cross-NUMA memory access and thus effectively improve performance.

[0091] In one embodiment, such as Figure 4 As shown, before obtaining the target optimization strategy, the method further includes the following steps:

[0092] Step 401: Obtain multiple candidate optimization strategies, and execute the optimization process based on each candidate optimization strategy. After executing the optimization process, determine the performance improvement parameters, which are used to characterize the improvement of the read and write performance of the storage server cluster where the server is located before and after the optimization process.

[0093] In one possible implementation, the various methods provided in the above embodiments can be combined to obtain multiple candidate optimization strategies. Then, the optimization process is executed based on each candidate optimization strategy. After the optimization process is executed, the client uses the FIO testing tool (rbd engine) to test the four basic indicators of 4k random read / write and 128k sequential read / write on the RBD of the server cluster under test. The optimized performance values ​​are recorded, and the performance improvement parameters are determined based on the baseline performance and the optimized performance values.

[0094] Step 402: Determine the target optimization strategy from the multiple candidate optimization strategies based on the performance improvement parameters corresponding to each candidate optimization strategy.

[0095] In one possible implementation, the performance improvement parameters of each candidate optimization strategy are compared, and the candidate optimization strategy with the largest performance improvement parameter is taken as the target optimization strategy.

[0096] The method described above for determining the target optimization strategy based on performance improvement parameters employs multiple strategies to execute the optimization process and selects the candidate optimization strategy with the largest performance improvement parameter as the target optimization strategy, thereby maximizing performance improvement.

[0097] In one embodiment, such as Figure 5 As shown, an optimization method is provided, including the following steps:

[0098] Step 501: Obtain the target optimization strategy and perform the optimization process on the server based on the target optimization strategy.

[0099] Step 502: Determine the binding script based on the type of the target network card.

[0100] Step 503: Use the binding script to bind the target network card to the target CPU core.

[0101] Step 504: Based on the number of target CPU cores, configure the target number of interrupt request queues for the target network card.

[0102] Step 505: Based on the distance between the target NUMA node and each of the storage devices, determine the target storage device that is closest to the target NUMA node from among the storage devices, and determine the storage device management process corresponding to the target storage device.

[0103] Step 506: Bind the target NUMA node to the storage device management process corresponding to the target storage device.

[0104] It should be noted that the inventors of this application have actually operated the method and obtained relevant data results. The relevant data results obtained by the inventors of this application through actual operation on a multi-NUMA CPU architecture server platform are shown in Table 1 below.

[0105] Table 1

[0106] Test metrics Basic performance After optimization Increase percentage 4k random reads (kIOPS) 641 800 24.80％ 4k random writes (kIOPS) 62 76.9 24.03％ 128k sequential read (MB / s) 2955 8234 178.65％ 128k sequential write (MB / s) 679 1449 113.40％

[0107] It should be noted that the optimization method provided in this application is not only applicable to Ceph block storage, but also to file storage and object storage interfaces.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this application also provides an optimization apparatus for implementing the optimization method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more optimization apparatus embodiments provided below can be found in the limitations of the optimization method described above, and will not be repeated here.

[0110] In one embodiment, such as Figure 6 As shown, an optimization device is provided, comprising: an acquisition module, wherein:

[0111] The acquisition module is used to acquire the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0112] In one embodiment, such as Figure 7 As shown, another optimization device is provided, which, in addition to the modules included in the optimization device 600, also includes a configuration module 602.

[0113] In one embodiment, the configuration module is configured to: configure a target number of interrupt request queues for the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0114] In one embodiment, the configuration module is specifically configured to: configure the target quantity to be the same as the target CPU core quantity; or, configure the target quantity to be less than the target CPU core quantity and the same as the logical CPU core quantity of the target NUMA node; or, configure the target quantity to be less than the target CPU core quantity and the same as the physical CPU core quantity of the target NUMA node.

[0115] In one embodiment, the acquisition module is specifically used to: bind the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each of the storage devices.

[0116] In one embodiment, the acquisition module is further configured to: determine the target storage device closest to the target NUMA node from among the storage devices based on the distance between the target NUMA node and each of the storage devices, and determine the storage device management process corresponding to the target storage device; and bind the target NUMA node to the storage device management process corresponding to the target storage device.

[0117] In one embodiment, the acquisition module is further configured to: directly bind the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0118] In one embodiment, the acquisition module is further configured to: bind the target network card to the target CPU core local to the target NUMA node based on the correspondence between the target network card and the target NUMA node; or, bind the target network card to the target CPU core contained in the entire CPU where the target NUMA node is located based on the correspondence between the target network card and the target NUMA node.

[0119] In one embodiment, the acquisition module is further specifically used to: determine a binding script based on the type of the target network card; and use the binding script to bind the target network card to the target CPU core.

[0120] In one embodiment, such as Figure 8 As shown, another optimization device is provided, which, in addition to the modules included in optimization device 600 and optimization device 700, also includes execution module 603.

[0121] In one embodiment, the execution module is configured to: obtain multiple candidate optimization strategies, execute the optimization process based on each candidate optimization strategy, and after executing the optimization process, determine performance improvement parameters, which are used to characterize the improvement of read and write performance of the storage server cluster where the server is located before and after the optimization process is executed; and determine the target optimization strategy from the multiple candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0122] Each module in the aforementioned optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0123] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. The computer program, when executed by the processor, implements an optimization method.

[0124] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0126] Obtain the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0127] In one embodiment, when the processor executes the computer program, it further performs the following steps: based on the number of target CPU cores, configure a target number of request queues for interrupt requests of the target network card, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0128] In one embodiment, based on the number of target CPU cores, a target number of interrupt request queues for the target network interface card is configured. When the processor executes the computer program, it further implements the following steps: configuring the target number to be the same as the number of target CPU cores; or configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node.

[0129] In one embodiment, the target NUMA node is bound to the target storage device based on the distance between the target NUMA node and each storage device in the server. When the processor executes the computer program, it further implements the following steps: binding the target NUMA node to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device.

[0130] In one embodiment, based on the distance between the target NUMA node and each of the storage devices, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the processor executes the computer program, it further implements the following steps: based on the distance between the target NUMA node and each of the storage devices, the target storage device closest to the target NUMA node is determined from among the storage devices, and the storage device management process corresponding to the target storage device is determined; the target NUMA node is bound to the storage device management process corresponding to the target storage device.

[0131] In one embodiment, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the processor executes the computer program, it further implements the following steps: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0132] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the NIC is bound to the target CPU core. When the processor executes the computer program, it further implements the following steps: based on the correspondence between the NIC and the target NUMA node, the NIC is bound to the target CPU core located on the local side of the target NUMA node; or, based on the correspondence between the NIC and the target NUMA node, the NIC is bound to the target CPU core contained in the entire CPU where the target NUMA node is located.

[0133] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core. When the processor executes the computer program, it further performs the following steps: determining the binding script according to the type of the target NIC; and using the binding script to bind the target NIC to the target CPU core.

[0134] In one embodiment, before obtaining the target optimization strategy, the processor executes the following steps when executing the computer program: obtaining multiple candidate optimization strategies, performing the optimization process based on each candidate optimization strategy, and after performing the optimization process, determining performance improvement parameters, which are used to characterize the improvement of the read and write performance of the storage server cluster where the server is located before and after the optimization process is performed; and determining the target optimization strategy from the multiple candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0135] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0136] Obtain the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0137] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: based on the number of target CPU cores, configuring a target number of request queues for interrupt requests of the target network card, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0138] In one embodiment, based on the number of target CPU cores, a target number of interrupt request queues for the target network interface card is configured. When the computer program is executed by the processor, the following steps are further implemented: configuring the target number to be the same as the number of target CPU cores; or configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node.

[0139] In one embodiment, the target NUMA node is bound to the target storage device based on the distance between the target NUMA node and each storage device in the server. When the computer program is executed by the processor, the following steps are also implemented: the target NUMA node is bound to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device.

[0140] In one embodiment, based on the distance between the target NUMA node and each of the storage devices, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the computer program is executed by the processor, the following steps are further implemented: based on the distance between the target NUMA node and each of the storage devices, the target storage device closest to the target NUMA node is determined from among the storage devices, and the storage device management process corresponding to the target storage device is determined; the target NUMA node is bound to the storage device management process corresponding to the target storage device.

[0141] In one embodiment, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the computer program is executed by the processor, the following steps are also performed: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0142] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core. When the computer program is executed by the processor, the following steps are further implemented: based on the correspondence between the target NIC and the target NUMA node, the target NIC is bound to the target CPU core local to the target NUMA node; or, based on the correspondence between the target NIC and the target NUMA node, the target NIC is bound to the target CPU core contained in the entire CPU where the target NUMA node is located.

[0143] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core. When the computer program is executed by the processor, the following steps are also performed: determining the binding script according to the type of the target NIC; and using the binding script to bind the target NIC to the target CPU core.

[0144] In one embodiment, before obtaining the target optimization strategy, the computer program, when executed by the processor, further performs the following steps: obtaining multiple candidate optimization strategies, performing the optimization process based on each candidate optimization strategy, and after performing the optimization process, determining performance improvement parameters, which are used to characterize the improvement of read and write performance of the storage server cluster where the server is located before and after the optimization process is performed; and determining the target optimization strategy from the multiple candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0145] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0146] Obtain the target optimization strategy and perform an optimization process on the server based on the target optimization strategy. The optimization process includes: determining the target NUMA node corresponding to the target network card in the server, binding the target network card to the target CPU core based on the correspondence between the target network card and the target NUMA node, and binding the target NUMA node to the target storage device according to the distance between the target NUMA node and each storage device in the server, so that the target CPU core can respond to the interrupt request of the target network card by accessing the target storage device.

[0147] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: based on the number of target CPU cores, configuring a target number of request queues for interrupt requests of the target network card, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues.

[0148] In one embodiment, based on the number of target CPU cores, a target number of interrupt request queues for the target network interface card is configured. When the computer program is executed by the processor, the following steps are further implemented: configuring the target number to be the same as the number of target CPU cores; or configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node.

[0149] In one embodiment, the target NUMA node is bound to the target storage device based on the distance between the target NUMA node and each storage device in the server. When the computer program is executed by the processor, the following steps are also implemented: the target NUMA node is bound to the storage device management process corresponding to the target storage device based on the distance between the target NUMA node and each storage device.

[0150] In one embodiment, based on the distance between the target NUMA node and each of the storage devices, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the computer program is executed by the processor, the following steps are further implemented: based on the distance between the target NUMA node and each of the storage devices, the target storage device closest to the target NUMA node is determined from among the storage devices, and the storage device management process corresponding to the target storage device is determined; the target NUMA node is bound to the storage device management process corresponding to the target storage device.

[0151] In one embodiment, the target NUMA node is bound to the storage device management process corresponding to the target storage device. When the computer program is executed by the processor, the following steps are also performed: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

[0152] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core. When the computer program is executed by the processor, the following steps are further implemented: based on the correspondence between the target NIC and the target NUMA node, the target NIC is bound to the target CPU core local to the target NUMA node; or, based on the correspondence between the target NIC and the target NUMA node, the target NIC is bound to the target CPU core contained in the entire CPU where the target NUMA node is located.

[0153] In one embodiment, based on the correspondence between the target network interface card (NIC) and the target NUMA node, the target NIC is bound to the target CPU core. When the computer program is executed by the processor, the following steps are also performed: determining the binding script according to the type of the target NIC; and using the binding script to bind the target NIC to the target CPU core.

[0154] In one embodiment, before obtaining the target optimization strategy, the computer program, when executed by the processor, further performs the following steps: obtaining multiple candidate optimization strategies, performing the optimization process based on each candidate optimization strategy, and after performing the optimization process, determining performance improvement parameters, which are used to characterize the improvement of read and write performance of the storage server cluster where the server is located before and after the optimization process is performed; and determining the target optimization strategy from the multiple candidate optimization strategies according to the performance improvement parameters corresponding to each candidate optimization strategy.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An optimization method, characterized in that, The method includes: An optimization process is performed based on multiple candidate optimization strategies to determine the performance improvement parameters corresponding to each candidate optimization strategy. Based on the performance improvement parameters corresponding to each candidate optimization strategy, a target optimization strategy is determined from the multiple candidate optimization strategies. The performance improvement parameters are used to characterize the improvement of the read and write performance of the storage server cluster where the server is located before and after the optimization process is executed. An optimization process is performed on the server based on the target optimization strategy, wherein the optimization process includes: The system identifies the target Non-Unified Memory Access Architecture (NUMA) node corresponding to the target network interface card (NIC) in the server. Based on the correspondence between the target NIC and the target NUMA node, it binds the target NIC to the target CPU core of the local CPU of the NUMA node, or binds the target NIC to the target CPU core contained in the entire CPU of the NUMA node. According to the distance between the target NUMA node and each storage device in the server, it identifies the target storage device closest to the target NUMA node and determines the storage device management process corresponding to the target storage device. Based on the service files and NUMA control commands of the storage device management process corresponding to the target storage device, it binds the target NUMA node to the storage device management process corresponding to the target storage device, so that the target CPU core responds to the interrupt request of the target NIC by accessing the target storage device. The method further includes: configuring a target number of request queues for interrupt requests of the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues; The step of configuring the target number of the interrupt request queue for the target network card based on the number of target CPU cores includes: configuring the target number to be the same as the number of target CPU cores; or, configuring the target number to be less than the number of target CPU cores and the same as the number of logical CPU cores of the target NUMA node; or, configuring the target number to be less than the number of target CPU cores and the same as the number of physical CPU cores of the target NUMA node. The step of binding the target NUMA node to the storage device management process corresponding to the target storage device includes: directly binding the target NUMA node to the storage device management process corresponding to the target storage device; or binding the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or binding the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

2. The method according to claim 1, characterized in that, The step of binding the target network interface card (NIC) to the target CPU core based on the correspondence between the target NIC and the target NUMA node includes: Determine the binding script based on the type of the target network interface card; The binding script is used to bind the target network card to the target CPU core.

3. The method according to claim 1, characterized in that, The step of determining the target optimization strategy from the plurality of candidate optimization strategies based on the performance improvement parameters corresponding to each of the candidate optimization strategies includes: The performance improvement parameters of each candidate optimization strategy are compared, and the candidate optimization strategy with the largest performance improvement parameter is selected as the target optimization strategy.

4. The method according to claim 1, characterized in that, Based on the number of target CPU cores, configure the target number of interrupt request queues for the target network interface card, including: Configure the target number to be the same as the target number of CPU cores.

5. The method according to claim 1, characterized in that, Based on the number of target CPU cores, configure the target number of interrupt request queues for the target network interface card, including: The target number is configured to be less than the target number of CPU cores, but the same as the number of logical CPU cores of the target NUMA node.

6. The method according to claim 1, characterized in that, Based on the number of target CPU cores, configure the target number of interrupt request queues for the target network interface card, including: The target number is configured to be less than the target number of CPU cores, but the same as the number of physical CPU cores of the target NUMA node.

7. An optimization device, characterized in that, The device includes: The acquisition module is used to perform an optimization process based on multiple candidate optimization strategies to determine the performance improvement parameters corresponding to each candidate optimization strategy, and to determine a target optimization strategy from the multiple candidate optimization strategies based on the performance improvement parameters. The performance improvement parameters are used to characterize the improvement in read and write performance of the storage server cluster where the server is located before and after the optimization process. The module then performs an optimization process on the server based on the target optimization strategy. This optimization process includes: determining the target non-uniform memory access architecture (NUMA) node corresponding to the target network interface card (NIC) in the server, and based on the correspondence between the target NIC and the target NUMA node, associating the target NIC with the target CPU local on the NUMA node. The core is bound, or the target network card is bound to the target CPU core contained in the entire CPU where the NUMA node is located; based on the distance between the target NUMA node and each storage device in the server, the target storage device closest to the target NUMA node is determined from among the storage devices, and the storage device management process corresponding to the target storage device is determined; based on the service file and non-uniform memory access architecture control commands of the storage device management process corresponding to the target storage device, the target NUMA node is bound to the storage device management process corresponding to the target storage device, so that the target CPU core responds to the interrupt request of the target network card by accessing the target storage device; The configuration module is used to configure a target number of interrupt request queues for the target network card based on the number of target CPU cores, so that the target CPU cores can respond to the interrupt requests of the target network card in parallel based on the target number of request queues; The configuration module is specifically configured to set the target number to be the same as the target number of CPU cores; or, set the target number to be less than the target number of CPU cores and the same as the number of logical CPU cores of the target NUMA node; or, set the target number to be less than the target number of CPU cores and the same as the number of physical CPU cores of the target NUMA node. The acquisition module is specifically used to directly bind the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the CPU core in the target NUMA node to the storage device management process corresponding to the target storage device; or, bind the L3 cache in the target NUMA node to the storage device management process corresponding to the target storage device.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images