An IO scheduling method and device

Abstract

The application discloses an IO scheduling method and device, and relates to the field of computer system structure. A specific embodiment of the method comprises the following steps: receiving an input / output (IO) request; wherein the IO request is a read data request or a write data request; when the IO request is the read data request, determining a target CPU corresponding to the read data request, so as to schedule an IO process or thread to a core of the target CPU to perform a data reading operation and return the read data; and when the IO request is the write data request, determining a target node corresponding to the write data request, so as to schedule the IO process or thread to write data into a CPU memory of the target node. The embodiment comprehensively considers read-write modes, IO sizes, performance characteristics and use conditions of each node / CPU, determines a mode of scheduling the IO process / thread, avoids scheduling errors, reduces the overhead of the IO process / thread in the back-and-forth scheduling on the CPU core, and thus improves the IO access performance of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer system structure, and in particular to an IO scheduling method and device. BACKGROUND

[0002] The distributed persistent memory system contains rich computing, network and storage resources, and these resources themselves have some performance differences. In the process of implementing the present application, the inventors found that at least the following problems exist in the prior art: the system cannot generally perceive the performance characteristics and other differences of different resources, and cannot optimally allocate appropriate resources for processes / threads performing IO (Input / Output) operations from a global perspective, and if the allocation is incorrect, it will seriously affect the access performance of the business. SUMMARY

[0003] Therefore, the embodiments of the present application provide an IO scheduling method and device, which can at least solve the problem that the prior art cannot perceive the performance characteristic differences of resources, resulting in unreasonable scheduling of IO processes / threads.

[0004] To achieve the above object, according to one aspect of the embodiments of the present application, an IO scheduling method is provided, comprising:

[0005] receiving a transmitted input / output (IO) request; wherein the IO request is a read data request or a write data request;

[0006] when the IO request is a read data request, determining a target CPU corresponding to the read data request, to schedule an IO process or thread to a core of the target CPU to perform a data reading operation, and returning the read data; or

[0007] when the IO request is a write data request, determining a target node corresponding to the write data request, to schedule an IO process or thread to write data into the CPU memory of the target node; wherein the write data request includes data to be written.

[0008] Optionally, the read data request includes a CPU memory logical access address.

[0009] The determining of the target CPU corresponding to the read data request, to schedule an IO process or thread to a core of the target CPU to perform a data reading operation, comprises:

[0010] locating to the target CPU according to the CPU memory logical access address, and judging whether the load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold;

[0011] according to the judgment result, scheduling an IO process or thread to a core of the corresponding CPU to perform a data reading operation.

[0012] Optionally, the scheduling the IO process or thread to a core of the target CPU for data reading operation according to the judgment result comprises:

[0013] if the load utilization rate of the target CPU is less than the preset load utilization rate threshold, scheduling the IO process or thread to a core of the target CPU for data reading operation; or

[0014] if the load utilization rate of the target CPU is greater than or equal to the preset load utilization rate threshold, scheduling the IO process or thread to a core of any idle CPU other than the target CPU for data reading operation by accessing the memory space of the target CPU remotely through the idle CPU; wherein the target CPU and the idle CPU are located in the same node.

[0015] Optionally, the locating the target CPU according to the CPU memory logical access address comprises:

[0016] processing the CPU memory logical access address using a consistent hashing algorithm to obtain a hash value, and querying a node corresponding to the hash value on a preset hash ring;

[0017] determining an actual memory access address corresponding to the CPU memory logical access address to locate the target CPU in the node according to the actual memory access address.

[0018] Optionally, the locating the target CPU in the node according to the actual memory access address comprises:

[0019] in response to all the data to be read being stored in a single node, locating the target CPU in the single node according to the actual memory access address; or

[0020] in response to all the data to be read being stored across nodes, counting the amount or size of data to be read in each node to filter out a first node with the largest amount or size of data, and locating the target CPU in the first node according to the corresponding actual memory access address;

[0021] after reading the data from the first node, filtering out a second node with the second largest amount or size of data, and repeating the operations of locating the target CPU, judging the load utilization rate, and scheduling the IO process or thread to perform data reading operation until all the data to be read is polled.

[0022] Optionally, the data reading request further comprises an IO size, and the IO size is the total size of the data to be read.

[0023] when all the data to be read is stored in a single node, the judging whether the load utilization rate of the target CPU is greater than or equal to the preset load utilization rate threshold comprises:

[0024] determining whether the IO size is greater than or equal to a preset size threshold, if less than, scheduling the IO process or thread to a core of the target CPU to perform a data reading operation; or

[0025] if greater than or equal to, determining whether a load utilization of the target CPU is greater than or equal to a preset load utilization threshold.

[0026] Optionally, the method further comprises:

[0027] acquiring a distribution of the current IO process or thread on the core of the CPU;

[0028] if the current IO process or thread is located on the core of the target CPU, scheduling the IO process or thread to the core of the target CPU to perform a data reading operation, otherwise, scheduling the IO process or thread to a core of any idle CPU to perform a remote data reading operation.

[0029] Optionally, the write data request comprises an IO size, an identification of the target node, and data to be written; wherein the IO size is a total size of the data to be written.

[0030] The scheduling of the IO process or thread to write data into the CPU memory of the target node comprises:

[0031] according to the identification of the target node, acquiring a current memory available space of the target node, and determining whether the memory available space is greater than or equal to the IO size;

[0032] according to the determination result, scheduling the IO process or thread to write data into the CPU memory of the corresponding node.

[0033] Optionally, the scheduling of the IO process or thread to write data into the CPU memory of the corresponding node according to the determination result comprises:

[0034] if greater than or equal to, scheduling the IO process or thread to write data into the CPU memory of the target node; wherein a node comprises a plurality of CPUs; or

[0035] if less than, determining a target CPU having a maximum memory available space under the target node, scheduling the IO process or thread to write data of the maximum memory available space into the memory of the target CPU, and writing the remaining data into the CPU memory of other nodes.

[0036] Optionally, the writing of the remaining data into the CPU memory of other nodes comprises:

[0037] selecting a node having a maximum current memory available space from other nodes, and determining whether the current memory available space is greater than or equal to an IO size of the remaining data.

[0038] If greater than or equal to, the IO process or thread is scheduled to write the remaining data into the CPU memory of the one node; or

[0039] If less than, the data of the current available memory space is written into the one node, and the above node screening and space size comparison operations are repeated to write the remaining data into the CPU memory of multiple nodes.

[0040] Optionally, the node comprises persistent memory;

[0041] After the data is written into the CPU memory of the corresponding node according to the judgment result, the method further comprises:

[0042] The persistent value located on the flag bit is obtained from the attribute of the data;

[0043] If the persistent value is a preset value, the current available space of the persistent memory of the node where the data is located is obtained, and it is judged whether the current available space is greater than or equal to the IO size of the data;

[0044] If greater than or equal to, the data in the CPU memory of the node is written into the persistent memory in an asynchronous manner; or

[0045] If less than, the data of the current available space in the CPU memory of the node is written into the persistent memory, and the remaining data is written into the persistent memory of other nodes.

[0046] To achieve the above object, according to another aspect of the embodiment of the present application, an IO scheduling device is provided, comprising:

[0047] A receiving module is configured to receive a transmitted input / output (IO) request; wherein the IO request is a read data request or a write data request;

[0048] An IO process or thread scheduling submodule is configured to, when the IO request is a read data request, determine a target CPU corresponding to the read data request, to schedule an IO process or thread to perform a data reading operation on a core of the target CPU, and return the read data; or

[0049] When the IO request is a write data request, a target node corresponding to the write data request is determined, to schedule an IO process or thread to write data into the CPU memory of the target node; wherein the write data request comprises data to be written.

[0050] Optionally, the read data request comprises a logical access address of the CPU memory;

[0051] The device further comprises a locating module configured to locate the target CPU according to the CPU memory logical access address;

[0052] a CPU load statistics submodule configured to obtain a load utilization of the target CPU;

[0053] The IO process or thread scheduling submodule is configured to determine whether the load utilization of the target CPU is greater than or equal to a preset load utilization threshold;

[0054] If less than, the IO process or thread is scheduled to the core of the target CPU for data reading operation; or

[0055] If greater than or equal to, the IO process or thread is scheduled to the core of any other idle CPU to access the memory space of the target CPU remotely through the idle CPU for data reading operation; wherein the target CPU and the other idle CPU are located in the same node.

[0056] Optionally, the locating module is configured to:

[0057] process the CPU memory logical access address using a consistent hashing algorithm to obtain a hash value, and query a node corresponding to the hash value on a preset hash ring;

[0058] determine an actual memory access address corresponding to the CPU memory logical access address to locate the target CPU in the node according to the actual memory access address.

[0059] Optionally, the locating module is configured to:

[0060] in response to all the data to be read being stored in a single node, locate the target CPU in the single node according to the actual memory access address; or

[0061] in response to all the data to be read being stored across nodes, count the amount or size of the data to be read in each node to filter out a first node with the largest amount or size of data, and locate the target CPU in the first node according to the corresponding actual memory access address;

[0062] after reading the data from the first node, filter out a second node with the second largest amount or size of data, and repeat the operations of locating the target CPU, determining the load utilization, and scheduling the IO process or thread to perform data reading operation until all the data to be read is polled.

[0063] Optionally, the data reading request further comprises an IO size, and the IO size is a total size of the data to be read.

[0064] When a single node stores all data to be read, the IO process or thread scheduling submodule is used for:

[0065] determining whether the IO size is greater than or equal to a preset size threshold, if less, scheduling the IO process or thread to the core of the target CPU for data reading operation; or

[0066] if greater than or equal to, determining whether the load utilization of the target CPU is greater than or equal to a preset load utilization threshold.

[0067] Optionally, the IO process or thread scheduling submodule is further used for:

[0068] obtaining the distribution of the current IO process or thread on the core of the CPU;

[0069] if the current IO process or thread is located on the core of the target CPU, scheduling the IO process or thread to the core of the target CPU for data reading operation, otherwise, scheduling the IO process or thread to the core of any idle CPU for remote data reading operation.

[0070] Optionally, the write data request comprises an IO size, an identification of a target node and data to be written; wherein the IO size is the total size of the data to be written.

[0071] The device further comprises a DRAM space statistics submodule, configured to obtain the current memory available space of the target node according to the identification of the target node.

[0072] The IO process or thread scheduling submodule is used for:

[0073] determining whether the memory available space is greater than or equal to the IO size;

[0074] if greater than or equal to, scheduling the IO process or thread to write data into the CPU memory of the target node; wherein a node comprises a plurality of CPUs; or

[0075] if less, determining a target CPU with the largest memory available space under the target node, scheduling the IO process or thread to write the data with the largest memory available space into the memory of the target CPU, and writing the remaining data into the CPU memory of other nodes.

[0076] Optionally, the IO process or thread scheduling submodule is used for:

[0077] selecting a node with the largest current memory available space from other nodes, and determining whether the current memory available space is greater than or equal to the IO size of the remaining data;

[0078] If greater than or equal to, the IO process or thread schedules the remaining data to be written into the CPU memory of the one node; or

[0079] If less than, the data of the current available memory space is written into the one node, and the above node screening and space size comparison operations are repeated to write the remaining data into the CPU memory of multiple nodes.

[0080] Optionally, the node comprises a persistent memory PM.

[0081] The device further comprises a PM space statistics submodule, configured to:

[0082] From the attribute of the data, a persistent value located on a flag bit is obtained.

[0083] If the persistent value is a preset value, a current available space of the persistent memory of the node where the data is located is obtained.

[0084] The IO process or thread scheduling submodule is configured to:

[0085] Judge whether the current available space is greater than or equal to the IO size of the data.

[0086] If greater than or equal to, the data in the CPU memory of the node is written into the persistent memory in an asynchronous manner; or

[0087] If less than, the data of the current available space in the CPU memory of the node is written into the persistent memory, and the remaining data is written into the persistent memory of other nodes.

[0088] To achieve the above object, according to another aspect of the embodiments of the present application, an IO scheduling electronic device is provided.

[0089] The electronic device of the embodiments of the present application comprises one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned IO scheduling methods.

[0090] To achieve the above object, according to another aspect of the embodiments of the present application, a computer readable medium having a computer program stored thereon is provided, when the program is executed by a processor, the program implements any of the above-mentioned IO scheduling methods.

[0091] To achieve the above object, according to another aspect of the embodiments of the present application, a computing program product is provided. The computing program product of the embodiments of the present application comprises a computer program, when the program is executed by a processor, the program implements the IO scheduling method provided by the embodiments of the present application.

[0092] According to the scheme provided in the application, one embodiment in the application has the following advantages or beneficial effects: according to the read-write mode and the performance characteristics of each hardware resource, appropriate computing, storage and network resources are allocated to the IO process / thread, so as to reduce the back-and-forth call overhead on different CPU cores and improve the overall access performance of the system. Meanwhile, in view of the poor PM write performance, data is preferentially written into the node memory, and then written into the PM in an asynchronous persistent manner, so as to reduce the write latency.

[0093] Further effects of the above-mentioned non-conventional optional mode will be described in the following in conjunction with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0094] The accompanying drawings are used to better understand the application and do not constitute an improper limitation on the application. Among them:

[0095] Figure 1 is a schematic diagram of a node in a NUMA structure;

[0096] Fig. 2(a) is a schematic diagram of the structure of a storage node node;

[0097] Fig. 2(b) is a schematic diagram of the structure of a distributed persistent memory system setting;

[0098] Figure 3 is a main flow diagram of an IO scheduling method according to an embodiment of the application;

[0099] Figure 4 is a flow diagram of a read data IO scheduling method according to an embodiment of the application;

[0100] Figure 5 is a flow diagram of a write data IO scheduling method according to an embodiment of the application;

[0101] Figure 6 is a flow diagram of an optional write data IO scheduling method according to an embodiment of the application;

[0102] Figure 7 is a main module diagram of an IO scheduling device according to an embodiment of the application;

[0103] Figure 8 is an exemplary system architecture diagram to which an embodiment of the application can be applied;

[0104] Figure 9 is a structural diagram of a computer system of a mobile device or a server suitable for implementing an embodiment of the application. DETAILED DESCRIPTION

[0105] Exemplary embodiments of the present application are described herein below with reference to the accompanying drawings, in which various specific details are set forth to assist in providing a thorough understanding of various examples. Those skilled in the art will understand, however, that the embodiments described and pictured herein are presented by way of example only, and are not intended to limit the scope of the application. Accordingly, the claims should not be limited to the various embodiments described and pictured herein.

[0106] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The acquisition, storage, use, processing, etc. of data in the technical solutions of the present application comply with the relevant provisions of national laws and regulations.

[0107] The distributed persistent memory system contains rich computing, network and storage resources, and these resources themselves have some performance differences. Here, the prior art is described in detail:

[0108] 1) The CPU in the NUMA (Non-Uniform Memory Access) structure has lower latency in accessing local memory than in accessing remote memory

[0109] Existing processors usually adopt the NUMA structure, and multiple CPUs are usually installed on the motherboard. Each CPU has its own DRAM (Dynamic Random Access Memory) memory at the physical level, which together forms a NUMA node. Each NUMA node accesses the DRAM memory of other CPUs through an IC (Interconnect Network).

[0110] Within the same node, CPU access to its own DRAM memory is usually referred to as accessing local memory, and access to the memory of other CPUs is referred to as accessing remote memory. The latency of each CPU accessing local DRAM is much lower than that of accessing remote DRAM. Referring to Figure 1 The NUMA structure contains multiple nodes, each CPU contains 4 cores and has independent LLC (Logical Link Control), local DRAM memory and IO interface, etc., and the MC in the figure is the Memory Controller. The speed of each node accessing the DRAM memory of other nodes through the IC is much lower than that of accessing local DRAM memory.

[0111] 2) Compared with ordinary Ethernet, RDMA (Remote Direct Memory Access) network has lower inter-node access latency and higher transmission bandwidth, and has the function of reducing CPU overhead.

[0112] 3) PM (Persistent Memory) as a new type of memory has been applied in distributed systems. Compared with DRAM memory, PM has higher access latency and lower bandwidth than DRAM, but the data is not lost after power failure. The read latency of persistent memory is lower than the write latency, and the read bandwidth is higher than the write bandwidth, that is, the read and write operations of the persistent memory itself have asymmetric characteristics.

[0113] It should be noted that the data not lost after power failure is a popular description opposite to the data lost after power failure. Storage is divided into volatile storage and non-volatile storage, such as commonly used magnetic disks and solid state disks, which belong to non-volatile storage devices and can save data for a long time, even if the system is powered off. The data stored in the DRAM memory will be lost. Whether the data is lost after power failure mainly refers to whether the storage device needs additional power support when saving data, such as DRAM memory, the data will be lost when the system is not continuously powered on, while the disk, solid state disk and persistent memory device will not be lost.

[0114] The current distributed system has certain limitations when allocating computing, storage and network resources to IO processes / threads. 1) When scheduling IO processes / threads within the node, time slice rotation, first-in-first-out and other strategies are usually used, so the IO processes / threads will constantly migrate on each CPU, which will prevent the IO processes / threads from utilizing the shared data of the CPU to improve access performance, and may also cause the IO processes / threads running on the local CPU to access the memory data of the remote CPU and affect the performance. 2) Between nodes, the existing distributed system is usually designed for volatile DRAM memory, which cannot perceive the performance characteristics of persistent memory, so it cannot optimally allocate computing, storage and network resources to IO processes / threads.

[0115] Referring to Fig. 2(a), a structure diagram of a storage node Node provided by the present scheme is shown, which includes a storage, a processor, a memory, and an interface. The memory is divided into volatile memory and non-volatile memory, and the interface is divided into a network interface, a user interface and an external device interface:

[0116] 1) The storage is used to store system data, application programs and user data, etc., and is the main storage device of the entire storage node, which has the characteristics of large capacity and low performance.

[0117] 2) Processor, designed with NUMA structure, usually installs multiple CPUs, performs main computing functions on storage nodes, and migrates freely between different processors or processor cores by scheduling IO processes / threads.

[0118] 3) Memory

[0119] ① Volatile memory: the main memory for program running, with high performance, small capacity, and data loss on power failure.

[0120] ② Non-volatile memory: performance close to volatile memory and no data loss on power failure, capacity larger than non-volatile memory, which can be used as memory and persistent storage device, but has disadvantages such as asymmetric read and write performance, specifically, lower read latency and higher read bandwidth than write bandwidth.

[0121] It should be noted that the node in the present scheme uses non-volatile memory (persistent memory PM, NVM (non-volatile memory)) which is a kind of device, and has good access performance compared with the existing large number of magnetic disks and solid state disks. Since the non-volatile memory is connected to the memory bus, the CPUs of different nodes can access each other's non-volatile memory data directly through the RDMA high-speed network.

[0122] 4) Interface

[0123] ① Network interface: used to connect high-speed communication networks such as RDMA, with high bandwidth and low latency.

[0124] ② User interface: a medium for users and storage systems to interact and exchange information, such as mouse, keyboard, display.

[0125] ③ External device interface: used to connect external devices, reduce processor overhead, and provide additional functions.

[0126] The structure of the distributed persistent memory system according to the present application is shown in Figure 2(b):

[0127] 1) User accesses the distributed persistent memory system through the network;

[0128] 2) The distributed persistent memory system is composed of multiple storage nodes, and the high-performance RDMA network is used for interconnection between different storage nodes and high-availability architecture deployment, that is, a single network switch failure does not affect the normal network communication access between nodes. The internal node usually refers to the access between different CPUs, which are connected by QPI (Quick Path Interconnect, a kind of architecture that can realize direct interconnection between chips) bus.

[0129] Referring to Figure 3 , a main flowchart of an IO scheduling method provided by an embodiment of the present application is shown, including the following steps:

[0130] S301: receiving a transmitted input / output (IO) request; wherein the IO request is a read data request or a write data request;

[0131] S302: when the IO request is a read data request, determining a target CPU corresponding to the read data request, to schedule an IO process or thread to a core of the target CPU to perform a data reading operation, and returning read data;

[0132] S303: when the IO request is a write data request, determining a target node corresponding to the write data request, to schedule an IO process or thread to write data into a CPU memory of the target node; wherein the write data request includes data to be written.

[0133] In the above embodiment, for steps S301-S303, the present application is mainly applied to a distributed persistent memory system, and is divided into a read data scenario and a write data scenario, so that the received IO requests are divided into read data requests and write data requests.

[0134] Embodiment one is for a read data scenario, and referring to Figure 4 , a specific process is shown:

[0135] S401: receiving a transmitted read data request; wherein the read data request includes a CPU memory logical access address and an IO size, and the IO size is a total size of data to be read;

[0136] S402: processing the CPU memory logical access address using a consistent hashing algorithm to obtain a hash value, and querying a node corresponding to the hash value on a preset hash ring;

[0137] S403: determining an actual memory access address corresponding to the CPU memory logical access address;

[0138] S404: in response to a single node storing all data to be read, locating a target CPU in the single node according to the actual memory access address, and judging whether the IO size is greater than or equal to a preset size threshold;

[0139] S405: if less, scheduling an IO process or thread to a core of the target CPU to perform a data reading operation;

[0140] S406: if greater than or equal to, judging whether a load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold;

[0141] S407: In response to storing all the data to be read across the nodes, counting the amount or size of data to be read in each node to screen out a first node with the largest amount or size of data, and locating the target CPU in the first node according to the corresponding actual memory access address;

[0142] S408: Judging whether the load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold;

[0143] S409: If less, scheduling an IO process or thread to a core of the target CPU for data reading operation; or

[0144] S410: If greater than or equal to, scheduling an IO process or thread to a core of any other idle CPU to access the memory space of the target CPU remotely through the any other idle CPU for data reading operation; wherein the target CPU and the any other idle CPU are located in the same node;

[0145] S411: After reading data from the first node, screening out a second node with the second largest amount or size of data, repeating the above target CPU locating, load utilization rate judging and IO process or thread scheduling operation to perform data reading operation until all the data to be read is polled.

[0146] In the embodiment, for step S401, the data size requested by different application programs is different, such as opening a text document with little content and opening a database document with large size, and the IO size requested by the application programs is different.

[0147] For step S402, considering that some nodes in the distributed system may fail and new nodes may be dynamically added, a consistent Hash algorithm is usually used to determine in which node the data is stored in the distributed system. The consistent Hash algorithm organizes the entire Hash value space into a virtual ring, i.e., a preset Hash ring, and the algorithm locates data access to the corresponding node: the data key (CPU memory logical access address in the scheme) is calculated using the same function Hash to determine the position of the data on the ring, and the first node encountered along the ring clockwise is the node to which the data should be located.

[0148] For step S403, the actual access address of the CPU memory is provided in the node, and the CPU memory logical access address and the actual memory access address have a corresponding relationship. The actual memory access address can be used to locate a specific CPU, such as CPU1.

[0149] The scheme only sets one IO process / thread. The distribution of the current IO process / thread can be considered to obtain its distribution on the core of the CPU in the node. If it is located on the core of the target CPU, the IO process or thread can be directly scheduled to the core of the target CPU for data reading operation subsequently, otherwise, the IO process or thread needs to be scheduled to any idle core of the CPU for remote data reading operation. For example, a text document is stored in the memory of CPU1, and the IO process / thread is currently scheduled to the core of CPU0 for execution. Because the IO of the text document itself is small, although the performance of the IO process / thread on CPU0 in accessing the memory data on CPU1 is poor, compared with the scheduling overhead of scheduling the IO process / thread to CPU1 for execution, this performance loss can be ignored. However, if the IO process / thread is currently scheduled to the core of CPU1 for execution, the reading operation can be directly performed.

[0150] For steps S404-S406, the processor principle is the same, and the relevant information can be seen by using a command, such as the numactl-s command under Linux. The memory in the whole system has a unified coding address, and the data storage in which NUMA CPU memory can be judged according to the coding address, that is, the target CPU. When the data to be read is completely stored in a single node, the CPU storing the data in the single node is first located, and then the next operation is selected according to the data IO size. It is judged whether the IO size is greater than or equal to the preset size threshold, such as 64K;

[0151] 1. When the IO size is greater than or equal to the preset size threshold, the IO process / thread is scheduled to the core of the NUMA CPU where the data is located, so as to trigger the IO process / thread to perform data reading operation;

[0152] 2. When the IO size is less than the preset size threshold, it is judged whether the utilization of the NUMA CPU where the data is located is greater than or equal to the preset utilization threshold, which can be set to 80%.

[0153] For steps S407 and S408, S411, the data is dispersedly stored because the target node storage resource is insufficient when writing, and the available space of other nodes needs to be found for storage. Corresponding to data reading, the data is stored across nodes. First, the amount or size of the data to be read in each node is counted to filter out the first node with the largest amount or size of data. The target CPU in the first node is determined, and it is judged whether the load utilization of the target CPU is greater than or equal to the preset load utilization threshold.

[0154] For example, Node0-data volume is 100, data size is 1G, Node1-data volume is 50, data size is 2G, Node2-data volume is 50, data size is 500M, if Node0-Node1-Node2 is obtained according to data volume sorting, data size is sorted when data volume is the same; but if Node1-Node0-Node2 is obtained according to data size sorting, data volume is sorted when data size is the same. The arrangement of NODE is different according to different ways. After reading data from the first node is completed, data from the remaining nodes needs to be read until all data is read.

[0155] For steps S409 and S410, if the utilization rate of the NUMA CPU where the data is located is not greater than or equal to the preset load utilization threshold, the IO process / thread is also scheduled to the core of the NUMA CPU where the data is located to trigger the IO process / thread to perform data reading operation; otherwise, the IO process / thread is scheduled to run on the core of any other idle NUMA CPU to access the data located NUMA CPU for remote reading operation.

[0156] For example, the data is stored in the memory of CPU0, when the data is small and the load of CPU0 is high, the IO process / thread may not run on CPU0 for a long time, and the access delay of CPU1 and CPU2 to the data of CPU0 is the same, so the IO process / thread is scheduled to any other idle CPU. In addition, the CPU is only the specific running place of the process / thread, and the IO process / thread has recorded the storage address of the data, and only the CPU for reading is replaced here, and the address of the data will not change.

[0157] The method provided by the embodiment one adopts different ways to schedule the IO process / thread according to the utilization rate of the NUMA CPU where the data is located, and the IO process / thread can quickly complete the reading operation regardless of whether the utilization rate of the NUMA CPU exceeds the limit, when the data is large and the IO process / thread is currently located on the core of other CPU, the IO process / thread can also be guaranteed to continuously read data on the same CPU in a way with minimum overhead, and the frequent migration of the IO process / thread is avoided.

[0158] Embodiment two is for a write data scenario, which is specifically described in Figure 5

[0159] S501: receiving a transmitted write data request; wherein the write data request includes IO size, identification of target node, data to be written, and IO size is the total size of data to be written;

[0160] ​S502: According to the identifier of the target node, the current memory available space of the target node is obtained, and it is judged whether the memory available space is greater than or equal to the IO size;

[0161] S503: If greater than or equal to, the IO process or thread is scheduled to write data into the CPU memory of the target node; wherein a node includes multiple CPUs;

[0162] S504: If less than, the target CPU with the maximum memory available space under the target node is determined, the IO process or thread is scheduled to write the data with the maximum memory available space into the memory of the target CPU, and the IO size of the remaining data is determined;

[0163] S505: One node with the maximum current memory available space is selected from other nodes, and it is judged whether the current memory available space is greater than or equal to the IO size of the remaining data;

[0164] S506: If greater than or equal to, the IO process or thread is scheduled to write the remaining data into the CPU memory of the one node;

[0165] S507: If less than, the data with the current memory available space is written into the one node, and the above node selection and space size comparison operations are repeated to write the remaining data into the CPU memory of multiple nodes.

[0166] In this embodiment, for steps S501-S503, the distributed persistent memory system receives the write data request transmitted by the application program, the write data request contains the IO size, the identifier of the target node, and the data to be written. According to the identifier of the target node, the memory available space of the target node is obtained, and it is judged whether the memory available space is greater than or equal to the IO size.

[0167] For example, the memory available spaces of node0, node1 and node2 are 2G, 15G and 21G respectively, the target node identifier transmitted by the application program is node2, and since 21G of node2>IO size 20G, it indicates that the IO process / thread can be scheduled to write all the data into the CPU memory of node2, but if the available space of node2 is 18G, it indicates that the data to be written this time needs to be sliced and stored, that is, cross-node storage.

[0168] For step S504, if the memory of a single node is insufficient, for the case of needing to slice and store, the remaining data can be stored in other nodes as much as possible after being stored in the node, such as in the above example, 18G of data is written into node2, and 2G of remaining data is written into other nodes node0 and node1 using RDMA network.

[0169] In this mode, after node2 stores the data, if there is still data to be written subsequently, since there is no available space, the data will be directly stored on other nodes. The present scheme preferably only considers the target CPU with the maximum available memory space under node2, such as CPU0-5G, CPU1-6G, and CPU2-7G. 7G of data is written to CPU2, and the remaining 13G of data is written to other nodes node0 and node1. Each NUMA CPU has its own local memory, and the available space of the local memory is recorded.

[0170] For steps S505-S507, there are many nodes in the system, and there are many ways to determine other nodes, such as random selection. For example, node0 is randomly selected, 2G of data is written to node0, and 11G of data is written to node1. Alternatively, the other nodes are sorted according to the current memory available space size, and the node with the maximum current memory available space is determined, such as 15G of node1 > 2G of node0. If 15G of node1 is greater than or equal to the IO size of the remaining data, it means that the remaining data can be written to node1.

[0171] However, if the current memory available space of node1 is 11G, which is less than the IO size of the remaining data, 11G of data can be written to node1, and there is still 2G of data remaining. Then, the node with the maximum current memory available space is selected from the remaining nodes to write 2G of data. In actual operation, there may be a situation that other nodes are insufficient, such as the memory available spaces of node0, node1, and node2 are 2G, 5G, and 18G respectively. After 7G of data is written to CPU2 under node2, there are still 13G of data remaining, which can only be stored in 7G of node0 and node1. At this time, node2 can be considered to achieve the purpose of writing all data.

[0172] The method provided by the second embodiment mainly refers to the write operation. When the storage resources of a node are insufficient to completely store all data to be written, the data needs to be stored across nodes, and the corresponding read data also needs to be read across nodes.

[0173] The method provided by the embodiment of the present application is mainly applied to the read-write mode of a distributed persistent memory system, and is mainly aimed at processing business access performance. The IO size, performance characteristics and use cases of each node / CPU are comprehensively considered to determine the scheduling mode of the IO process / thread, avoid scheduling errors, and reduce the overhead of the IO process / thread moving back and forth on the CPU core, so as to improve the processing performance of the system.

[0174] Referring to Figure 6, shows an optional IO scheduling method flowchart according to an embodiment of the application, comprising the following steps:

[0175] S601: From the attribute of the data, obtain the persistent value located on the flag bit;

[0176] S602: If the persistent value is a preset value, obtain the current available space of the persistent memory of the node where the data is located, and determine whether the current available space is greater than or equal to the IO size of the data;

[0177] S603: If it is greater than or equal to, use an asynchronous mode to write the data in the node CPU memory into the persistent memory;

[0178] S604: If it is less than, write the data in the current available space of the node CPU memory into the persistent memory, and write the remaining data into the persistent memory of other nodes.

[0179] In the above embodiment, for steps S601-S604, since the PM write delay is high and the write bandwidth is low, directly writing data into PM will result in poor write performance, affecting the overall performance of the business, therefore, data is preferentially written into the node CPU memory, and then the data in the CPU memory is written into the PM.

[0180] When data is written, it will be set in the attribute flag bit whether it needs to be persisted. If it needs to be persisted, determine whether the current available space size of the persistent memory PM of the node where the data is located is greater than or equal to the IO size of the data to be persisted:

[0181] 1) If it is greater than or equal to, when the NUMA CPU local PM space of the IO process / thread belongs to is sufficient, use an asynchronous mode to directly write the data in the memory into the local PM to complete the persistent operation.

[0182] 2) If it is less than, after writing the data in the available space size of the node CPU memory into the local persistent memory PM, use RDMA network to transmit the remaining data to other node PMs for persistent storage.

[0183] The distributed system has the ability to integrate global resources, and a single node storage space is insufficient but the entire distributed system space is sufficient. There is a module in the entire distributed system that is responsible for statistics of the use of each resource. In addition, PM is different from DRAM, and if DRAM data needs to be persisted, it needs to be written into PM, and deleting data in PM is similar to deleting data on a disk or solid state disk.

[0184] The method provided by the above embodiment, when IO data needs to be persisted, preferentially utilizes the high performance of DRAM, writes data into the CPU memory of a node to complete a write operation, and then writes the data into PM asynchronously, so that the write bottleneck of PM is effectively avoided.

[0185] Referring to Figure 7 , a main module schematic diagram of an IO scheduling device 700 provided by an embodiment of the application is shown, which comprises:

[0186] A receiving module 701 is configured to receive a transmitted input / output (IO) request; wherein the IO request is a read data request or a write data request.

[0187] An IO process or thread scheduling submodule 702 is configured to, when the IO request is a read data request, determine a target CPU corresponding to the read data request, to schedule an IO process or thread to a core of the target CPU to perform a data reading operation and return the read data; or

[0188] When the IO request is a write data request, a target node corresponding to the write data request is determined, to schedule an IO process or thread to write data into the CPU memory of the target node; wherein the write data request comprises data to be written.

[0189] In the device, the read data request comprises a CPU memory logical access address.

[0190] The device further comprises a positioning module configured to position to the target CPU according to the CPU memory logical access address.

[0191] A CPU load statistics submodule is configured to acquire a load utilization rate of the target CPU.

[0192] The IO process or thread scheduling submodule 702 is configured to judge whether the load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold.

[0193] According to the judgment result, an IO process or thread is scheduled to a core of a corresponding CPU to perform a data reading operation.

[0194] In the device, the IO process or thread scheduling submodule 702 is configured to:

[0195] If less than, an IO process or thread is scheduled to a core of the target CPU to perform a data reading operation; or

[0196] If greater than or equal to, the IO process or thread is scheduled to the core of any other idle CPU to perform a data read operation by remote access to the target CPU memory space through the any other idle CPU; wherein the target CPU and the any other idle CPU are located under the same node.

[0197] In the device, the positioning module is configured to:

[0198] process the CPU memory logical access address to obtain a hash value, and query a node corresponding to the hash value on a preset hash ring;

[0199] determine an actual memory access address corresponding to the CPU memory logical access address, and locate the target CPU in the node according to the actual memory access address.

[0200] In the device, the positioning module is configured to:

[0201] in response to all the data to be read being stored in a single node, locate the target CPU in the single node according to the actual memory access address; or

[0202] in response to all the data to be read being stored across nodes, count the amount or size of the data to be read in each node to filter out a first node with the largest amount or size of data, and locate the target CPU in the first node according to the corresponding actual memory access address;

[0203] after reading the data from the first node, filter out a second node with the second largest amount or size of data, and repeat the operations of locating the target CPU, judging the load utilization rate, and scheduling the IO process or thread to perform the data read operation until all the data to be read is polled.

[0204] In the device, the read data request further includes an IO size, and the IO size is a total size of the data to be read.

[0205] When all the data to be read is stored in a single node, the IO process or thread scheduling submodule 702 is configured to:

[0206] determine whether the IO size is greater than or equal to a preset size threshold, if less than, schedule the IO process or thread to the core of the target CPU to perform a data read operation; or

[0207] if greater than or equal to, determine whether the load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold.

[0208] In the device, the IO process or thread scheduling submodule 702 is further configured to:

[0209] acquire the distribution of the current IO process or thread on the core of the CPU;

[0210] If the current IO process or thread is located on the core of the target CPU, the IO process or thread is scheduled to the core of the target CPU for data reading operation, otherwise, the IO process or thread is scheduled to the core of any idle CPU for remote data reading operation.

[0211] In the embodiment of the application, the write data request comprises an IO size, an identification of a target node and data to be written; wherein the IO size is the total size of the data to be written.

[0212] The device further comprises a DRAM space statistics submodule for acquiring the current memory available space of the target node according to the identification of the target node.

[0213] The IO process or thread scheduling submodule 702 is configured to:

[0214] determine whether the memory available space is greater than or equal to the IO size;

[0215] According to the determination result, the IO process or thread is scheduled to write data into the CPU memory of the corresponding node.

[0216] In the embodiment of the application, the IO process or thread scheduling submodule 702 is configured to:

[0217] If greater than or equal to, the IO process or thread is scheduled to write data into the CPU memory of the target node; wherein a node comprises a plurality of CPUs; or

[0218] If less than, a target CPU with the largest memory available space under the target node is determined, the IO process or thread is scheduled to write the data with the largest memory available space into the memory of the target CPU, and the remaining data is written into the CPU memory of other nodes.

[0219] In the embodiment of the application, the IO process or thread scheduling submodule 702 is configured to:

[0220] From other nodes, a node with the largest current memory available space is selected, and it is determined whether the current memory available space is greater than or equal to the IO size of the remaining data.

[0221] If greater than or equal to, the IO process or thread is scheduled to write the remaining data into the CPU memory of the node; or

[0222] If less, write the data of the current available memory space into the one node, repeat the node screening and the space size comparison operation to write the remaining data into the CPU memory of the plurality of nodes.

[0223] In the device, the node comprises a persistent memory PM;

[0224] The device further comprises a PM space statistics submodule, configured to:

[0225] From the attribute of the data, obtain the persistent value located on the flag bit;

[0226] If the persistent value is a preset value, obtain the current available space of the persistent memory of the node where the data is located;

[0227] The IO process or thread scheduling submodule 702 is configured to:

[0228] Judge whether the current available space is greater than or equal to the IO size of the data;

[0229] If greater than or equal to, write the data in the node CPU memory into the persistent memory in an asynchronous manner; or

[0230] If less, write the data of the current available memory space into the one node, repeat the node screening and the space size comparison operation to write the remaining data into the CPU memory of the plurality of nodes.

[0231] The present scheme can be provided with an RDMA network management submodule: responsible for establishing a high-performance RDMA communication link between nodes, and realizing the read and write operations on a remote node.

[0232] In addition, the specific implementation of the device in the embodiment of the present application has been described in detail in the above method, and therefore the repeated content will not be described here.

[0233] Figure 8 An exemplary system architecture 800 to which embodiments of the application can be applied is shown, including terminal devices 801, 802, 803, a network 804, and a server 805 (only as an example).

[0234] The terminal devices 801, 802, 803 can be various electronic devices with a display screen and supporting web browsing, installed with various communication client applications, and the user can use the terminal devices 801, 802, 803 to interact with the server 805 through the network 804 to receive or send messages, etc.

[0235] The network 804 is a medium for providing a communication link between the terminal devices 801, 802, 803 and the server 805. The network 804 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0236] The server 805 can be a server providing various services. It should be noted that the method provided by the embodiments of the present application is generally executed by the server 805, and accordingly, the apparatus is generally arranged in the server 805.

[0237] It should be understood that the number of terminal devices, networks and servers in the system is merely illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers. Figure 8

[0238] Reference is made below to Figure 9 which shows a structural schematic diagram of a computer system 900 suitable for implementing the terminal device of the embodiments of the present application. Figure 9 The terminal device shown is merely an example and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0239] As shown in Figure 9 , the computer system 900 includes a central processing unit (CPU) 901 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 902 or programs loaded from a storage portion 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902 and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0240] The following components are connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable recording medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 910 as needed, so that a computer program read therefrom is installed in the storage portion 908 as needed.

[0241] ​In particular, the processes described above with reference to the flow charts can be implemented as a computer software program in accordance with the embodiments disclosed herein. For example, embodiments disclosed herein include a computer program product which includes a computer program tangibly embodied on a computer readable medium, the computer program including program code for executing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 909 and / or installed from the removable media 911. When the computer program is executed by the central processing unit (CPU) 901, the above-described functions defined in the system of the present application are executed.

[0242] It should be noted that the computer readable medium shown in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a carrier wave in a propagated data signal, in which the computer readable program code is carried. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0243] The computer program product of the present application comprises a computer program which, when executed by a processor, implements the IO scheduling method in the embodiments of the present application.

[0244] The modules described in the embodiments of the present application can be implemented in the form of software or hardware. The described modules can also be arranged in a processor, for example, a processor can be described as comprising a receiving module, an IO process or thread scheduling sub-module. In some cases, the names of the modules do not constitute a limitation on the modules themselves, for example, the receiving module can also be described as an "IO request receiving module".

[0245] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the device to perform the above IO scheduling method.

[0246] The computer program product of the present application comprises a computer program which, when executed by a processor, implements the IO scheduling method in the embodiments of the present application.

[0247] According to the technical solutions of the embodiments of the present application, the read-write mode, the IO size, the performance characteristics and use of each node / CPU are comprehensively considered to determine the mode of scheduling IO processes / threads, avoid scheduling errors, and reduce the overhead of moving IO processes / threads back and forth on the CPU core, so as to improve the processing performance of the system. At the same time, in view of the poor PM write performance, the IO data is preferentially written into the DRAM memory, and then written into the PM in an asynchronous persistent manner, so as to reduce the write latency.

[0248] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of specific terminology. However, embodiments thereof can be practiced with the exact description not being presented in detail. The term "device" should be understood to encompass devices operating in various modes, such as active mode, sleep mode, hibernate mode, and the like. The terms "coupled" and "connected," along with derivatives thereof, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, particular circuitry that can be said to be coupled to, or connected with, other circuitry can be some of the other circuitry that can communicate in some way, while other circuitry can be some of the other circuitry that can not be in communication with that particular circuitry.

Claims

1. An IO scheduling method, characterized in that, The method comprises: receiving an input / output (IO) request; wherein the IO request is a read data request or a write data request; when the IO request is the read data request, determining a target CPU corresponding to the read data request to schedule an IO process or thread to a core of the target CPU to perform a data reading operation and return read data, comprising: processing the CPU memory logical access address using a consistent hash algorithm to obtain a hash value, and querying a node corresponding to the hash value on a preset hash ring; determining an actual memory access address corresponding to the CPU memory logical access address; in response to all data to be read being stored in a single node, locating the target CPU in the single node according to the actual memory access address, judging whether the IO size is greater than or equal to a preset size threshold; if yes, judging whether the load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold; if no, scheduling the IO process or thread to the core of the target CPU to perform the data reading operation; in response to all data to be read being stored across nodes, counting the amount or size of data to be read in each node to filter out a first node with the largest amount or size of data, locating the target CPU in the first node according to the corresponding actual memory access address; judging whether the load utilization rate of the target CPU is greater than or equal to the preset load utilization rate threshold; if yes, scheduling the IO process or thread to a core of any idle CPU to perform the data reading operation by remotely accessing the memory space of the target CPU through the idle CPU; if no, scheduling the IO process or thread to the core of the target CPU to perform the data reading operation; or when the IO request is the write data request, determining a target node corresponding to the write data request to schedule an IO process or thread to write data into the CPU memory of the target node; wherein the write data request comprises data to be written, comprising: obtaining the current memory available space of the target node according to the identifier of the target node, judging whether the memory available space is greater than or equal to the IO size; if yes, scheduling the IO process or thread to write data into the CPU memory of the target node, if no, determining a target CPU with the largest memory available space in the target node, scheduling the IO process or thread to write the data with the largest memory available space into the memory of the target CPU, and determining the IO size of the remaining data; filtering out a node with the largest current memory available space from other nodes, judging whether the current memory available space is greater than or equal to the IO size of the remaining data; if yes, scheduling the IO process or thread to write the remaining data into the CPU memory of the one node, if no, writing the data with the current memory available space into the one node, and repeating the node filtering and space size comparison operations to write the remaining data into the CPU memories of multiple nodes.

2. The method of claim 1, wherein, The read data request comprises a CPU memory logical access address. The determining a target CPU corresponding to the read data request comprises: locating the target CPU according to the CPU memory logical access address, and judging whether a load utilization of the target CPU is greater than or equal to a preset load utilization threshold; scheduling an IO process or thread to a core of the corresponding CPU for a data read operation according to a judgment result.

3. The method of claim 2, wherein, The scheduling an IO process or thread to a core of the corresponding CPU for a data read operation according to a judgment result comprises: if less than, scheduling an IO process or thread to a core of the target CPU for a data read operation; or if greater than or equal to, scheduling an IO process or thread to a core of any other idle CPU to remotely access the target CPU memory space through the any other idle CPU for a data read operation; wherein the target CPU and the any other idle CPU are located in the same node.

4. The method of claim 1, wherein, The locating the target CPU in the node according to the actual memory access address comprises: in response to all to-be-read data being stored in a single node, locating the target CPU in the single node according to the actual memory access address; or in response to all to-be-read data being stored across nodes, counting an amount or size of to-be-read data in each node to filter out a first node with the largest amount or size of to-be-read data, and locating the target CPU in the first node according to a corresponding actual memory access address; after reading data from the first node is completed, filtering out a second node with the second largest amount or size of to-be-read data, and repeating the operations of locating the target CPU, judging the load utilization, and scheduling the IO process or thread to perform a data read operation until all to-be-read data is polled.

5. The method of claim 4, wherein, The read data request further comprises an IO size, which is a total size of to-be-read data. When all to-be-read data is stored in a single node, the judging whether the load utilization of the target CPU is greater than or equal to the preset load utilization threshold comprises: judging whether the IO size is greater than or equal to a preset size threshold, and if less than, scheduling an IO process or thread to a core of the target CPU for a data read operation; or if greater than or equal to, judging whether the load utilization of the target CPU is greater than or equal to the preset load utilization threshold. The method further comprises:

6. The method according to any one of claims 2-5, characterized in that, acquiring a distribution of a current IO process or thread on a core of a CPU; if the current IO process or thread is located on a core of the target CPU, scheduling an IO process or thread to the core of the target CPU for a data read operation, otherwise, scheduling an IO process or thread to a core of any other idle CPU for a remote data read operation. The write data request comprises an IO size, an identifier of a target node, and to-be-written data; wherein the IO size is a total size of the to-be-written data.

7. The method of claim 1, wherein, The scheduling an IO process or thread to write data into a CPU memory of the target node comprises: ​ According to the identification of the target node, the current memory available space of the target node is obtained, and it is judged whether the memory available space is greater than or equal to the IO size; According to the judgment result, the IO process or thread is scheduled to write data into the CPU memory of the corresponding node.

8. The method of claim 7, wherein, According to the judgment result, the IO process or thread is scheduled to write data into the CPU memory of the corresponding node, comprising: If greater than or equal to, the IO process or thread is scheduled to write data into the CPU memory of the target node; wherein a node includes multiple CPUs; or If less than, the target CPU with the largest memory available space under the target node is determined, the IO process or thread is scheduled to write the data with the largest memory available space into the memory of the target CPU, and the remaining data is written into the CPU memory of other nodes.

9. The method of claim 8, wherein, The remaining data is written into the CPU memory of other nodes, comprising: From other nodes, the node with the largest current memory available space is selected, and it is judged whether the current memory available space is greater than or equal to the IO size of the remaining data; If greater than or equal to, the IO process or thread is scheduled to write the remaining data into the CPU memory of the node; or If less than, the data with the current memory available space is written into the node, and the above node selection and space size comparison operations are repeated to write the remaining data into the CPU memory of multiple nodes.

10. The method according to any one of claims 7-9, characterized in that, The node includes persistent memory; After the method of scheduling the IO process or thread to write data into the CPU memory of the corresponding node according to the judgment result, the method further comprises: From the attributes of the data, the persistent value located on the flag bit is obtained; If the persistent value is a preset value, the current available space of the persistent memory of the node where the data is located is obtained, and it is judged whether the current available space is greater than or equal to the IO size of the data; If greater than or equal to, the data in the CPU memory of the node is written into the persistent memory in an asynchronous manner; or If less than, the data with the current available space in the CPU memory of the node is written into the persistent memory, and the remaining data is written into the persistent memory of other nodes.

11. An IO scheduling apparatus, comprising: Comprising: The receiving module is used for receiving the transmitted input / output (IO) request; wherein the IO request is a read data request or a write data request; The IO process or thread scheduling submodule is configured to, when the IO request is a read data request, determine a target CPU corresponding to the read data request, schedule an IO process or thread to a core of the target CPU for a data reading operation, and return read data, including: processing the CPU memory logical access address using a consistent hash algorithm to obtain a hash value, and querying a node corresponding to the hash value on a preset hash ring; determining an actual memory access address corresponding to the CPU memory logical access address; in response to a single node storing all data to be read, locating a target CPU in the single node according to the actual memory access address, determining whether the IO size is greater than or equal to a preset size threshold, and if so, determining whether a load utilization rate of the target CPU is greater than or equal to a preset load utilization rate threshold; if not, scheduling the IO process or thread to the core of the target CPU for the data reading operation; in response to cross-node storage of all data to be read, counting the amount or size of data to be read in each node to filter out a first node with the largest amount or size of data, locating the target CPU in the first node according to the corresponding actual memory access address, determining whether the load utilization rate of the target CPU is greater than or equal to the preset load utilization rate threshold, and if so, scheduling the IO process or thread to a core of any idle CPU to perform a data reading operation by remote access to the memory space of the target CPU through the idle CPU; if not, scheduling the IO process or thread to the core of the target CPU for the data reading operation; or When the IO request is a write data request, a target node corresponding to the write data request is determined to schedule an IO process or thread to write data into the CPU memory of the target node; wherein the write data request includes data to be written, including: obtaining a current memory available space of the target node according to an identifier of the target node, determining whether the memory available space is greater than or equal to the IO size, and if so, scheduling the IO process or thread to write data into the CPU memory of the target node, and if not, determining a target CPU with the largest memory available space in the target node, scheduling the IO process or thread to write data with the largest memory available space into the memory of the target CPU, and determining an IO size of remaining data; filtering out a node with the largest current memory available space from other nodes, determining whether the current memory available space is greater than or equal to the IO size of the remaining data, and if so, scheduling the IO process or thread to write the remaining data into the CPU memory of the one node, and if not, writing data with the current memory available space into the one node, and repeating the node filtering and space size comparison operations to write the remaining data into the CPU memories of multiple nodes.

12. The apparatus of claim 11, wherein, The read data request includes a CPU memory logical access address. The device further includes a locating module configured to locate the target CPU according to the CPU memory logical access address. A CPU load statistics submodule is configured to obtain a load utilization rate of the target CPU. The IO process or thread scheduling submodule is configured to determine whether the load utilization of the target CPU is greater than or equal to a preset load utilization threshold. According to the determination result, the IO process or thread is scheduled to perform a data reading operation on a core of the corresponding CPU.

13. The apparatus of claim 12, wherein, The IO process or thread scheduling submodule is configured to: If the load utilization of the target CPU is less than the preset load utilization threshold, the IO process or thread is scheduled to perform a data reading operation on a core of the target CPU; or If the load utilization of the target CPU is greater than or equal to the preset load utilization threshold, the IO process or thread is scheduled to perform a data reading operation on a core of any idle CPU, so as to access the memory space of the target CPU remotely through the idle CPU.

14. The apparatus of claim 11, wherein, The positioning module is configured to: In response to all the data to be read being stored in a single node, the target CPU is located in the single node according to the actual memory access address; or In response to all the data to be read being stored across nodes, the amount or size of the data to be read in each node is counted, and a first node with the largest amount or size of data is filtered out, and the target CPU is located in the first node according to the corresponding actual memory access address. After the data reading from the first node is completed, a second node with the second largest amount or size of data is filtered out, and the above operations of locating the target CPU, determining the load utilization, and scheduling the IO process or thread are repeated to perform a data reading operation, until all the data to be read is polled.

15. The apparatus of claim 14, wherein, The read data request further includes an IO size, which is the total size of the data to be read. When all the data to be read is stored in a single node, the IO process or thread scheduling submodule is configured to: Determine whether the IO size is greater than or equal to a preset size threshold, if the IO size is less than the preset size threshold, the IO process or thread is scheduled to perform a data reading operation on a core of the target CPU; or If the IO size is greater than or equal to the preset size threshold, it is determined whether the load utilization of the target CPU is greater than or equal to a preset load utilization threshold.

16. The apparatus of any one of claims 12-15, wherein, The IO process or thread scheduling submodule is further configured to: Obtain the distribution of the current IO process or thread on the cores of the CPUs; If the current IO process or thread is located on a core of the target CPU, the IO process or thread is scheduled to perform a data reading operation on the core of the target CPU, otherwise, the IO process or thread is scheduled to perform a remote data reading operation on a core of any idle CPU.

17. The apparatus of claim 11, wherein, The write data request includes an IO size, an identifier of a target node, and data to be written, wherein the IO size is the total size of the data to be written. The apparatus further includes a dynamic random access memory (DRAM) space statistical submodule configured to obtain the available memory space of the target node according to the identifier of the target node. The IO process or thread scheduling submodule is configured to: Determine whether the available memory space is greater than or equal to the IO size; According to the determination result, the IO process or thread is scheduled to write data into the CPU memory of the corresponding node.

18. The apparatus of claim 17, wherein, The IO process or thread scheduling submodule is configured to: If greater than or equal to, the IO process or thread is scheduled to write data into the CPU memory of the target node; wherein one node includes multiple CPUs; If less than, the target CPU with the largest available memory space in the target node is determined, the IO process or thread is scheduled to write data with the largest available memory space into the memory of the target CPU, and the remaining data is written into the CPU memory of other nodes.

19. The apparatus of claim 18, wherein, The IO process or thread scheduling submodule is configured to: select one node with the largest current available memory space from other nodes, and determine whether the current available memory space is greater than or equal to the IO size of the remaining data; If greater than or equal to, the IO process or thread is scheduled to write the remaining data into the CPU memory of the one node; Or If less than, the data with the current available memory space is written into the one node, and the above node selection and space size comparison operations are repeated to write the remaining data into the CPU memory of multiple nodes.

20. The apparatus of any one of claims 17-19, wherein, The node includes a persistent memory PM. The device further includes a PM space statistics submodule configured to: obtain a persistent value located on a flag bit from the attribute of the data; If the persistent value is a preset value, obtain the current available space of the persistent memory of the node where the data is located. The IO process or thread scheduling submodule is configured to: determine whether the current available space is greater than or equal to the IO size of the data; If greater than or equal to, the data in the CPU memory of the node is written into the persistent memory in an asynchronous manner; Or If less than, the data with the current available space in the CPU memory of the node is written into the persistent memory, and the remaining data is written into the persistent memory of other nodes.

21. An electronic device, comprising: It includes: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-10.

22. A computer readable medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-10.

23. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-10.

Images