Data writing method and device, storage medium and electronic equipment
By determining the CPU and NUMA corresponding to the data write request in the NUMA architecture and directly writing the data into available memory, the problem of low data writing efficiency caused by the service nodes across different CPUs is solved, and more efficient reading and writing is achieved performance.
Patent Information
- Application Number
- CN202411045458.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-08
AI Technical Summary
In the NUMA architecture, service nodes will jump across different CPUs when data is written, resulting in the problem of low data writing efficiency not yet effectively solved.
By determining the corresponding CPU to the data write request and its corresponding NUMA, and writing the data directly into the memory when determining that the memory of the NUMA is available, avoiding jumps between different CPUs, and using the cache of the distributed storage system The kernel binding method caches data to the NUMA cache corresponding to the CPU according to the CPU where the read and write process is located.
Improves data writing efficiency, reduces switching between NUMAs, reduces system latency and improves read and write performance.
Smart Images

Figure CN118915973A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of big data, and more specifically, to a method and device for writing data, a storage medium, and an electronic device. Background Art
[0002] NUMA (Non-Uniform Memory Access) technology is a multi-processor system architecture designed to solve the memory access problem in multi-processor systems. Before the emergence of NUMA architecture, the CPU (central processing unit) frequency continued to rise (vertical development), but with the challenge of physical limits, it turned to the direction of increasing the number of cores (horizontal development). In the early days, all CPUs had to access memory through the north bridge, which would cause processors to compete for memory bandwidth, thus affecting system performance. To address this problem, NUMA technology came into being. It can enable many servers to operate like a single system while retaining the advantages of small systems that are easy to program and manage. The key to NUMA technology is to divide memory and processors into different nodes, each with its own local memory and processor. Communication between nodes is achieved through a high-speed interconnection network, while communication within the node is carried out through local memory. This design helps to reduce memory access competition between processors and improve system performance. However, in related technologies, when writing data, the service node will jump between multiple CPUs in different NUMAs, and then switch between NUMAs.
[0003] Therefore, the problem in the related art that a service node jumps between multiple CPUs in different NUMAs when writing data, resulting in low data writing efficiency, has not yet been effectively solved. Summary of the invention
[0004] The embodiments of the present application provide a data writing method and device, a storage medium and an electronic device to at least solve the problem of low data writing efficiency caused by the service node jumping between multiple CPUs in different NUMA when writing data in the related art.
[0005] According to an embodiment of the present application, a method for writing data is provided, comprising: upon receiving a data write request, determining a first CPU corresponding to the data write request, and determining a first NUMA corresponding to the first CPU, wherein the first NUMA comprises: a plurality of CPUs and a first memory, the plurality of CPUs comprising: the first CPU; upon determining that the first memory of the first NUMA is an available memory, writing the first data corresponding to the data write request into the first memory.
[0006] In an exemplary embodiment, determining the first CPU corresponding to the data write request includes: determining a first process corresponding to the data write request; obtaining a preset CPU number corresponding to the first process; and determining the first CPU according to the preset CPU number.
[0007] In an exemplary embodiment, after determining the first NUMA corresponding to the first CPU, the method further includes: determining a first size relationship between a first data amount of the first memory and a second data amount of the first data, wherein the first data amount is the remaining data amount of the first memory; when the first size relationship indicates that the first data amount is greater than or equal to the second data amount, determining that the first memory is an available memory; writing the first data to the first memory; when the first size relationship indicates that the first data amount is less than the second data amount, determining that the first memory is an unavailable memory; determining a second NUMA, and determining a second size relationship between a third data amount of the second memory and the second data amount, wherein the second NUMA includes: the second memory, and the third data amount is the remaining data amount of the second memory; when the second size relationship indicates that the third data amount is greater than or equal to the second data amount, determining that the second memory is an available memory; and writing the first data to the second memory.
[0008] In an exemplary embodiment, determining the second NUMA includes: determining the distance between the first NUMA and other NUMAs in a NUMA group except the first NUMA, wherein the NUMA group includes: multiple NUMAs, and the multiple NUMAs include: the first NUMA; determining the NUMA with the smallest distance to the first NUMA as the second NUMA.
[0009] In an exemplary embodiment, writing the first data into the second memory includes: determining multiple second CPUs included in the second NUMA; determining a target CPU that has a communication relationship with the first CPU among the multiple second CPUs; forwarding the data write request from the first CPU to the target CPU, and writing the first data into the second memory according to the target CPU.
[0010] In an exemplary embodiment, writing the first data corresponding to the data write request into the first memory includes: applying for a first data space in the first memory, wherein the amount of data contained in the first data space is consistent with the second amount of data of the first data, and the first data space is a blank data space in the first memory; writing the first data into the first data space through the first CPU.
[0011] In an exemplary embodiment, after writing the first data corresponding to the data write request into the first memory, the method also includes: upon receiving a data read request, determining a third memory corresponding to the data read request, and determining a third NUMA including the third memory; obtaining multiple CPUs in the third NUMA; determining a third CPU among the multiple CPUs in the third NUMA, and reading data in the third memory based on the third CPU.
[0012] According to another embodiment of the present application, a data writing device is provided, including: a determination module, for determining, upon receiving a data write request, a first CPU corresponding to the data write request, and determining a first NUMA corresponding to the first CPU, wherein the first NUMA includes: a plurality of CPUs and a first memory, wherein the plurality of CPUs include: the first CPU; a writing module, for writing the first data corresponding to the data write request into the first memory upon determining that the first memory of the first NUMA is an available memory.
[0013] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps in the above method embodiment when running.
[0014] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in the above method embodiment.
[0015] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiment are implemented.
[0016] Through the data writing method of the present application, when a data write request is received, the first CPU corresponding to the data write request is determined, and the first NUMA including the first memory and multiple CPUs corresponding to the first CPU is determined, wherein the multiple CPUs include the first CPU. When it is determined that the first memory of the first NUMA is an available memory, the first data corresponding to the data write request is written into the first memory. That is to say, when the data write request is received, the embodiment of the present application directly writes the first data into the first memory in the first NUMA corresponding to the first CPU corresponding to the data write request, thereby avoiding jumps between different CPUs. Through the embodiment of the present application, the problem of low data writing efficiency caused by the service node jumping between multiple CPUs in different NUMAs when writing data in the related technology is solved, thereby avoiding the switching between NUMAs caused by the jumps between different CPUs, and improving the read and write efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for writing data in an embodiment of the present application;
[0018] Figure 2 is a flow chart of a method for writing data according to an embodiment of the present application;
[0019] Figure 3 This is the architecture diagram of NUMA in the related art;
[0020] Figure 4 This is a schematic diagram of a NUMA memory and a CPU after core binding according to an embodiment of the present application;
[0021] Figure 5 It is a structural block diagram of a data writing device according to an embodiment of the present application. DETAILED DESCRIPTION
[0022] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0024] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a data writing method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the interactive state in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0026] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0027] Figure 2 is a flow chart of a method for writing data according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:
[0028] Step S202, when a data write request is received, determining a first CPU corresponding to the data write request, and determining a first NUMA corresponding to the first CPU, wherein the first NUMA includes: a plurality of CPUs and a first memory, and the plurality of CPUs includes: the first CPU;
[0029] Each NUMA includes a memory and multiple CPUs, and each CPU is bound to the NUMA, thereby implementing core binding.
[0030] Step S204: When it is determined that the first memory of the first NUMA is an available memory, write the first data corresponding to the data write request into the first memory.
[0031] Through the data writing method of the present application, when a data write request is received, the first CPU corresponding to the data write request is determined, and the first NUMA including the first memory and multiple CPUs corresponding to the first CPU is determined, wherein the multiple CPUs include the first CPU. When it is determined that the first memory of the first NUMA is an available memory, the first data corresponding to the data write request is written into the first memory. That is to say, when the data write request is received, the embodiment of the present application directly writes the first data into the first memory in the first NUMA corresponding to the first CPU corresponding to the data write request, thereby avoiding jumps between different CPUs. Through the embodiment of the present application, the problem of low data writing efficiency caused by the service node jumping between multiple CPUs in different NUMAs when writing data in the related technology is solved, thereby avoiding the switching between NUMAs caused by the jumps between different CPUs, and improving the read and write efficiency.
[0032] Optionally, determining the first CPU corresponding to the data write request in the above step S202 includes: determining a first process corresponding to the data write request; obtaining a preset CPU number corresponding to the first process; and determining the first CPU according to the preset CPU number.
[0033] It is understandable that it is necessary to determine the first process corresponding to the data write request, and the first process is the process that completes the data write request. The system will randomly assign a CPU number to the first process. It is necessary to determine the first CPU corresponding to this CPU number, and then determine which NUMA the first CPU exists in, for example: the first NUMA.
[0034] Through the above technical solution, the first CPU is determined according to the CPU number corresponding to the first process, so that data can be accessed and processed more quickly and the delay of the system can be reduced.
[0035] Optionally, after determining the first NUMA corresponding to the first CPU in the above step S202, the method also includes: determining a first size relationship between the first data amount of the first memory and the second data amount of the first data, wherein the first data amount is the remaining data amount of the first memory; when the first size relationship indicates that the first data amount is greater than or equal to the second data amount, determining that the first memory is available memory; writing the first data to the first memory; when the first size relationship indicates that the first data amount is less than the second data amount, determining that the first memory is unavailable memory; determining a second NUMA, and determining a second size relationship between a third data amount of the second memory and the second data amount, wherein the second NUMA includes: the second memory, the third data amount is the remaining data amount of the second memory; when the second size relationship indicates that the third data amount is greater than or equal to the second data amount, determining that the second memory is available memory; writing the first data to the second memory.
[0036] It can be understood that the above-mentioned first data may be cache data.
[0037] After determining the first NUMA corresponding to the first CPU, it is also necessary to determine the first size relationship between the first data amount of the first memory and the second data amount of the first data, and determine whether the first data can be written to the first memory through the first size relationship. When the first data amount is greater than or equal to the second data amount, it can be determined that the memory space of the first memory can meet the needs of the first data, that is, the first data can be written to the first memory. When it is determined that the first data amount is less than the second data amount, it can be determined that the memory space of the first memory cannot meet the needs of the first data, that is, the first data cannot be written to the first memory. At this time, it is necessary to determine a second memory whose memory space can meet the needs of the first data, that is, the second memory in the second NUMA, and then write the first data to the second memory.
[0038] According to the memory size required by the current request, it can be determined whether the memory space of the first memory of the first MA node is sufficient to meet the demand of the first data, thereby avoiding the situation of insufficient or wasted memory.
[0039] Determining the second NUMA includes: determining the distance between the first NUMA and other NUMAs in the NUMA group except the first NUMA, wherein the NUMA group includes: multiple NUMAs, and the multiple NUMAs include: the first NUMA; determining the NUMA with the smallest distance to the first NUMA as the second NUMA.
[0040] The second NUMA can be determined based on the distance between the first NUMA and other NUMAs. Specifically, the fourth NUMA with the smallest distance from the first NUMA is determined, and whether the memory in the fourth NUMA with the smallest distance meets the needs of the first data. If it does not meet the needs, the fifth NUMA with the smallest distance from the first NUMA other than the fourth NUMA can also be determined. In an embodiment of the present application, it can be stipulated that a maximum of two NUMAs are allowed to obtain the memory of the NUMA. If the memory for the first data is still not met across the two NUMAs, the data can be written according to the system allocation.
[0041] Through the embodiments of the present application, memory allocation can be performed based on the distance information between NUMA nodes to minimize the latency of memory access and avoid obtaining memory across multiple NUMA nodes as much as possible to ensure the stability and reliability of system performance.
[0042] Among them, writing the first data into the second memory includes: determining multiple second CPUs included in the second NUMA; determining a target CPU that has a communication relationship with the first CPU among the multiple second CPUs; forwarding the data write request from the first CPU to the target CPU, and writing the first data into the second memory according to the target CPU.
[0043] It can be seen from the above technical solution that if the first memory of the first NUMA cannot meet the demand of the first data, and the second memory of the second NUMA with the smallest distance from the first NUMA meets the demand of the first data, the data write request can be forwarded to any CPU in the second NUMA through the first CPU, so that the first data can be written to the second memory in the second NUMA through any CPU in the second NUMA.
[0044] Optionally, the above step S204 of writing the first data corresponding to the data write request into the first memory includes: applying for a first data space in the first memory, wherein the amount of data contained in the first data space is consistent with the second data amount of the first data, and the first data space is a blank data space in the first memory; writing the first data into the first data space through the first CPU.
[0045] It is understandable that, when it is determined that the first data can be written into the first memory, a certain amount of data space needs to be applied in the first memory, and the amount of space of this data space needs to be at least consistent with the amount of data of the first data. Storing data in the memory can reduce the delay of data access, thereby improving the running efficiency of the program.
[0046] Optionally, after writing the first data corresponding to the data write request to the first memory in the above step S204, the method also includes: when receiving a data read request, determining a third memory corresponding to the data read request, and determining a third NUMA including the third memory; obtaining multiple CPUs in the third NUMA; determining a third CPU among the multiple CPUs in the third NUMA, and reading data in the third memory according to the third CPU.
[0047] It can be understood that the embodiment of the present application not only defines the processing method corresponding to the data write request, but also defines the processing method corresponding to the data read request. Specifically, when a data read request is received, the third memory corresponding to the second data corresponding to the data read request can be determined, and the third NUMA corresponding to the third memory can be determined. Since the embodiment of the present application performs a core binding operation between the CPU and the NUMA, any CPU in the third NUMA can be determined as the CPU for reading the second data.
[0048] In order to better understand the process of the above data writing method, the implementation process of the above data writing method is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.
[0049] Currently, the main cache allocation method for distributed file systems is distributed cache. The so-called distributed cache is a cache technology that runs on multiple machines. Although it requires serialization and network transmission, which makes it relatively slow, in theory the number and size of caches can be infinitely expanded, so it is more suitable for processing large-scale data. Distributed cache is often used in scenarios such as cross-server data sharing and distributed computing. The existing distributed cache allocation strategies include:
[0050] (1) Hash-based allocation strategy: The hash-based allocation strategy maps the key-value pair to a hash function to determine which node the key-value pair should be stored on. The advantage of the hash-based allocation strategy is that it is simple and easy to implement, but it may cause load imbalance problems.
[0051] (2) Range-based allocation strategy: The range-based allocation strategy determines which node to store the data on based on the range of the data. For example, data within a certain range can be stored on the same node. The advantage of the range-based allocation strategy is that it can achieve load balancing, but it may cause uneven data distribution.
[0052] (3) List-based allocation strategy: The list-based allocation strategy stores data on different nodes in a certain order. The advantage of the list-based allocation strategy is that it is easy to implement, but it may cause uneven data distribution and cache hot spots.
[0053] (4) Directory-based allocation strategy: Directory-based allocation strategy stores data on different nodes according to a certain directory structure. The advantage of directory-based allocation strategy is that it can solve the cache hotspot problem, but it may cause uneven data distribution and complex management.
[0054] (5) LRU (least recently used) expiration policy: LRU is a common cache replacement policy. When the cache is full, the least recently used cache item will be removed first. The LRU expiration policy will lead to uneven data distribution.
[0055] (6) Shared cache mechanism: The shared cache mechanism can take advantage of the most important feature of distributed cache, that is, multiple application systems can share a set of cache data. In other words, when the cache of a process changes, the changed content is notified to other processes, so that the data cached by any process can be synchronized to the cache of other processes. However, this may lead to uneven data distribution and complex management.
[0056] NUMA (Non-Uniform Memory Access) technology is a multi-processor system architecture that aims to solve the memory access problem in the multi-processor system in the above-mentioned related technologies. Figure 3 This is the architecture diagram of NUMA in related technologies, such as Figure 3 As shown in the figure, NUMA can be described as a physical CPU (generally including multiple logical CPUs: CPU0, CPU1 or multiple cores) constituting a node. This node includes not only the CPU but also a set of memory slots, that is, a physical CPU and a piece of memory (i.e. Figure 3 Each CPU can access the memory of its own node and the memory of other nodes, but the access speed is different. The access speed of the memory of its own node is faster.
[0057] In order to solve the problem of low data writing efficiency caused by service nodes jumping between multiple CPUs in different NUMAs when writing data in the related technology, the service nodes can be reduced from jumping back and forth between CPUs when reading and writing data, thereby improving the cache reading and writing performance of the system. The optional embodiment of the present application proposes a cache binding core method based on a distributed storage system, which caches data to the NUMA cache corresponding to the CPU according to the CPU where the read and write process is located, so that when the cache data is obtained, there is no need to expand the NUMA data reading or flushing, which can improve the reading and writing performance and reduce the switching of caches between NUMAs.
[0058] Figure 4 is a schematic diagram of a NUMA memory and a CPU after being bound to a core according to an embodiment of the present application, such as Figure 4 As shown, Figure 4 The numa0 part is a schematic diagram after the core binding operation of the optional embodiment of the present application is completed, and the numa1 part is a schematic diagram before the core binding operation. By comparing numa0 and numa1, it can be determined that after the core is bound, the thread will not copy and access the cache across NUMA; when there is no core binding, the data is copied and accessed across NUMA memory.
[0059] An optional embodiment of the present application provides a cache core binding method based on a distributed storage system, including:
[0060] (1) The protocol layer accesses the client and sends the data read and write request (i.e., the data write request and data read request of this application) to the service node. The service node obtains the current thread (i.e., the first process of this application) and the logical CPU number where the process is located (i.e., the preset CPU number of this application).
[0061] (2) Calculate the NUMA where the logical CPU is located (i.e., the first NUMA of this application) according to the logical CPU number, and calculate whether the memory space of the NUMA (i.e., the first memory of this application) meets the size required by the current request.
[0062] (3) If the memory space of NUMA is insufficient, the memory of the NUMA with the smallest NUMA spacing (i.e., the second NUMA of this application) is selected as the cache for the request. Memory is allowed to be obtained across a maximum of 2 NUMAs. Otherwise, it is performed according to the system allocation.
[0063] According to the process of the service node read and write request, the NUMA memory is analyzed to see whether it is available, and its cache space is bound to its nearest NUMA memory. When the NUMA memory does not meet the principle, the system allocation principle is followed to ensure that the system performance is not reduced while maximizing the performance. (4) After obtaining the memory, if it is a data write request, a certain amount of space is applied in the memory and the data is filled into the cache; if it is a data read request, the data in the data memory is used.
[0064] (5) After the data is filled, subsequent data access can directly access the NUMA memory of the process.
[0065] (6) Cache cleaning remains consistent with the original logic.
[0066] (7) Subsequent data cache reading and writing reduces data cache copy and access time, improving the system's read and write performance.
[0067] According to the optional embodiment of the present application, when accessing, the client can directly obtain data from the NUMA memory corresponding to the service node, avoiding mutual copying between NUMA memories, reducing the time consumption caused by switching between NUMAs, and ultimately achieving the effect of improving the read and write performance of the system. Even if the NUMA memory where the process is located is insufficient, it will not affect the system performance. When the number of NUMAs that need to be crossed exceeds 2, it is necessary to follow the cache allocation principle of the system to minimize the impact on the overall performance of the system while improving the performance of the system.
[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0069] In the present embodiment, a data writing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0070] Figure 5 is a structural block diagram of a data writing device according to an embodiment of the present application, such as Figure 5 As shown, the device comprises:
[0071] The determination module 52 is used to determine, when receiving a data write request, a first CPU corresponding to the data write request, and determine a first NUMA corresponding to the first CPU, wherein the first NUMA includes: a plurality of CPUs and a first memory, and the plurality of CPUs includes: the first CPU;
[0072] The writing module 54 is used to write the first data corresponding to the data writing request into the first memory when it is determined that the first memory of the first NUMA is an available memory.
[0073] Through the data writing device of the present application, when a data write request is received, the first CPU corresponding to the data write request is determined, and the first NUMA including the first memory and multiple CPUs corresponding to the first CPU is determined, wherein the multiple CPUs include the first CPU. When it is determined that the first memory of the first NUMA is an available memory, the first data corresponding to the data write request is written into the first memory. That is to say, when the data write request is received, the embodiment of the present application directly writes the first data into the first memory in the first NUMA corresponding to the first CPU corresponding to the data write request, thereby avoiding jumps between different CPUs. Through the embodiment of the present application, the problem of low data writing efficiency caused by the service node jumping between multiple CPUs in different NUMAs when writing data in the related technology is solved, thereby avoiding the switching between NUMAs caused by jumps between different CPUs, and improving the reading and writing efficiency.
[0074] In an exemplary embodiment, the determination module 52 is further configured to determine a first process corresponding to the data write request; obtain a preset CPU number corresponding to the first process; and determine the first CPU according to the preset CPU number.
[0075] It is understandable that it is necessary to determine the first process corresponding to the data write request, and the first process is the process that completes the data write request. The system will randomly assign a CPU number to the first process. It is necessary to determine the first CPU corresponding to this CPU number, and then determine which NUMA the first CPU exists in, for example: the first NUMA.
[0076] Through the above technical solution, the first CPU is determined according to the CPU number corresponding to the first process, so that data can be accessed and processed more quickly and the delay of the system can be reduced.
[0077] In an exemplary embodiment, the determination module 52 is also used to determine a first size relationship between a first data amount of the first memory and a second data amount of the first data, wherein the first data amount is the remaining data amount of the first memory; when the first size relationship indicates that the first data amount is greater than or equal to the second data amount, determine that the first memory is available memory; write the first data to the first memory; when the first size relationship indicates that the first data amount is less than the second data amount, determine that the first memory is unavailable memory; determine a second NUMA, and determine a second size relationship between a third data amount of the second memory and the second data amount, wherein the second NUMA includes: the second memory, the third data amount is the remaining data amount of the second memory; when the second size relationship indicates that the third data amount is greater than or equal to the second data amount, determine that the second memory is available memory; write the first data to the second memory.
[0078] It can be understood that the above-mentioned first data may be cache data.
[0079] After determining the first NUMA corresponding to the first CPU, it is also necessary to determine the first size relationship between the first data amount of the first memory and the second data amount of the first data, and determine whether the first data can be written to the first memory through the first size relationship. When the first data amount is greater than or equal to the second data amount, it can be determined that the memory space of the first memory can meet the needs of the first data, that is, the first data can be written to the first memory. When it is determined that the first data amount is less than the second data amount, it can be determined that the memory space of the first memory cannot meet the needs of the first data, that is, the first data cannot be written to the first memory. At this time, it is necessary to determine a second memory whose memory space can meet the needs of the first data, that is, the second memory in the second NUMA, and then write the first data to the second memory.
[0080] According to the memory size required by the current request, it can be determined whether the memory space of the first memory of the first MA node is sufficient to meet the demand of the first data, thereby avoiding the situation of insufficient or wasted memory.
[0081] In an exemplary embodiment, the determination module 52 is further used to determine the distance between the first NUMA and other NUMAs in the NUMA group except the first NUMA, wherein the NUMA group includes: multiple NUMAs, and the multiple NUMAs include: the first NUMA; and determine the NUMA with the smallest distance to the first NUMA as the second NUMA.
[0082] The second NUMA can be determined based on the distance between the first NUMA and other NUMAs. Specifically, the fourth NUMA with the smallest distance from the first NUMA is determined, and whether the memory in the fourth NUMA with the smallest distance meets the needs of the first data. If it does not meet the needs, the fifth NUMA with the smallest distance from the first NUMA other than the fourth NUMA can also be determined. In an embodiment of the present application, it can be stipulated that a maximum of two NUMAs are allowed to obtain the memory of the NUMA. If the memory for the first data is still not met across the two NUMAs, the data can be written according to the system allocation.
[0083] Through the embodiments of the present application, memory allocation can be performed based on the distance information between NUMA nodes to minimize the latency of memory access and avoid obtaining memory across multiple NUMA nodes as much as possible to ensure the stability and reliability of system performance.
[0084] In an exemplary embodiment, the determination module 52 is also used to determine multiple second CPUs included in the second NUMA; determine a target CPU that has a communication relationship with the first CPU among the multiple second CPUs; forward the data write request from the first CPU to the target CPU, and write the first data to the second memory according to the target CPU.
[0085] It can be seen from the above technical solution that if the first memory of the first NUMA cannot meet the demand of the first data, and the second memory of the second NUMA with the smallest distance from the first NUMA meets the demand of the first data, the data write request can be forwarded to any CPU in the second NUMA through the first CPU, so that the first data can be written to the second memory in the second NUMA through any CPU in the second NUMA.
[0086] In an exemplary embodiment, the write module 54 is also used to apply for a first data space in the first memory, wherein the amount of data contained in the first data space is consistent with the second data amount of the first data, and the first data space is a blank data space in the first memory; the first data is written into the first data space through the first CPU.
[0087] It is understandable that, when it is determined that the first data can be written into the first memory, a certain amount of data space needs to be applied in the first memory, and the amount of space of this data space needs to be at least consistent with the amount of data of the first data. Storing data in the memory can reduce the delay of data access, thereby improving the running efficiency of the program.
[0088] In an exemplary embodiment, the write module 54 is also used to, upon receiving a data read request, determine a third memory corresponding to the data read request, and determine a third NUMA including the third memory; obtain multiple CPUs in the third NUMA; determine a third CPU among the multiple CPUs in the third NUMA, and read data in the third memory based on the third CPU.
[0089] It can be understood that the embodiment of the present application not only defines the processing method corresponding to the data write request, but also defines the processing method corresponding to the data read request. Specifically, when a data read request is received, the third memory corresponding to the second data corresponding to the data read request can be determined, and the third NUMA corresponding to the third memory can be determined. Since the embodiment of the present application performs a core binding operation between the CPU and the NUMA, any CPU in the third NUMA can be determined as the CPU for reading the second data.
[0090] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps in the above method embodiment when running.
[0091] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0092] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the above method embodiment.
[0093] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0094] An embodiment of the present application further provides a computer program product, including a computer program, which is implemented to perform the steps in the above method embodiment when executed by a processor.
[0095] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0096] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0097] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for writing data, characterized in that: include: In the case of receiving a data write request, determining a first CPU corresponding to the data write request, and determining a first NUMA corresponding to the first CPU, wherein the first NUMA includes: a plurality of CPUs and a first memory, and the plurality of CPUs includes: the first CPU; When it is determined that the first memory of the first NUMA is an available memory, the first data corresponding to the data write request is written into the first memory.
2. The method according to claim 1, characterized in that Determining the first CPU corresponding to the data write request includes: Determine a first process corresponding to the data write request; Obtaining a preset CPU number corresponding to the first process; The first CPU is determined according to the preset CPU number.
3. The method according to claim 1, characterized in that: After determining the first NUMA corresponding to the first CPU, the method further includes: Determine a first size relationship between a first data amount of the first memory and a second data amount of the first data, wherein the first data amount is a remaining data amount of the first memory; In a case where the first size relationship indicates that the first data amount is greater than or equal to the second data amount, determining that the first memory is available memory; Writing the first data into the first memory; In a case where the first size relationship indicates that the first data amount is smaller than the second data amount, determining that the first memory is an unavailable memory; Determine a second NUMA, and determine a second size relationship between a third data amount of the second memory and the second data amount, wherein the second NUMA includes: the second memory, and the third data amount is a remaining data amount of the second memory; In a case where the second size relationship indicates that the third data amount is greater than or equal to the second data amount, determining that the second memory is available memory; The first data is written into the second memory.
4. The method according to claim 3, characterized in that Determine the secondary NUMA, including: Determine a distance between the first NUMA and other NUMAs in a NUMA group except the first NUMA, wherein the NUMA group includes: a plurality of NUMAs, and the plurality of NUMAs includes: the first NUMA; The NUMA with the shortest distance from the first NUMA is determined as the second NUMA.
5. The method according to claim 3, characterized in that: Writing the first data into the second memory includes: Determine a plurality of second CPUs included in the second NUMA; determining a target CPU having a communication relationship with the first CPU among the plurality of second CPUs; The data write request is forwarded from the first CPU to the target CPU, and the first data is written into the second memory according to the target CPU.
6. The method according to claim 1, characterized in that Writing first data corresponding to the data write request into the first memory includes: Applying for a first data space in the first memory, wherein the amount of data contained in the first data space is consistent with the second amount of data of the first data, and the first data space is a blank data space in the first memory; The first data is written into the first data space by the first CPU.
7. The method according to claim 1, characterized in that After writing the first data corresponding to the data write request into the first memory, the method further includes: In the case of receiving a data read request, determining a third memory corresponding to the data read request, and determining a third NUMA including the third memory; Obtain multiple CPUs in the third NUMA; A third CPU is determined from among a plurality of CPUs in the third NUMA, and data in the third memory is read according to the third CPU.
8. A data writing device, characterized in that: include: A determination module, configured to, upon receiving a data write request, determine a first CPU corresponding to the data write request, and determine a first NUMA corresponding to the first CPU, wherein the first NUMA includes: a plurality of CPUs and a first memory, and the plurality of CPUs includes: the first CPU; A writing module is used to write the first data corresponding to the data writing request into the first memory when it is determined that the first memory of the first NUMA is an available memory.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
