Memory allocation method and device, system on chip, computing equipment and program product

Abstract

The invention provides a memory allocation method and device, a system-on-chip, computing equipment and a program product, the method is applied to the system-on-chip comprising a plurality of system nodes, and a performance monitoring unit is arranged in the system-on-chip and used for monitoring memory access events in the plurality of system nodes; the method comprises the following steps: under the condition that a first thread in a first system node requests to allocate a memory, acquiring a memory access bandwidth of each system node, and determining distances between the first system node and other system nodes; based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, determining a target system node which enables the memory access speed to be fastest when the first thread accesses the memory from the plurality of system nodes; the target system node is used for providing the required memory for the first thread. According to the method, memory allocation can be more reasonable, the threads can access the memory more smoothly, memory access blockage is avoided, and system performance can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a memory allocation method, apparatus, system on chip, computing device and program product. Background Art

[0002] In a computer system based on the Non-Uniform Memory Access (NUMA) architecture, each system node is equipped with corresponding memory. However, in actual operation, not all kernel threads in each system node use only the memory of the system node where they are located. When the memory of the system node is insufficient, memory will be allocated to them from other system nodes.

[0003] This makes memory allocation in computer systems based on NUMA architecture more complicated, and the access channel bandwidth of memory in each system node is limited. If the memory allocation is unreasonable, it will cause memory access blocking and affect system performance. Summary of the invention

[0004] Based on the above technical problems, the present application proposes a memory allocation method, device, system on chip, computing device and program product, which can make memory allocation more reasonable, make thread access to memory smoother, avoid memory access blocking, and help improve system performance.

[0005] In a first aspect, the present application provides a memory allocation method, which is applied to a system on chip, wherein the system on chip includes a plurality of system nodes, the system nodes include memory, and a performance monitoring unit is provided in the system on chip for monitoring memory access events in the plurality of system nodes; the method includes:

[0006] When a first thread in a first system node requests memory allocation, obtaining a memory access bandwidth of each system node, and determining a distance between the first system node and other system nodes; wherein the memory access bandwidth of the system node is determined based on a memory access event in the system node;

[0007] Based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, a target system node is determined from the multiple system nodes to enable the first thread to access the memory at the fastest memory access speed; the target system node is used to provide the required memory for the first thread.

[0008] In some implementations, the method further includes:

[0009] Obtaining memory access events of respective system nodes monitored by the performance monitoring unit;

[0010] Based on the memory access events of each system node, the total amount of memory read and write data of each system node is determined by calculating the sum of the total amount of write data and the total amount of read data of each system node;

[0011] The memory access bandwidth of each system node is determined by calculating the ratio of the total amount of memory read and write data and the operation cycle of each system node.

[0012] In some implementations, determining the distance between the first system node and other system nodes includes:

[0013] Determine a topological structure between the plurality of system nodes based on a connection relationship between the plurality of system nodes;

[0014] Based on the topological structure between the multiple system nodes, a distance matrix for representing the distance between each system node is generated, and based on the distance matrix, the distance between the first system node and other system nodes is determined.

[0015] In some implementations, based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, determining, from the multiple system nodes, a target system node that enables the first thread to access the memory at the fastest memory access speed includes:

[0016] Compare the memory access bandwidth of the first system node with the memory access bandwidth of other system nodes, and search for a candidate system node from the other system nodes; wherein the memory access bandwidth of the candidate system node is smaller than the memory access bandwidth of the first system node, and the difference between the memory access bandwidth of the first system node and the memory access bandwidth of the candidate system node is greater than a set bandwidth threshold;

[0017] In the case where the candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, a candidate system node closest to the first system node is determined from the candidate system nodes as the target system node.

[0018] In some implementations, the method further includes:

[0019] In the case that the candidate system node is not found from the other system nodes, the first system node is determined as the target system node.

[0020] In some implementations, when the candidate system node is found from the other system nodes, determining, based on the distance between the first system node and the other system nodes, a candidate system node that is closest to the first system node from the candidate system nodes as the target system node includes:

[0021] In the case where the i-th candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, determining whether the distance between the i-th candidate system node and the first system node is less than an i-1th distance threshold; wherein i is an integer starting from 1, and when i=1, the i-1th distance threshold is a preset distance threshold;

[0022] When the distance between the i-th candidate system node and the first system node is less than a distance threshold, the i-th candidate system node is determined as a target system node, and the distance threshold is updated to an i-th distance threshold, where the i-th distance threshold is equal to the distance between the i-th candidate system node and the first system node.

[0023] In some implementations, the method further includes:

[0024] In response to a bandwidth query request sent by an application, feeding back to the application a memory access bandwidth of each of the plurality of system nodes;

[0025] and / or,

[0026] In response to a monitoring period setting command sent by an application, a monitoring period of the performance monitoring unit is set.

[0027] In a second aspect, the present application provides a memory allocation device, which is applied to a system on chip, wherein the system on chip includes a plurality of system nodes, each of which includes a memory, and a performance monitoring unit is provided in the system on chip for monitoring memory access events in the plurality of system nodes; the device includes:

[0028] A data acquisition unit, configured to obtain a memory access bandwidth of each system node and determine a distance between the first system node and other system nodes when a first thread in the first system node requests memory allocation; wherein the memory access bandwidth of the system node is determined based on a memory access event in the system node;

[0029] A judgment and selection unit is used to determine, from the multiple system nodes, a target system node with the fastest memory access speed when the memory is accessed by the first thread based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes; the target system node is used to allocate memory to the first thread.

[0030] In a third aspect of the present application, a system on chip is proposed, comprising a plurality of system nodes, wherein the system nodes include memory, and a performance monitoring unit is provided in the system on chip for monitoring memory access events in the plurality of system nodes; the system on chip is configured to implement the above-mentioned memory allocation method.

[0031] A fourth aspect of the present application provides a computing device, comprising the above-mentioned system on chip.

[0032] A fifth aspect of the present application provides a computer program product, comprising computer program instructions, which, when executed by a processor, cause the processor to execute the above-mentioned memory allocation method.

[0033] The memory allocation method proposed in the present application comprehensively considers the memory access bandwidth in each system node when allocating memory for the first thread, that is, the load of the memory access channel in each system node is considered, and the distance between the system node where the first thread is located and other system nodes is also considered. Based on the above two aspects, a target system node with the fastest memory access speed when the first thread accesses the memory is selected from multiple system nodes, that is, a system node that can make the memory access speed of the first thread the fastest is selected to provide the required memory for the first thread. The above-mentioned memory allocation method comprehensively considers the memory access bandwidth of each system node and the distance between each system node when allocating memory for the thread, so that the target system node with the best memory access performance can be selected to allocate memory for the thread, so that the memory allocation can be more reasonable, the thread access to the memory can be smoother, and the memory access blocking can be avoided, which is helpful to improve the system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 A schematic diagram of the structure of a NUMA architecture provided in an embodiment of the present application.

[0036] Figure 2 A schematic diagram of the architecture of the memory bandwidth awareness decision module provided in an embodiment of the present application.

[0037] Figure 3 A flowchart of a memory allocation method provided in an embodiment of the present application.

[0038] Figure 4 A schematic diagram of the topological structure of a computer system provided in an embodiment of the present application.

[0039] Figure 5 A schematic diagram of a distance matrix provided in an embodiment of the present application.

[0040] Figure 6 A schematic diagram of a processing flow for determining a target system node provided in an embodiment of the present application.

[0041] Figure 7 A schematic diagram of the structure of a memory allocation device provided in an embodiment of the present application.

[0042] Figure 8 A schematic diagram of a system on a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] NUMA (Non-Uniform Memory Access) is a computer architecture that is derived from the development of multi-processor or multi-processor core technology and is used to optimize the processor's access performance to memory.

[0044] In the NUMA architecture, processors or processor cores are divided into multiple NUMA nodes. Each node has its own independent memory space and bus system, and the NUMA nodes are also interconnected through the bus.

[0045] In one implementation, the NUMA architecture includes a processor including multiple processor cores. The multiple processor cores are divided into different groups, each group includes at least one processor core, and each group serves as a NUMA node.

[0046] In another embodiment, if Figure 1 As shown, the NUMA architecture includes multiple processors, each processor includes multiple processor cores, and the processor cores of each processor are divided into different NUMA nodes.

[0047] Regardless of the method used, each NUMA node is also configured with an independent memory controller. The processor cores in the same NUMA node can share the memory controller. At the same time, the corresponding physical memory is configured for each NUMA node. Each NUMA node can access the memory space of its own configured local memory through its corresponding memory controller. At the same time, it can also access the memory space of other NUMA nodes.

[0048] In the NUMA architecture, each NUMA node corresponds to its own memory controller and memory. Any NUMA node can access its own memory, and can also access the memory of any other node in the architecture. According to the relationship between NUMA nodes, NUMA nodes can be divided into local nodes, neighbor nodes, and remote nodes. The speed at which the processor core accesses the memory of different types of nodes is different. The speed of accessing local nodes is the fastest, and the speed of accessing remote nodes is the slowest.

[0049] In existing applications, the execution of any thread depends on memory space. That is, when the operating system allocates a thread to a processor core for executing a task, the memory space required for the thread to execute the task must also be allocated in the memory. This process is called memory allocation.

[0050] In the NUMA architecture, the problem of memory allocation is also involved. Based on the basic principles of the NUMA architecture, any processor core of any NUMA node in the NUMA architecture can not only access the memory corresponding to its own node, but also access the memory corresponding to other NUMA nodes. Therefore, when allocating memory based on the NUMA architecture, more factors need to be considered.

[0051] Currently, when allocating physical memory for the NUMA architecture, the operating system will select the NUMA node where the current process is located or the NUMA node specified by the user for memory allocation by default. Only when the current NUMA node or the NUMA node specified by the user cannot meet the memory allocation requirements will other NUMA nodes be selected for physical memory allocation. Since the memory channel bandwidth for the memory controller in each NUMA node to access the memory space is limited, if new memory continues to be allocated and used in the node after the bandwidth reaches the upper limit, it will cause memory channel congestion, and the read and write performance of the system memory will drop sharply.

[0052] In response to the above technical problems, the present application proposes a new memory allocation scheme, which is applied to a system-on-chip including multiple system nodes, and can achieve more reasonable and efficient memory allocation in the system-on-chip, and effectively avoid the problem of memory access blockage caused by unreasonable memory allocation.

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] The embodiment of the present application first proposes a memory allocation method, which is applied to a system on chip including multiple system nodes, that is, a system on chip using a NUMA architecture, in which each system node in the system on chip includes a memory, a processor core or a processor, and a memory controller, so that the processor core or the processor in each system node can access the local memory through the local memory controller. In addition, the system nodes of the system on chip are also interconnected through a system on chip bus, so that the processor or the processor core of any system node can access the memory in other system nodes through the system on chip bus.

[0055] In an embodiment of the present application, a performance monitoring unit (PMU) is set in the system on chip to monitor memory access events in multiple system nodes of the system on chip. In this embodiment, the memory access events specifically include write memory events, read memory events, and processor core operation cycles.

[0056] Among them, PMU monitors memory write events by monitoring the sum of the data volume of all write data channels, namely, bandwidth. The bandwidth per unit time reflects the write bandwidth of the memory.

[0057] PMU monitors memory read events by monitoring the number of times the memory sends data, txdat. The memory sends data to the CPU, which means the CPU reads data from the memory. The number of txdat per unit time reflects the memory read bandwidth.

[0058] The PMU monitors the processor core operating cycle by counting the number of CPU cycles. A CPU cycle is the minimum time unit for a CPU to execute instructions, and the value of a CPU cycle is the inverse of the current CPU operating frequency.

[0059] In addition, in order to execute the memory allocation method proposed in the embodiment of the present application, see Figure 2 As shown, the embodiment of the present application sets a memory bandwidth perception decision module in the kernel of the system on chip, which can obtain memory access events obtained by monitoring the PMU in the SOC, and based on the obtained memory access events, execute the memory allocation method proposed in the embodiment of the present application to allocate memory space for the thread to be allocated memory. At the same time, the user software in the user state can also obtain the memory bandwidth related information of the system node from the memory bandwidth perception decision module by way of system call, and can also set the start and close of the memory bandwidth perception decision module, so as to control whether the memory allocation method of the embodiment of the present application is implemented in the system on chip, and can also set the strategy of the memory bandwidth perception decision module, the memory access event monitoring cycle of the PMU, and the relevant thresholds applied to execute the method of this embodiment.

[0060] Continue to see Figure 2 As shown, in the above-mentioned memory bandwidth perception decision module, a DDR PMU driver is set, which is a driver program set corresponding to the PMU in the SOC. The PMU driver provides a function interface to set or read the registers of the PMU, such as the register that controls the clock enable, the register that controls the start and stop sampling of the PMU, the register that stores various event counting results, etc. The kernel thread can use these function interfaces to control the PMU clock enable, start and stop counting, and read the event counting results.

[0061] The above-mentioned kernel thread is a thread running in the memory bandwidth perception decision module for executing the memory allocation method of the embodiment of the present application. The kernel thread uses the DDR PMU driver to collect real-time data, and calculates the current real-time memory bandwidth based on the collected data, and then saves the calculation result to the real-time bandwidth buffer. Finally, the memory bandwidth balancing decision module determines the final memory allocation strategy based on the data in the real-time bandwidth buffer. The specific processing process will be described in detail in the subsequent introduction to the memory allocation method proposed in the embodiment of the present application.

[0062] The above kernel thread is created when the memory bandwidth awareness decision module is initialized. The module initialization process mainly includes:

[0063] 1. Register the character device. After registering the character device, the user software can send commands to the character device through the ioctl system call to interact, such as policy setting commands, real-time bandwidth query commands, sampling period setting or query commands, etc. The character device corresponds to Figure 2 The policy setting or query module, node real-time bandwidth query module, sampling period setting or query module, and threshold setting or query module shown in.

[0064] 2. Configure the default policy. The default policy is not to use the memory allocation policy based on memory bandwidth perception. When the user wants to use it, he can send a policy setting command through the ioctl system call to set the policy to use the memory allocation policy based on memory bandwidth perception. At this time, the memory bandwidth perception decision module allocates memory for the system thread by executing the memory allocation method proposed in the embodiment of the present application.

[0065] 3. Enable the PMU clock by using the function interface provided by the PMU driver. The driver software will write the value of the clock register to 1 according to the address of the PMU clock register in the manual, which is valid. At this time, the PMU is enabled, and the PMU will count all supported events by default.

[0066] 4. A default bandwidth threshold and a default sampling period t will be set during initialization. User software can set new parameters by sending commands to set bandwidth threshold and sampling period through system call ioctl.

[0067] 5. Create a kernel thread.

[0068] In the above-mentioned memory bandwidth perception decision module, there are also setting and query function modules, including a policy setting or query module, a node real-time bandwidth query module, a sampling period setting or query module, and a threshold setting or query module. These modules can be called by user software through system calls, so that users can continue to set or query the policy, bandwidth, sampling period, and threshold of the memory bandwidth perception decision module through these modules.

[0069] The policy setting or query module is used to set or query the memory allocation policy, which will serve as a policy benchmark for memory allocation by the memory bandwidth balancing decision module.

[0070] The node real-time bandwidth query module is used to query the real-time bandwidth of each system node. When the module receives a real-time bandwidth query request, the kernel thread feeds back the collected real-time data to the module so that the module feeds back the query result to the user software.

[0071] The sampling period setting or querying module is used to set or query the memory access event sampling period of the PMU.

[0072] A threshold setting or query module is used to set or query threshold parameters when executing the memory allocation method proposed in the embodiment of the present application.

[0073] Based on the above-mentioned performance monitoring unit PMU, memory bandwidth perception decision module and the setting of the kernel thread of the module, the memory allocation method proposed in the embodiment of the present application can be executed in the on-chip system to realize memory allocation.

[0074] like Figure 3 As shown, the specific processing process of the memory allocation method proposed in the embodiment of the present application includes:

[0075] S101. When a first thread in a first system node requests memory allocation, obtain the memory access bandwidth of each system node, and determine the distance between the first system node and other system nodes.

[0076] Specifically, the first system node mentioned above represents any system node in the system on chip, and the first thread mentioned above represents any thread in the first system node.

[0077] When the first thread requests to allocate content, the operating system sends a memory allocation request to the memory bandwidth awareness decision module, or sends information of the first thread to the memory bandwidth awareness decision module, so that the memory bandwidth awareness decision module determines that the first thread is requesting to allocate memory.

[0078] At this time, the memory bandwidth perception decision module obtains the current memory access bandwidth of each system node, which refers to the memory access bandwidth that has been occupied by the system node. In the case of the same memory bandwidth, if the memory access bandwidth of a system node is larger, it means that the memory access bandwidth that has been occupied is larger and the remaining free memory access bandwidth is smaller; on the contrary, if the memory access bandwidth of a system node is smaller, it means that the memory access bandwidth that has been occupied is smaller and the remaining free memory access bandwidth is larger.

[0079] In some implementations, such as Figure 2 As shown, the kernel thread in the memory bandwidth awareness decision module obtains the memory access events of each system node monitored by the PMU by calling the DDR PMU driver, such as the write memory event bandwidth, the read memory event number txdat, and the processor core operation cycle number cycles of each system node monitored by the PMU.

[0080] Then, based on the memory access events of each system node, the kernel thread calculates the sum of the total amount of written data and the total amount of read data of each system node to determine the total amount of read and write data of each system node. That is, for each system node, the kernel thread calculates and determines the total amount of memory read and write data of the system node based on the memory access events of the system node. Specifically, through the formula The total amount of memory read and write data of the system node can be calculated, where bandwith represents the total amount of write data. Indicates the total amount of read data, and datawidth indicates the data bandwidth.

[0081] Finally, the kernel thread determines the memory access bandwidth of each system node by calculating the ratio of the total amount of memory read and write data and the operation cycle of each system node.

[0082] Specifically, the kernel thread calculates and determines the memory access bandwidth MNL of each system node using the following formula:

[0083]

[0084] in, It indicates that the memory read and write data corresponding to the memory access events monitored by the PMU are summed to obtain the total amount of memory read and write data.

[0085] Through the above processing, the kernel thread in the memory bandwidth awareness decision module calculates and determines the memory access bandwidth of each system node according to the real-time data collected by the PMU.

[0086] In some embodiments, the kernel thread starts from the first system node and calls the interface in the driver of the DDR PMU to start counting the PMU, so that the PMU starts counting according to the set sampling period t, and the kernel thread sleeps for a time period t to prevent the kernel thread from occupying CPU resources. When the kernel thread ends the sleep time t, the PMU has been continuously sampling for a time period t. After the kernel thread ends the sleep time, it calls the function interface provided by the PMU driver to stop the PMU counting, and calculates the current real-time memory access bandwidth of the first system node according to the counting result and the above calculation method, and updates the memory access bandwidth value of the node to Figure 2 The real-time bandwidth buffer is shown.

[0087] Then, the kernel module continues to update the memory access bandwidth of other system nodes according to the above process until the memory access bandwidth of all system nodes is updated, and then continues to sample from the first system node, and the cycle continues.

[0088] Through the above cycle, the kernel thread will continuously update the memory access bandwidth of each system node in the real-time bandwidth buffer, that is, the real-time bandwidth buffer always stores the latest memory access bandwidth of each system node.

[0089] In other implementations, the kernel thread in the memory bandwidth perception decision module obtains the memory access events of each system node monitored by the PMU in real time by calling the DDRPMU driver, and then calculates and determines the memory access bandwidth of each system node through the calculation method introduced in the above embodiment, and stores the memory access bandwidth calculated in real time to Figure 2 Furthermore, when the kernel module calculates a new memory access bandwidth of a system node, the new memory access bandwidth is used to update the historical memory access bandwidth of the system node stored in the real-time bandwidth buffer, so that the memory access bandwidth of each system node stored in the real-time bandwidth buffer is the memory access bandwidth updated in real time.

[0090] Based on the above operations, when the first thread in the first system node requests memory allocation, the memory bandwidth awareness decision module accesses the above real-time bandwidth buffer and reads the stored memory access bandwidth of each system node from the real-time bandwidth buffer.

[0091] In addition to obtaining the memory access bandwidth of each system node in the above manner, the memory bandwidth awareness decision module also determines the distance between the first system node and other system nodes.

[0092] In some implementations, the memory bandwidth awareness decision module determines the distance between the first system node and the other system nodes based on the communication time between the first system node and the other system nodes. In general, the longer the communication time between the first system node and the other system nodes, the longer the distance between the first system node and the other system nodes, and the longer the communication time between the first system node and the other system nodes, the shorter the distance between the first system node and the other system nodes. Among them, the communication time between the first system node and the other system nodes refers to the communication data routing time for the first system node to send communication data to other system nodes at a time during the historical communication process between the first system node and the other system nodes, that is, the time it takes for the sent communication data to reach other system nodes from the first system node.

[0093] The memory bandwidth awareness decision module can pre-set the correspondence between the communication time and the distance between nodes. When determining the communication time between the first system node and other system nodes, the distance between the first system node and other system nodes can be determined in combination with the correspondence.

[0094] In some other implementations, the memory bandwidth awareness decision module quantitatively determines the distance between the first system node and the other system nodes according to the connection relationship between the various system nodes.

[0095] Specifically, the memory bandwidth awareness decision module determines the topology between the system nodes based on the connection relationship between multiple system nodes on the chip. Figure 4 As shown, in this topology, there are nodes and connection edges between nodes. The nodes represent system nodes, and the connection edges between nodes represent communication links between system nodes.

[0096] Then, based on the topological structure between the multiple system nodes, a distance matrix for representing the distance between each system node is generated, and based on the distance matrix, the distance between the first system node and other system nodes is determined.

[0097] Specifically, Figure 4 The weight of the connecting edge in the topological structure shown is set to 1. The number of connecting edges between two nodes represents the distance between the two nodes, which can be generated as follows: Figure 5The distance matrix distance is shown. The rows and columns of the distance matrix represent the system nodes respectively, and the elements in the matrix represent the shortest distance between the row nodes and the column nodes. The element in the i-th row and j-th column is represented by distance(i,j). For example, the value of distance(1,2) is 1, which means that the shortest distance between node 1 and node 2 is 1.

[0098] Based on the above distance matrix, the shortest distance between each system node can be determined intuitively. Therefore, through the distance matrix, the distance (shortest distance) between the first system node and each other system node can be determined.

[0099] S102. Based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, determine, from the multiple system nodes, a target system node that enables the first thread to access the memory at the fastest memory access speed; the target system node is used to provide the required memory for the first thread.

[0100] Specifically, the above-mentioned target system node that enables the first thread to access the memory with the fastest memory access speed refers to a system node that can enable the first thread in the first system node to access its memory at a memory access speed faster than the access speed when the first thread accesses the memory of other system nodes.

[0101] Combining the memory access method and characteristics of the NUMA architecture, it can be understood that if the memory access bandwidth of a system node is smaller, the remaining free memory access bandwidth is larger, and the memory access to the system node is smoother. At the same time, the faster a thread on a system node accesses the memory of a system node that is closer to the system node, the faster it accesses the memory of a system node that is farther away from the system node, and the slower it accesses the memory of a system node that is farther away from the system node.

[0102] Therefore, based on the above rules, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module selects the system node with the smallest memory access bandwidth and / or the closest distance to the first system node from multiple system nodes of the system on chip as the target system node based on the memory access bandwidth of each system node in the system on chip and the distance between the first system node and other system nodes.

[0103] After the target system node is determined in the above manner, the target system node can provide the required memory for the first thread, that is, allocate memory space for the first thread from the memory of the target system node.

[0104] Through the above introduction, it can be seen that the memory allocation method proposed in the embodiment of the present application comprehensively considers the memory access bandwidth in each system node when allocating memory for the first thread, that is, the load of the memory access channel in each system node is considered, and the distance between the system node where the first thread is located and other system nodes is also considered. Based on the above two aspects, a target system node with the fastest memory access speed when the first thread accesses the memory is selected from multiple system nodes, that is, a system node that can make the memory access speed of the first thread the fastest is selected to provide the required memory for the first thread. The above-mentioned memory allocation method comprehensively considers the memory access bandwidth of each system node and the distance between each system node when allocating memory for the thread, so that the target system node that makes the thread access memory performance optimal can be selected to allocate memory for the thread, so that the memory allocation can be made more reasonable, the thread access to memory can be made smoother, and memory access blocking can be avoided, which helps to improve system performance.

[0105] In other embodiments, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module determines, from the multiple system nodes, a target system node that enables the first thread to access the memory with the fastest memory access speed based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes. The memory access bandwidth of the other system nodes is compared with that of the first system node, and the target system node is selected in combination with the distance between the other system nodes and the first system node.

[0106] Specifically, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module compares the memory access bandwidth of the first system node with the memory access bandwidth of other system nodes, and searches for candidate system nodes from other system nodes.

[0107] Assume that the memory access bandwidth of the first system node is MNL cur , the memory access bandwidth of other system nodes is MNL i , where i=0,1,L n-1, and n represents the total number of system nodes.

[0108] The memory bandwidth load balancing decision module in the memory bandwidth awareness decision module converts the memory access bandwidth MNL of the first system node into cur The memory access bandwidth MNL of other system nodes i For comparison, if the memory access bandwidth MNL of a system node in other system nodes i Less than the memory access bandwidth MNL of the first system node cur , and the difference between the memory access bandwidth of the first system node and the memory access bandwidth of the system node is greater than the set bandwidth threshold MNL balance, then the system node is determined as a candidate system node. That is, if for the i-th system node, there is MNL cur >MNL i +MNL balance , then the i-th system node is determined as a candidate system node.

[0109] The bandwidth threshold MNL mentioned above balance , can be flexibly set according to experience or demand. For example, when the remaining free memory access bandwidth of other system nodes is a certain amount more than the remaining free memory access bandwidth of the first system node where the first thread is located, the performance of the first thread accessing the memory of the other system node can be equivalent to the performance of the first thread accessing the memory of the first system node. Then, this certain amount can be used as the bandwidth threshold MNL balance .

[0110] If no candidate system node is found from other system nodes according to the above scheme, the memory bandwidth load balancing decision module in the memory bandwidth perception decision module uses the first system node as the target system node. In this case, it means that there is no candidate system node in other system nodes whose remaining free memory bandwidth is significantly more sufficient than the remaining free memory bandwidth of the first system node where the first thread is located. Since the speed at which the first thread accesses the memory of other system nodes is bound to be lower than the speed at which it accesses the memory of the first system node, in this case, the memory bandwidth load balancing decision module determines the first system node where the first thread is located as the target system node, and allocates memory to the first thread in the target system node, which can ensure the memory access speed and efficiency of the first thread.

[0111] If a candidate system node is found from other system nodes according to the above scheme, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module determines, based on the distance between the first system node and the other system nodes, a candidate system node that is closest to the first system node from the candidate system nodes as the target system node.

[0112] When there is only one candidate system node, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module directly determines the candidate system node as the target system node.

[0113] When there are multiple candidate system nodes, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module selects the system node closest to the first system node from the multiple candidate system nodes as the target system node. The target system node determined in this way is closest to the first system node, which can improve the access speed of the first thread to the memory in the target system node.

[0114] In some embodiments, the memory bandwidth load balancing decision module in the memory bandwidth awareness decision module determines the target system node by traversing other system nodes to search for and update candidate system nodes therefrom.

[0115] See also Figure 6 As shown, the memory bandwidth load balancing decision module in the memory bandwidth perception decision module first inputs the first system node where the first thread is located, that is, inputs the identifier cur of the current node, and then determines whether the data MNL_valid in the current real-time bandwidth buffer is valid, that is, determines whether the data currently stored in the real-time bandwidth buffer is the current memory access bandwidth of each system node. If the data MNL_valid in the current real-time bandwidth buffer is valid, that is, MNL_valid==false, the memory bandwidth load balancing decision module outputs the identifier of the first system node, that is, determines the first system node as the target system node. If the data MNL_valid in the current real-time bandwidth buffer is valid, that is, MNL_valid==true, then traversal and corresponding operations are performed on each system node.

[0116] First, starting from the first node, that is, starting from i=0, the output node is first set to the first system node, that is, result=cur, and the distance threshold is equal to the preset distance threshold INT_MAX, that is, min_distance=INT_MAX.

[0117] For i=0, determine MNL cur >MNL i +MNL balance Is it true? If not, set i=i+1 and return to judge MNL cur >MNL i +MNL balance The step of whether it is established is executed, that is, continue to judge the next system node.

[0118] If it is true, the system node is determined as the i-th candidate system node, and then further judge whether distance(result,i)<min_distance is true, that is, based on the distance between the first system node and other system nodes, judge whether the distance between the i-th candidate system node and the first system node is less than the i-1th distance threshold min_distance (the distance threshold determined last time). If distance(result,i)<min_distance is not true, set i=i+1 and return to judge MNL cur >MNL i +MNL balance Whether the steps are executed.

[0119] If distance(result,i)<min_distance holds, the i-th candidate system node is determined as the target system node, and the distance threshold is updated to the i-th distance threshold, which is equal to the distance between the i-th candidate system node and the first system node. That is, when distance(result,i)<min_distance holds, let result=i, and let min_distance=distance(result,i).

[0120] Then set i=i+1 and return to judge MNL cur >MNL i +MNL balance The steps of whether it is established are executed until i<node_max is not established (node_max represents the maximum number of system nodes), and the latest target system node is output, that is, the latest result is output.

[0121] Through the above traversal process, the selected target system node can be continuously updated until a target system node with sufficient remaining idle memory access bandwidth and closest to the first system node is selected from other system nodes.

[0122] In other embodiments, the memory bandwidth awareness decision module may also receive a bandwidth query request sent by an application, and feedback the memory access bandwidth of each of the multiple system nodes to the application; and / or, in response to a monitoring period setting command sent by the application, set the monitoring period of the performance monitoring unit.

[0123] The above-mentioned query request and setting command can be implemented through functional modules such as a policy setting or query module, a node real-time bandwidth query module, a sampling period setting or query module, and a threshold setting or query module.

[0124] Corresponding to the above-mentioned memory allocation method, an embodiment of the present application further provides a memory allocation device, which is applied to a system on chip, the system on chip comprising a plurality of system nodes, the system nodes comprising memory, and the system on chip being provided with a performance monitoring unit for monitoring memory access events in the plurality of system nodes, see Figure 7 As shown, the device comprises:

[0125] The data acquisition unit 100 is used to obtain the memory access bandwidth of each system node and determine the distance between the first system node and other system nodes when a first thread in the first system node requests memory allocation; wherein the memory access bandwidth of the system node is determined based on the memory access event in the system node;

[0126] The judgment and selection unit 110 is used to determine, from the multiple system nodes, a target system node with the fastest memory access speed when the memory is accessed by the first thread based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes; the target system node is used to allocate memory to the first thread.

[0127] In some implementations, the data collection unit 100 is further configured to:

[0128] Obtaining memory access events of respective system nodes monitored by the performance monitoring unit;

[0129] Based on the memory access events of each system node, the total amount of memory read and write data of each system node is determined by calculating the sum of the total amount of write data and the total amount of read data of each system node;

[0130] The memory access bandwidth of each system node is determined by calculating the ratio of the total amount of memory read and write data and the operation cycle of each system node.

[0131] In some implementations, the data acquisition unit 100 determines the distance between the first system node and other system nodes, including:

[0132] Determine a topological structure between the plurality of system nodes based on a connection relationship between the plurality of system nodes;

[0133] Based on the topological structure between the multiple system nodes, a distance matrix for representing the distance between each system node is generated, and based on the distance matrix, the distance between the first system node and other system nodes is determined.

[0134] In some implementations, the determination and selection unit 110 determines, from the plurality of system nodes, a target system node that enables the first thread to access the memory at the fastest memory access speed based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, including:

[0135] Compare the memory access bandwidth of the first system node with the memory access bandwidth of other system nodes, and search for a candidate system node from the other system nodes; wherein the memory access bandwidth of the candidate system node is smaller than the memory access bandwidth of the first system node, and the difference between the memory access bandwidth of the first system node and the memory access bandwidth of the candidate system node is greater than a set bandwidth threshold;

[0136] In the case where the candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, a candidate system node closest to the first system node is determined from the candidate system nodes as the target system node.

[0137] In some implementations, the determination and selection unit 110 is further configured to:

[0138] In the case that the candidate system node is not found from the other system nodes, the first system node is determined as the target system node.

[0139] In some implementations, when the determination and selection unit 110 finds the candidate system node from the other system nodes, based on the distance between the first system node and the other system nodes, determines, from the candidate system nodes, a candidate system node that is closest to the first system node as the target system node, including:

[0140] In the case where the i-th candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, determining whether the distance between the i-th candidate system node and the first system node is less than an i-1th distance threshold; wherein i is an integer starting from 1, and when i=1, the i-1th distance threshold is a preset distance threshold;

[0141] When the distance between the i-th candidate system node and the first system node is less than a distance threshold, the i-th candidate system node is determined as a target system node, and the distance threshold is updated to an i-th distance threshold, where the i-th distance threshold is equal to the distance between the i-th candidate system node and the first system node.

[0142] In some implementations, the apparatus further includes:

[0143] A query unit, configured to respond to a bandwidth query request sent by an application and feed back to the application a memory access bandwidth of each of the plurality of system nodes;

[0144] and / or,

[0145] The setting unit is used to set the monitoring period of the performance monitoring unit in response to a monitoring period setting command sent by an application.

[0146] The memory allocation device provided in this embodiment belongs to the same application concept as the memory allocation method provided in the above embodiments of this application, and can execute the memory allocation method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For the technical details not described in detail in this embodiment, please refer to the specific processing content of the memory allocation method provided in the above embodiments of this application, which will not be repeated here.

[0147] The functions implemented by the above data acquisition unit 100 and the judgment and selection unit 110 may be implemented by the same or different processors, respectively, and the embodiments of the present application are not limited thereto.

[0148] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0149] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0150] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0151] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0152] Optionally, the present application embodiment further provides a system on chip, see Figure 8 As shown, the system on chip includes multiple system nodes, each of which includes a memory. In addition, each system node may also include a memory controller and a processor or a processor core.

[0153] A performance monitoring unit is provided in the system on chip for monitoring memory access events in multiple system nodes.

[0154] The system on chip is configured to implement the memory allocation method described in any of the above embodiments.

[0155] In addition, another embodiment of the present application further provides a computing device, which includes the above-mentioned system on chip. The computing device can be a computer, a laptop, a smart terminal, a server, a handheld terminal, a wearable device, etc.

[0156] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the memory allocation method described in any of the above embodiments of this specification.

[0157] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0158] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the memory allocation method described in any of the above embodiments of this specification.

[0159] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0160] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0161] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0162] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0164] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0166] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0167] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0168] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0169] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A memory allocation method, characterized in that: Applied to a system on chip, the system on chip includes a plurality of system nodes, the system nodes include memory, and the system on chip is provided with a performance monitoring unit for monitoring memory access events in the plurality of system nodes; the method includes: When a first thread in a first system node requests memory allocation, obtaining a memory access bandwidth of each system node, and determining a distance between the first system node and other system nodes; wherein the memory access bandwidth of the system node is determined based on a memory access event in the system node; Based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, a target system node is determined from the multiple system nodes to enable the first thread to access the memory at the fastest memory access speed; the target system node is used to provide the required memory for the first thread.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining memory access events of respective system nodes monitored by the performance monitoring unit; Based on the memory access events of each system node, the total amount of memory read and write data of each system node is determined by calculating the sum of the total amount of write data and the total amount of read data of each system node; The memory access bandwidth of each system node is determined by calculating the ratio of the total amount of memory read and write data and the operation cycle of each system node.

3. The method according to claim 1, characterized in that Determining the distance between the first system node and other system nodes includes: Determine a topological structure between the plurality of system nodes based on a connection relationship between the plurality of system nodes; Based on the topological structure between the multiple system nodes, a distance matrix for representing the distance between each system node is generated, and based on the distance matrix, the distance between the first system node and other system nodes is determined.

4. The method according to any one of claims 1 to 3, characterized in that Based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes, determining, from the multiple system nodes, a target system node having the fastest memory access speed when the first thread accesses the memory, including: Compare the memory access bandwidth of the first system node with the memory access bandwidth of other system nodes, and search for a candidate system node from the other system nodes; wherein the memory access bandwidth of the candidate system node is smaller than the memory access bandwidth of the first system node, and the difference between the memory access bandwidth of the first system node and the memory access bandwidth of the candidate system node is greater than a set bandwidth threshold; In the case where the candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, a candidate system node closest to the first system node is determined from the candidate system nodes as the target system node.

5. The method according to claim 4, characterized in that The method further comprises: In the case that the candidate system node is not found from the other system nodes, the first system node is determined as the target system node.

6. The method according to claim 4, characterized in that In the case where the candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, determining from the candidate system nodes a candidate system node that is closest to the first system node as a target system node, including: In the case where the i-th candidate system node is found from the other system nodes, based on the distance between the first system node and the other system nodes, determining whether the distance between the i-th candidate system node and the first system node is less than an i-1th distance threshold; wherein i is an integer starting from 1, and when i=1, the i-1th distance threshold is a preset distance threshold; When the distance between the i-th candidate system node and the first system node is less than a distance threshold, the i-th candidate system node is determined as a target system node, and the distance threshold is updated to an i-th distance threshold, where the i-th distance threshold is equal to the distance between the i-th candidate system node and the first system node.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to a bandwidth query request sent by an application, feeding back to the application a memory access bandwidth of each of the plurality of system nodes; and / or, In response to a monitoring period setting command sent by an application, a monitoring period of the performance monitoring unit is set.

8. A memory allocation device, characterized in that: Applied to a system on chip, the system on chip includes a plurality of system nodes, the system nodes include memory, and the system on chip is provided with a performance monitoring unit for monitoring memory access events in the plurality of system nodes; the device includes: A data acquisition unit, configured to obtain a memory access bandwidth of each system node and determine a distance between the first system node and other system nodes when a first thread in the first system node requests memory allocation; wherein the memory access bandwidth of the system node is determined based on a memory access event in the system node; A judgment and selection unit is used to determine, from the multiple system nodes, a target system node with the fastest memory access speed when the memory is accessed by the first thread based on the memory access bandwidth of each system node and the distance between the first system node and other system nodes; the target system node is used to allocate memory to the first thread.

9. A system on chip, characterized in that: The system comprises a plurality of system nodes, each of which comprises a memory, and the system on chip is provided with a performance monitoring unit for monitoring memory access events in the plurality of system nodes; The system on chip is configured to implement the memory allocation method according to any one of claims 1 to 7.

10. A computing device, characterized in that Comprising the system on chip as claimed in claim 9.

11. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to execute the memory allocation method according to any one of claims 1 to 7.

Images