Tensor management method and device based on DRAM-NVM heterogeneous memory

Abstract

The invention relates to a tensor management method and device based on a DRAM-NVM heterogeneous memory, and the method comprises the steps: obtaining training task information and DRAM limitation information of a preset DNN model, and analyzing the training task information and the DRAM limitation information through Profiler to obtain an analysis result; constructing a DRL optimizer according to an analysis result, and determining an access mode, an expected access frequency and a tensor size of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory; and carrying out continuous page migration operation on the target tensor meeting a preset expected access frequency requirement or a preset access mode requirement through the expected access frequency and the tensor size so as to distribute each target tensor to the corresponding target page. Therefore, the problems that the existing heterogeneous memory technology neglects optimization of the actual bandwidth utilization rate of the heterogeneous memory, is difficult to adapt to change requirements of each kernel in a deep learning scene, and has error risks such as inaccuracy, high calculation overhead and insufficient memory are solved.

Description

Technical Field

[0001] This application relates to the field of tensor management technology, and in particular to a tensor management method and apparatus based on DRAM-NVM heterogeneous memory. Background Technology

[0002] Deep neural networks have been widely used in various fields such as image classification, object detection, and natural language processing. However, training large-scale DNN models often encounters significant memory bottlenecks, which requires efficient management of a large number of tensors.

[0003] Existing heterogeneous memory technologies combine persistent memory modules and traditional DRAM in heterogeneous memory systems to address the tensor management challenges during DNN training. However, existing heterogeneous memory technologies suffer from the following problems:

[0004] (1) Characteristics and analysis of persistent memory: Existing technologies only analyze the basic bandwidth and random / sequential access latency of persistent memory in multi-threaded scenarios, but they are insufficient in reducing the high latency of multiple persistent memory accesses and the low bandwidth of persistent memory read and write, and ignore the challenge of optimizing the actual bandwidth utilization of heterogeneous memory.

[0005] (2) Application characteristic analysis: When existing technologies replace hot and cold data in heterogeneous memory systems, they often only consider a limited set of characteristics; static feature analysis cannot adapt to the changing requirements of each kernel in deep learning scenarios and hinders performance improvement.

[0006] (3) Real-time adaptability: Offline solutions are difficult to adapt to changes in the environment quickly. Some existing technologies use crude mathematical techniques to calculate allocation strategies, but they are prone to inaccuracy, high computational overhead and memory shortage errors.

[0007] In summary, existing heterogeneous memory technologies neglect to optimize the actual bandwidth utilization of heterogeneous memory, making it difficult to adapt to the changing requirements of various kernels in deep learning scenarios. Furthermore, they pose risks of inaccuracy, high computational overhead, and insufficient memory, which urgently need to be addressed. Summary of the Invention

[0008] This application provides a tensor management method and apparatus based on DRAM-NVM heterogeneous memory to solve the problems of existing heterogeneous memory technologies neglecting to optimize the actual bandwidth utilization of heterogeneous memory, making it difficult to adapt to the changing requirements of various kernels in deep learning scenarios, and having the risk of errors such as inaccuracy, high computational overhead and insufficient memory.

[0009] The first aspect of this application provides a tensor management method based on DRAM-NVM heterogeneous memory, comprising the following steps: obtaining training task information and DRAM constraint information of a preset DNN model, and analyzing the training task information and the DRAM constraint information through a Profiler to obtain analysis results; constructing a DRL optimizer based on the analysis results, determining the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory, and obtaining the expected access count and tensor size of each target tensor; based on the DRL optimizer, a preset allocator and a migrationer, performing batch processing operations on target tensors that meet the preset expected access count requirements through the expected access count and the tensor size, and simultaneously performing continuous page migration operations on target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page.

[0010] Optionally, in one embodiment of this application, the step of constructing a DRL optimizer based on the analysis results includes: obtaining model information of the preset DNN model and the real-time state of the DRL optimizer, and constructing the DRL optimizer environment based on the model information and the real-time state; obtaining the number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determining the time step of the DRL optimizer based on the number; determining the round information of the DRL optimizer based on the training process of the preset DNN model; determining the state space, action space, and reward function of the DRL optimizer, and constructing the DRL optimizer based on the DRL optimizer environment, the time step, the round information, the state space, the action space, and the reward function.

[0011] Optionally, in one embodiment of this application, the step of performing batch processing operations on target tensors that meet the preset expected access count requirements based on the DRL optimizer, a preset allocator, and a migrater, and performing continuous page migration operations on target tensors that meet the preset access mode requirements, to allocate each target tensor to a corresponding target page, includes: controlling a preset DRL agent to take tensor placement actions according to the real-time state, so that a preset reward generator generates rewards according to the tensor placement actions; based on the rewards, causing the allocator and the migrater to perform batch processing operations and continuous page migration operations to allocate each target tensor in the current kernel to a corresponding target page; updating the DRL optimizer environment and the real-time state, so as to allocate target tensors in other kernels to corresponding target pages according to the updated DRL optimizer environment and real-time state.

[0012] Optionally, in one embodiment of this application, the step of batch processing the target tensors that meet the preset expected access count requirement based on the expected access count and the tensor size includes: sorting each target tensor in descending order based on the expected access count to generate a sorting result; allocating target tensors with the same expected access count to the same page according to the sorting result; and, if a page cannot accommodate target tensors with the same expected access count, allocating each target tensor to the target page according to the tensor size.

[0013] Optionally, in one embodiment of this application, the step of performing a continuous page migration operation on the target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page includes: determining the future reuse distance and future reuse count of each target tensor; and performing a batch processing operation on the target tensors with the same future reuse distance based on the future reuse count and a preset ascending allocation strategy to allocate each target tensor to the contiguous memory space corresponding to the contiguous pages.

[0014] A second aspect of this application provides a tensor management device based on DRAM-NVM heterogeneous memory, comprising: an analysis module, configured to acquire training task information and DRAM constraint information of a preset DNN model, and analyze the training task information and the DRAM constraint information through a Profiler to obtain analysis results; a construction module, configured to construct a DRL optimizer based on the analysis results, determine the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory, and acquire the expected access count and tensor size of each target tensor; and an allocation module, configured to perform batch processing operations on target tensors that meet the preset expected access count requirements based on the DRL optimizer, a preset allocator and a migrationer, and the expected access count and tensor size, and simultaneously perform continuous page migration operations on target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page.

[0015] Optionally, in one embodiment of this application, the construction module includes: a first acquisition unit, configured to acquire model information of the preset DNN model and the real-time state of the DRL optimizer, and construct the DRL optimizer environment based on the model information and the real-time state; a second acquisition unit, configured to acquire the number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determine the time step of the DRL optimizer based on the number; a first determination unit, configured to determine the round information of the DRL optimizer based on the training process of the preset DNN model; and a second determination unit, configured to determine the state space, action space, and reward function of the DRL optimizer, and construct the DRL optimizer based on the DRL optimizer environment, the time step, the round information, the state space, the action space, and the reward function.

[0016] Optionally, in one embodiment of this application, the allocation module includes: a generation unit, configured to control a preset DRL agent to take tensor placement actions according to the real-time state, so that a preset reward generator generates rewards according to the tensor placement actions; an aggregation unit, configured to, based on the rewards, cause the allocator and the migrater to perform batch processing operations and continuous page migration operations, so as to allocate each target tensor in the current kernel to the corresponding target page; and an update unit, configured to update the DRL optimizer environment and the real-time state, so as to allocate target tensors in other kernels to the corresponding target pages according to the updated DRL optimizer environment and real-time state.

[0017] Optionally, in one embodiment of this application, the allocation module further includes: a sorting unit, configured to sort each target tensor in descending order based on the expected number of visits to generate a sorting result; and a management unit, configured to allocate target tensors with the same expected number of visits to the same page according to the sorting result, and, if a page cannot accommodate target tensors with the same expected number of visits, allocate each target tensor to the target page according to the size of the tensor.

[0018] Optionally, in one embodiment of this application, the allocation module further includes: a third determining unit, configured to determine the future reuse distance and future reuse count of each target tensor; and a processing unit, configured to perform batch processing operations on the target tensors with the same future reuse distance based on the future reuse count and a preset ascending allocation strategy, so as to allocate each target tensor to a contiguous memory space corresponding to a contiguous page.

[0019] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the tensor management method based on DRAM-NVM heterogeneous memory as described in the above embodiments.

[0020] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the tensor management method based on DRAM-NVM heterogeneous memory as described above.

[0021] A fifth aspect of this application provides a computer program product, including a computer program that is executed to implement the above-described tensor management method based on DRAM-NVM heterogeneous memory.

[0022] Therefore, the embodiments of this application have the following beneficial effects:

[0023] The embodiments of this application obtain training task information and DRAM constraint information of a preset DNN model, and analyze the training task information and DRAM constraint information using a profiler to obtain analysis results. Based on the analysis results, a DRL optimizer is constructed, and the access pattern of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory is determined, along with the expected access count and tensor size of each target tensor. Based on the DRL optimizer, a preset allocator, and a migrationer, batch processing operations are performed on target tensors that meet the preset expected access count requirements based on the expected access count and tensor size. Simultaneously, continuous page migration operations are performed on target tensors that meet the preset access pattern requirements to allocate each target tensor to the corresponding target page. This reduces the differences in access latency, read / write bandwidth, and capacity of computing memory, bridging the performance differences of heterogeneous memory and improving DNN training performance. Therefore, this solves the problems of existing heterogeneous memory technologies neglecting to optimize the actual bandwidth utilization of heterogeneous memory, making it difficult to adapt to the changing requirements of various kernels in deep learning scenarios, and posing risks of inaccuracy, high computational overhead, and insufficient memory.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1A flowchart illustrating a tensor management method based on DRAM-NVM heterogeneous memory, provided as an embodiment of this application;

[0027] Figure 2 A schematic diagram of the overall framework of a DRL optimizer is provided for one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the logical architecture of a tensor management method based on DRAM-NVM heterogeneous memory provided according to an embodiment of this application;

[0029] Figure 4 A schematic diagram of the computational flow of a tensor management method based on DRAM-NVM heterogeneous memory provided for one embodiment of this application;

[0030] Figure 5 A schematic diagram of tensor aggregation processing is provided for one embodiment of this application;

[0031] Figure 6 This is an example diagram of a tensor management device based on DRAM-NVM heterogeneous memory according to an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0033] Among them, 10-Tensor management device based on DRAM-NVM heterogeneous memory; 100-Analysis module, 200-Construction module, 300-Allocation module; 701-Memory, 702-Processor, 703-Communication interface. Detailed Implementation

[0034] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0035] The following describes a tensor management method and apparatus based on DRAM-NVM heterogeneous memory according to embodiments of this application, with reference to the accompanying drawings. Addressing the problems mentioned in the background section, this application provides a tensor management method based on DRAM-NVM heterogeneous memory. In this method, training task information and DRAM constraint information of a preset DNN model are obtained, and the training task information and DRAM constraint information are analyzed using a profiler to obtain analysis results. A DRL optimizer is constructed based on the analysis results, and the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory is determined, along with the expected access count and tensor size of each target tensor. Based on the DRL optimizer, a preset allocator, and a migrationer, batch processing operations are performed on target tensors that meet the preset expected access count requirements based on the expected access count and tensor size. Simultaneously, continuous page migration operations are performed on target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page. This reduces the differences in access latency, read / write bandwidth, and capacity of computing memory, bridges the performance differences of heterogeneous memory, and improves DNN training performance. This solves the problems of existing heterogeneous memory technologies, such as neglecting to optimize the actual bandwidth utilization of heterogeneous memory, being unable to adapt to the changing requirements of various kernels in deep learning scenarios, and having the risk of errors such as inaccuracy, high computational overhead, and insufficient memory.

[0036] Specifically, Figure 1 This is a flowchart illustrating a tensor management method based on DRAM-NVM heterogeneous memory, provided as an embodiment of this application.

[0037] like Figure 1 As shown, the tensor management method based on DRAM-NVM heterogeneous memory includes the following steps:

[0038] In step S101, the training task information and DRAM limitation information of the preset DNN model are obtained, and the training task information and DRAM limitation information are analyzed by Profiler to obtain the analysis results.

[0039] In step S102, a DRL optimizer is constructed based on the analysis results, and the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory is determined, and the expected number of accesses and tensor size of each target tensor are obtained.

[0040] This embodiment first provides the DNN model training task and DRAM constraint information to the Profiler, enabling the Profiler to analyze and process the DNN model training task and DRAM constraint information. In addition, this embodiment also needs to obtain the expected number of accesses and tensor size of each tensor t accessed in the current kernel k from feature analysis, analyze the access pattern of the tensors, and consider the number and distance of future accesses in each static computation graph, thereby providing reliable data support for the subsequent implementation of DRL hierarchical memory optimization management.

[0041] Optionally, in one embodiment of this application, constructing a DRL optimizer based on the analysis results includes: obtaining model information of a preset DNN model and the real-time state of the DRL optimizer, and constructing a DRL optimizer environment based on the model information and the real-time state; obtaining the number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determining the time step of the DRL optimizer based on the number; determining the round information of the DRL optimizer based on the training process of the preset DNN model; determining the state space, action space, and reward function of the DRL optimizer, and constructing the DRL optimizer based on the DRL optimizer environment, time step, round information, state space, action space, and reward function.

[0042] Those skilled in the art should understand that the heterogeneous memory optimization problem, whether in synchronous or asynchronous scenarios, is an NP-complete problem, with a solution complexity of O(2^3). N The fundamental insight into the DNN training problem indicates that it is consistent with Markov Decision Processes (MDPs). Therefore, in order to solve the given optimization problem more efficiently, embodiments of this application can process the optimization problem through Deep Reinforcement Learning (DRL).

[0043] Understandably, DRL has the following advantages:

[0044] (1) DRL enables DRL agents to dynamically sense and adapt to environmental changes, enhancing their ability to analyze constantly changing model characteristics and make informed decisions.

[0045] (2) DRL can systematically consider the potential benefits of each operation. Unlike traditional heuristics that usually lack foresight, DRL is good at optimizing the future and can manage heterogeneous memory more effectively.

[0046] Figure 2 This is a schematic diagram of the overall framework of the DRL optimizer. Figure 2As shown, the DRL agent obtains crucial data from TensorFlow (TF) kernels, which include tensors and kernels in a static computation graph. During the continuous decision-making process, the DRL agent operates periodically. At each time step, the DRL agent obtains the current state from the environment and selects an action from the action space. The DRL optimizer employs a neural network to approximate the Q-function, which is crucial for mapping the state to the expected reward of all possible actions. This network architecture includes one flattening layer and three fully connected layers, designed to minimize the difference between the predicted Q-value and the target Q-value derived from the Bellman equation. Through this method, the DRL agent progressively optimizes its policy to prioritize actions that maximize cumulative rewards, thereby optimizing the results of each training round. Subsequently, a reward generator evaluates the effectiveness of the executed actions.

[0047] Specifically, the key components of the DRL optimizer are as follows:

[0048] 1. Environment: The environment includes information from the DNN model and the real-time state obtained from the Profiler; important parameters include tensor size, available memory resources, execution order and execution time of each kernel in different tensor states, number of tensors and kernels in the computation graph, and their respective tensor inputs and outputs;

[0049] 2. Time step: Given the extensive search space associated with intermediate tensors, the time step strategy of this application embodiment includes considering all active tensors in memory during the kernel with DRAM bottleneck, which can result in a maximum of T×K steps in a computation graph containing T tensors and K kernels;

[0050] 3. Agent: The agent is the key decision-maker for tensor assignment and transfer during training. At each time step, it obtains information from the environment to select an action and advance to the next state.

[0051] 4. Round: A round represents the entire DNN training process, from the start of the first kernel to the completion of the last kernel; a round may also terminate prematurely if an OOM error caused by tensor operations occurs.

[0052] 5. State Space: The state space in the training scenario is a multi-dimensional discrete space, which can be represented as a one-dimensional vector: [p t H t D left ,P left ]; where p t H represents the current location of the tensor in memory. t It represents a weighted combination of four important tensor features: size s t Future frequency f t Future time rt and lifespan t In addition, D left and P left Indicates the memory resources available during the current kernel period;

[0053] 6. Action Space: The action space represents the placement of tensors within the kernel, and can be concisely represented as a discrete one-dimensional variable.

[0054] 7. Reward Function: After the agent selects an action, it will receive an immediate reward. Depending on the location of the input tensor in the current kernel, the benefit obtained from the allocation or migration operation can be calculated:

[0055]

[0056] When a tensor is moved from one memory location to another, a reward is given based on the normalized migration cost. To prevent OOM errors during training, a significant negative reward is applied when tensor operations result in insufficient available memory. Upon successful completion of a round, a round reward is granted, which represents an additional reward for completing a single round of training.

[0057] Therefore, the embodiments of this application construct a DRL optimizer to achieve global performance optimization and utilize the potential of non-volatile memory to meet the demand for cheaper byte-addressable memory, thus solving the shortcomings of existing memory optimization methods such as lack of efficiency and accuracy, and high overhead.

[0058] In step S103, based on the DRL optimizer, the preset allocator and the migrater, the target tensors that meet the preset expected access number requirements are batch processed according to the expected access number and tensor size. At the same time, the target tensors that meet the preset access mode requirements are continuously migrated to allocate each target tensor to the corresponding target page.

[0059] Furthermore, the embodiments of this application can analyze four key features affecting tensor heat and design a dynamic adaptation algorithm to fine-tune the weights of tensor heat, and quantify the performance impact of tensor assignment and migration; then, a global memory management solution is derived through a DRL optimizer; in order to adapt to the real-world DNN training environment and minimize system overhead, the embodiments of this application can adopt a collaborative offline-online training method to avoid OOM errors; in addition, the embodiments of this application configure DRL parameters to provide a cost-effective DNN training solution.

[0060] It should be noted that, in order to improve the cache hit rate of frequently accessed tensors, the embodiments of this application introduce batch processing operations to group micro-tensors with similar access patterns to the same page; at the same time, the embodiments of this application propose an adjacency operation to allocate tensors with similar access frequencies to consecutive pages; finally, when tensor migration is required, the embodiments of this application can implement consecutive page migration operations to optimize memory bandwidth utilization.

[0061] Therefore, when faced with frequent reads and writes of small tensors and fragmented copying of tensors during DNN training, the embodiments of this application can improve bandwidth utilization and cache hit rate through page-level tensor aggregation and continuous migration, thereby alleviating cache misses and maximizing memory bandwidth utilization. In addition, the embodiments of this application optimize data exchange in heterogeneous memory systems by fully analyzing various tensor and kernel features during training.

[0062] Optionally, in one embodiment of this application, based on a DRL optimizer, a preset allocator, and a migrater, target tensors that meet the preset expected access count requirements are batch processed according to the expected access count and tensor size, while target tensors that meet the preset access mode requirements are continuously migrated to allocate each target tensor to the corresponding target page. This includes: controlling a preset DRL agent to take tensor placement actions according to the real-time state, so that a preset reward generator generates rewards according to the tensor placement actions; based on the rewards, causing the allocator and migrater to perform batch processing operations and continuous page migration operations to allocate each target tensor in the current kernel to the corresponding target page; and updating the DRL optimizer environment and real-time state to allocate target tensors in other kernels to the corresponding target pages according to the updated DRL optimizer environment and real-time state.

[0063] Specifically, the process of performing DRL hierarchical memory optimization management for each kernel in the embodiments of this application is as follows:

[0064] (1) Use the real-time status information provided by Profiler to build the DRL environment and initialize the system state;

[0065] (2) The DRL agent takes tensor placement action based on the current state;

[0066] (3) Based on the agent's actions, the reward generator produces rewards;

[0067] (4) The allocator and migrater perform allocation, release, or migration operations based on the agent selection, such as... Figure 3 As shown;

[0068] (5) When allocating and migrating tensors in memory, perform aggregation operations;

[0069] (6) TensorFlow continues the training task, such as Figure 4 As shown;

[0070] (7) If there are still kernels and tensors remaining to be processed, the embodiments of this application may update the environment and state accordingly and continue to execute;

[0071] (8) When all kernels have finished executing, the agent receives a one-time reward.

[0072] In summary, this application embodiment uses a page-level aggregation method to group micro-tensors with similar access patterns onto the same page, thereby improving cache hit rate, reducing differences in access latency, read / write bandwidth, and capacity of computing memory, bridging performance differences in heterogeneous memory, and enhancing DNN training performance. Furthermore, this application embodiment allocates large tensors with similar access frequencies to consecutive pages to effectively optimize memory bandwidth utilization during tensor migration. It also quantifies tensor heat through four key tensor features, enabling dynamic adjustment of tensor heat to the current state and analysis of the DNN model's computation graph for global performance optimization.

[0073] Optionally, in one embodiment of this application, a batch processing operation is performed on target tensors that meet the preset expected access count requirements based on the expected access count and tensor size, including: sorting each target tensor in descending order based on the expected access count to generate a sorting result; allocating target tensors with the same expected access count to the same page according to the sorting result; and, if a page cannot accommodate target tensors with the same expected access count, allocating each target tensor to a target page according to the tensor size.

[0074] Those skilled in the art will understand that multiple accesses to heterogeneous memory can lead to numerous cache misses, severely impacting performance. To reduce access latency and improve cache hit rate, embodiments of this application can optimize the tensor organization within the virtual address space using a page-level aggregation method, such as... Figure 5 As shown.

[0075] It should be noted that during DNN training, many micro tensors smaller than 4KB are frequently used. These small tensors have a very short lifespan, usually lasting only one kernel cycle. They are distributed across different memory pages, resulting in low memory access efficiency. Therefore, embodiments of this application may introduce batch processing operations on these micro short-term tensors to improve the temporal locality of data during training.

[0076] Specifically, embodiments of this application first obtain the expected number of accesses and size of each tensor t to be accessed in the current kernel k from feature analysis; secondly, embodiments of this application can sort the tensors in descending order of access count (if the access counts are the same, then sort by tensor size). At the start of the kernel, embodiments of this application can allocate the most frequently accessed tensors to the same page in order, ensuring that tensors with the same number of accesses are grouped together. If a page cannot accommodate all tensors with the same number of accesses, embodiments of this application can allocate them in descending order of tensor size until all tensors are allocated in memory.

[0077] Therefore, the batch processing of micro-temporal tensors in this application can effectively enhance the temporal locality of data access, increase the cache hit probability when accessing the page, reduce the possibility of fragmentation within the page, and optimize the utilization of memory space.

[0078] Optionally, in one embodiment of this application, a continuous page migration operation is performed on the target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page, including: determining the future reuse distance and future reuse count of each target tensor; and performing a batch processing operation on target tensors with the same future reuse distance based on the future reuse count and a preset ascending allocation strategy to allocate each target tensor to the continuous memory space corresponding to the continuous page.

[0079] Tensor joins: In addition to batch processing, we also addressed the impact of buffer size on bandwidth utilization.

[0080] Furthermore, embodiments of this application optimize memory bandwidth in tensor copying operations by employing a tensor join mechanism. This mechanism is particularly effective for large, infrequently accessed, and long-lived tensors, as these tensors often involve migration operations. At the beginning of each kernel k, embodiments of this application can define the future reuse distance and future reuse count for each tensor. Long-lived tensors are typically feature maps generated by kernel computation; these tensors exist in memory for extended periods and therefore require efficient management.

[0081] The embodiments of this application employ tensor join operations to allocate tensors to contiguous pages in ascending order of future reuse distance. To ensure memory contiguity, the embodiments of this application batch tensors with the same reuse distance and allocate them to contiguous memory spaces in ascending order of reuse count. Then, a join operation is performed, specifically for tensors in the current kernel that require memory allocation. When the system enters the next kernel, the embodiments of this application continue this process, allocating them to new addresses. Furthermore, tensors allocated in adjacent kernels are typically arranged adjacently in memory, thereby further improving the spatial locality of memory access, promoting sequential read / write operations, and improving memory bandwidth utilization.

[0082] The tensor management method based on DRAM-NVM heterogeneous memory proposed in this application obtains the training task information and DRAM constraint information of a preset DNN model, and analyzes the training task information and DRAM constraint information using a profiler to obtain analysis results. A DRL optimizer is constructed based on the analysis results, and the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory is determined, along with the expected access count and tensor size of each target tensor. Based on the DRL optimizer, a preset allocator, and a migrationer, batch processing operations are performed on target tensors that meet the preset expected access count requirements based on the expected access count and tensor size. Simultaneously, continuous page migration operations are performed on target tensors that meet the preset access mode requirements to allocate each target tensor to the corresponding target page. This reduces the differences in access latency, read / write bandwidth, and capacity of computing memory, bridges the performance differences of heterogeneous memory, and improves DNN training performance.

[0083] Secondly, a tensor management device based on DRAM-NVM heterogeneous memory proposed according to an embodiment of this application will be described with reference to the accompanying drawings.

[0084] Figure 6 This is a block diagram of a tensor management device based on DRAM-NVM heterogeneous memory according to an embodiment of this application.

[0085] like Figure 6 As shown, the tensor management device 10 based on DRAM-NVM heterogeneous memory includes: an analysis module 100, a construction module 200, and an allocation module 300.

[0086] The analysis module 100 is used to obtain the training task information and DRAM limitation information of the preset DNN model, and to analyze the training task information and DRAM limitation information through the Profiler to obtain the analysis results.

[0087] The construction module 200 is used to build a DRL optimizer based on the analysis results, determine the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory, and obtain the expected number of accesses and the tensor size of each target tensor.

[0088] The allocation module 300 is used to perform batch processing operations on target tensors that meet the preset expected access number requirements based on the DRL optimizer, preset allocators and migraters, and tensor size based on the expected access number and tensor size. At the same time, it performs continuous page migration operations on target tensors that meet the preset access mode requirements, so as to allocate each target tensor to the corresponding target page.

[0089] Optionally, in one embodiment of this application, the construction module 200 includes: a first acquisition unit, a second acquisition unit, a first determination unit, and a second determination unit.

[0090] The first acquisition unit is used to acquire the model information of the preset DNN model and the real-time state of the DRL optimizer, and to construct the DRL optimizer environment based on the model information and the real-time state.

[0091] The second acquisition unit is used to acquire the number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determine the time step of the DRL optimizer based on the number.

[0092] The first determining unit is used to determine the round information of the DRL optimizer based on the training process of the preset DNN model.

[0093] The second determining unit is used to determine the state space, action space, and reward function of the DRL optimizer, and to construct the DRL optimizer based on the DRL optimizer environment, time step, round information, state space, action space, and reward function.

[0094] Optionally, in one embodiment of this application, the allocation module 300 includes: a generation unit, an aggregation unit, and an update unit.

[0095] The generation unit is used to control the preset DRL agent to take tensor placement actions according to the real-time status, so that the preset reward generator generates rewards according to the tensor placement actions.

[0096] The aggregation unit, based on rewards, enables the allocator and migrater to perform batch and sequential page migration operations to assign each target tensor in the current kernel to its corresponding target page.

[0097] The update unit is used to update the DRL optimizer environment and real-time state, so as to allocate the target tensors in other kernels to the corresponding target pages according to the updated DRL optimizer environment and real-time state.

[0098] Optionally, in one embodiment of this application, the allocation module 300 further includes a sorting unit and a management unit.

[0099] The sorting unit is used to sort each target tensor in descending order based on the expected number of accesses to generate a sorting result.

[0100] The management unit is used to allocate target tensors with the same expected number of visits to the same page according to the sorting results, and when a page cannot accommodate target tensors with the same expected number of visits, it allocates each target tensor to a target page according to the size of the tensor.

[0101] Optionally, in one embodiment of this application, the allocation module 300 further includes a third determining unit and a processing unit.

[0102] The third determining unit is used to determine the future reuse distance and future reuse count of each target tensor.

[0103] The processing unit is used to perform batch processing operations on target tensors with the same future reuse distance based on the number of future reuses and a preset ascending allocation strategy, so as to allocate each target tensor to the contiguous memory space corresponding to the contiguous pages.

[0104] It should be noted that the foregoing explanation of the tensor management method embodiment based on DRAM-NVM heterogeneous memory also applies to the tensor management device based on DRAM-NVM heterogeneous memory in this embodiment, and will not be repeated here.

[0105] The tensor management device based on DRAM-NVM heterogeneous memory proposed in this application includes an analysis module for acquiring training task information and DRAM constraint information of a preset DNN model, and analyzing the training task information and DRAM constraint information through a profiler to obtain analysis results; a construction module for constructing a DRL optimizer based on the analysis results, determining the access mode of each target tensor corresponding to the current kernel in the target DRAM-NVM heterogeneous memory, and acquiring the expected access count and tensor size of each target tensor; and an allocation module for performing batch processing operations on target tensors that meet the preset expected access count requirements based on the DRL optimizer, a preset allocator, and a migrationer, and performing continuous page migration operations on target tensors that meet the preset access mode requirements, thereby allocating each target tensor to the corresponding target page, thereby reducing the differences in access latency, read / write bandwidth, and capacity of computing memory, bridging the performance differences of heterogeneous memory, and improving DNN training performance.

[0106] Figure 7A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0107] The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0108] When the processor 702 executes the program, it implements the tensor management method based on DRAM-NVM heterogeneous memory provided in the above embodiments.

[0109] Furthermore, electronic devices also include:

[0110] Communication interface 703 is used for communication between memory 701 and processor 702.

[0111] The memory 701 is used to store computer programs that can run on the processor 702.

[0112] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0113] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0114] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0115] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0116] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described tensor management method based on DRAM-NVM heterogeneous memory.

[0117] This application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-described tensor management method based on DRAM-NVM heterogeneous memory.

[0118] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0120] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0122] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0123] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0125] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A tensor management method based on DRAM-NVM heterogeneous memory, characterized in that, The method comprises the following steps: obtaining training task information and DRAM restriction information of a preset DNN model, and analyzing the training task information and the DRAM restriction information through a Profiler to obtain an analysis result; constructing a DRL optimizer according to the analysis result, and determining an access mode of each target tensor corresponding to a current kernel in a target DRAM-NVM heterogeneous memory, and obtaining an expected access frequency and a tensor size of each target tensor; based on the DRL optimizer, a preset allocator and a migrator, and through the expected access frequency and the tensor size, performing batch processing operation on the target tensor meeting a preset expected access frequency requirement, and performing continuous page migration operation on the target tensor meeting a preset access mode requirement, so as to allocate each target tensor to a corresponding target page.

2. The method of claim 1, wherein, The DRL optimizer is constructed according to the analysis result, comprising: obtaining model information of the preset DNN model and a real-time state of the DRL optimizer, and constructing a DRL optimizer environment according to the model information and the real-time state; obtaining the number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determining the time step of the DRL optimizer according to the number; determining the round information of the DRL optimizer based on the training process of the preset DNN model; determining the state space, action space and reward function of the DRL optimizer, and constructing the DRL optimizer based on the DRL optimizer environment, the time step, the round information, the state space, the action space and the reward function.

3. The method of claim 2, wherein, The DRL optimizer, a preset allocator and a migrator are based on the expected access frequency and the tensor size, and the target tensor meeting the preset expected access frequency requirement is subjected to batch processing operation, and the target tensor meeting the preset access mode requirement is subjected to continuous page migration operation, so as to allocate each target tensor to a corresponding target page, comprising: controlling a preset DRL agent to take a tensor placement action according to the real-time state, so that a preset reward generator generates a reward according to the tensor placement action; based on the reward, the allocator and the migrator perform batch processing operation and continuous page migration operation to allocate each target tensor in the current kernel to a corresponding target page; updating the DRL optimizer environment and the real-time state to allocate target tensors in other kernels to corresponding target pages according to the updated DRL optimizer environment and real-time state.

4. The method of claim 1, wherein, The target tensor meeting the preset expected access frequency requirement is subjected to batch processing operation based on the expected access frequency and the tensor size, comprising: based on the expected access frequency, sorting each target tensor in descending order to generate a sorting result; according to the sorting result, allocating target tensors with the same expected access frequency in the same page, and in the case that one page cannot accommodate target tensors with the same expected access frequency, allocating each target tensor to the target page according to the tensor size.

5. The method of claim 4, wherein, The target tensor satisfying the preset access mode requirement is subjected to a continuous page migration operation to allocate each target tensor to a corresponding target page, including: determining the future reuse distance and the future reuse times of each target tensor; based on the future reuse times and a preset ascending allocation strategy, performing a batch processing operation on the target tensors with the same future reuse distance to allocate each target tensor to a continuous memory space corresponding to the continuous page.

6. A tensor management apparatus based on DRAM-NVM heterogeneous memory, characterized in that, including: an analysis module configured to obtain training task information and DRAM limitation information of a preset DNN model, and analyze the training task information and the DRAM limitation information through a Profiler to obtain an analysis result; a construction module configured to construct a DRL optimizer according to the analysis result, determine an access mode of each target tensor corresponding to a current kernel in a target DRAM-NVM heterogeneous memory, and obtain an expected access times and a tensor size of each target tensor; an allocation module configured to perform a batch processing operation on a target tensor satisfying a preset expected access times requirement based on the DRL optimizer, a preset allocator and a migrator, and simultaneously perform a continuous page migration operation on a target tensor satisfying a preset access mode requirement to allocate each target tensor to a corresponding target page through the expected access times and the tensor size.

7. The apparatus of claim 6, wherein, The construction module includes: a first obtaining unit configured to obtain model information of the preset DNN model and a real-time state of the DRL optimizer, and construct a DRL optimizer environment according to the model information and the real-time state; a second obtaining unit configured to obtain a number of target tensors and kernels in the target DRAM-NVM heterogeneous memory, and determine a time step of the DRL optimizer according to the number; a first determining unit configured to determine round information of the DRL optimizer based on a training process of the preset DNN model; a second determining unit configured to determine a state space, an action space and a reward function of the DRL optimizer, and construct the DRL optimizer based on the DRL optimizer environment, the time step, the round information, the state space, the action space and the reward function.

8. An electronic device, comprising: including: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the tensor management method based on DRAM-NVM heterogeneous memory according to any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the tensor management method based on DRAM-NVM heterogeneous memory according to any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed to implement the tensor management method based on DRAM-NVM heterogeneous memory according to any one of claims 1-5.

Images