DMA sparse memory access method, intelligent computing subsystem and intelligent computing platform

Abstract

Provided in the present application are a DMA sparse memory access method, an intelligent computing subsystem and an intelligent computing platform. The method is applied to a DMA engine of an intelligent computing platform, and comprises: when a data access request for a memory of the intelligent computing platform is received, acquiring accessed data information of current accessed data; acquiring data storage information of the memory, and inputting the data storage information and the accessed data information into an initial policy optimization model to acquire predicted policy information; inputting the predicted policy information, the data storage information and the data information into a policy simulation module, so as to execute a first data memory access action on the memory, and outputting execution result information; acquiring reward information corresponding to the execution result information, and completing optimization of the initial policy optimization model to obtain a target policy optimization model; and on the basis of the target policy optimization model and the data access request, outputting corresponding target policy information, so as to execute a second data memory access action on the memory. Thus, the operation stability and efficiency of the intelligent computing platform can be ensured.

Description

A DMA sparse memory access method, intelligent computing subsystem and intelligent computing platform Technical Field

[0001] The present application relates to the field of data processing, and in particular to a DMA sparse memory access method, an intelligent computing subsystem, and an intelligent computing platform. Background Art

[0002] With the rapid development of artificial intelligence, intelligent computing platforms with various functions are constantly emerging. The requirements for the computing and operating capabilities of the central processing unit (CPU) used for operations in intelligent computing platforms are also increasing. The CPU of the intelligent computing platform is mainly used to run various deep learning models. The weight tensors of various models are very large. When the size of the deep learning model is larger than the external memory capacity of the CPU, one way to circumvent the storage limitation of the CPU is to divide the model into multiple parts and run them on multiple CPUs. However, this type of solution affects the model operation effect due to cost and communication problems between the CPUs, resulting in long delays and low throughput in model operation.

[0003] Although the weight tensors used by the CPU to run deep learning models may be large, many weight tensors are sparse because many weight values ​​are zero. Current direct memory access (DMA) engines with online decompression units can process and decompress weight tensors in real time. By performing decompression and data transmission in the DMA engine according to preset decompression strategies and transmission parameters, the weight values ​​can be expanded back to the original weight tensor size in parallel while the processing unit is performing other neural network calculations. However, due to the excessive amount of data processed within the intelligent computing platform, the processing efficiency of the DMA engine continues to decline as the data content and type change, affecting the operating efficiency of the intelligent computing platform.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a DMA sparse memory access method, an intelligent computing subsystem, and an intelligent computing platform, which can solve the problem that in the traditional process of performing decompression and data transmission in the DMA engine according to preset decompression strategies and transmission parameters, when the amount of data processed in the intelligent computing platform is too large, the processing efficiency based on the DMA engine will continue to decline as the data content and type change, thereby affecting the operating efficiency of the intelligent computing platform.

[0006] In a first aspect, an embodiment of the present application provides a DMA sparse memory access method, which is applied to a DMA engine of an intelligent computing platform, wherein the intelligent computing platform also includes a memory, and the DMA engine is equipped with a policy simulation module; the method includes:

[0007] Upon receiving a data access request to the memory, obtaining access data information of current access data corresponding to the data access request; the access data information includes access data size, access data sparsity information, access information, and association information;

[0008] Acquire data storage information of the memory, input the data storage information and the access data information into an initial strategy optimization model, the initial strategy optimization model outputting prediction strategy information based on the data storage information and the access data information, the prediction strategy information including prediction prefetch information and prediction compression information;

[0009] Inputting the prediction strategy information, the data storage information, and the data information into the strategy simulation module, the strategy simulation module being configured to simulate executing a first data access action on the memory according to the prediction strategy information, the data storage information, and the data information, and outputting execution result information;

[0010] Obtaining reward information corresponding to the execution result information, optimizing the initial strategy optimization model according to the reward information, and obtaining a target strategy optimization model;

[0011] Based on the target policy optimization model, the corresponding target policy information is output according to the sparse information, access information and associated information of the current access data corresponding to the data access request. The target policy information includes target prefetch information and target compression information to perform a second data access action on the memory according to the target policy information.

[0012] In some embodiments, the predicted prefetch information includes predicted prefetch duration and predicted prefetch data size, and the predicted compression information includes compression strategy, encoding strategy and transmission parameters; the initial strategy optimization model outputs predicted strategy information based on the data storage information and the access data information, including: the initial strategy optimization model generates the predicted prefetch data size based on the sparsity information, access information and associated information and the data size, and generates the predicted prefetch duration based on the prefetch data size and the data storage information of the memory; the initial strategy optimization model outputs the compression strategy based on the sparse information, outputs the encoding strategy based on the access information and the associated information, and outputs the transmission parameters based on the compression strategy and the encoding strategy; the transmission parameters include sparsity-adjusted transmission priority and transmission speed.

[0013] Exemplarily, before the initial policy optimization model generates the predicted pre-fetched data size based on the sparse information, access information, associated information and the data size, it also includes: judging whether the current access data corresponding to the data access request meets the preset pre-fetch conditions based on the sparse information, access information and associated information; if the current access data meets the preset pre-fetch conditions, the initial policy optimization model generates the predicted pre-fetched data size based on the sparse information, access information, associated information and the data size.

[0014] In some embodiments, the initial policy optimization model includes a feature extraction network, a policy network and a value network; inputting the data storage information and the access data information into the initial policy optimization model includes: inputting the sparse information, the access information and the association information into the feature extraction network to obtain feature information; inputting the data storage information and the feature information into the policy network to obtain multiple prediction policy information; inputting the prediction policy information and the feature information into the value network, and the value network is used to calculate the value information corresponding to each prediction policy information; outputting the prediction policy information corresponding to the maximum value of multiple value information.

[0015] In some embodiments, after the target policy optimization model outputs the corresponding target policy information according to the sparse information, access information and associated information of the current access data corresponding to the data access request, it also includes: obtaining transmission parameter information for executing the target policy information to access data on the memory, the transmission parameter information including transmission speed information and delay information; sending the transmission speed information and delay information to the target policy optimization model, and the target policy optimization model updates the policy information according to the transmission speed information and delay information.

[0016] In some embodiments, obtaining the reward information corresponding to the execution result information includes: obtaining change information of the memory according to the execution result information and the data storage information; and calculating the reward information corresponding to the execution result information according to a preset reward function, where the expression of the preset reward function is:

[0017] In the formula is the value of the reward information, t is the time when the prediction strategy information is executed, h is the data storage information of the memory, a is the prediction strategy information, F i (h, a) is the change information of the memory, i is the identifier of the memory, A eq is the equality constraint coefficient of the prediction strategy information a, y eq is the constant of the equality constraint.

[0018] In some embodiments, optimizing the initial strategy optimization model according to the reward information includes: optimizing the initial strategy optimization model according to the reward information using a reinforcement learning algorithm.

[0019] Exemplarily, the optimizing the initial policy optimization model according to the reward information using a reinforcement learning algorithm includes optimizing the initial policy optimization model according to the reward information using an approximate policy optimization algorithm.

[0020] In a second aspect, an embodiment of the present application provides an intelligent computing subsystem, which is applied to a DMA engine of an intelligent computing platform, wherein the intelligent computing platform further includes a memory, and the DMA engine is equipped with a policy simulation module; the intelligent computing subsystem includes:

[0021] an acquisition module, configured to, upon receiving a data access request to the memory, acquire access data information of current access data corresponding to the data access request; the access data information including access data size, access data sparsity information, access information, and associated information;

[0022] a prediction module, configured to obtain data storage information of the memory, input the data storage information and the access data information into an initial strategy optimization model, and output prediction strategy information based on the data storage information and the access data information, wherein the prediction strategy information includes prediction prefetching information and prediction compression information;

[0023] an output module, configured to input the prediction strategy information, the data storage information, and the data information into the strategy simulation module, wherein the strategy simulation module is configured to simulate executing a first data access action on the memory according to the prediction strategy information, the data storage information, and the data information, and output execution result information;

[0024] an optimization module, configured to obtain reward information corresponding to the execution result information, optimize the initial strategy optimization model according to the reward information, and obtain a target strategy optimization model;

[0025] An action module is used to output corresponding target policy information based on the target policy optimization model according to the sparse information, access information and associated information of the current access data corresponding to the data access request, wherein the target policy information includes target prefetch information and target compression information, so as to perform a second data access action on the memory according to the target policy information.

[0026] In a third aspect, an embodiment of the present application provides an intelligent computing platform, including a DMA engine and a memory, wherein the DMA engine is equipped with a policy simulation module;

[0027] The DMA engine includes a memory, a processor, and a computer program stored in the memory and runnable on the processor, and is characterized in that when the processor executes the computer program, it implements the DMA sparse memory access method provided in any embodiment of the present application.

[0028] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0029] In the technical solution provided by the embodiment of the present application, first, when a data access request to the memory is received, access data information of the current access data corresponding to the data access request is obtained; the access data information includes access data size, access data sparsity information, access information and associated information; then, the data storage information of the memory is obtained, and the data storage information and the access data information are input into an initial policy optimization model, and the initial policy optimization model outputs prediction policy information based on the data storage information and the access data information, and the prediction policy information includes prediction prefetch information and prediction compression information; then, the prediction policy information, the data storage information and the data information are input into the policy simulation module, and the policy simulation module is used to simulate the execution of a first data access action on the memory according to the prediction policy information, the data storage information and the data information, and output execution result information; further, based on the target policy optimization model, the corresponding target policy information is output according to the sparsity information, access information and associated information of the current access data corresponding to the data access request, and the target policy information includes target prefetch information and target compression information, so as to execute a second data access action on the memory according to the target policy information.

[0030] In an embodiment of the present application, the provided method can intelligently determine corresponding prediction policy information based on the data storage information and access data information of the accessed data through an initial policy optimization model when data in the memory of the intelligent computing platform is accessed. Simulation is then performed based on the prediction policy information to determine corresponding reward information, completing the optimization of the initial policy optimization model. Ultimately, a target policy optimization model for accurately and dynamically predicting data access policies is obtained. This ensures the operational stability of the intelligent computing platform and improves the operational efficiency of data within the intelligent computing platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] FIG1 is a schematic flow chart of a DMA sparse memory access method provided in an embodiment of the present application;

[0033] FIG2 is a schematic diagram of the structure of the intelligent computing subsystem provided in an embodiment of the present application;

[0034] FIG3 is a schematic diagram of the structure of the DMA engine provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0036] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0037] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0038] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0039] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0041] To facilitate understanding of the embodiments of the present application, the following briefly explains the terms involved in the embodiments of the present application.

[0042] Reinforcement Learning (RL): Inspired by behavioral psychology, RL focuses on how intelligent agents can take different actions within an environment to maximize cumulative rewards. RL primarily consists of an agent, an environment, a state, an action, and a reward. After an agent performs an action, the environment transitions to a new state, for which the environment provides a reward signal (positive or negative). The agent then executes new actions according to a specific strategy based on the new state and the reward provided by the environment.

[0043] The technical solutions of the embodiments of this application are introduced below.

[0044] With the rapid development of artificial intelligence, intelligent computing platforms with various functions are constantly emerging. The requirements for the computing and operating capabilities of the central processing unit (CPU) used for operations in intelligent computing platforms are also increasing. The CPU of the intelligent computing platform is mainly used to run various deep learning models. The weight tensors of various models are very large. When the size of the deep learning model is larger than the external memory capacity of the CPU, one way to circumvent the storage limitation of the CPU is to divide the model into multiple parts and run them on multiple CPUs. However, this type of solution affects the model operation effect due to cost and communication problems between the CPUs, resulting in long delays and low throughput in model operation.

[0045] Although the weight tensors used by the CPU to run deep learning models may be large, many weight tensors are sparse because many weight values ​​are zero. Current direct memory access (DMA) engines with online decompression units can process and decompress weight tensors in real time. By performing decompression and data transmission in the DMA engine according to preset decompression strategies and transmission parameters, the weight values ​​can be expanded back to the original weight tensor size in parallel while the processing unit is performing other neural network calculations. However, due to the excessive amount of data processed within the intelligent computing platform, the processing efficiency of the DMA engine continues to decline as the data content and type change, affecting the operating efficiency of the intelligent computing platform.

[0046] To address the above issues, please refer to Figure 1, which is a schematic flowchart of a DMA sparse memory access method provided in an embodiment of the present application. As an example and not a limitation, this method can be applied to a DMA engine of an intelligent computing platform, which also includes a memory and a DMA engine equipped with a policy simulation module. The method includes the following steps:

[0047] S101. When receiving a data access request to a memory, obtain access data information of current access data corresponding to the data access request; the access data information includes access data size, access data sparse information, access information, and association information.

[0048] Specifically, when the DMA engine receives a data access request to any memory in the intelligent computing platform, it can obtain the access data information of the access data corresponding to the data access request by parsing the data access request. In order to improve the accuracy of the method provided by this application, the access data size, access data sparse information, access information and associated information are obtained. Among them, the access data size refers to the memory occupied by the current access data, the sparse information of the access data is the number of zero elements or useless elements of the data, the access information is the access situation of the data, and the associated information is the correlation between the data and the surrounding data.

[0049] In some embodiments, a method for obtaining access data sparse information includes: obtaining zero element information and useless element information of the access data, and obtaining data sparse information of the access data using a sparsity calculation formula based on the zero element information and the useless element information. The sparsity calculation formula is as follows: Where X is the value corresponding to the data sparsity information, c is the number of zero elements in the accessed data, d is the number of useless elements in the accessed data, and Z is the total number of elements in the accessed data. By using the value corresponding to the data sparsity information, the DMA engine can easily determine the sparsity of the accessed data, thereby improving the accuracy of the strategy.

[0050] S102. Obtain data storage information of the memory, input the data storage information and access data information into the initial strategy optimization model, and the initial strategy optimization model outputs prediction strategy information based on the data storage information and access data information. The prediction strategy information includes prediction prefetch information and prediction compression information.

[0051] Specifically, the DMA engine obtains the data storage information of the memory, that is, the size, type and remaining storage space information of the data stored in the memory at this time. The data storage information and access data information are then input into the initial policy optimization model. By constructing the initial policy optimization model, the present application can accurately determine the corresponding policy information for different data access requests. In the process of constructing the initial policy optimization model, the initialization policy optimization model first outputs the predicted policy information based on the data storage information and the access data information. In order to improve the timeliness of data access requests, the present application predicts the strategy by pre-fetching and compressing the data. By designing a suitable pre-fetching strategy, the data access delay can be reduced and the data transmission efficiency can be improved. By predicting the access pattern of the data, the data can be loaded into the cache in advance when needed, reducing the waiting time of the DMA engine data transmission. By compressing and encoding sparse data, the data transmission volume can be reduced and the transmission efficiency can be improved. Selecting a suitable compression algorithm and encoding method can save bandwidth and reduce transmission delay during the DMA engine data transmission process.

[0052] In some embodiments, the predicted prefetch information includes predicted prefetch duration and predicted prefetch data size, and the predicted compression information includes compression strategy, encoding strategy and transmission parameters; the initial strategy optimization model outputs predicted strategy information based on the data storage information and the access data information, including: the initial strategy optimization model generates the predicted prefetch data size based on the sparsity information, access information and associated information and the data size, and generates the predicted prefetch duration based on the prefetch data size and the data storage information of the memory; the initial strategy optimization model outputs the compression strategy based on the sparsity information, outputs the encoding strategy based on the access information and the associated information, and outputs the transmission parameters based on the compression strategy and the encoding strategy; the transmission parameters include sparsity-adjusted transmission priority and transmission speed. Therefore, the initial strategy optimization model can accurately generate prefetch parameters and compression parameters, reduce data access delay, improve data transmission efficiency and save bandwidth.

[0053] Exemplarily, before the initial policy optimization model generates the predicted pre-fetched data size based on the sparse information, access information, associated information and the data size, it also includes: judging whether the current access data corresponding to the data access request meets the preset pre-fetch conditions based on the sparse information, access information and associated information; if the current access data meets the preset pre-fetch conditions, the initial policy optimization model generates the predicted pre-fetched data size based on the sparse information, access information, associated information and the data size.

[0054] When the amount of accessed data is large, the data is loaded into the cache in advance by selecting an appropriate pre-fetched data size, thereby reducing the waiting time of the DMA engine data transmission.

[0055] In some embodiments, the initial policy optimization model includes a feature extraction network, a policy network and a value network; inputting the data storage information and the access data information into the initial policy optimization model includes: inputting the sparse information, the access information and the association information into the feature extraction network to obtain feature information; inputting the data storage information and the feature information into the policy network to obtain multiple prediction policy information; inputting the prediction policy information and the feature information into the value network, and the value network is used to calculate the value information corresponding to each prediction policy information; outputting the prediction policy information corresponding to the maximum value of multiple value information.

[0056] The reinforcement learning method of the present application constructs an intelligent agent that performs data access actions on the memory, and inputs the sparse information, the access information and the associated information as the environmental information corresponding to the intelligent agent into the feature extraction network of the initial policy optimization model, and the extracted feature information is input into the policy network, and the policy network outputs multiple prediction strategies. The prediction strategy and feature information are then input into the value network, and the value network outputs the value information corresponding to each prediction strategy. Finally, the output prediction strategy information is determined among multiple prediction strategies based on the value information corresponding to each prediction strategy. This enables the method provided by the present application to obtain the most valuable prediction strategy information of the intelligent agent at the moment, thereby improving the accuracy of the strategy generated after the initial policy optimization model is constructed.

[0057] Exemplarily, the feature extraction network includes a first fully connected layer, a second fully connected layer, a third fully connected layer, and a splicing layer; the step of inputting the sparse information, the access information, and the associated information into the feature extraction network to obtain feature information includes: inputting the sparse information, the access information, and the associated information into the first fully connected layer, the second fully connected layer, and the third fully connected layer, respectively, to output first feature information, second feature information, and third feature information, respectively; inputting the first feature information, the second feature information, and the third feature information into the splicing layer to splice the first feature information, the second feature information, and the third feature information to obtain the feature information. The features are then processed, and the processed features can be used for accurate acquisition strategies.

[0058] In some embodiments, the intelligent computing platform includes multiple memories, and obtaining data storage information of the memories includes: obtaining identification information of the memories corresponding to the data access request; and obtaining data storage information corresponding to the memories based on the identification information. When the intelligent computing platform includes multiple memories, the provided method can accurately determine the memories corresponding to the data access request, thereby enabling real-time, dynamic, and accurate memory access to the multiple memories.

[0059] S103: Input the prediction strategy information, data storage information and data information into a strategy simulation module, which is used to simulate executing a first data access action on the memory according to the prediction strategy information, data storage information and data information, and output execution result information.

[0060] Specifically, after the initial policy optimization model outputs prediction policy information based on data storage information and access data information, the DMA engine configures a policy simulation module to simulate the first data access operation corresponding to the prediction policy information. The policy simulation module simulates memory access operations based on the prediction policy information generated by the initial policy optimization model. This enables timely execution of the corresponding prediction policy information in the context of one or more memories, allowing for rapid verification of the accuracy of the prediction policy information.

[0061] S104. Obtain reward information corresponding to the execution result information, optimize the initial strategy optimization model based on the reward information, and obtain the target strategy optimization model.

[0062] Specifically, when the strategy simulation module executes the first data access action, the DMA engine measures the accuracy of the predicted strategy information by obtaining the reward information corresponding to the first data access action, and can determine the impact of executing the first data access action. By calculating the reward information, the DMA engine can iteratively optimize the initial strategy optimization model, thereby significantly improving the performance of the target strategy optimization model. In some embodiments, obtaining the reward information corresponding to the execution result information includes: obtaining the change information of the memory based on the execution result information and the data storage information; calculating the reward information corresponding to the execution result information based on a preset reward function, and the expression of the preset reward function is:

[0063] In the formula is the value of the reward information, t is the time when the prediction strategy information is executed, h is the data storage information of the memory, a is the prediction strategy information, F i (h, a) is the change information, i is the identifier of the memory, for example, when there are multiple memories, i is the serial number corresponding to the memory, A eq is the equality constraint coefficient of the prediction strategy information a, y eq is the constant of the equality constraint.

[0064] Exemplarily, the reward function is the cumulative time of memory change information minus the constraint violation penalty. The step of obtaining the memory change information based on the execution result information and the data storage information includes: obtaining the memory transmission speed change information and storage space change information based on the execution result information and the data storage information; and calculating the memory change information based on the transmission speed change information and the storage space change information using a preset change information calculation formula, wherein the change information calculation formula includes: In the formula, △ is the value of the change information, V1 and V2 are the transmission speed change information and storage space change information respectively, U1 is the transmission speed of the memory after executing the first data access action, U2 is the transmission speed of the memory before executing the first data access action, T1 is the storage space size of the memory after executing the first data access action, T2 is the storage space size of the memory before executing the first data access action, α and β are the thresholds corresponding to the transmission speed change information and storage space change information respectively. This application does not limit the size of α and β.

[0065] In some embodiments, optimizing the initial strategy optimization model according to the reward information includes: optimizing the initial strategy optimization model according to the reward information using a reinforcement learning algorithm.

[0066] The DMA engine can obtain the corresponding prediction strategy information in real time and optimize the initial strategy optimization model through reinforcement learning algorithms, such as the PPO algorithm, thereby significantly improving the accuracy of the strategy.

[0067] Exemplarily, the optimizing the initial policy optimization model according to the reward information using a reinforcement learning algorithm includes optimizing the initial policy optimization model according to the reward information using an approximate policy optimization algorithm.

[0068] The training module uses the Proximal Policy Optimization (PPO) algorithm for reinforcement learning training. The PPO algorithm iteratively optimizes the model's value function and policy using the generated reward information. The mean squared error (MSE) is used as the value loss function to optimize value estimation, while the Kullback-Leibler divergence (KL divergence) is used as the information entropy loss function to maintain a certain level of policy exploration capability.

[0069] S105. Based on the target policy optimization model, the corresponding target policy information is output according to the sparse information, access information and associated information of the current access data corresponding to the data access request. The target policy information includes target prefetch information and target compression information to perform a second data access action on the memory according to the target policy information.

[0070] Specifically, after building the target policy optimization model, the DMA engine can output the corresponding target policy information in real time based on the sparsity information, access information, and associated information of the currently accessed data corresponding to the data access request. This allows it to accurately obtain the corresponding target compression information and complete precise memory access to the memory data. This ensures the stability of the intelligent computing platform and improves the operational efficiency of the data within the intelligent computing platform.

[0071] In some embodiments, after the target policy optimization model outputs the corresponding target policy information according to the sparse information, access information and associated information of the current access data corresponding to the data access request, it also includes: obtaining transmission parameter information for executing the target policy information to access data on the memory, the transmission parameter information including transmission speed information and delay information; sending the transmission speed information and delay information to the target policy optimization model, and the target policy optimization model updates the policy information according to the transmission speed information and delay information.

[0072] During the use of the target strategy optimization model, the DMA engine monitors system performance indicators such as data transfer speed and latency in real time and dynamically adjusts the DMA data transfer strategy based on real-time feedback. By continuously optimizing and adjusting the DMA control strategy, the efficiency and performance of sparse memory access DMA can be maximized.

[0073] In the embodiments of the present application, when data in the memory of the intelligent computing platform is accessed, the initial policy optimization model is intelligently determined based on the data storage information of the memory and the access data information of the accessed data. Simulation is then performed based on the predicted policy information to determine the corresponding reward information, thereby optimizing the initial policy optimization model. Ultimately, a target policy optimization model for accurately and dynamically predicting data access policies is obtained. This ensures the operational stability of the intelligent computing platform and improves the operational efficiency of data within the intelligent computing platform.

[0074] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] Corresponding to the DMA sparse memory access method described in the above embodiment, FIG2 shows a block diagram of the structure of the intelligent computing subsystem provided by the embodiment of the present application. The device can be applied to the DMA engine of the intelligent computing platform shown in FIG2. The intelligent computing platform also includes a memory, and the DMA engine 11 is equipped with a policy simulation module. For ease of explanation, only the parts related to the embodiment of the present application are shown. Referring to FIG2, the device includes:

[0076] The acquisition module 201 is used to acquire access data information of current access data corresponding to the data access request when receiving the data access request to the memory; the access data information includes access data size, access data sparse information, access information and association information.

[0077] The prediction module 202 is used to obtain the data storage information of the memory, input the data storage information and the access data information into the initial strategy optimization model, and the initial strategy optimization model outputs prediction strategy information based on the data storage information and the access data information, and the prediction strategy information includes prediction prefetching information and prediction compression information.

[0078] The output module 203 is used to input the prediction strategy information, the data storage information and the data information into the strategy simulation module. The strategy simulation module is used to simulate the execution of the first data access action on the memory according to the prediction strategy information, the data storage information and the data information, and output the execution result information.

[0079] The optimization module 204 is used to obtain reward information corresponding to the execution result information, and optimize the initial strategy optimization model according to the reward information to obtain a target strategy optimization model.

[0080] The action module 205 is used to output corresponding target policy information based on the sparse information, access information and associated information of the current access data corresponding to the data access request based on the target policy optimization model, and the target policy information includes target prefetch information and target compression information, so as to perform a second data access action on the memory according to the target policy information.

[0081] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0082] The intelligent computing platform provided in this application includes a DMA engine 11 and a memory, and the DMA engine 11 is equipped with a strategy simulation module.

[0083] Please also refer to Figure 3, which is a schematic diagram of the structure of the DMA engine provided in an embodiment of the present application. The DMA engine 11 of this embodiment includes: at least one processor 111, a memory 112, and a computer program 113 stored in the memory 112 and executable on the at least one processor 111. When the processor 111 executes the computer program 113, the steps of any of the above-mentioned method embodiments are implemented.

[0084] The DMA engine can be installed on a computing device such as a desktop computer, a notebook computer, a PDA, or a cloud server. The computer device in which the DMA engine is installed may include, but is not limited to, a processor 111 and a memory 112. Those skilled in the art will appreciate that FIG3 is merely an example of a DMA engine 11 and does not limit the DMA engine 11. The DMA engine 11 may include more or fewer components than shown, or may combine certain components or different components. For example, it may also include input and output devices, network access devices, etc.

[0085] The processor 111 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0086] In some embodiments, the memory 112 may be an internal storage unit of the DMA engine 11, such as a hard disk or memory of the DMA engine 11. In other embodiments, the memory 112 may also be an external storage device of the DMA engine 11, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the DMA engine 11. Furthermore, the memory 112 may also include both an internal storage unit of the DMA engine 11 and an external storage device. The memory 112 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 112 may also be used to temporarily store data that has been output or is about to be output.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0088] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the DMA engine, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0090] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0091] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0093] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0094] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A DMA sparse memory access method, characterized in that: A DMA engine is applied to an intelligent computing platform, wherein the intelligent computing platform further includes a memory and the DMA engine is equipped with a policy simulation module; the method includes: Upon receiving a data access request to the memory, obtaining access data information of current access data corresponding to the data access request; the access data information includes access data size, access data sparsity information, access information, and association information; Acquire data storage information of the memory, input the data storage information and the access data information into an initial strategy optimization model, the initial strategy optimization model outputting prediction strategy information based on the data storage information and the access data information, the prediction strategy information including prediction prefetch information and prediction compression information; Inputting the prediction strategy information, the data storage information, and the data information into the strategy simulation module, the strategy simulation module being configured to simulate executing a first data access action on the memory according to the prediction strategy information, the data storage information, and the data information, and outputting execution result information; Obtaining reward information corresponding to the execution result information, optimizing the initial strategy optimization model according to the reward information, and obtaining a target strategy optimization model; Based on the target policy optimization model, the corresponding target policy information is output according to the sparse information, access information and associated information of the current access data corresponding to the data access request. The target policy information includes target prefetch information and target compression information to perform a second data access action on the memory according to the target policy information.

2. The method according to claim 1, characterized in that The predicted prefetch information includes a predicted prefetch duration and a predicted prefetch data size; the predicted compression information includes a compression strategy, an encoding strategy, and transmission parameters; the initial strategy optimization model outputs predicted strategy information based on the data storage information and the access data information, including: The initial strategy optimization model generates the predicted prefetch data size according to the sparse information, access information, association information and the data size, and generates the predicted prefetch duration according to the prefetch data size and data storage information of the memory; The initial strategy optimization model outputs the compression strategy according to the sparse information, outputs the encoding strategy according to the access information and the associated information, and outputs the transmission parameters according to the compression strategy and the encoding strategy; the transmission parameters include sparsity-adjusted transmission priority and transmission speed.

3. The method according to claim 2, characterized in that Before the initial policy optimization model generates the predicted prefetched data size according to the sparse information, the access information, the association information and the data size, the method further includes: Determining whether the current access data corresponding to the data access request meets a preset pre-fetch condition according to the sparse information, the access information and the associated information; If the currently accessed data meets the preset pre-fetch condition, the initial strategy optimization model generates the predicted pre-fetched data size according to the sparse information, access information, association information and the data size.

4. The method according to claim 1, wherein The initial strategy optimization model includes a feature extraction network, a strategy network and a value network; Inputting the data storage information and the access data information into the initial strategy optimization model includes: Inputting the sparse information, the access information and the association information into the feature extraction network to obtain feature information; Inputting the data storage information and the feature information into a strategy network to obtain a plurality of prediction strategy information; Inputting the prediction strategy information and the feature information into the value network, wherein the value network is used to calculate the value information corresponding to each prediction strategy information; The prediction strategy information corresponding to the maximum value of the multiple value information is output.

5. The method according to claim 1, wherein After outputting corresponding target policy information according to the sparse information, access information, and associated information of the current access data corresponding to the data access request based on the target policy optimization model, the method further includes: Acquire transmission parameter information for executing the target policy information to access data from the memory, the transmission parameter information including transmission speed information and delay information; The transmission speed information and the delay information are sent to the target policy optimization model, and the target policy optimization model updates the policy information according to the transmission speed information and the delay information.

6. The method according to claim 1, characterized in that The obtaining of reward information corresponding to the execution result information includes: Acquiring change information of the memory according to the execution result information and the data storage information; The reward information corresponding to the execution result information is calculated according to a preset reward function, where the expression of the preset reward function is: In the formula is the value of the reward information, t is the time when the prediction strategy information is executed, h is the data storage information of the memory, a is the prediction strategy information, F i (h, a) is the change information of the memory, i is the identifier of the memory, A eq is the equality constraint coefficient of the prediction strategy information a, y eq is the constant of the equality constraint.

7. The method according to claim 1, characterized in that The optimizing the initial strategy optimization model according to the reward information includes: A reinforcement learning algorithm is used to optimize the initial strategy optimization model according to the reward information.

8. The method according to claim 7, characterized in that The optimization of the initial strategy optimization model using a reinforcement learning algorithm according to the reward information includes: An approximate policy optimization algorithm is used to optimize the initial policy optimization model according to the reward information.

9. An intelligent computing subsystem, characterized in that: A DMA engine applied to an intelligent computing platform, the intelligent computing platform also including a memory, the DMA engine being equipped with a policy simulation module; The intelligent computing subsystem includes: an acquisition module, configured to, upon receiving a data access request to the memory, acquire access data information of current access data corresponding to the data access request; the access data information including access data size, access data sparsity information, access information, and associated information; a prediction module, configured to obtain data storage information of the memory, input the data storage information and the access data information into an initial strategy optimization model, and output prediction strategy information based on the data storage information and the access data information, wherein the prediction strategy information includes prediction prefetching information and prediction compression information; an output module, configured to input the prediction strategy information, the data storage information, and the data information into the strategy simulation module, wherein the strategy simulation module is configured to simulate executing a first data access action on the memory according to the prediction strategy information, the data storage information, and the data information, and output execution result information; an optimization module, configured to obtain reward information corresponding to the execution result information, optimize the initial strategy optimization model according to the reward information, and obtain a target strategy optimization model; An action module is used to output corresponding target policy information based on the target policy optimization model according to the sparse information, access information and associated information of the current access data corresponding to the data access request, wherein the target policy information includes target prefetch information and target compression information, so as to perform a second data access action on the memory according to the target policy information.

10. An intelligent computing platform, characterized in that: It includes a DMA engine and a memory, wherein the DMA engine is equipped with a strategy simulation module; The DMA engine includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images