GPU-based memory dynamic allocation method, device, computer equipment and medium

Abstract

The present application relates to a GPU-based memory dynamic allocation method, device, computer equipment and medium, the method comprising real-time monitoring and obtaining the memory access behavior of computing tasks; predicting the memory access mode of computing tasks using a preset memory prediction model according to the memory access behavior; determining the data access frequency of computing tasks according to the memory access mode, and judging the storage trend of data according to the data access frequency, and allocating the corresponding computing tasks to the corresponding storage environment based on the storage trend; obtaining corresponding real-time feedback information according to the allocation action of the computing tasks, and dynamically adjusting the memory allocation strategy based on the real-time feedback information. The present application has the effect of improving the efficiency of memory allocation.

Description

Technical Field

[0001] The present application relates to the technical field of memory management, and in particular to a GPU-based memory dynamic allocation method, device, computer equipment and medium. Background Art

[0002] At present, as the complexity of GPU computing tasks continues to increase, GPU memory management has become an important factor affecting computing efficiency and performance. GPU memory resources are limited and have high bandwidth requirements. When performing tasks such as deep learning, image processing, and data analysis, how to effectively allocate memory resources to improve task execution efficiency has become an urgent problem to be solved.

[0003] Existing GPU memory management methods mainly rely on static configuration or memory allocation based on preset rules, usually through static parameters such as task priority and memory size. However, these methods fail to fully consider the timing and locality characteristics of memory access during the execution of computing tasks, and lack the ability to adapt to the dynamic changes of memory access behavior in real time. Therefore, there are problems such as low memory resource utilization, serious memory bandwidth bottleneck and insufficient computing efficiency.

[0004] The above-mentioned existing technical solutions have the following defects: the existing GPU memory management method fails to dynamically optimize according to the memory access mode of the computing task, resulting in waste of memory resources and performance bottlenecks, so there is room for improvement. Summary of the invention

[0005] In order to improve the efficiency of memory allocation, the present application provides a GPU-based memory dynamic allocation method, device, computer equipment and medium.

[0006] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A GPU-based memory dynamic allocation method, the GPU-based memory dynamic allocation method comprising: Monitor and obtain the memory access behavior of computing tasks in real time; Predicting a memory access pattern of the computing task using a preset memory prediction model according to the memory access behavior; Determine the data access frequency of the computing task according to the memory access mode, judge the storage trend of the data according to the data access frequency, and allocate the corresponding computing task to the corresponding storage environment based on the storage trend; Corresponding real-time feedback information is obtained according to the allocation action of the computing task, and a memory allocation strategy is dynamically adjusted based on the real-time feedback information.

[0007] By adopting the above technical solution, by real-time monitoring and obtaining the memory access behavior of the computing task, the memory access process of the task can be dynamically tracked, and the usage of the task memory can be understood in real time, thereby providing data support for subsequent memory allocation; by predicting the memory access mode of the computing task based on the memory access behavior using a preset memory prediction model, the memory access characteristics of the computing task can be pre-identified, the memory allocation strategy can be optimized, and the waste of memory resources and access conflicts can be avoided; by determining the data access frequency of the computing task based on the memory access mode, and judging the storage direction of the data based on the data access frequency, high-frequency data can be preferentially stored in an efficient storage area such as GPU video memory according to the data access frequency, and low-frequency data can be stored in a less efficient storage area, thereby improving the utilization efficiency of the memory; by obtaining the corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjusting the memory allocation strategy based on the real-time feedback information, the allocation of memory resources can be timely adjusted according to the task execution situation, memory bottlenecks can be avoided, and the execution efficiency of the computing task and the response speed of the memory can be improved.

[0008] In one example, the present application may be further configured as follows: the construction of the preset memory prediction model specifically includes: Obtain historical memory access behavior, including the memory access frequency of the computing task, the task execution stage, etc.; According to the memory access frequency and the task execution stage, the historical memory access data is divided into access types to obtain a model training data set; According to the model training data set, the preset timing model is iteratively trained to learn the rules and patterns of data access, thereby generating the memory prediction model.

[0009] By adopting the above technical solution, by obtaining historical memory access behaviors, including the memory access frequency of computing tasks, the task execution stage, etc., it is possible to provide comprehensive historical data support for building a memory prediction model, thereby improving the accuracy of model predictions; by dividing the historical memory access data into access types according to the memory access frequency and the task execution stage, it is possible to provide a more accurate data set for model training according to different types of memory access behaviors, thereby improving the accuracy of memory access pattern predictions; by iteratively training the preset timing model according to the model training data set, it is possible to continuously optimize the model's prediction capability, thereby improving the intelligence level of memory resource allocation and making the use of memory resources more efficient.

[0010] In one example, the present application may be further configured as follows: determining the data access frequency of the computing task according to the memory access mode, and judging the storage direction of the data according to the data access frequency, specifically including: Analyzing the temporal and spatial characteristics of memory access according to the memory access pattern, and identifying high-frequency access data and low-frequency access data; For the high-frequency access data, the computing task is preferentially stored in the GPU memory to reduce memory access latency and improve computing efficiency; For the low-frequency access data, the computing task is stored in the host memory or remote storage to save GPU video memory resources and optimize memory usage efficiency.

[0011] By adopting the above technical solution, by analyzing the time and spatial characteristics of memory access according to the memory access mode, identifying high-frequency access data and low-frequency access data, it is possible to accurately distinguish the access frequency of different data, thereby adjusting the data storage location according to the frequency; by preferentially storing computing tasks in the GPU video memory for high-frequency access data, it is possible to reduce memory access latency and improve computing efficiency and task execution speed; by storing computing tasks in the host memory or remote storage for low-frequency access data, it is possible to save GPU video memory resources, thereby optimizing the use of memory resources and avoiding excessive occupation of video memory resources.

[0012] In one example, the present application may be further configured as follows: analyzing the temporal characteristics and spatial characteristics of memory access according to the memory access pattern to identify high-frequency access data and low-frequency access data, specifically including: Analyzing the access frequency of the computing task on the time axis according to the memory access pattern, and calculating the access frequency of the computing task in different time periods by using a sliding window algorithm, thereby identifying the high-frequency access data and the low-frequency access data; According to the memory access pattern, the access location and access pattern of the computing task are analyzed, and based on the locality principle of the memory access pattern, the access path and hot spot area of ​​the computing task are tracked, so as to identify the high-frequency access data and the low-frequency access data.

[0013] By adopting the above technical solution, by analyzing the access frequency of computing tasks on the timeline according to the memory access pattern, and using the sliding window algorithm to calculate the access frequency of computing tasks in different time periods, it is possible to dynamically evaluate the changing trend of memory access frequency, thereby optimizing data storage decisions; by analyzing the access location and access pattern of computing tasks according to the memory access pattern, and based on the locality principle of memory access pattern, it is possible to effectively identify the hot spots of computing tasks, thereby improving the efficiency of data storage and access, and avoiding excessive allocation or waste of resources.

[0014] In one example, the present application may be further configured as follows: obtaining corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjusting the memory allocation strategy based on the real-time feedback information, specifically including: Obtaining a memory access exception during the execution of the current computing task, and determining the resource occupancy of the computing task based on the memory access exception; According to the resource usage, the corresponding memory allocation strategy is adjusted.

[0015] By adopting the above technical solution, by obtaining the memory access anomalies during the execution of the current computing task, memory access problems in the task execution, such as frequent cache misses, can be discovered in time, thereby preventing performance bottlenecks; by judging the resource usage of the computing task based on the memory access anomalies, it is possible to accurately evaluate whether the resources required for the task exceed the system's carrying capacity, thereby avoiding resource overload or shortage during task execution and improving the stability and reliability of the system.

[0016] In one example, the present application may be further configured as follows: adjusting the corresponding memory allocation strategy according to the resource occupancy situation specifically includes: When the resource occupancy condition is that the memory bandwidth utilization of the high-frequency access data is insufficient and the GPU video memory has not reached the upper limit, dynamically expand the GPU video memory pool and migrate the high-frequency access data to the GPU video memory; In the case where the resource occupancy situation is that low-frequency access data occupies video memory but is not frequently accessed, dynamically migrating the low-frequency access data to host memory or remote storage; When the resource occupancy condition is that the memory bandwidth reaches a preset saturation value, the access parallelism of the computing task is adjusted.

[0017] By adopting the above technical solution, when the resource occupancy is that the memory bandwidth utilization of high-frequency access data is insufficient and the GPU video memory has not reached the upper limit, the GPU video memory pool is dynamically expanded and the high-frequency access data is migrated to the GPU video memory, so that the memory bandwidth utilization can be improved, the memory access delay can be reduced, and the access performance of the high-frequency access data can be optimized; when the resource occupancy is that the low-frequency access data occupies the video memory but is not frequently accessed, the low-frequency access data is dynamically migrated to the host memory or remote storage, so that the GPU video memory resources can be effectively released and the utilization efficiency of the video memory resources can be improved; when the resource occupancy is that the memory bandwidth reaches the preset saturation value, the access parallelism of the computing task is adjusted, so that the congestion of the memory bandwidth can be effectively avoided, the utilization rate of the memory resources can be improved, and the smooth execution and efficient calculation of the task can be ensured; by dynamically adjusting the memory access parallelism of the computing task, the utilization efficiency of the memory bandwidth can be optimized, thereby improving the processing speed and resource utilization of the computing task, avoiding the impact of the memory bottleneck on the system performance, and improving the computing efficiency.

[0018] The second object of the invention is achieved by the following technical solutions: A GPU-based memory dynamic allocation device, the GPU-based memory dynamic allocation device comprising: Memory access monitoring module, used to monitor and obtain the memory access behavior of computing tasks in real time; A memory access prediction module, used to predict the memory access mode of the computing task using a preset memory prediction model according to the memory access behavior; A memory allocation decision module is used to determine the data access frequency of the computing task according to the memory access mode, and to judge the storage trend of the data according to the data access frequency, and to allocate the corresponding computing task to the corresponding storage environment based on the storage trend; The dynamic memory adjustment module is used to obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

[0019] By adopting the above technical solution, by real-time monitoring and obtaining the memory access behavior of the computing task, the memory access process of the task can be dynamically tracked, and the usage of the task memory can be understood in real time, thereby providing data support for subsequent memory allocation; by predicting the memory access mode of the computing task based on the memory access behavior using a preset memory prediction model, the memory access characteristics of the computing task can be pre-identified, the memory allocation strategy can be optimized, and the waste of memory resources and access conflicts can be avoided; by determining the data access frequency of the computing task based on the memory access mode, and judging the storage direction of the data based on the data access frequency, high-frequency data can be preferentially stored in an efficient storage area such as GPU video memory according to the data access frequency, and low-frequency data can be stored in a less efficient storage area, thereby improving the utilization efficiency of the memory; by obtaining the corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjusting the memory allocation strategy based on the real-time feedback information, the allocation of memory resources can be timely adjusted according to the task execution situation, memory bottlenecks can be avoided, and the execution efficiency of the computing task and the response speed of the memory can be improved.

[0020] The third objective of the present application is achieved through the following technical solutions: A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned GPU-based dynamic memory allocation method when executing the computer program.

[0021] The fourth objective of the present application is achieved through the following technical solutions: A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned GPU-based dynamic memory allocation method.

[0022] In summary, this application includes the following beneficial technical effects: 1. By real-time monitoring and obtaining the memory access behavior of computing tasks, it is possible to dynamically track the memory access process of tasks and understand the memory usage of tasks in real time, thereby providing data support for subsequent memory allocation; by using a preset memory prediction model to predict the memory access mode of computing tasks based on memory access behavior, it is possible to pre-identify the memory access characteristics of computing tasks, optimize memory allocation strategies, and avoid waste of memory resources and access conflicts; by determining the data access frequency of computing tasks based on the memory access mode, and judging the storage direction of data based on the data access frequency, it is possible to store high-frequency data in high-efficiency storage areas such as GPU video memory and low-frequency data in less efficient storage areas based on the data access frequency, thereby improving memory utilization efficiency; by obtaining corresponding real-time feedback information based on the allocation action of computing tasks, and dynamically adjusting the memory allocation strategy based on the real-time feedback information, it is possible to timely adjust the allocation of memory resources according to the task execution situation, avoid memory bottlenecks, and improve the execution efficiency of computing tasks and the response speed of memory; 2. By obtaining historical memory access behaviors, including the memory access frequency of computing tasks, task execution stages, etc., it can provide comprehensive historical data support for building memory prediction models, thereby improving the accuracy of model predictions; by classifying historical memory access data into access types based on memory access frequency and task execution stages, it can provide more accurate data sets for model training based on different types of memory access behaviors, thereby improving the accuracy of memory access pattern predictions; by iteratively training the preset timing model based on the model training data set, it can continuously optimize the model's prediction capabilities, thereby improving the level of intelligence in memory resource allocation and making memory resource usage more efficient; 3. By analyzing the temporal and spatial characteristics of memory access according to the memory access pattern and identifying high-frequency access data and low-frequency access data, it is possible to accurately distinguish the access frequency of different data, thereby adjusting the data storage location according to the frequency; by preferentially storing computing tasks in the GPU video memory for high-frequency access data, it is possible to reduce memory access latency and improve computing efficiency and task execution speed; by storing computing tasks in the host memory or remote storage for low-frequency access data, it is possible to save GPU video memory resources, thereby optimizing the use of memory resources and avoiding excessive occupation of video memory resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1is a flow chart of a GPU-based memory dynamic allocation method in an embodiment of the present application; Figure 2 It is a flowchart for implementing the construction of a preset memory prediction model in a GPU-based memory dynamic allocation method in an embodiment of the present application; Figure 3 is a flowchart for implementing step S30 in the GPU-based dynamic memory allocation method in one embodiment of the present application; Figure 4 is a flowchart for implementing step S31 in the GPU-based dynamic memory allocation method in one embodiment of the present application; Figure 5 is a flowchart for implementing step S40 in the GPU-based dynamic memory allocation method in one embodiment of the present application; Figure 6 is a flowchart for implementing step S42 in the GPU-based dynamic memory allocation method in one embodiment of the present application; Figure 7 It is a principle block diagram of a GPU-based memory dynamic allocation device in one embodiment of the present application; Figure 8 It is a schematic diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The present application is further described in detail below in conjunction with the accompanying drawings.

[0025] In one embodiment, if Figure 1 As shown, the present application discloses a GPU-based memory dynamic allocation method, which specifically includes the following steps: S10: Monitor and obtain the memory access behavior of computing tasks in real time.

[0026] Specifically, the monitoring system uses hardware counters or software analysis tools to track the memory access of computing tasks in real time, including information such as the memory access frequency, access location, and access duration of each computing task. For example, the system obtains memory access behavior by directly reading the log of the memory controller or sampling memory access traffic data. These behavior data include not only the frequency of reading data, but also the access patterns to different memory areas, such as whether the locality principle is presented, whether continuous memory blocks are accessed, etc., thereby fully reflecting the specific memory requirements of the computing task.

[0027] S20: Predict the memory access pattern of the computing task using a preset memory prediction model according to the memory access behavior.

[0028] Specifically, the memory prediction model uses historical data and current memory access behavior, combined with timing analysis methods or machine learning models, to infer future memory access trends of computing tasks. For example, through regression analysis of historical memory access behavior, the model can identify which data will be accessed at a certain moment in the future, thereby predicting the access pattern of the data. The model can update and optimize the prediction results in real time based on multiple input features, such as memory access frequency, task execution stage, and time characteristics, to provide accurate predictions of task memory requirements.

[0029] S30: Determine the data access frequency of the computing task according to the memory access mode, judge the storage direction of the data according to the data access frequency, and allocate the corresponding computing task to the corresponding storage environment based on the storage direction.

[0030] Specifically, after predicting the memory access pattern, the access frequency of the data is analyzed and classified according to the frequency. High-frequency access data will be identified as data that needs to be accessed frequently. This data should be retained in the GPU video memory first to reduce frequent memory read delays and improve computing efficiency. Low-frequency access data does not need to be accessed frequently and can be migrated to the host memory or remote storage to save video memory resources and avoid video memory being occupied by low-frequency data, optimize overall resource allocation, and dynamically adjust the storage structure of the memory according to the predicted storage trend. Data is effectively allocated between different storage environments to ensure task execution efficiency and maximize the utilization of memory resources.

[0031] S40: Obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

[0032] Specifically, after the computing task starts executing, its memory access is monitored in real time, and the resource usage during its execution is tracked. If the feedback data shows that the actual memory usage of the task deviates from the result of the prediction model, for example, some high-frequency access data is not migrated to the video memory in time, or the video memory usage rate is abnormal, the memory allocation strategy is adjusted according to the feedback information. This can be done by re-evaluating memory usage, updating the estimate of memory access frequency, or dynamically adjusting memory allocation to ensure that the computing task obtains the best memory support during execution and avoid performance degradation due to improper memory resource configuration.

[0033] In one embodiment, if Figure 2 As shown, in step S20, the construction of the preset memory prediction model specifically includes: S201: Obtain historical memory access behaviors, including memory access frequency of computing tasks, task execution phase, etc.

[0034] Specifically, the acquisition of historical memory access behavior includes collecting memory usage at each time point during the execution of computing tasks, recording the time and frequency of memory access and the resource consumption of tasks at different stages, and summarizing the memory data collected by monitoring tools or performance analysis instruments to provide detailed memory usage information during task execution. These data will be used for subsequent model training.

[0035] S202: According to the memory access frequency and the task execution stage, the historical memory access data is divided into access types to obtain a model training data set.

[0036] Specifically, memory access data is divided into high-frequency data and low-frequency data according to the access frequency, and the data is divided according to the task execution stage. For example, the memory access behavior in the training stage may be different from that in the inference stage. Based on this, the access behavior can be classified by stage, and the data in each stage can be assigned to different data sets for training. Through this process, the memory demand characteristics under different task stages are clarified, and a training data set suitable for prediction is constructed.

[0037] S203: Iteratively train the preset time series model according to the model training data set to learn the rules and patterns of data access, and then generate a memory prediction model.

[0038] Specifically, the timing model will perform multiple iterations based on the training data set, optimize the model's prediction effect by continuously adjusting the model's weights and biases, update the model parameters using the back-propagation algorithm, and learn the laws and patterns of data access. For example, some data access behaviors may show periodic fluctuations, or access to certain memory areas may be spatially local. The model learns these laws and generates a memory prediction model that can accurately predict future memory access patterns based on historical behaviors.

[0039] In one embodiment, if Figure 3 As shown, in step S30, the data access frequency of the computing task is determined according to the memory access mode, and the storage direction of the data is determined according to the data access frequency, which specifically includes: S31: Analyze the temporal and spatial characteristics of memory access according to the memory access pattern, and identify high-frequency access data and low-frequency access data.

[0040] Specifically, through in-depth analysis of memory access patterns, we first analyze from a temporal perspective, use a sliding window algorithm to segment access behaviors, calculate the access frequency of data blocks in each time period, and identify frequently accessed data blocks through temporal features; secondly, from a spatial perspective, combined with the principle of data access locality, we analyze the data access areas and access paths, and identify high-frequency and low-frequency data by tracking the memory access hotspots of computing tasks. This can effectively determine which data needs to be retained in the video memory and which data can be migrated to the host memory.

[0041] S32: For high-frequency access data, the computing tasks are stored in the GPU memory first to reduce memory access latency and improve computing efficiency.

[0042] Specifically, for high-frequency access data, the access speed of video memory is much faster than that of host memory or remote storage. By storing this data in GPU video memory first, memory access latency can be reduced and computing efficiency can be improved. Especially in tasks such as deep learning and image processing that require frequent reading of the same data, the high-bandwidth characteristics of GPU video memory can significantly improve the speed and response time of task execution.

[0043] S33: For low-frequency access data, store computing tasks in host memory or remote storage to save GPU memory resources and optimize memory usage efficiency.

[0044] Specifically, for low-frequency access data, since the access frequency of this data is low and will not have a significant impact on the execution efficiency of computing tasks, it can be stored in the host memory or remote storage to avoid occupying valuable GPU video memory resources, thereby freeing up more video memory space for high-frequency access data. This not only optimizes the utilization efficiency of memory resources, but also reduces the pressure on memory bandwidth and improves the overall performance of the system.

[0045] In one embodiment, if Figure 4 As shown, in step S31, the temporal and spatial characteristics of memory access are analyzed according to the memory access mode to identify high-frequency access data and low-frequency access data, which specifically includes: S311: Analyze the access frequency of the computing task on the time axis according to the memory access pattern, and use the sliding window algorithm to calculate the access frequency of the computing task in different time periods, and then identify high-frequency access data and low-frequency access data.

[0046] Specifically, the sliding window algorithm can divide the memory access behavior into time slices, calculate the access frequency in each slice, and identify the data blocks with higher access frequency according to the ratio of the access frequency of each window to the overall access pattern, and then determine them as high-frequency data blocks. Otherwise, they are identified as low-frequency data blocks. This helps to adjust the memory allocation strategy in real time and optimize the configuration of memory resources.

[0047] S312: Analyze the access location and access pattern of the computing task according to the memory access pattern, and track the access path and hot spot area of ​​the computing task based on the locality principle of the memory access pattern, so as to identify high-frequency access data and low-frequency access data.

[0048] Specifically, through the principle of locality of memory access patterns, we analyze the access paths of computing tasks in memory. For example, we analyze the access correlation between data blocks. By tracking changes in the access paths and hot spots of computing tasks and combining location features, we determine which data are frequently accessed. These data will be classified as high-frequency data, and the others will be identified as low-frequency data. Based on this process, we can effectively improve the accuracy of memory allocation.

[0049] In one embodiment, if Figure 5 As shown, in step S40, corresponding real-time feedback information is obtained according to the allocation action of the computing task, and the memory allocation strategy is dynamically adjusted based on the real-time feedback information, specifically including: S41: Obtain the memory access exception during the execution of the current computing task, and determine the resource usage of the computing task based on the memory access exception.

[0050] Specifically, memory access anomalies may include memory bandwidth overload, excessive memory latency, excessive video memory usage, etc. By monitoring these abnormal information, we can timely judge the resource usage of computing tasks, evaluate whether the current memory allocation is reasonable, and whether the strategy needs to be adjusted to avoid waste or overload of memory resources.

[0051] S42: Adjust the corresponding memory allocation strategy according to the resource usage situation.

[0052] Specifically, based on resource usage, a variety of strategies can be used to adjust memory allocation. For example, if the video memory bandwidth is insufficient, the video memory pool can be dynamically expanded to accommodate more high-frequency data. If the video memory has reached its upper limit, low-frequency data needs to be migrated to the host memory or remote storage to make room for more important computing task data. When the memory bandwidth is close to saturation, it may be necessary to adjust the parallelism of the task to avoid bandwidth competition between concurrent computing tasks.

[0053] In one embodiment, if Figure 6As shown, in step S42, the corresponding memory allocation strategy is adjusted according to the resource occupancy, specifically including: S421: When the resource usage is that the memory bandwidth utilization of the high-frequency access data is insufficient and the GPU video memory has not reached the upper limit, dynamically expand the GPU video memory pool and migrate the high-frequency access data to the GPU video memory.

[0054] Specifically, when it is detected that the memory bandwidth utilization of frequently accessed data is insufficient, the bandwidth pressure can be alleviated by increasing the allocation of GPU video memory, for example, by using memory extension technology or dividing the additional memory area into a GPU video memory pool, dynamically adjusting the allocation when the video memory capacity is not full, and migrating frequently accessed data blocks from the host memory or remote storage to the GPU video memory to reduce memory access latency and ensure that frequently accessed data can be read efficiently, thereby improving the execution efficiency of computing tasks.

[0055] S422: When the resource occupancy situation is that the low-frequency access data occupies the video memory but is not frequently accessed, dynamically migrate the low-frequency access data to the host memory or remote storage.

[0056] Specifically, when some low-frequency access data occupies the GPU memory space, but its access frequency is low, in order to more effectively utilize the memory resources, these low-frequency access data can be migrated to the host memory or remote storage. The migration operation is usually based on the output results of the memory access prediction model. When it is predicted that the access frequency of certain data will decrease, these data will be unloaded from the GPU memory in time to free up space for other high-frequency data, so as to avoid the GPU memory being occupied by low-frequency data for too long, improve the utilization of memory resources, and reduce resource waste.

[0057] S423: When the resource usage condition is that the memory bandwidth reaches a preset saturation value, adjust the access parallelism of the computing task.

[0058] Specifically, when the system detects that the memory bandwidth is close to saturation and can no longer effectively support the parallel execution of more computing tasks, it should dynamically adjust the access parallelism of the computing tasks. It can reduce the pressure on the memory bandwidth by reducing the task parallelism, reducing the number of simultaneous memory accesses, or reducing the memory access frequency of some computing tasks. For example, the memory access frequency of each task can be reduced or the memory load can be dispersed through batch processing to ensure that the system can maintain high execution efficiency and stability when bandwidth resources are limited.

[0059] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0060] In one embodiment, a GPU-based memory dynamic allocation device is provided, and the GPU-based memory dynamic allocation device corresponds to the GPU-based memory dynamic allocation method in the above embodiment. Figure 7 As shown, the GPU-based memory dynamic allocation device includes a memory access monitoring module, a memory access prediction module, a memory allocation decision module, and a dynamic memory adjustment module. The detailed description of each functional module is as follows: Memory access monitoring module, used to monitor and obtain the memory access behavior of computing tasks in real time; A memory access prediction module is used to predict the memory access mode of a computing task based on the memory access behavior using a preset memory prediction model; The memory allocation decision module is used to determine the data access frequency of the computing task according to the memory access mode, and judge the storage trend of the data according to the data access frequency, and allocate the corresponding computing task to the corresponding storage environment based on the storage trend; The dynamic memory adjustment module is used to obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

[0061] Optionally, build a preset memory prediction model, including: The data collection module is used to obtain historical memory access behaviors, including the memory access frequency of computing tasks, task execution stages, etc. The data preprocessing module is used to classify the historical memory access data into access types according to the memory access frequency and the task execution stage to obtain the model training data set; The model training module is used to iteratively train the preset time series model according to the model training data set to learn the rules and patterns of data access and then generate an in-memory prediction model.

[0062] Optionally, the memory allocation decision module specifically includes: The access pattern analysis submodule is used to analyze the temporal and spatial characteristics of memory access according to the memory access pattern, and identify high-frequency access data and low-frequency access data; The high-access storage allocation submodule is used to store computing tasks in the GPU memory first for high-frequency access data, so as to reduce memory access latency and improve computing efficiency. The low-access storage allocation submodule is used to store computing tasks in host memory or remote storage for low-frequency access data to save GPU video memory resources and optimize memory usage efficiency.

[0063] Optionally, the access pattern analysis submodule specifically includes: The time feature analysis unit is used to analyze the access frequency of computing tasks on the time axis according to the memory access pattern, and use the sliding window algorithm to calculate the access frequency of computing tasks in different time periods, thereby identifying high-frequency access data and low-frequency access data.

[0064] The spatial feature analysis unit is used to analyze the access location and access pattern of the computing task according to the memory access pattern, and to track the access path and hot spot area of ​​the computing task based on the locality principle of the memory access pattern, so as to identify high-frequency access data and low-frequency access data.

[0065] Optionally, the dynamic memory adjustment module specifically includes: The feedback adjustment submodule is used to obtain the abnormal memory access during the execution of the current computing task, and judge the resource usage of the computing task based on the abnormal memory access; The policy adjustment submodule is used to adjust the corresponding memory allocation policy according to resource usage.

[0066] Optionally, the policy adjustment submodule specifically includes: A resource expansion unit is used to dynamically expand the GPU memory pool and migrate the high-frequency access data to the GPU memory when the resource usage is insufficient for the memory bandwidth utilization of the high-frequency access data and the GPU memory has not reached the upper limit; A resource migration unit, used for dynamically migrating low-frequency access data to a host memory or remote storage when the resource occupancy situation is that the low-frequency access data occupies the video memory but is not frequently accessed; The parallelism adjustment unit is used to adjust the access parallelism of the computing task when the resource usage condition is that the memory bandwidth reaches a preset saturation value.

[0067] For the specific limitations of the GPU-based dynamic memory allocation device, please refer to the limitations of the GPU-based dynamic memory allocation method above, which will not be repeated here. Each module in the above-mentioned GPU-based dynamic memory allocation device can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0068] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a GPU-based memory dynamic allocation method is implemented.

[0069] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented: Monitor and obtain the memory access behavior of computing tasks in real time; According to the memory access behavior, the memory access pattern of the computing task is predicted using a preset memory prediction model; Determine the data access frequency of the computing task according to the memory access mode, and judge the storage direction of the data according to the data access frequency, and assign the corresponding computing task to the corresponding storage environment based on the storage direction; Obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

[0070] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Monitor and obtain the memory access behavior of computing tasks in real time; According to the memory access behavior, the memory access pattern of the computing task is predicted using a preset memory prediction model; Determine the data access frequency of the computing task according to the memory access mode, and judge the storage direction of the data according to the data access frequency, and assign the corresponding computing task to the corresponding storage environment based on the storage direction; Obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

[0071] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0072] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0073] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A GPU-based memory dynamic allocation method, characterized in that: The GPU-based dynamic memory allocation method includes: Monitor and obtain the memory access behavior of computing tasks in real time; Predicting a memory access pattern of the computing task using a preset memory prediction model according to the memory access behavior; Determine the data access frequency of the computing task according to the memory access mode, judge the storage trend of the data according to the data access frequency, and allocate the corresponding computing task to the corresponding storage environment based on the storage trend; Corresponding real-time feedback information is obtained according to the allocation action of the computing task, and a memory allocation strategy is dynamically adjusted based on the real-time feedback information.

2. The GPU-based dynamic memory allocation method according to claim 1, characterized in that: The construction of the preset memory prediction model specifically includes: Obtain historical memory access behavior, including the memory access frequency of the computing task, the task execution stage, etc.; According to the memory access frequency and the task execution stage, the historical memory access data is divided into access types to obtain a model training data set; According to the model training data set, the preset timing model is iteratively trained to learn the rules and patterns of data access, thereby generating the memory prediction model.

3. The GPU-based dynamic memory allocation method according to claim 1, characterized in that: Determining the data access frequency of the computing task according to the memory access mode, and judging the storage direction of the data according to the data access frequency, specifically includes: Analyzing the temporal and spatial characteristics of memory access according to the memory access pattern, and identifying high-frequency access data and low-frequency access data; For the high-frequency access data, the computing task is preferentially stored in the GPU memory to reduce memory access latency and improve computing efficiency; For the low-frequency access data, the computing task is stored in the host memory or remote storage to save GPU video memory resources and optimize memory usage efficiency.

4. The GPU-based dynamic memory allocation method according to claim 3, characterized in that: The analyzing the time characteristics and spatial characteristics of memory access according to the memory access pattern to identify high-frequency access data and low-frequency access data specifically includes: Analyzing the access frequency of the computing task on the time axis according to the memory access pattern, and calculating the access frequency of the computing task in different time periods by using a sliding window algorithm, thereby identifying the high-frequency access data and the low-frequency access data; According to the memory access pattern, the access location and access pattern of the computing task are analyzed, and based on the locality principle of the memory access pattern, the access path and hot spot area of ​​the computing task are tracked, so as to identify the high-frequency access data and the low-frequency access data.

5. The GPU-based dynamic memory allocation method according to claim 1, characterized in that: The obtaining corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjusting the memory allocation strategy based on the real-time feedback information specifically includes: Obtaining a memory access exception during the execution of the current computing task, and determining the resource occupancy of the computing task based on the memory access exception; According to the resource usage, the corresponding memory allocation strategy is adjusted.

6. The GPU-based dynamic memory allocation method according to claim 5, characterized in that: The adjusting the corresponding memory allocation strategy according to the resource occupancy situation specifically includes: When the resource occupancy condition is that the memory bandwidth utilization of the high-frequency access data is insufficient and the GPU video memory has not reached the upper limit, dynamically expand the GPU video memory pool and migrate the high-frequency access data to the GPU video memory; When the resource occupancy situation is that low-frequency access data occupies video memory but is not frequently accessed, dynamically migrating the low-frequency access data to host memory or remote storage; When the resource occupancy condition is that the memory bandwidth reaches a preset saturation value, the access parallelism of the computing task is adjusted.

7. A GPU-based memory dynamic allocation device, characterized in that: The GPU-based dynamic memory allocation device includes: Memory access monitoring module, used to monitor and obtain the memory access behavior of computing tasks in real time; A memory access prediction module, used to predict the memory access mode of the computing task using a preset memory prediction model according to the memory access behavior; A memory allocation decision module is used to determine the data access frequency of the computing task according to the memory access mode, and to judge the storage trend of the data according to the data access frequency, and to allocate the corresponding computing task to the corresponding storage environment based on the storage trend; The dynamic memory adjustment module is used to obtain corresponding real-time feedback information according to the allocation action of the computing task, and dynamically adjust the memory allocation strategy based on the real-time feedback information.

8. The GPU-based dynamic memory allocation device according to claim 7, characterized in that: The construction of the preset memory prediction model specifically includes: A data collection module, used to obtain historical memory access behaviors, including the memory access frequency of the computing task, the task execution stage, etc.; A data preprocessing module, used to classify the historical memory access data into access types according to the memory access frequency and the task execution stage, so as to obtain a model training data set; The model training module is used to iteratively train the preset timing model according to the model training data set to learn the rules and patterns of data access, and then generate the memory prediction model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the GPU-based dynamic memory allocation method are implemented as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the GPU-based dynamic memory allocation method as claimed in any one of claims 1 to 6 are implemented.

Images