Runtime memory optimization method for real-time self-learning management

Abstract

This invention discloses a runtime memory optimization method based on real-time self-learning management. By combining the fragmentation evolution patterns of historical tasks with the real-time fragmentation growth rate, the system can predict the development trend of memory fragmentation in advance, avoiding passively triggering defragmentation operations during high-load task phases. Initiating fragment merging during identified task gaps effectively integrates scattered free memory blocks into a continuous pool of available resources without interfering with the real-time execution of core tasks. This fundamentally reduces resource allocation failures caused by memory fragmentation, ensuring system stability and smoothness, and continuously optimizing the adaptability of memory management strategies. Operational data such as the timing of defragmentation and the priority of merging are continuously fed back to the system to update the parameters of the prediction model and the historical pattern database, making subsequent fragmentation trend predictions more accurate, task gap identification more timely, and merging operations more efficient.

Description

Technical Field

[0001] This invention belongs to the field of memory optimization technology, specifically a runtime memory optimization method with real-time self-learning management. Background Technology

[0002] Video memory (VRAM) is a dedicated high-speed cache on a graphics card used for temporary storage and processing of graphics rendering data, texture mapping, frame buffers, and complex computational instructions. As the "workbench" of the graphics processor, VRAM capacity directly determines the upper limit of the amount of data the GPU can access at ultra-high resolutions, while its bandwidth determines the throughput speed of data interaction between the GPU core and the GPU. Together, these factors determine the performance ceiling of a device when facing heavy 3D rendering or parallel computing. With technological evolution, VRAM types have progressed from early GDDR and GDDR3 to the widely adopted GDDR6 and GDDR6X, and finally to the HBM architecture, which dominates the high-end computing field. In recent years, with the explosion of artificial intelligence deep learning, large language model training, and ray tracing technologies, VRAM has transformed from a simple image output buffer into a core memory pool capable of handling massive tensor matrix operations. Insufficient VRAM capacity often becomes a key "VRAM wall" restricting model scale and computational power. As an indispensable underlying component of modern visual computing and AI computing infrastructure, continuous breakthroughs in VRAM performance are profoundly driving the iterative leap forward of graphics technology and the general artificial intelligence industry.

[0003] However, existing memory management methods are often based on static or predefined rules, making it difficult to dynamically adapt to changing task requirements. Specifically, they suffer from the following problems: 1. Unpredictable task requirements: Different tasks have significantly different memory needs, making it difficult to meet them with fixed allocation strategies. 2. Severe fragmentation: When dynamically allocating memory, prolonged operation can lead to memory fragmentation, reducing resource utilization. 3. Lack of real-time optimization capabilities: Traditional methods struggle to dynamically adjust based on runtime task behavior, resulting in performance degradation.

[0004] Therefore, a real-time self-learning memory optimization method is needed, which can dynamically adjust memory allocation strategies based on data during task execution to improve memory utilization and task performance. Summary of the Invention

[0005] The purpose of this invention is to provide a runtime memory optimization method for real-time self-learning management in order to solve the problems mentioned above.

[0006] The technical solution adopted in this invention is as follows: a runtime memory optimization method for real-time self-learning management, comprising the following steps:

[0007] S1: Task initialization, loading historical memory usage data of similar tasks, generating an initial allocation strategy and synchronizing it to S2, while passing the fragmentation pattern to S4 to provide a basis for decision-making in subsequent operations.

[0008] S2: Memory is partitioned and resources are allocated according to short-term buffer, long-term parameter area, and dynamic shared area, with an embedded monitoring probe. Real-time access data in S3 will trigger dynamic adjustments to partition boundaries, and free memory reclaimed in S5 will be preferentially added to the shared area.

[0009] S3: During runtime, data such as memory access frequency and priority changes are collected, and optimization strategies are iteratively optimized. The optimization results are synchronized to S2 to adjust partitions, pushed to S4 to indicate the fragmentation growth rate, and input to S6 to show the video memory usage curve.

[0010] S4: Based on the fragmentation rate of S3 and the historical patterns of S1, fragmentation defragmentation is initiated during task intervals to merge free blocks. The information of the defragmented free areas is synchronized to S5 to trigger reclamation, and contiguous space is fed back to S2 for subsequent allocation.

[0011] S5: Reclaim expired data in layers according to temporary buffer → shared area → parameter area. The released video memory is replenished to the resource pool of S2 in real time. The reclamation time window refers to the task load curve of S3.

[0012] S6: Based on the memory utilization curve of S3, predict the overflow risk, migrate low-priority task data to external storage, free up space for S2 to allocate high-priority tasks, and synchronize the migration status to S3 to track performance changes.

[0013] In a preferred embodiment, in step S1, complete runtime logs of similar tasks within the last 30 days are extracted as historical data. Total video memory resources are pre-allocated based on 80% of the peak average video memory usage of the tasks. The initial partitioning ratio of the short-term buffer, long-term parameter area, and dynamic shared area is set with reference to the historical average of similar tasks. The system analyzes the fragmentation generation rate and merging efficiency of historical tasks on an hourly basis to extract fragmentation patterns. Simultaneously, a task update threshold is set. When the number of newly added tasks of the same type exceeds 5, the initial allocation strategy is automatically regenerated and the fragmentation patterns are updated, providing a decision-making basis for subsequent steps that fits the current task scenario.

[0014] In a preferred embodiment, step S2, based on the historical data of the same type of task loaded in S1, completes the three-zone partitioning and initial resource allocation of the video memory, specifically including:

[0015] The short-term buffer mainly stores the original video frames to be inferred and the preprocessed tensor data. The initial allocation ratio is 25% of the total video memory, which is temporarily expanded to 30% when the core detection task is running.

[0016] The long-term parameter area stores the backbone network weights, detector head parameters, and anchor box configurations of YOLOv8. It allocates 40% of the video memory space and locks it to prevent it from being squeezed out by other areas, which could cause abnormal model loading.

[0017] The dynamic shared area reserves 35% of the video memory to store intermediate feature maps generated during inference and semantic information reused in multiple frames. The allocation ratio will be dynamically adjusted in real time according to the data reuse rate, and feature maps with high reuse rates will be given priority in space guarantee.

[0018] In a preferred embodiment, in step S3, during the runtime phase of the real-time video frame analysis task, a Deep Q-Network (DQN) is used as the real-time learning model, and data acquisition and feature construction are completed every 100ms: the memory access frequency of the short-term buffer is collected. Weight update frequency of the long-term parameter region Data reuse rate in dynamic shared areas Task priority coefficient (Core tasks with high real-time requirements are rated at 0.8-1.0, and background auxiliary tasks at 0.2-0.5), VRAM usage (u) and fragmentation growth rate. There are 6 types of features. After performing min-max normalization on each type of feature, the results are concatenated to obtain the state vector. The normalization operation maps the original features to the [0,1] interval, eliminating dimensional differences to adapt to the input requirements of the DQN model.

[0019] In a preferred embodiment, in step S3, the DQN model is based on the current state vector. Output memory allocation actions The action space contains three types of executable operations: adjusting the proportion of the short-run buffer. Adjust the proportion of dynamic shared areas Marking redundant data areas triggers data reclamation. After each action is performed, the system calculates an immediate reward. To evaluate the effectiveness of actions, the reward function comprehensively considers three core objectives: improved memory utilization, reduced task latency, and fragmentation suppression. Multi-objective collaborative optimization is achieved through weighted summation. Subsequently... The quadruplets are stored in the experience replay pool. After accumulating 1000 samples, batch data is randomly sampled to update the network parameters of DQN. By minimizing the mean square error between the current Q value and the target Q value, the memory allocation strategy is iteratively optimized so that the system can maximize the cumulative reward in long-term operation and achieve a dynamic balance between memory utilization and task performance.

[0020] The feature normalization formula is:

[0021]

[0022] f represents the original collected memory behavior feature value;

[0023] This is the minimum value of this feature in the historical operating data;

[0024] This is the maximum value of this feature in historical operational data;

[0025] These are the normalized feature values, ranging from [0,1], used to eliminate the dimensional differences between different features and improve the stability of model training;

[0026] The multi-objective instant reward function is:

[0027]

[0028] α is a weighting coefficient for improving video memory utilization, with a value range of [0,1]. The value can be increased in scenarios where video memory is scarce.

[0029] β is the weighting coefficient for reducing task latency, with a value range of [0,1]. In scenarios with high real-time requirements, the value can be increased.

[0030] γ is the weighting coefficient for suppressing fragmentation growth rate, with a value range of [0,1]. The value can be increased for long-term operating scenarios.

[0031] The increase in video memory utilization after the action is performed is calculated as follows: ;

[0032] The reduction in task inference latency after the action is performed is calculated as follows: ;

[0033] The amount of suppression of the fragmentation growth rate after the action is executed is calculated as follows:

[0034] The DQN network parameter update loss function is:

[0035]

[0036] This is the loss function, which measures the error between the current Q value and the target Q value.

[0037] These are the trainable parameters of the current Q-network;

[0038] For the parameters of the target Q network, periodically from Copying to avoid training oscillations;

[0039] D is the experience replay pool, which stores historical state-action-reward-next state samples;

[0040] γ is a discount factor, with a value range of [0,1], which balances the importance of immediate rewards and long-term cumulative rewards;

[0041] This is the current Q-network's value estimate for action A in state S;

[0042] This is the value estimate of the optimal action in the next state S′ for the target Q-network.

[0043] In a preferred embodiment, in step S4, historical fragment data of the same type of task is first loaded from S1, including the fragment growth rate, free block size distribution, and sorting time patterns at different task stages (such as model initialization, frame inference, and result output), to construct a fragment evolution feature library. Simultaneously, dynamic data such as the current task fragment growth rate, memory access frequency, and task load curve transmitted in real-time from S3 are received. Historical features and real-time data are concatenated into a multi-dimensional input vector, which is then input into a pre-trained LSTM prediction model to output a predicted fragment growth trend value for the next 10 seconds. When the predicted value exceeds a preset fragment warning threshold, the sorting preparation process is triggered.

[0044] In a preferred embodiment, in step S4, task gaps are identified by combining the task running status data from S3: by monitoring abrupt changes in the video memory access frequency, when the video memory access frequency drops by more than 60% from its peak for three consecutive sampling cycles (each cycle being 50ms), and the CPU utilization is below 20%, it is determined to be a valid task gap (such as the result output waiting stage after video frame inference is completed). At this time, a greedy merging algorithm is started. Based on the rule summarized in the historical feature library of S1 that "merging adjacent small free blocks has a higher priority than merging scattered large free blocks", free blocks with consecutive addresses are merged first. During the merging process, only necessary boundary data is moved, rather than a full scan of the video memory, which greatly shortens the processing time. After processing is completed, the merged consecutive free block information is synchronized to the video memory resource pool of S2 for subsequent task allocation, and the processing result is fed back to S3 to update the real-time fragment data.

[0045] The formula for predicting the growth trend of LSTM fragmentation is:

[0046]

[0047] In the formula: The fragmentation growth rate predicted by the LSTM model for the next 10 seconds;

[0048] The input feature vector at time t includes the historical fragmentation growth rate of S1, the real-time fragmentation growth rate of S3, the video memory access frequency, and the task load coefficient.

[0049] Here is the weight matrix of the LSTM hidden layer;

[0050] The bias vector of the LSTM hidden layer;

[0051] The formula for identifying task gaps is:

[0052]

[0053] This is the task gap identifier at time t, where 1 indicates a valid gap and 0 indicates a non-gap.

[0054] Let be the memory access frequency at time t;

[0055] This represents the peak memory access time three sampling cycles prior.

[0056] Let be the CPU utilization at time t;

[0057] The formula for calculating the revenue from merging free blocks is as follows:

[0058]

[0059] B represents the benefit value for merging free blocks, used to sort the merge priority;

[0060] This represents the total size of the merged free blocks;

[0061] Let be the size of the i-th free block to be merged;

[0062] The estimated time for the merge operation is calculated based on historical data from S1.

[0063] As a priority weight, adjacent free blocks are assigned a weight of 1.2, and scattered free blocks are assigned a weight of 0.8.

[0064] In a preferred embodiment, in step S5, during the real-time video target detection task, the S5 module strictly follows a hierarchical recycling logic prioritizing temporary buffers. First, it scans the frame buffer data in the temporary buffer. Based on the historical task patterns loaded in S1, expired frame data exceeding the 30-frame retention threshold is marked for recycling, and the corresponding video memory space is directly released. Next, it processes the reused feature maps in the dynamic shared area. Combining this with the task load curve transmitted in real-time in S3, when the reuse rate of a certain type of feature map is less than 10% within five consecutive sampling periods, it is determined to be low-value shared data, and batch recycling is initiated. Finally, it reaches the long-term parameter area, recycling only the frozen model branch weights. These weights typically correspond to low-priority detection targets. Before recycling, it verifies whether the current task still needs to call this branch, and releases the video memory only after confirming there is no dependency.

[0065] In a preferred embodiment, in step S6, real-time video multi-target tracking is performed using the Transformer architecture. First, the system receives a continuous 1-minute time-series data of video memory usage transmitted in S3. This data is then input into a pre-trained LSTM prediction model to generate a usage trend curve for the next 30 seconds. When the prediction curve indicates that the video memory usage will exceed the 90% warning threshold within 15 seconds, the system immediately initiates an overflow risk response process. At this time, the module iterates through all currently running tasks, filters out low-priority tasks based on preset priority tags, such as background video statistical analysis tasks, and migrates their non-real-time computing data, such as historical tracking result caches and feature backups, to local NVMe SSD storage. The migration process uses incremental transfer, transferring only cold data blocks not referenced by the current task to avoid interfering with task execution.

[0066] In a preferred embodiment, a runtime memory optimization system with real-time self-learning management includes:

[0067] Real-time learning module: Analyzes task behavior (such as memory access patterns and task lifecycle) in real time through machine learning models, and optimizes memory allocation strategies online to adapt to current task requirements.

[0068] Dynamic memory partitioning module: Based on task characteristics, the video memory is divided into multiple regions (short-term buffer, long-term parameter area and dynamic shared area).

[0069] Defragmentation and Reclamation Module: Based on a self-learning model, this module identifies memory fragmentation trends and proactively defragments memory during idle periods. It provides a segmented reclamation mechanism to gradually release unused memory areas.

[0070] Memory overflow prediction and migration module: Utilizes historical data and real-time monitoring to predict the risk of video memory overflow. When the risk of overflow is high, it prioritizes migrating low-priority task data to main storage or other hardware devices.

[0071] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0072] 1. In this invention, by combining the fragmentation evolution patterns of historical tasks with the real-time fragmentation growth rate, the system can predict the development trend of memory fragmentation in advance, avoiding passively triggering defragmentation operations during high-load task phases. Initiating fragment merging during identified task gaps effectively integrates scattered free memory blocks into a continuous pool of available resources without interfering with the real-time execution of core tasks. This fundamentally reduces resource allocation failures caused by memory fragmentation, ensuring the stability and smoothness of system operation.

[0073] 2. In this invention, a self-learning closed-loop mechanism continuously optimizes the adaptability of memory management strategies. Operational data such as the timing of defragmentation and the priority of merging are constantly fed back to the system to update the parameters of the prediction model and the historical pattern database, making subsequent fragmentation trend predictions more accurate, task gap identification more timely, and merging operations more efficient. Over long-term operation, the system will gradually develop memory management logic adapted to specific task scenarios, reducing unnecessary memory consumption and defragmentation overhead, improving the overall utilization efficiency of memory resources, and providing stable and reliable memory support for continuously running real-time tasks. Attached Figure Description

[0074] Figure 1 This is a schematic diagram illustrating the process principle of the present invention;

[0075] Figure 2 This is a block diagram of the overall system in this invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0077] Example:

[0078] Reference Figure 1-2 A runtime memory optimization method for real-time self-learning management, the method comprising the following steps:

[0079] S1: Task initialization, loading historical memory usage data of similar tasks, generating an initial allocation strategy and synchronizing it to S2, while passing the fragmentation pattern to S4 to provide a basis for decision-making in subsequent operations.

[0080] S2: Memory is partitioned and resources are allocated according to short-term buffer, long-term parameter area, and dynamic shared area, with an embedded monitoring probe. Real-time access data in S3 will trigger dynamic adjustments to partition boundaries, and free memory reclaimed in S5 will be preferentially added to the shared area.

[0081] S3: During runtime, data such as memory access frequency and priority changes are collected, and optimization strategies are iteratively optimized. The optimization results are synchronized to S2 to adjust partitions, pushed to S4 to indicate the fragmentation growth rate, and input to S6 to show the video memory usage curve.

[0082] S4: Based on the fragmentation rate of S3 and the historical patterns of S1, fragmentation defragmentation is initiated during task intervals to merge free blocks. The information of the defragmented free areas is synchronized to S5 to trigger reclamation, and contiguous space is fed back to S2 for subsequent allocation.

[0083] S5: Reclaim expired data in layers according to temporary buffer → shared area → parameter area. The released video memory is replenished to the resource pool of S2 in real time. The reclamation time window refers to the task load curve of S3.

[0084] S6: Based on the memory utilization curve of S3, predict the overflow risk, migrate low-priority task data to external storage, free up space for S2 to allocate high-priority tasks, and synchronize the migration status to S3 to track performance changes.

[0085] In step S1, complete runtime logs of similar tasks within the last 30 days are extracted as historical data. Total video memory resources are pre-allocated based on 80% of the peak average video memory usage of the tasks. The initial partitioning ratio of the short-term buffer, long-term parameter area, and dynamic shared area is set with reference to the historical average of similar tasks. The system analyzes the fragmentation generation rate and merging efficiency of historical tasks hourly to extract fragmentation patterns. Simultaneously, a task update threshold is set. When the number of newly added tasks of the same type exceeds 5, the initial allocation strategy is automatically regenerated and the fragmentation patterns are updated, providing a decision-making basis for subsequent steps that fits the current task scenario.

[0086] In step S2, based on the historical data of similar tasks loaded in S1, the three-zone partitioning and initial resource allocation of the video memory are completed, specifically including:

[0087] The short-term buffer mainly stores the original video frames to be inferred and the preprocessed tensor data. The initial allocation ratio is 25% of the total video memory, which is temporarily expanded to 30% when the core detection task is running.

[0088] The long-term parameter area stores the backbone network weights, detector head parameters, and anchor box configurations of YOLOv8. It allocates 40% of the video memory space and locks it to prevent it from being squeezed out by other areas, which could cause abnormal model loading.

[0089] The dynamic shared area reserves 35% of the video memory to store intermediate feature maps generated during inference and semantic information reused in multiple frames. The allocation ratio will be dynamically adjusted in real time according to the data reuse rate, and feature maps with high reuse rates will be given priority in space guarantee.

[0090] In step S3, during the runtime phase of the real-time video frame analysis task, a Deep Q-Network (DQN) is used as the real-time learning model, and data acquisition and feature construction are completed every 100ms: the memory access frequency of the short-term buffer is collected. Weight update frequency of the long-term parameter region Data reuse rate in dynamic shared areas Task priority coefficient For core tasks with high real-time requirements, use 0.8-1.0; for background auxiliary tasks, use 0.2-0.5. Memory usage (u) and fragmentation growth rate (u) are also considered. There are 6 types of features. After performing min-max normalization on each type of feature, the results are concatenated to obtain the state vector. The normalization operation maps the original features to the [0,1] interval, eliminating dimensional differences to adapt to the input requirements of the DQN model.

[0091] In step S3, the DQN model is based on the current state vector. Output memory allocation actions The action space contains three types of executable operations: adjusting the proportion of the short-run buffer. Adjust the proportion of dynamic shared areas Marking redundant data areas triggers data reclamation. After each action is performed, the system calculates an immediate reward. To evaluate the effectiveness of actions, the reward function comprehensively considers three core objectives: improved memory utilization, reduced task latency, and fragmentation suppression. Multi-objective collaborative optimization is achieved through weighted summation. Subsequently... The quadruplets are stored in the experience replay pool. After accumulating 1000 samples, batch data is randomly sampled to update the network parameters of DQN. By minimizing the mean square error between the current Q value and the target Q value, the memory allocation strategy is iteratively optimized so that the system can maximize the cumulative reward in long-term operation and achieve a dynamic balance between memory utilization and task performance.

[0092] The feature normalization formula is:

[0093]

[0094] f represents the original collected memory behavior feature value;

[0095] This is the minimum value of this feature in the historical operating data;

[0096] This is the maximum value of this feature in historical operational data;

[0097] These are the normalized feature values, ranging from [0,1], used to eliminate the dimensional differences between different features and improve the stability of model training;

[0098] The multi-objective instant reward function is:

[0099]

[0100] α is a weighting coefficient for improving video memory utilization, with a value range of [0,1]. The value can be increased in scenarios where video memory is scarce.

[0101] β is the weighting coefficient for reducing task latency, with a value range of [0,1]. In scenarios with high real-time requirements, the value can be increased.

[0102] γ is the weighting coefficient for suppressing fragmentation growth rate, with a value range of [0,1]. The value can be increased for long-term operating scenarios.

[0103] The increase in video memory utilization after the action is performed is calculated as follows: ;

[0104] The reduction in task inference latency after the action is performed is calculated as follows: ;

[0105] The amount of suppression of the fragmentation growth rate after the action is executed is calculated as follows:

[0106] The DQN network parameter update loss function is:

[0107]

[0108] This is the loss function, which measures the error between the current Q value and the target Q value.

[0109] These are the trainable parameters of the current Q-network;

[0110] For the parameters of the target Q network, periodically from Copying to avoid training oscillations;

[0111] D is the experience replay pool, which stores historical state-action-reward-next state samples;

[0112] γ is a discount factor, with a value range of [0,1], which balances the importance of immediate rewards and long-term cumulative rewards;

[0113] This is the current Q-network's value estimate for action A in state S;

[0114] This is the value estimate of the optimal action in the next state S′ for the target Q-network.

[0115] In step S4, historical fragment data of the same type of task is first loaded from S1, including the fragment growth rate, free block size distribution, and sorting time patterns at different task stages (such as model initialization, frame inference, and result output), to construct a fragment evolution feature library. Simultaneously, dynamic data such as the current task fragment growth rate, memory access frequency, and task load curve transmitted in real-time from S3 are received. Historical features and real-time data are concatenated into a multi-dimensional input vector, which is then input into a pre-trained LSTM prediction model to output a predicted fragment growth trend for the next 10 seconds. When the predicted value exceeds a preset fragment warning threshold, the sorting preparation process is triggered.

[0116] In step S4, task gaps are identified by combining the task running status data from S3: by monitoring abrupt changes in the video memory access frequency, when the video memory access frequency drops by more than 60% from its peak for three consecutive sampling cycles (each cycle being 50ms), and the CPU utilization is below 20%, it is determined to be a valid task gap (such as the result output waiting stage after video frame inference is completed). At this time, a greedy merging algorithm is started. Based on the rule summarized in the historical feature library of S1 that "merging adjacent small free blocks has a higher priority than merging scattered large free blocks", free blocks with consecutive addresses are merged first. During the merging process, only necessary boundary data is moved, rather than a full scan of the video memory, which greatly shortens the processing time. After processing is completed, the information of the merged consecutive free blocks is synchronized to the video memory resource pool of S2 for subsequent task allocation, and the processing results are fed back to S3 to update the real-time fragment data.

[0117] The formula for predicting the growth trend of LSTM fragmentation is:

[0118]

[0119] In the formula: The fragmentation growth rate predicted by the LSTM model for the next 10 seconds;

[0120] The input feature vector at time t includes the historical fragmentation growth rate of S1, the real-time fragmentation growth rate of S3, the video memory access frequency, and the task load coefficient.

[0121] Here is the weight matrix of the LSTM hidden layer;

[0122] The bias vector of the LSTM hidden layer;

[0123] The formula for identifying task gaps is:

[0124]

[0125] This is the task gap identifier at time t, where 1 indicates a valid gap and 0 indicates a non-gap.

[0126] Let be the memory access frequency at time t;

[0127] This represents the peak memory access time three sampling cycles prior.

[0128] Let be the CPU utilization at time t;

[0129] The formula for calculating the revenue from merging free blocks is as follows:

[0130]

[0131] B represents the benefit value for merging free blocks, used to sort the merge priority;

[0132] This represents the total size of the merged free blocks;

[0133] Let be the size of the i-th free block to be merged;

[0134] The estimated time for the merge operation is calculated based on historical data from S1.

[0135] As a priority weight, adjacent free blocks are assigned a weight of 1.2, and scattered free blocks are assigned a weight of 0.8.

[0136] In step S5, during the real-time video target detection task, the S5 module strictly follows a hierarchical recycling logic prioritizing temporary buffers. First, it scans the frame buffer data in the temporary buffer. Based on the historical task patterns loaded in S1, expired frame data exceeding the 30-frame retention threshold is marked for recycling, and the corresponding GPU memory space is directly released. Next, it processes the reused feature maps in the dynamic shared area. Combining this with the task load curve transmitted in real-time in S3, when the reuse rate of a certain type of feature map is less than 10% within five consecutive sampling periods, it is determined to be low-value shared data, and batch recycling is initiated. Finally, it reaches the long-term parameter area, recycling only the frozen model branch weights. These weights typically correspond to low-priority detection targets. Before recycling, it verifies whether the current task still needs to call this branch, and releases GPU memory only after confirming there is no dependency.

[0137] In step S6, real-time video multi-target tracking is performed using the Transformer architecture. First, it receives one minute of continuous video memory usage time-series data transmitted in S3, and inputs it into a pre-trained LSTM prediction model to generate a usage trend curve for the next 30 seconds. When the prediction curve shows that the video memory usage will exceed the 90% warning threshold within 15 seconds, the system immediately initiates the overflow risk response process. At this time, the module iterates through all currently running tasks, filters out low-priority tasks based on preset priority labels, such as background video statistical analysis tasks, and migrates their non-real-time computing data, such as historical tracking result cache and feature backups, to local NVMe SSD storage. The migration process uses incremental transfer, only transferring cold data blocks not referenced by the current task to avoid interfering with task operation.

[0138] A runtime memory optimization system with real-time self-learning management, wherein the system uses a runtime memory optimization method with real-time self-learning management during runtime, specifically including:

[0139] Real-time learning module: Analyzes task behavior (such as memory access patterns and task lifecycle) in real time through machine learning models, and optimizes memory allocation strategies online to adapt to current task requirements.

[0140] Dynamic memory partitioning module: Based on task characteristics, the video memory is divided into multiple regions (short-term buffer, long-term parameter area and dynamic shared area).

[0141] Defragmentation and Reclamation Module: Based on a self-learning model, this module identifies memory fragmentation trends and proactively defragments memory during idle periods. It provides a segmented reclamation mechanism to gradually release unused memory areas.

[0142] Memory overflow prediction and migration module: Utilizes historical data and real-time monitoring to predict the risk of video memory overflow. When the risk of overflow is high, it prioritizes migrating low-priority task data to main storage or other hardware devices.

[0143] As can be seen from the above, in this invention, by combining the fragmentation evolution patterns of historical tasks with the real-time fragmentation growth rate, the system can predict the development trend of memory fragmentation in advance, avoiding passively triggering defragmentation operations during high-load task phases. Initiating fragment merging during identified task gaps effectively integrates scattered free memory blocks into a continuous pool of available resources without interfering with the real-time execution of core tasks. This fundamentally reduces resource allocation failures caused by memory fragmentation, ensuring the stability and smoothness of system operation.

[0144] In this invention, a self-learning closed-loop mechanism continuously optimizes the adaptability of memory management strategies. Operational data such as the timing of defragmentation and the priority of merging are constantly fed back to the system to update the parameters of the prediction model and the historical pattern database. This results in more accurate fragmentation trend predictions, more timely task gap identification, and more efficient merging operations. Over long-term operation, the system gradually develops memory management logic adapted to specific task scenarios, reducing unnecessary memory consumption and defragmentation overhead, improving the overall utilization efficiency of memory resources, and providing stable and reliable memory support for continuously running real-time tasks.

[0145] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A runtime memory optimization method for real-time self-learning management, characterized by: The method includes the following steps: S1: Task initialization, loading historical memory usage data of similar tasks, generating an initial allocation strategy and synchronizing it to S2, while passing the fragmentation pattern to S4 to provide a basis for decision-making in subsequent operations; S2: Divide the video memory into short-term buffer, long-term parameter area and dynamic shared area and allocate resources, and embed monitoring probes; S3: Collect data such as memory access frequency and priority changes during runtime, and iteratively optimize strategies; synchronize the optimization results to S2 to adjust partitions, push the fragmentation growth rate to S4, and input the video memory usage curve to S6; S4: Based on the fragmentation rate of S3 and the historical patterns of S1, fragment defragmentation is initiated during task intervals to merge idle blocks; S5: Reclaim expired data in layers according to temporary buffer → shared area → parameter area. The released video memory is replenished to the resource pool of S2 in real time. The reclamation time window refers to the task load curve of S3. S6: Based on the memory utilization curve of S3, predict the overflow risk, migrate low-priority task data to external storage, and free up space for S2 to allocate high-priority tasks.

2. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S1, complete running logs of similar tasks within the last 30 days are extracted as historical data. Total video memory resources are pre-allocated based on 80% of the peak average video memory usage of the tasks. The initial division ratio of the short-term buffer, long-term parameter area, and dynamic shared area is set with reference to the historical running average of similar tasks. The system extracts fragmentation patterns by statistically analyzing the fragmentation generation rate and merging efficiency of historical tasks on an hourly basis. At the same time, a task update threshold is set. When the number of new tasks of the same type exceeds 5, the initial allocation strategy is automatically regenerated and the fragmentation patterns are updated to provide decision-making basis for subsequent steps that fit the current task scenario.

3. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S2, based on the historical data of similar tasks loaded in S1, the three-zone partitioning and initial resource allocation of the video memory are completed, specifically including: The short-term buffer mainly stores the original video frames to be inferred and the preprocessed tensor data. The initial allocation ratio is 25% of the total video memory, which is temporarily expanded to 30% when the core detection task is running. The long-term parameter area stores the backbone network weights, detector head parameters, and anchor box configurations of YOLOv8. It allocates 40% of the video memory space and locks it to prevent it from being squeezed out by other areas, which would cause abnormal model loading. The dynamic shared area reserves 35% of the video memory to store intermediate feature maps generated during inference and semantic information reused in multiple frames. The allocation ratio will be dynamically adjusted in real time according to the data reuse rate, and feature maps with high reuse rates will be given priority in space guarantee.

4. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S3, during the runtime phase of the real-time video frame analysis task, a deep Q-network is used as the real-time learning model, and data acquisition and feature construction are completed every 100ms: the memory access frequency of the short-term buffer is collected. Weight update frequency of the long-term parameter region Data reuse rate in dynamic shared areas Task priority coefficient Memory usage rate (u) and fragmentation growth rate There are 6 types of features. After performing min-max normalization on each type of feature, the results are concatenated to obtain the state vector. The normalization operation maps the original features to the [0,1] interval, eliminating dimensional differences to adapt to the input requirements of the DQN model.

5. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S3, the DQN model is based on the current state vector. Output memory allocation actions The action space contains three types of executable operations: adjusting the proportion of the short-run buffer. Adjust the proportion of dynamic shared areas Mark redundant data areas to trigger data reclamation; calculate immediate rewards after each action is performed. To evaluate the effectiveness of actions, the reward function comprehensively considers three core objectives: improved memory utilization, reduced task latency, and fragmentation suppression. Multi-objective collaborative optimization is achieved through weighted summation. Subsequently, The quadruplets are stored in the experience replay pool. After accumulating 1000 samples, the network parameters of DQN are updated by randomly sampling batch data: the memory allocation strategy is iteratively optimized by minimizing the mean square error between the current Q value and the target Q value. The feature normalization formula is: f represents the original collected memory behavior feature value; This is the minimum value of this feature in the historical operating data; This is the maximum value of this feature in historical operational data; These are the normalized feature values, ranging from [0,1], used to eliminate the dimensional differences between different features and improve the stability of model training; The multi-objective instant reward function is: α is a weighting coefficient for improving video memory utilization, with a value range of [0,1]. The value can be increased in scenarios where video memory is scarce. β is the weighting coefficient for reducing task latency, with a value range of [0,1]. In scenarios with high real-time requirements, the value can be increased. γ is the weighting coefficient for suppressing fragmentation growth rate, with a value range of [0,1]. The value can be increased for long-term operating scenarios. The increase in video memory utilization after the action is performed is calculated as follows: ; The reduction in task inference latency after the action is performed is calculated as follows: ; The amount of suppression of the fragmentation growth rate after the action is executed is calculated as follows: The DQN network parameter update loss function is: This is the loss function, which measures the error between the current Q value and the target Q value. These are the trainable parameters of the current Q-network; For the parameters of the target Q network, periodically from Copying to avoid training oscillations; D is the experience replay pool, which stores historical state-action-reward-next state samples; γ is a discount factor, with a value range of [0,1], which balances the importance of immediate rewards and long-term cumulative rewards; This is the current Q-network's value estimate for action A in state S; This is the value estimate of the optimal action in the next state S′ for the target Q-network.

6. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S4, historical fragment data of the same type of task is first loaded from S1, including the fragment growth rate, free block size distribution, and sorting time of different task stages, and a fragment evolution feature library is constructed. At the same time, dynamic data such as the current task fragment growth rate, memory access frequency, and task load curve transmitted in real time from S3 are received. The historical features and real-time data are concatenated into a multi-dimensional input vector, which is input into a pre-trained LSTM prediction model and outputs the fragment growth trend prediction value for the next 10 seconds. When the predicted value exceeds the preset fragmentation warning threshold, the sorting and preparation process is triggered.

7. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S4, task gaps are identified by combining the task running status data from S3: by monitoring the sudden change points of the video memory access frequency, when the video memory access frequency drops by more than 60% from the peak for three consecutive sampling cycles and the CPU utilization is less than 20%, it is determined to be a valid task gap; at this time, the greedy merging algorithm is started, based on the rule summarized in the historical feature library of S1 that "the merging priority of adjacent small free blocks is higher than that of scattered large free blocks", and free blocks with consecutive addresses are merged first. During the merging process, only the necessary boundary data is moved, rather than a full scan of the video memory, which greatly shortens the processing time; After the consolidation is completed, the merged contiguous free block information will be synchronized to the video memory resource pool of S2 for subsequent task allocation. At the same time, the consolidation results will be fed back to S3 to update the real-time fragment data. The formula for predicting the growth trend of LSTM fragmentation is: In the formula: The fragmentation growth rate predicted by the LSTM model for the next 10 seconds; The input feature vector at time t includes the historical fragmentation growth rate of S1, the real-time fragmentation growth rate of S3, the video memory access frequency, and the task load coefficient. Here is the weight matrix of the LSTM hidden layer; The bias vector of the LSTM hidden layer; The formula for identifying task gaps is: This is the task gap identifier at time t, where 1 indicates a valid gap and 0 indicates a non-gap. Let be the memory access frequency at time t; This represents the peak memory access time three sampling cycles prior. Let be the CPU utilization at time t; The formula for calculating the revenue from merging free blocks is as follows: B represents the benefit value for merging free blocks, used to sort the merge priority; This represents the total size of the merged free blocks; Let be the size of the i-th free block to be merged; The estimated time for the merge operation is calculated based on historical data from S1. As a priority weight, adjacent free blocks are assigned a weight of 1.2, and scattered free blocks are assigned a weight of 0.

8.

8. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S5, during the real-time video target detection task, the S5 module strictly follows the hierarchical recycling logic that prioritizes temporary buffers. First, it scans the frame buffer data in the temporary buffer. Based on the historical task patterns loaded in S1, it marks expired frame data exceeding the 30-frame retention threshold as data to be recycled and directly releases the corresponding video memory space. Next, it processes the reused feature maps of the dynamic shared area. Combining the task load curve transmitted in real time in S3, when the reuse rate of a certain type of feature map is less than 10% within 5 consecutive sampling periods, it is determined to be low-value shared data, and batch recycling is initiated.

9. The runtime memory optimization method for real-time self-learning management as described in claim 1, characterized in that: In step S6, the Transformer architecture is used for real-time video multi-target tracking. First, the system receives the continuous 1-minute time series data of video memory usage transmitted in S3, and inputs it into the pre-trained LSTM prediction model to generate the usage trend curve for the next 30 seconds. When the prediction curve shows that the video memory usage will exceed the warning threshold of 90% within 15 seconds, the system immediately starts the overflow risk response process.

10. A runtime memory optimization system for real-time self-learning management, characterized in that: The system uses the runtime memory optimization method for real-time self-learning management as described in any one of claims 1 to 9 during runtime, specifically including: Real-time learning module: Analyzes task behavior in real time through machine learning models and optimizes memory allocation strategies online to adapt to current task requirements; Dynamic memory partitioning module: Divides video memory into multiple regions based on task characteristics; Defragmentation and Recycling Module: Based on a self-learning model, it identifies memory fragmentation trends and proactively defragments during idle time; it provides a segmented recycling mechanism to gradually release unused memory areas; Memory overflow prediction and migration module: Utilizes historical data and real-time monitoring to predict the risk of video memory overflow; when the risk of overflow is high, it prioritizes migrating low-priority task data to main storage or other hardware devices.

Images