GPU video memory dynamic isolation method based on container technology

Abstract

The invention relates to the technical field of containers, in particular to a GPU (Graphics Processing Unit) video memory dynamic isolation method based on a container technology, which comprises the following steps of: intercepting a video memory allocation request instruction of a GPU driving layer according to a container virtualization runtime environment, establishing a circular queue structure, generating a container video memory occupation time sequence queue, and calling the container video memory occupation time sequence queue. And calculating and generating a video memory pressure sensing fluctuation index. According to the method, scattered occupied video memory blocks are moved and merged on the physical level by hanging up the container instruction stream and calling the GPU internal copy instruction, fragment holes in a physical video memory are eliminated, then a virtual address mapping pointer is updated, instruction execution is recovered, the space layout of the video memory is dynamically optimized on the premise that the container does not need to be restarted or service logic does not need to be interrupted, and therefore the space layout of the video memory is optimized. The continuous distribution capability and the overall utilization efficiency of the GPU video memory are improved, and the calculation stability and the resource isolation effect in a multi-task concurrent scene are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of container technology, and in particular to a method for dynamic isolation of GPU memory based on container technology. Background Technology

[0002] Container technology is a lightweight virtualization technology at the operating system level. It uses namespace technology to achieve logical isolation of system resources such as processes, networks, and file systems, and uses the Cgroups (Control Groups) mechanism to limit and audit the usage quotas of physical resources such as CPU, memory, and I / O. Thus, it builds an independent, closed, and portable runtime environment for applications while sharing the host operating system kernel.

[0003] Current technologies primarily rely on the Cgroups mechanism to statically limit the usage quota of physical resources. This management model focuses on setting a total upper limit for resource usage. In complex scenarios where multiple containers share a GPU, when a container suddenly initiates a large-scale or high-frequency request for video memory allocation, even if the total quota limit is not reached, it can instantly fill up the video memory bandwidth or cause driver layer lag, thereby triggering task stuttering or even crashes in neighboring containers. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a dynamic isolation method for GPU memory based on container technology.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a dynamic isolation method for GPU memory based on container technology, comprising the following steps: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instructions, establish a circular queue structure, generate a container memory usage time-series queue, call the container memory usage time-series queue, and calculate and generate a memory pressure perception fluctuation index. The memory pressure sensing fluctuation index is compared with a preset fluctuation threshold parameter. When the memory pressure sensing fluctuation index exceeds the fluctuation threshold parameter, the container is determined to be in a high-frequency jitter state, and a jitter suppression state identifier is generated. Based on the jitter suppression state identifier, a forced shaping mechanism for the container's memory request frequency is activated, a maximum value for memory increment per unit time is set, the difference between the current request and the previous request is calculated and compared with the increment limit value to generate a memory change rate limit parameter. Based on the memory change rate limit parameter, the new memory requests of the container are thresholded and filtered, and a virtualized memory resource pool is constructed to generate a virtual memory fragment pool. The minimum available free area that meets the container request size and the memory change rate limit parameter constraint is retrieved in the virtual memory fragment pool, and a virtual address mapping table entry is established. The fragmentation distribution of the virtual memory fragment pool is monitored. When there are insufficient consecutive free blocks and the virtual address mapping table entry indicates memory dispersion, an instruction stream suspension signal is sent to the container to generate a container execution suspension instruction. In response to the container execution suspension instruction, the GPU internal copy instruction is called to move the occupied memory blocks in the physical memory, merge the scattered free blocks into a continuous region, resume the container instruction stream execution, and obtain a compact memory address space.

[0006] Preferably, the step of obtaining the memory pressure sensing fluctuation index is as follows: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instruction, establish a circular queue structure, continuously collect the real-time memory usage of the container according to a fixed time slice period, and write the memory usage of each time slice into the circular queue in chronological order to generate a container memory usage time sequence queue. Based on the memory usage time sequence queue of the container, the memory usage of each time slice is indexed and numbered, the memory usage difference sequence between adjacent time slices is calculated, and the average value of memory usage and the average trend of the difference sequence are obtained to generate memory dynamic analysis data. Based on the aforementioned dynamic analysis of the video memory, the video memory pressure sensing fluctuation index is calculated.

[0007] Preferably, the step of obtaining the jitter suppression state identifier is as follows: The fluctuation index of the memory pressure perception is compared with the fluctuation threshold parameter. The comparison direction and result flag are recorded. The container is determined to be in a high-frequency jitter state or a non-high-frequency jitter state according to the result flag. The low-pass filter logic is triggered and written into the control channel to generate a jitter suppression state flag.

[0008] Preferably, the steps for obtaining the memory rate-of-change limit parameter are as follows: Based on the jitter suppression status flag, the forced shaping mechanism of the container memory request frequency is activated, the upper limit value of the memory increment per unit time is set and bound to the time slice number, the difference between the current request and the previous request is calculated and compared with the upper limit value of the memory increment, the over-limit entries and the non-over-limit entries are marked, and the memory increment comparison result is generated. Based on the comparison results of the incremental memory, a delayed processing queue position is set for the over-limit entries, the timestamp and request identifier are recorded and the corresponding integer rule fragment is generated, the non-over-limit entries are merged to generate a release list for the same time slice, and the memory change rate limit parameter is generated.

[0009] Preferably, the step of obtaining the virtual video memory fragment pool is as follows: The new memory requests of the container are thresholded and filtered according to the memory change rate limit parameter. The request size and time slice number of the new memory request are parsed, the request size of the previous moment is read, the difference between the two requests is calculated and compared with the memory change rate limit parameter, and the allowed or rejected entries are marked to obtain the list of new memory requests after threshold filtering. Based on the list of newly added video memory requests after threshold filtering, the physical GPU video memory management interface is called in the virtualization layer. The set of thick slice specifications is selected according to the size distribution of the requests in the list, the physical GPU video memory is pre-occupied and divided according to the thick slice specifications, the slice metadata and free bitmap are registered, and a virtual video memory slice pool is generated.

[0010] Preferably, the step of obtaining the virtual address mapping table entry is as follows: Based on the virtual memory fragment pool, the available fragment combinations are searched one by one according to the list of new memory requests after threshold filtering. The smallest free area that meets the request size and the memory change rate limit parameter is selected first. The correspondence between virtual address and physical memory offset is established and written into the virtual memory page table to generate virtual address mapping table entries.

[0011] Preferably, the step of obtaining the suspend instruction for the container is as follows: Based on the free bitmap and fragment metadata of the virtual memory fragment pool, the length of consecutive free blocks is scanned in physical address order. The comparison results between the maximum consecutive free block and the target request threshold are calculated. The virtual address mapping entries are traversed, non-contiguous mappings are identified, an instruction stream suspension signal is sent and the execution of GPU instructions in the container is paused, and a container execution suspension instruction is generated.

[0012] Preferably, the step of obtaining the compact video memory address space is as follows: According to the container, the suspend instruction is executed, the migration list of occupied video memory blocks is locked, sorted by physical address and the starting point of the target contiguous region is defined, the GPU internal copy instruction is called block by block to complete the relocation and reclaim the source free blocks, and adjacent free blocks are merged to form a single contiguous region, generating a contiguous free region. Based on the continuous free area, the new physical address offset of the moved video memory block is extracted, the corresponding virtual address mapping table entry is located one by one and the pointer offset is rewritten, the page table cache is refreshed and the container instruction stream is resumed to generate a compact video memory address space.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, the GPU driver-level memory allocation request instructions are intercepted based on the container virtualization runtime environment, and a circular queue structure is established. The high-frequency changing memory usage behavior of the container is continuously sampled, and a memory pressure perception fluctuation index is calculated by combining static dispersion, dynamic change rate, and system load weight. This index quantifies the resource contention risk and jitter intensity under high load scenarios. Based on this index, a forced shaping mechanism is dynamically activated, and an incremental upper limit is set per unit time. Sudden requests exceeding the safety threshold are delayed or rejected, smoothing the container's resource request curve and preventing memory overflow or memory bus issues caused by a single process's instantaneous excessive demand. To address congestion, a virtualized memory resource pool and address mapping table are constructed to decouple physical memory from logical addresses, providing a foundation for physical memory reorganization. When high memory dispersion is detected, the container instruction stream is suspended and the GPU's internal copy instruction is called to move and merge scattered occupied memory blocks at the physical level, eliminating fragmentation holes in physical memory. Subsequently, the virtual address mapping pointer is updated and instruction execution is resumed. This dynamically optimizes the memory space layout without restarting the container or interrupting business logic, improving the continuous allocation capability and overall utilization efficiency of GPU memory, and ensuring computational stability and resource isolation in multi-task concurrent scenarios. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the steps of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] Please see Figure 1 This invention provides a technical solution, a method for dynamic isolation of GPU memory based on container technology, comprising the following steps: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instructions, establish a circular queue structure, generate a container memory usage time-series queue, call the container memory usage time-series queue, and calculate and generate a memory pressure perception fluctuation index. The memory pressure sensing fluctuation index is compared with the preset fluctuation threshold parameter. When the memory pressure sensing fluctuation index exceeds the fluctuation threshold parameter, it is determined that the container is in a high-frequency jitter state, and a jitter suppression status flag is generated. Based on the jitter suppression status flag, the forced shaping mechanism for the container's memory request frequency is activated, the upper limit value of the memory increment per unit time is set, the difference between the current request and the previous request is calculated and compared with the upper limit value of the increment to generate a memory change rate limit parameter. Based on the memory change rate limit parameter, the new memory requests of the container are thresholded and filtered, and a virtualized memory resource pool is constructed. A virtual memory fragment pool is generated, and the smallest available free area that meets the container request size and the memory change rate limit parameter constraint is retrieved in the virtual memory fragment pool. A virtual address mapping table entry is then established. The fragmentation distribution of the virtual memory pool is monitored. When there are insufficient consecutive free blocks and the virtual address mapping table entries indicate memory dispersion, an instruction stream suspension signal is sent to the container, generating a container execution suspension instruction. In response to the container execution suspension instruction, the GPU's internal copy instruction is invoked to move the occupied memory blocks in the physical memory, merging the scattered free blocks into a contiguous region, resuming the container instruction stream execution, and obtaining a compact memory address space.

[0017] The steps to obtain the memory pressure sensing fluctuation index are as follows: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instruction, establish a circular queue structure, continuously collect the real-time memory usage of the container according to a fixed time slice period, and write the memory usage of each time slice into the circular queue in chronological order to generate a container memory usage time sequence queue. Based on the memory usage time sequence queue of the container, the memory usage of each time slice is indexed and numbered, the memory usage difference sequence between adjacent time slices is calculated, and the average value of memory usage and the average trend of the difference sequence are obtained to generate the dynamic analysis quantity of memory. Based on the dynamic analysis of the video memory, the video memory pressure perception fluctuation index is calculated using the following formula: ; in, This is a metric for sensing fluctuations in memory pressure. This represents the total number of sampling points in the container's memory usage time queue. For the first Video memory usage per time slice This represents the average memory usage across all time slices. The average trend of the difference in video memory usage between adjacent time slices is calculated using the following formula: , This represents the total physical video memory capacity of the current GPU. This is a constant pressure adjustment coefficient used to regulate the sensitivity of pressure increases.

[0018] Specifically, based on the container virtualization runtime environment, the GPU driver-level memory allocation request instructions are intercepted. Utilizing dynamic link library preloading technology, interception functions are inserted during the CUDA driver library loading phase. Specifically, symbol redirection is performed on memory management interfaces such as cudaMalloc and cudaFree. A fixed-length circular buffer based on shared memory is constructed as a circular queue structure. The depth of this queue is set to accommodate data points from the past several seconds; for example, if the sampling frequency is set to 2Hz, the initial queue length is 10 units. An independent high-precision timer thread is started, with a time slice period of 500 milliseconds. At the end of each time slice... By calling GPU management interfaces such as NVML or directly reading the memory counter at the driver level, the current physical video memory usage in the context of the container process is obtained. If the underlying interface returns units in bytes, the value is immediately divided by 1048576 to convert it to megabytes (MB). The collected real-time video memory usage is combined with the current timestamp and written to the tail pointer position of the circular queue in chronological order. If the queue is full, the earliest data at the head is overwritten. The read and write pointer positions are updated through atomic operations to prevent data races during concurrent writes. After completing one round of sampling, the current complete time window data is extracted from the queue to generate a container video memory usage time sequence queue.

[0019] Based on the memory usage time-series queue of the container, the memory region of the circular queue is locked to prevent write operations. Starting from the current write pointer position, backtracking is performed to read the data records of the most recent N time slices. The memory usage of each time slice is assigned an integer index number from 1 to N according to the time sequence. A temporary floating-point array of length N-1 is created. The memory usage data with indices from 1 to N-1 is traversed. The memory usage of the (k+1)th time slice is subtracted from the memory usage of the kth time slice to obtain the memory change increment between adjacent time steps. This increment value is then sequentially filled into the temporary array. Within the group, the memory usage difference sequence between adjacent time slices is obtained. All values ​​in the memory usage sequence are summed and divided by the total number of sampling points N to obtain the average memory usage. At the same time, all values ​​in the memory usage difference sequence are summed and divided by the sequence length N-1 to obtain the average trend of the difference sequence. This trend value reflects whether the memory usage is increasing or decreasing within the current time window. The difference sequence, average usage, and trend value obtained above are packaged and encapsulated as the basic input for subsequent fluctuation calculations to generate dynamic memory analysis quantities.

[0020] The formula for calculating the video memory pressure perception fluctuation index combines the dispersion of video memory usage (the first variance) with the dispersion of video memory change rate (the second variance) to more comprehensively capture the severe fluctuations of the video memory. At the same time, it introduces a video memory pressure weight term in the form of an exponential form, so that when the video memory usage is close to the total capacity, even if the fluctuation amplitude is the same, the calculated pressure index will be amplified exponentially. This makes it more sensitive to fluctuations when video memory resources are scarce, and achieves differentiated isolation control. This parameter represents the total number of sampling points in the timing queue used by the container's video memory. It is obtained by reading the effective data length of the circular queue established in the preceding steps. Specifically, the timing analysis window length is set in the configuration file. (e.g., 2.5 seconds) and sampling time slice interval (For example, 0.5 seconds), the calculation formula is as follows: For example, if the analysis window is set to 2.5 seconds and the sampling interval to 0.5 seconds, then... This value represents the number of historical data points involved in the calculation of this fluctuation.

[0021] For the first The video memory usage per time slice is obtained by traversing the container's video memory usage time queue, and the unit is megabytes (MB). The specific acquisition process involves extracting this data from real-time video memory usage intercepted and recorded by the GPU driver layer. For example, in an Nvidia GPU environment, this is achieved by calling the nvmlDeviceGetMemoryInfo interface to obtain the current process's video memory usage value (in bytes). The calculation formula is as follows: And store them in a queue in chronological order. That is, the first in the queue The specific value stored in each location.

[0022] This is the average memory usage across all time slices. This parameter is obtained by averaging the memory usage across all time slices in the timing queue.

[0023] The average trend of the difference in video memory usage between adjacent time slices is obtained by calculating the difference in video memory usage between the previous and subsequent time slices in the sequence and averaging the results. The unit is kept in MB, and it is used to characterize the overall growth or decrease trend of video memory usage.

[0024] This refers to the total physical video memory capacity of the current GPU. This parameter is obtained by querying the GPU hardware specifications and is measured in megabytes (MB). The specific process is as follows: During system initialization, the Total Memory field is read by calling a GPU device query command (such as nvidia-smi -q -d MEMORY) (e.g., returning 25769803776 Bytes). The calculation formula is... For example, for an RTX 3090 graphics card, the converted... The value is 24576MB.

[0025] This is a constant pressure adjustment coefficient used to regulate the sensitivity of pressure increases. This parameter is obtained through experimental testing and empirical fitting. The specific process involves: deploying a standard load container in an experimental environment, gradually increasing the memory utilization from 10% to 90%, and simulating memory allocation and release jitter at different frequencies. The fluctuation values ​​without the coefficient are recorded. A trigger threshold is set when the memory utilization reaches 80% and high-frequency jitter occurs (e.g., a target trigger value of 100). This is then used to deduce... The value range was determined through multiple tests. When high video memory usage is required, a more sensitive response is needed. Therefore, the setting... The value of 0.6 causes the exponent term to increase significantly when memory usage is high. The value of .

[0026] Calculations based on parameters: A set of actual collected container memory usage data was selected for calculation, and the number of sampling points was set. .

[0027] The obtained video memory usage sequence (Unit: MB) is: , , , , .

[0028] Total GPU memory MB.

[0029] Pressure adjustment coefficient .

[0030] The first step is to calculate the average video memory usage. : MB.

[0031] The second step is to calculate the sequence of occupancy differences between adjacent time slices: ; ; ; ; The third step is to calculate the average trend of the difference. : MB.

[0032] The fourth step is to calculate the first part under the square root in the formula (the variance term of video memory usage): ; ; ; ; Step 5: Calculate the second part under the square root in the formula (the variance term of the difference in memory usage): The difference sequence is: 500, -300, 800, -200. .

[0033] ; ; ; ; Summation: ; ; Step 6: Calculate the sum within the square root and take the square root: ; ; Step 7: Calculate the exponential stress term: ; ; Step 8: Calculate the final indicators : ; The results indicate that, under conditions where current video memory usage is approximately 50% of the total capacity and there are fluctuations in memory allocation and deallocation, the perceived fluctuation index of video memory pressure... The calculation result is 1382.78, which quantifies the current unstable state of the video memory. If the set fluctuation threshold parameter is 1200, the result exceeds the threshold, and the system will determine that the container is in a high-frequency jitter state, thereby triggering the subsequent video memory shaping mechanism. In this way, the usage level of video memory can be combined with the usage fluctuation to achieve accurate identification of high-risk video memory operations.

[0034] The steps to obtain the jitter suppression status indicator are as follows: The memory pressure sensing fluctuation index is compared with the fluctuation threshold parameter. The comparison direction and result flag are recorded. The container is determined to be in a high-frequency jitter state or a non-high-frequency jitter state according to the result flag. The low-pass filter logic is triggered and written to the control channel to generate a jitter suppression status flag.

[0035] Specifically, the memory pressure perceived fluctuation index is compared with the fluctuation threshold parameter to read the memory pressure perceived fluctuation index. The real-time calculated value reads the fluctuation threshold parameter pre-configured in the global register. Among them, the fluctuation threshold parameter The setup process involves starting a benchmark test program during the initial container deployment phase to simulate the container's memory allocation and release behavior under normal business loads. Memory pressure perceptual fluctuation data is continuously collected over 10 minutes, and the maximum value in the sample set is obtained. To tolerate occasional, non-malicious fluctuations, a safety redundancy factor is set. (For example, if the value is 1.2), the calculation formula is: For example, measuring the maximum volatility indicator during testing. If it is 1000, then set Execute the comparison instruction to determine Is it greater than ,like If the comparison direction is positive, the comparison direction is recorded as positive overflow, and the result flag is temporarily set to 1; otherwise, it is set to 0. To avoid misjudgment caused by a single instantaneous spike, the low-pass filtering logic is called to smooth the result flag. A 5-bit shift register is established, and the latest result flag is shifted into the least significant bit of the register. The number of 1s in the register is counted. Only when the number of 1s is greater than or equal to 3 is it finally confirmed that the current state is jittering. The confirmed state boolean value is written to the shared memory control channel of the GPU driver layer. This channel is mapped to the address space of the virtualization manager for the scheduler to read in real time and generate a jitter suppression status flag.

[0036] The steps to obtain the memory rate of change limit parameter are as follows: Based on the jitter suppression status flag, the forced shaping mechanism of the container memory allocation frequency is activated, the upper limit value of memory increment per unit time is set and bound to the time slice number, the difference between the current request and the previous request is calculated and compared with the upper limit value of memory increment, the over-limit and non-over-limit entries are marked, and the memory increment comparison result is generated. Based on the incremental comparison results of video memory, a delayed processing queue position is set for the entries that exceed the limit, the timestamp and request identifier are recorded and the corresponding integer rule fragment is generated, the entries that do not exceed the limit are merged to generate a release list for the same time slice, and the video memory change rate limit parameter is generated.

[0037] Specifically, based on the jitter suppression status flag, the status bits in the control channel are scanned. When the status bit is high, the forced reshaping mechanism for the container's memory allocation frequency is immediately activated, intercepting all cudaMalloc requests issued by the container and setting the upper limit of memory increment per unit time. The upper limit value is set based on the maximum bandwidth of the GPU memory controller and the container's Quality of Service (QoS) level. The setting process involves obtaining the theoretical upper limit of the physical GPU's memory write bandwidth. (e.g., 900GB / s), the proportion of bandwidth quota allocated to the current container. (e.g., 0.1%), and time slice granularity. (For example, 10ms), the calculation formula is as follows: Substituting the numerical values, we obtain This means that the allowed memory increment within each 10ms time slice is 9MB. This value is bound to the current time slice number index to extract the current memory allocation request amount. And read the request quantity of the same container at the previous moment from the history table. Calculate the absolute value of the difference between the two. ,Will With the maximum value of video memory increment Perform a line-by-line comparison, if If the request violates the smooth application principle, the request entry will be marked as "exceeding the limit". If the value is not exceeded, it is marked as "not exceeded". All marked entries are summarized and a list of structures containing request ID, application size and exceedance status bits are constructed to generate the incremental comparison result of video memory.

[0038] Based on the incremental comparison results of the video memory, the structure list is traversed to identify entries marked as "out of bounds". For each out-of-bounds request, the required delay time is calculated. The computational strategy employs a linear penalty mechanism, setting a basic delay step size. (For example, 5ms), the penalty multiplier is calculated based on the over-limit ratio, and the calculation formula is as follows: For example, the difference exceeding the limit 18MB maximum If it is 9MB, then the delay time is The request ID, original request parameters, and calculated delay timestamp are encapsulated into an integer rule fragment and stored at the tail of the delay processing queue. For entries marked as "not exceeded" in the list, they are directly added to the release list of the current time slice, maintaining their original execution priority. After completing one round of traversal, the release list and the integer rule fragment are merged and serialized to construct a comprehensive control strategy parameter package containing immediate execution instructions and delayed scheduling instructions. This parameter package defines the specific processing actions (immediate allocation or suspension and waiting) of each request by the memory allocator in the next stage, and generates the video memory change rate limit parameters.

[0039] The steps to obtain a virtual memory fragment pool are as follows: The new memory requests of the container are thresholded and filtered according to the memory change rate limit parameter. The request size and time slice number of the new memory request are parsed, the request size of the previous moment is read, the difference between the two requests is calculated and compared with the memory change rate limit parameter, and the entry is marked as allowed or rejected to obtain the list of new memory requests after threshold filtering. Based on the list of new video memory requests after threshold filtering, the physical GPU video memory management interface is called in the virtualization layer. The set of thick slice specifications is selected according to the size distribution of the requests in the list, the physical GPU video memory is pre-occupied and divided according to the thick slice specifications, the slice metadata and free bitmap are registered, and a virtual video memory slice pool is generated.

[0040] Specifically, new memory requests to the container are thresholded and filtered based on the memory change rate limit parameter. First, each received cudaMalloc request is parsed, and the request size field is extracted. (Unit: MB) and the timestamp when the request occurred, by performing a modulo operation on the timestamp (the modulus is the time slice period). Map the request to the corresponding time slice number. The system retrieves the request size record for the container from the historical request record table at the previous moment (i.e., the moment the request was last successfully submitted). Calculate the difference between the current request and the previous request. Read the integrated control strategy parameter package from the memory rate of change limit parameters, extract the release list and shaping rule fragment corresponding to the time slice, and then... The request is compared with the allowable fluctuation range defined in the parameter package. If the request is on the release list, or if it is not on the release list but its... It did not exceed the instantaneous increment threshold defined in the integer shaping rules (this threshold is determined by the upper limit of the video memory increment). If the decision is made, the request entry will be marked as "allowed". If the request is in an integer rule fragment and is marked as requiring a delay, or its... If the fluctuation exceeds the allowable range, the delay time will be calculated according to the shaping rules. The request is temporarily stored and marked as "rejected" or "delayed". For requests marked as "delayed", an asynchronous scheduled task is started. After a certain time, the request is reinserted into the head of the processing queue. Finally, all request entries marked "allowed" are aggregated to generate a list of new video memory requests after threshold filtering.

[0041] Based on the list of newly added video memory requests filtered by the threshold, the virtualization layer uses ioctl to call the underlying physical GPU memory management interface (such as the NVIDIA UVM driver interface) to read the current GPU's memory page table layout information and iterate through the size of each request in the list. ,according to Select appropriate slab class based on the numerical distribution characteristics, and define the slab class set as follows: Using the best fit principle, select those greater than or equal to And the closest thickness specification For example, if the requested size is 3.5MB, then a 4MB thick block specification is selected, requesting a contiguous large page of physical video memory (e.g., a 2GB large page) from the physical memory manager, and then processing it according to the selected... Perform equal partitioning, initialize the partition metadata structure, record the physical start address, length, major page ID, and allocation status of each partition, and create a corresponding free bitmap. Initialize the bits corresponding to all partitioned partitions to 0 (indicating free). Mount this metadata and bitmap structure to the memory pool object of the virtual memory manager to generate a virtual memory partition pool.

[0042] The steps for obtaining virtual address mapping table entries are as follows: Based on the virtual memory fragment pool, the available fragment combinations are searched one by one according to the list of new memory requests after threshold filtering. The smallest free area that meets the request size and the memory change rate limit parameter is selected first. The correspondence between virtual address and physical memory offset is established and written into the virtual memory page table to generate virtual address mapping table entries.

[0043] Specifically, based on the virtual memory fragment pool, the list of new memory requests after threshold filtering is iterated again. For each request, the free bitmap in the corresponding virtual memory fragment pool is locked, and bit manipulation instructions (such as ffs or __builtin_ctz) are used to quickly find the first bit index that is 0 in the bitmap. This index corresponds to an available physical memory fragment, and the size of the fragment is checked to see if it meets the requirements. The system then verifies whether the allocation operation complies with the total amount constraint in the memory change rate limit parameter (i.e., checks whether the current allocated amount plus the new request amount exceeds the container's quota limit). If the condition is met, the bit corresponding to the bitmap index is set to 1 (indicating occupancy), and the physical base address of the fragment is extracted. Calculate its offset relative to the video memory base address. Allocate a contiguous range of virtual addresses within the container's virtual address space. Construct page table entries (PTEs) Mapped to (or Set the page table attributes to read-write and disable caching, write the constructed page table entries into the IOMMU or GPU MMU page table, refresh the TLB to make the mapping effective, record the mapping relationship between virtual address and physical offset in the hash table, and generate virtual address mapping table entries.

[0044] The steps to obtain the container's suspend instruction are as follows: Based on the free bitmap and fragment metadata of the virtual memory fragment pool, scan the length of consecutive free blocks in physical address order, calculate the comparison results between the maximum consecutive free block and the target request threshold, traverse the virtual address mapping table entries, identify non-contiguous mappings, send an instruction stream suspension signal and pause the execution of GPU instructions in the container, and generate a container execution suspension instruction.

[0045] Specifically, based on the free bitmap and fragment metadata of the virtual memory fragment pool, the memory pool object is locked to prevent concurrent modification. The bitmap data is extracted and linearly scanned sequentially to identify consecutive 0-bit sequences (representing consecutive free blocks). The starting address and length of each consecutive free region are recorded, and the segment with the largest length is selected as the starting address. Read the size of the current pending requests as the target request threshold. ,Will and Perform a comparison, if If the total free space is sufficient, it indicates that there is a lack of sufficiently large contiguous space (i.e., external fragmentation has occurred). At this point, the dispersion detection logic is initiated, traversing all established virtual address mapping entries, calculating the degree of physical address discontinuity, and counting the number of non-contiguous mapping pairs of physical page frame numbers. When this number exceeds 30% of the total number of mappings (this threshold is set based on experience; for example, if there are 1000 total mappings and more than 300 are non-contiguous), it is confirmed that memory dispersion is severe. A high-priority signal is immediately sent to the container's instruction scheduler to trigger the instruction stream suspension mechanism. This mechanism uses CUDA's cudaStreamQuery interface to poll the stream status and injects a pause instruction at a safe point to temporarily block the startup of new GPU kernels and generate a container execution suspension instruction.

[0046] The steps for obtaining the compact video memory address space are as follows: According to the container execution suspension instruction, the migration list of occupied video memory blocks is locked, sorted by physical address and the starting point of the target contiguous region is defined. The GPU internal copy instruction is called block by block to complete the relocation and reclaim the source free blocks. Adjacent free blocks are merged to form a single contiguous region, generating a contiguous free region. Based on the continuous free areas, extract the new physical address offset of the moved video memory block, locate the corresponding virtual address mapping table entry one by one and rewrite the pointer offset, refresh the page table cache and restore the execution of the container instruction stream to generate a compact video memory address space.

[0047] Specifically, based on the container's execution suspend instruction, it confirms that all GPU instruction streams within the container have entered an idle or waiting state. It then extracts a list of all currently marked "occupied" memory blocks from the metadata management structure, reads the physical start address, length, and process ID of each occupied block, and sorts these blocks in ascending order based on their physical start addresses to obtain an ordered sequence of occupied blocks. The starting point of the target contiguous region is defined as the base address of the physical memory. Starting from the base address, the new physical address of each occupied block after compact arrangement is calculated sequentially. For example, the new address of the first block is the base address, the new address of the second block is the new address of the first block plus the length of the first block, and so on. This generates a migration list containing the source address, target address, and copy length, utilizing the GPU's internal high-bandwidth memory copy engine (such as Copy...). The Engine constructs a series of device-to-device memory copy instructions (such as cudaMemcpyAsync or low-level DMA transfer instructions), moving memory blocks from the source location to the target location one by one in the list order. After each block is moved, the physical address record in the metadata is updated immediately, and the original physical space marker is released. After all occupied blocks have been moved, the free space that was originally scattered in various places will naturally merge into the high address end of the physical memory, forming a single and huge continuous free area.

[0048] Based on the contiguous free regions, after the physical relocation operation is completed, the virtual memory view must be updated synchronously to ensure the correct operation of the program. The old and new physical address pairs of each memory block recorded in the migration list are extracted. All virtual address mapping entries stored in the hash table are traversed, and the old physical address is used as the index key to quickly locate the corresponding virtual address entry. The Physical Page Frame Number (PFN) field in the entry is updated to the page frame number corresponding to the new physical address after the relocation. To ensure that the hardware MMU can detect this change, a TLB (Translation Lookaside Buffer) invalidation operation (e.g., using the tlb_invalidate instruction) is performed on the affected page table range. This forces the GPU to reload the page table entries from memory the next time it accesses these virtual addresses. After confirming that all page tables have been updated and the TLB has been refreshed, a recovery signal is sent to the container's instruction scheduler to release the instruction stream from its suspended state, allowing the container to continue submitting and executing GPU computing tasks. At this point, because the physical memory has become compact, subsequent large memory requests can be directly satisfied from the contiguous free regions, generating a compact memory address space.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for dynamic isolation of GPU memory based on container technology, characterized in that, Includes the following steps: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instruction, establish a circular queue structure, generate a container memory usage time-series queue, call the container memory usage time-series queue, and calculate and generate a memory pressure perception fluctuation index. The memory pressure sensing fluctuation index is compared with a preset fluctuation threshold parameter. When the memory pressure sensing fluctuation index exceeds the fluctuation threshold parameter, the container is determined to be in a high-frequency jitter state, and a jitter suppression state identifier is generated. Based on the jitter suppression state identifier, a forced shaping mechanism for the container's memory request frequency is activated, a maximum value for memory increment per unit time is set, the difference between the current request and the previous request is calculated and compared with the increment limit value to generate a memory change rate limit parameter. Based on the memory change rate limit parameter, the new memory requests of the container are thresholded and filtered, and a virtualized memory resource pool is constructed to generate a virtual memory fragment pool. The minimum available free area that meets the container request size and the memory change rate limit parameter constraint is retrieved in the virtual memory fragment pool, and a virtual address mapping table entry is established. The fragmentation distribution of the virtual memory fragment pool is monitored. When there are insufficient consecutive free blocks and the virtual address mapping table entry indicates memory dispersion, an instruction stream suspension signal is sent to the container to generate a container execution suspension instruction. In response to the container execution suspension instruction, the GPU internal copy instruction is called to move the occupied memory blocks in the physical memory, merge the scattered free blocks into a continuous region, resume the container instruction stream execution, and obtain a compact memory address space.

2. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the memory pressure sensing fluctuation index are as follows: Based on the container virtualization runtime environment, intercept the GPU driver-level memory allocation request instruction, establish a circular queue structure, continuously collect the real-time memory usage of the container according to a fixed time slice period, and write the memory usage of each time slice into the circular queue in chronological order to generate a container memory usage time sequence queue. Based on the memory usage time sequence queue of the container, the memory usage of each time slice is indexed and numbered, the memory usage difference sequence between adjacent time slices is calculated, and the average value of memory usage and the average trend of the difference sequence are obtained to generate memory dynamic analysis data. Based on the aforementioned dynamic analysis of the video memory, the video memory pressure sensing fluctuation index is calculated.

3. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the jitter suppression status identifier are as follows: The fluctuation index of the memory pressure perception is compared with the fluctuation threshold parameter. The comparison direction and result flag are recorded. The container is determined to be in a high-frequency jitter state or a non-high-frequency jitter state according to the result flag. The low-pass filter logic is triggered and written into the control channel to generate a jitter suppression state flag.

4. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the memory rate of change limit parameter are as follows: Based on the jitter suppression status flag, the forced shaping mechanism of the container memory request frequency is activated, the upper limit value of memory increment per unit time is set and bound to the time slice number, the difference between the current request and the previous request is calculated and compared with the upper limit value of memory increment, the over-limit entries and not over-limit entries are marked, and the memory increment comparison result is generated. Based on the comparison results of the incremental memory, a delayed processing queue position is set for the over-limit entries, the timestamp and request identifier are recorded and the corresponding integer rule fragment is generated, the non-over-limit entries are merged to generate a release list for the same time slice, and the memory change rate limit parameter is generated.

5. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the virtual video memory fragment pool are as follows: The new memory requests of the container are thresholded and filtered according to the memory change rate limit parameter. The request size and time slice number of the new memory request are parsed, the request size of the previous moment is read, the difference between the two requests is calculated and compared with the memory change rate limit parameter, and the allowed or rejected entries are marked to obtain the list of new memory requests after threshold filtering. Based on the list of newly added video memory requests after threshold filtering, the physical GPU video memory management interface is called in the virtualization layer. The set of thick slice specifications is selected according to the size distribution of the requests in the list, the physical GPU video memory is pre-occupied and divided according to the thick slice specifications, the slice metadata and free bitmap are registered, and a virtual video memory slice pool is generated.

6. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the virtual address mapping table entry are as follows: Based on the virtual memory fragment pool, the available fragment combinations are searched one by one according to the list of new memory requests after threshold filtering. The smallest free area that meets the request size and the memory change rate limit parameter is selected first. The correspondence between virtual address and physical memory offset is established and written into the virtual memory page table to generate virtual address mapping table entries.

7. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the container's suspend instruction are as follows: Based on the free bitmap and fragment metadata of the virtual memory fragment pool, the length of consecutive free blocks is scanned in physical address order. The comparison results between the maximum consecutive free block and the target request threshold are calculated. The virtual address mapping entries are traversed, non-contiguous mappings are identified, an instruction stream suspension signal is sent and the execution of GPU instructions in the container is paused, and a container execution suspension instruction is generated.

8. The GPU memory dynamic isolation method based on container technology according to claim 1, characterized in that, The steps for obtaining the compact video memory address space are as follows: According to the container, the suspend instruction is executed, the migration list of occupied video memory blocks is locked, sorted by physical address and the starting point of the target contiguous region is defined, the GPU internal copy instruction is called block by block to complete the relocation and reclaim the source free blocks, and adjacent free blocks are merged to form a single contiguous region, generating a contiguous free region. Based on the continuous free area, the new physical address offset of the moved video memory block is extracted, the corresponding virtual address mapping table entry is located one by one and the pointer offset is rewritten, the page table cache is refreshed and the container instruction stream is resumed to generate a compact video memory address space.

Images