GPU pooling video memory resource allocation method and system based on BFS

Abstract

The invention provides a GPU pooling video memory resource allocation method and system based on a BFS, and relates to the technical field of computer resource allocation. The method comprises the following steps: monitoring a video memory state of each GPU node in real time, and recording a mapping relationship between a service and a video memory through a hash table; obtaining video memory use states of all current GPUs based on video memory resource occupation conditions recorded by the hash table; comparing the video memory demand of the new service with the video memory use state of each GPU, and marking the GPUs of which the residual video memories are greater than or equal to the new service demand as candidate resources; traversing all GPUs to judge whether candidate resources exist or not; if the candidate resource exists, directly scheduling the new service to the GPU corresponding to the candidate resource; if not, the system starts a video memory re-planning process; and scheduling each existing service and new service to the corresponding GPU based on the re-planned scheduling scheme. The utilization rate of GPU video memory resources is remarkably improved, and resource waste is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of computer resource allocation, and in particular to a GPU pooled video memory resource allocation method and system based on BFS. Background Art

[0002] With the rapid development of artificial intelligence and large-scale deep learning models, the demand for GPU memory resources is growing. In cloud computing, edge computing, and high-performance computing scenarios, concurrent multi-task execution has become the norm. Efficiently managing and allocating limited GPU memory resources has become a key challenge. Traditional memory allocation methods typically employ static allocation strategies that cannot dynamically adjust the deployment of memory services, leading to low memory utilization and resource contention. Especially in large-model training and highly concurrent inference scenarios, a single task can consume a large amount of memory, while other tasks cannot start due to insufficient memory, severely impacting overall computing efficiency and service quality.

[0003] Therefore, there is an urgent need for a comprehensive solution that can dynamically evaluate the memory status, intelligently plan resource allocation, and efficiently schedule tasks to maximize GPU memory utilization and meet diverse task requirements.

[0004] The above problems are in urgent need of resolution. Summary of the Invention

[0005] The present invention aims to overcome at least one technical problem existing in the prior art and provides a GPU pooled memory resource allocation method and system based on BFS.

[0006] On the one hand, an embodiment of the present invention provides a GPU pooled memory resource allocation method based on BFS, the method comprising: step S1: monitoring the memory status of each GPU node in real time, and recording the mapping relationship between services and memory through a hash table; step S2: obtaining the current memory usage status of all GPUs based on the memory resource occupancy recorded in the hash table; step S3: comparing the memory requirement of a new service with the memory usage status of each GPU, and marking GPUs whose remaining memory is greater than or equal to the new service requirement as candidate resources; step S4: traversing all GPUs to determine whether there are candidate resources; step S5: in response to the existence of candidate resources, directly scheduling the new service to the GPU corresponding to the candidate resource; Step S6: In response to the absence of candidate resources, the system starts a memory re-planning process, including: Step S601: Using the BFS strategy to simulate the scheduling of existing services that have allocated memory to other GPUs; Step S602: Each time the scheduling is simulated, evaluate whether the memory status of the current GPU node meets the memory requirements of the new service; Step S603: In response to the memory status of the current GPU node meeting the memory requirements of the new service, exit the simulation scheduling and output the scheduling plan; Step S604: In response to the memory status of the current GPU node not meeting the memory requirements of the new service, go to step S601; Step S7: Based on the scheduling plan output in step S603, schedule each existing service and the new service to the corresponding GPU.

[0007] Furthermore, the step S1 includes: step S101: polling the memory usage status of each GPU node in real time, and recording the service information running on each GPU through a global hash table, wherein the service information includes one or a combination of service ID, memory usage size and service status; step S102: updating the hash table in real time, including: step S1021: periodically polling the memory status of the GPU node, and updating the remaining memory capacity of the corresponding GPU in the hash table; step S1022: when a service completes migration or termination, dynamically remove the corresponding service record in the hash table and recalculate the remaining memory.

[0008] Furthermore, the step S601 includes: step S6011: obtaining the current memory usage status of all GPUs based on the memory occupancy recorded in the hash table; step S6012: initializing the BFS queue, taking the current memory allocation status as the starting node, and marking it as accessed; step S6013: creating a scheduling record list at the same time to store detailed information of each simulation scheduling.

[0009] Furthermore, the step S601 also includes: step S6014: for each service in the current state, arrange them in ascending order according to their required video memory size; step S6015: traverse the arranged services and try to schedule them to other target GPUs in the order of arrangement of the services.

[0010] Furthermore, the step S6015 includes: step S60151: traversing the services in order from small to large based on the remaining video memory capacity, and checking whether the remaining video memory of the target GPU can accommodate the service; step S60152: in response to the remaining video memory of the target GPU being able to accommodate the service, simulating scheduling and generating a new video memory allocation status node.

[0011] Furthermore, the step S602 includes: step S6021: adding the state node after the simulated scheduling to the BFS queue and recording the scheduling path; step S6022: evaluating whether the memory usage status of the GPU after the simulated scheduling can meet the memory demand of the new service.

[0012] Furthermore, step S6 also includes: during the BFS traversal process, optimizing the search efficiency through pruning strategies and deduplication mechanisms; the pruning strategies include: discarding status nodes whose video memory utilization exceeds a preset threshold; discarding status nodes whose scheduling path length exceeds a preset level; the deduplication mechanism includes: generating a unique hash value for each status node, and recording the visited nodes through a hash table; if the hash value of the newly generated node is repeated with that of the recorded node, further searching of the node is skipped.

[0013] Furthermore, the step S7 further includes: during the process of scheduling each existing service and the new service to the corresponding GPU, if an abnormal rollback mechanism is triggered, restoring the system to an initial state.

[0014] Furthermore, the abnormal rollback mechanism includes: recording the current video memory allocation status and scheduling path before executing the migration; if a service startup failure or video memory allocation abnormality is detected during the migration process, undoing the executed migration operations in reverse order and restoring the hash table to the previous valid state.

[0015] In the second aspect, an embodiment of the present invention provides a GPU pooled memory resource allocation system based on BFS, the system adopts the above-mentioned GPU pooled memory resource allocation method based on BFS, and the system includes: a memory status monitoring and recording module, which is suitable for monitoring the memory status of each GPU node in real time, and recording the mapping relationship between service and memory through a hash table; an existing allocation scheme evaluation module, which is suitable for obtaining the memory resource occupancy recorded in the hash table and the memory usage status of all current GPUs; comparing the memory demand of the new service with the memory usage status of each GPU, marking the GPU with remaining memory greater than or equal to the new service demand as a candidate resource; traversing all GPUs to determine whether there is a candidate resource; a direct scheduling module, which is suitable for directly dispatching the new service to the target server in response to the existence of a candidate resource. The system then schedules the service to the GPU corresponding to the candidate resource; a memory re-planning module is configured to initiate a memory re-planning process by the system in response to the absence of candidate resources, including: using a BFS strategy to simulate scheduling of existing services that have been allocated memory to other GPUs; each time a scheduling simulation is performed, evaluating whether the memory status of the current GPU node meets the memory requirements of the new service; in response to the memory status of the current GPU node meeting the memory requirements of the new service, exiting the simulated scheduling and outputting a scheduling plan; in response to the memory status of the current GPU node not meeting the memory requirements of the new service, using a BFS strategy to simulate scheduling of existing services that have been allocated memory to other GPUs; a re-scheduling module is configured to schedule each existing service and the new service to the corresponding GPU based on the scheduling plan output by the memory re-planning module.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned BFS-based GPU pooled memory resource allocation method when executed by the processor.

[0017] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned BFS-based GPU pooled memory resource allocation method.

[0018] The beneficial effects of the present invention are: (1) Global optimal solution: By systematically traversing all possible memory allocation schemes through the breadth-first search (BFS) algorithm, we can avoid local optimal problems, find the global optimal memory allocation strategy, and significantly improve resource utilization.

[0019] (2) Dynamic adaptability: Supports real-time evaluation of video memory status and task requirements, and can quickly respond to the video memory requirements of new tasks or changes in existing tasks, ensuring flexibility and efficiency in resource allocation.

[0020] (3) Multi-objective optimization: During the search process, multi-dimensional objectives such as video memory utilization, task migration cost, and service priority are comprehensively considered. This can minimize the impact on existing tasks while meeting the requirements of new tasks, thereby ensuring overall service quality.

[0021] (4) Reduce hardware costs: Through memory pooling technology, maximize the use of existing GPU resources, reduce the need for additional hardware procurement, and reduce the overall cost of computing infrastructure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below with reference to the accompanying drawings and examples.

[0023] Figure 1 This is a flow chart of a BFS-based GPU pooled memory resource allocation method provided in Example 1 of the present invention.

[0024] Figure 2 This is a flow chart of another BFS-based GPU pooled memory resource allocation method provided in Example 1 of the present invention.

[0025] Figure 3 This is a flow chart of another BFS-based GPU pooled memory resource allocation method provided in Example 1 of the present invention.

[0026] Figure 4 This is a structural diagram of a BFS-based GPU pooled memory resource allocation system provided in Example 2 of the present invention.

[0027] Figure 5 This is a partial block diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0028] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0030] The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0031] Example 1 For ease of understanding, the following is an overall description of the inventive concept before describing the embodiments of the present invention in detail: This paper provides a method and system for allocating GPU pooled memory resources based on breadth-first search (BFS). This method monitors existing GPU memory usage to assess whether the existing solution can directly accommodate new services. If so, it directly schedules the new service to an appropriate GPU. If not, it re-plans the memory using a BFS algorithm and schedules related services based on the final plan. This significantly improves GPU memory utilization and avoids resource waste.

[0032] The specific implementation is as follows: like Figure 1 , which is a flow chart of a GPU pooled video memory resource allocation method based on BFS provided by the present invention.

[0033] As an example, the method includes: step S1: monitoring the video memory status of each GPU node in real time, and recording the mapping relationship between the service and the video memory through a hash table; step S2: obtaining the video memory usage status of all current GPUs based on the video memory resource occupancy recorded in the hash table; step S3: comparing the video memory requirement of the new service with the video memory usage status of each GPU, and marking the GPU whose remaining video memory is greater than or equal to the new service requirement as a candidate resource; step S4: traversing all GPUs to determine whether there is a candidate resource; step S5: in response to the existence of a candidate resource, directly scheduling the new service to the GPU corresponding to the candidate resource; step S6: in response to the absence of a candidate resource The system starts a memory re-planning process, including: step S601: using the BFS strategy to simulate the scheduling of existing services that have allocated memory to other GPUs; step S602: each time the scheduling is simulated, it is evaluated whether the memory status of the current GPU node meets the memory requirements of the new service; step S603: in response to the memory status of the current GPU node meeting the memory requirements of the new service, exit the simulation scheduling and output a scheduling plan; step S604: in response to the memory status of the current GPU node not meeting the memory requirements of the new service, go to step S601; step S7: based on the scheduling plan output in step S603, schedule each existing service and the new service to the corresponding GPU.

[0034] In some feasible implementations, the step S1 includes: step S101: polling the video memory usage status of each GPU node in real time, and recording the service information running on each GPU through a global hash table, the service information including one or a combination of service ID, video memory occupancy size and service status; step S102: updating the hash table in real time, including: step S1021: periodically polling the video memory status of the GPU node, and updating the remaining video memory capacity of the corresponding GPU in the hash table; step S1022: when a service completes migration or termination, dynamically remove the corresponding service record in the hash table and recalculate the remaining video memory.

[0035] Preferably, first, the services scheduled on each GPU and their memory usage are recorded. A separate record unit is maintained for each GPU to store information about currently running services, including the service ID, memory usage, and service status (e.g., running, completed). To support efficient read and write operations in highly concurrent environments, a thread-safe concurrent hash table (Concurrent HashMap) is used as the storage structure. The hash table key is the service ID, and the value is the service's memory usage information and the GPU number to which it belongs. This design ensures high throughput and low latency for updating and querying memory records when multiple tasks are concurrently scheduled. Second, the status changes of each service are monitored in real time. Service status is monitored through periodic polling or event-driven mechanisms. When a service completes or terminates abnormally, the system triggers a memory release process, deleting the service record from the hash table and updating the corresponding GPU's remaining memory information. This process ensures the real-time and accuracy of memory records and avoids resource waste caused by service offline. Specifically, during system startup, an initialization phase is performed to create a separate thread-safe hash table for each GPU to store service information. At the same time, the total memory size of each GPU is obtained, and the initial remaining memory of all GPUs is recorded as the total memory size. When a new service request is dispatched, the system begins recording information about scheduled services on each GPU. Using an event-driven mechanism, it monitors service status changes in real time. When a service is completed or abnormally terminated, the system immediately deletes the service record from the hash table and updates the remaining memory information of the corresponding GPU.

[0036] In some feasible implementations, steps S2-S4 include: obtaining the current memory usage status of all GPUs based on the memory resource occupancy recorded in the hash table. The hash table stores the services running on each GPU and their memory occupancy information, including service ID, memory size, and GPU number. By traversing the hash table, the remaining memory capacity of each GPU is calculated, that is, the total memory minus the allocated memory. This process can reflect the resource utilization of each GPU in real time and provide accurate data support for subsequent evaluation. Secondly, traverse all GPUs to determine whether there is a graphics card that meets the memory requirements of the new service. Specifically, compare the memory requirements of the new service with the remaining memory capacity of each GPU. If the remaining memory of a GPU is greater than or equal to the new service requirement, mark the GPU as a candidate resource, and record its number and remaining memory size. This step can quickly screen out available GPU resources through simple traversal and comparison operations, avoiding complex computing overhead.

[0037] In some possible implementations, combined Figure 2As shown, step S601 includes: step S6011: based on the memory occupancy recorded in the hash table, obtaining the current memory usage status of all GPUs; step S6012: initializing the BFS queue, using the current memory allocation status as the starting node, and marking it as accessed; step S6013: simultaneously creating a scheduling record list for storing detailed information for each simulated scheduling. Step S601 also includes: step S6014: for each service in the current state, arranging them in ascending order according to their required memory size; step S6015: traversing the arranged services and attempting to schedule them to other target GPUs in the order in which they are arranged. Preferably, when the existing GPU memory resources cannot directly meet the needs of the new service, a memory replanning process is initiated. Based on the memory occupancy recorded in the hash table, obtaining the current memory usage status of all GPUs; initializing the BFS queue, using the current memory allocation status as the starting node, and marking it as accessed; and simultaneously creating a scheduling record list for storing detailed information for each simulated scheduling. Starting from the starting node, all possible memory allocation solutions are traversed layer by layer, including: Sorting by existing service memory usage: For each service in the current state, sort it by its required memory size, from smallest to largest. This way, the first solution found by the BFS search will be to move the service with the smallest memory usage first, reducing the overall migration cost of the system. Traversing existing services: For each service in the current state (sorted), attempt to schedule it to a different GPU. Here, traversing the GPUs in ascending order of remaining memory is attempted. The main purpose is to leave more overall memory space for the new service.

[0038] In some possible implementations, combined Figure 3 As shown, step S6015 includes: step S60151: traversing services in ascending order of remaining video memory capacity to check whether the remaining video memory of the target GPU can accommodate the service; step S60152: simulating scheduling and generating a new video memory allocation status node in response to the remaining video memory of the target GPU being able to accommodate the service. Step S602 includes: step S6021: adding the status node after simulated scheduling to the BFS queue and recording the scheduling path; step S6022: evaluating whether the video memory usage status of the GPU after simulated scheduling can meet the video memory requirements of the new service.

[0039] Preferably, the target GPU's remaining memory is checked to see if it can accommodate the service. If so, a scheduling simulation is performed, generating a new memory allocation status node. The simulated scheduling status node is added to the BFS queue, and the scheduling path is recorded. Simultaneously, the new status node is evaluated to see if it can meet the memory requirements of the new service. If so, the search is exited and the scheduling record is output.

[0040] In some feasible implementations, during the BFS traversal process, in order to avoid state space explosion, the following optimization strategy is adopted: the step S6 also includes: during the BFS traversal process, optimizing the search efficiency through a pruning strategy and a deduplication mechanism; the pruning strategy includes: discarding state nodes whose memory utilization exceeds a preset threshold; discarding state nodes whose scheduling path length exceeds a preset level; the deduplication mechanism includes: generating a unique hash value for each state node, and recording the visited nodes through a hash table; if the hash value of the newly generated node is the same as that of the recorded node, further searching of the node is skipped. Specifically, the preset threshold value based on the utilization setting can be 95%, and the preset level based on the scheduling path setting can be 5 layers, that is, during the BFS traversal process, discarding state nodes whose memory utilization exceeds 95%, discarding state nodes whose scheduling path length exceeds 5 layers, and effectively reducing invalid searches through the pruning strategy. The deduplication mechanism effectively avoids repeated calculations.

[0041] In some feasible implementations, the BFS search process terminates in the following situations: A feasible solution is found: When a state node can meet the memory requirements of the new service, the scheduling record is output and the search ends. Traversal is completed: When the BFS queue is empty and no feasible solution is found, the scheduling is determined to have failed and the process ends.

[0042] In some feasible implementations, step S7 further includes restoring the system to its initial state if an exception rollback mechanism is triggered during the scheduling of each existing service and new service to the corresponding GPU. The exception rollback mechanism includes recording the current memory allocation state and scheduling path before executing the migration; if a service startup failure or memory allocation anomaly is detected during the migration process, the executed migration operations are reversed and the hash table is restored to its previous valid state. The rollback mechanism ensures system stability; if no viable solution is found, the scheduling and deployment task for the service is rejected.

[0043] In some feasible implementations, the method further includes dynamically adjusting the polling frequency, such as: when the GPU cluster load is lower than a first threshold, reducing the polling frequency to once per minute; when a new service request is detected or the memory utilization is higher than a second threshold, increasing the polling frequency to once per second.

[0044] In the above embodiment, the present invention uses a thread-safe hash table to record video memory information, thereby ensuring efficient read and write operations in a high-concurrency environment and improving the efficiency of updating and querying video memory records. By monitoring the service status in real time and releasing video memory in a timely manner, resource waste is avoided and the real-time and accuracy of video memory records are ensured. Through the BFS search and optimization strategy, it is possible to efficiently find feasible video memory re-planning solutions when existing GPU video memory resources are insufficient, thereby improving the success rate of service scheduling and reducing the overall migration cost of the system. The rollback mechanism provided enhances the stability of the system, ensuring that the system can maintain normal operation when anomalies occur during service migration.

[0045] Example 2 See also Figure 4 , this embodiment provides a structural diagram of a GPU pooled memory resource allocation system based on BFS.

[0046] As an example, the system adopts the BFS-based GPU pooled memory resource allocation method described in Example 1, and the system includes: The video memory status monitoring and recording module 400 is adapted to monitor the video memory status of each GPU node in real time, and record the mapping relationship between services and video memory through a hash table.

[0047] The existing allocation scheme evaluation module 410 is applicable to the memory resource occupancy recorded in the hash table to obtain the memory usage status of all current GPUs; compares the memory requirement of the new service with the memory usage status of each GPU, and marks the GPUs whose remaining memory is greater than or equal to the new service requirement as candidate resources; and traverses all GPUs to determine whether there are candidate resources.

[0048] The direct scheduling module 420 is adapted to directly schedule the new service to the GPU corresponding to the candidate resource in response to the existence of the candidate resource.

[0049] The memory re-planning module 430 is adapted to initiate a memory re-planning process in response to the absence of candidate resources, including: using the BFS strategy to simulate scheduling of existing services that have been allocated memory to other GPUs; evaluating whether the memory status of the current GPU node meets the memory requirements of the new service each time the scheduling is simulated; in response to the memory status of the current GPU node meeting the memory requirements of the new service, exiting the simulated scheduling and outputting a scheduling plan; in response to the memory status of the current GPU node not meeting the memory requirements of the new service, using the BFS strategy to simulate scheduling of existing services that have been allocated memory to other GPUs.

[0050] The rescheduling module 440 is adapted to schedule each existing service and new service to a corresponding GPU based on the scheduling solution output by the graphics memory replanning module.

[0051] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0052] It is worth noting that all modules involved in this embodiment are logical units. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovations of this invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by this invention. However, this does not mean that other units do not exist in this embodiment.

[0053] Example 3 See also Figure 5 An embodiment of the present invention further provides an electronic device, comprising: a memory and a processor; the memory stores at least one program instruction; the processor implements the BFS-based GPU pooled memory resource allocation method provided in Example 1 by loading and executing the at least one program instruction.

[0054] The memory 702 and processor 701 are connected using a bus. The bus can include any number of interconnected buses and bridges, connecting various circuits of one or more processors 701 and memory 702. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and, therefore, are not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 701 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor 701.

[0055] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 702 can be used to store data used by the processor 701 when performing operations.

[0056] Example 4 An embodiment of the present invention further provides a storage medium storing a BFS-based GPU pooled memory resource allocation method. When executed by a processor, the BFS-based GPU pooled memory resource allocation program implements the steps of the BFS-based GPU pooled memory resource allocation method described above. Because this storage medium utilizes all the technical solutions of all the aforementioned embodiments, it at least has all the beneficial effects brought about by the technical solutions of the aforementioned embodiments, and no further details are given here.

[0057] The above is only an embodiment of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A GPU pooled memory resource allocation method based on BFS, characterized in that: The method comprises: Step S1: monitor the memory status of each GPU node in real time, and record the mapping relationship between services and memory through a hash table; Step S2: Based on the memory resource occupancy recorded in the hash table, obtain the current memory usage status of all GPUs; Step S3: Compare the memory requirement of the new service with the memory usage status of each GPU, and mark the GPUs whose remaining memory is greater than or equal to the new service requirement as candidate resources; Step S4: traverse all GPUs to determine whether there are candidate resources; Step S5: In response to the existence of candidate resources, directly scheduling the new service to the GPU corresponding to the candidate resources; Step S6: In response to the absence of candidate resources, the system initiates a video memory replanning process, including: Step S601: Using the BFS strategy to simulate scheduling of existing services that have been allocated video memory to other GPUs; Step S602: Each time scheduling is simulated, evaluate whether the memory status of the current GPU node meets the memory requirements of the new service; Step S603: In response to the video memory status of the current GPU node meeting the video memory requirement of the new service, exit the simulation scheduling and output the scheduling plan; Step S604: In response to the current GPU node's memory status not meeting the new service's memory requirement, go to step S601; Step S7: Based on the scheduling solution output in step S603, schedule each existing service and new service to the corresponding GPU.

2. The GPU pooling memory resource allocation method based on BFS according to claim 1, characterized in that: The step S1 comprises: Step S101: polling the memory usage status of each GPU node in real time, and recording service information running on each GPU through a global hash table, wherein the service information includes one or a combination of service ID, memory usage size, and service status; Step S102: updating the hash table in real time, including: Step S1021: Periodically poll the memory status of the GPU node and update the remaining memory capacity of the corresponding GPU in the hash table; Step S1022: When a service completes migration or terminates, the corresponding service record in the hash table is dynamically removed and the remaining video memory is recalculated.

3. The GPU pooling memory resource allocation method based on BFS according to claim 1, characterized in that: The step S601 includes: Step S6011: Based on the video memory occupancy recorded in the hash table, obtain the current video memory usage status of all GPUs; Step S6012: Initialize the BFS queue, use the current video memory allocation state as the starting node, and mark it as visited; Step S6013: Simultaneously create a scheduling record list for storing detailed information of each simulation scheduling.

4. The GPU pooling memory resource allocation method based on BFS according to claim 3, characterized in that: The step S601 further includes: Step S6014: sort each service in the current state in ascending order according to its required video memory size; Step S6015: traverse the arranged services and try to schedule them to other target GPUs in the order in which they are arranged.

5. The GPU pooling memory resource allocation method based on BFS according to claim 4, characterized in that: The step S6015 includes: Step S60151: Traverse the services in ascending order based on the remaining video memory capacity to check whether the remaining video memory of the target GPU can accommodate the service; Step S60152: In response to the remaining video memory of the target GPU being able to accommodate the service, simulate scheduling and generate a new video memory allocation status node.

6. The GPU pooling memory resource allocation method based on BFS according to claim 5, characterized in that: The step S602 includes: Step S6021: Add the state node after simulated scheduling to the BFS queue and record the scheduling path; Step S6022: Evaluate whether the GPU memory usage status after the simulated scheduling can meet the memory demand of the new service.

7. The GPU pooling memory resource allocation method based on BFS according to claim 1, characterized in that: The step S6 further includes: During the BFS traversal process, the search efficiency is optimized through pruning strategies and deduplication mechanisms; The pruning strategy includes: discarding state nodes whose memory utilization exceeds a preset threshold; discarding state nodes whose scheduling path length exceeds a preset level; The deduplication mechanism includes: generating a unique hash value for each state node, recording the visited nodes through a hash table; if the hash value of the newly generated node is the same as that of the recorded node, further searching of the node is skipped.

8. The GPU pooling memory resource allocation method based on BFS according to claim 1, characterized in that: The step S7 further includes: during the process of scheduling each existing service and the new service to the corresponding GPU, if an abnormal rollback mechanism is triggered, restoring the system to an initial state.

9. The GPU pooling memory resource allocation method based on BFS according to claim 8, characterized in that: The abnormal rollback mechanism includes: Record the current video memory allocation status and scheduling path before executing migration; If a service startup failure or abnormal video memory allocation is detected during the migration process, the executed migration operations are undone in reverse order and the hash table is restored to the previous valid state.

10. A GPU pooled memory resource allocation system based on BFS, the system adopting the GPU pooled memory resource allocation method based on BFS according to any one of claims 1 to 9, characterized in that: The system comprises: The memory status monitoring and recording module is suitable for real-time monitoring of the memory status of each GPU node and recording the mapping relationship between services and memory through a hash table; The existing allocation scheme evaluation module is applied to the memory resource occupancy recorded in the hash table to obtain the current memory usage status of all GPUs; the memory requirement of the new service is compared with the memory usage status of each GPU, and the GPUs with remaining memory greater than or equal to the new service requirement are marked as candidate resources; and all GPUs are traversed to determine whether there are candidate resources. a direct scheduling module, adapted to, in response to the existence of candidate resources, directly schedule the new service to the GPU corresponding to the candidate resources; The memory re-planning module is adapted to initiate a memory re-planning process by the system in response to the absence of candidate resources, including: using a BFS strategy to simulate scheduling existing services that have been allocated memory to other GPUs; evaluating whether the memory status of the current GPU node meets the memory requirements of the new service during each scheduling simulation; in response to the memory status of the current GPU node meeting the memory requirements of the new service, exiting the simulation scheduling and outputting a scheduling plan; in response to the memory status of the current GPU node not meeting the memory requirements of the new service, using a BFS strategy to simulate scheduling existing services that have been allocated memory to other GPUs; The rescheduling module is adapted to schedule each existing service and new service to a corresponding GPU based on the scheduling solution output by the video memory replanning module.

Images