MCM-GPU-oriented resource storage optimization method, device and equipment

Abstract

The invention provides an MCM-GPU-oriented resource storage optimization method, device and equipment, and the method comprises the steps: carrying out the static analysis of an operator-level data flow diagram of a calculation task in a compiling stage, and extracting the data dependence relation and life cycle characteristics of each operator; generating a storage optimization scheme based on the data dependency relationship and life cycle characteristics of each operator; in the operation stage, task and data collaborative allocation is carried out according to a data prefetching strategy and a data replacement strategy, task and data collaborative allocation and the calculation task are executed in parallel, and a data flow diagram and an operator dependency relationship of the task are analyzed by utilizing a compiling period; in combination with the data locality characteristics of the MCM-GPU multi-level storage architecture and the concurrency requirements of different modules for data access, an appropriate GPU module is selected to carry out data optimization management, so that frequent data exchange between a GPU and a CPU is reduced, and the performance potential of the MCM-GPU architecture in a super-large-scale task is brought into full play.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a resource storage optimization method, device and equipment for MCM-GPU. BACKGROUND

[0002] With the semiconductor manufacturing process approaching the physical limit, the transistor density per unit area increases slowly, and the computing power expansion of traditional single-chip GPU encounters significant bottlenecks. Multi-chip module GPU (MCM-GPU) breaks through the above limitations through chip granulation design, integrates multiple smaller chiplets in a single processor package, each chiplet only contains part of the resources of a traditional single-chip system-on-chip (SoC), thereby effectively reducing chip manufacturing complexity and cost. Chiplets work together through high-bandwidth, low-latency interconnection channels within the package, and appear to programmers as a single logical GPU at the programming interface level, which can effectively meet the computing power needs of high-computing-density tasks such as machine learning. However, due to physical packaging space and power consumption constraints, the memory capacity of MCM-GPU grows much slower than the rapid growth of storage demand for large-scale computing tasks. In related technologies, a unified memory management mechanism is used to solve the problem of insufficient memory capacity. This mechanism allows the GPU to swap data pages to the CPU main memory when the memory is exhausted, and dynamically call back when needed. Although this method expands the available storage space, the data migration is triggered passively, usually when the memory is insufficient, causing the computing task to stall due to waiting for data loading, introducing additional delay; and the unified memory lacks awareness of the data access pattern of the computing task, which may frequently migrate unnecessary data, causing bandwidth waste. In addition, when the data is located in the CPU main memory, access needs to go through the PCIe or CXL bus, which has much lower latency and bandwidth than the interconnection within the package, further exacerbating the performance bottleneck. SUMMARY

[0003] The present application provides a resource storage optimization method, device and equipment for MCM-GPU, to solve the defects of passive data migration, lack of task awareness, unbalanced resource utilization, and computing and transmission serialization in traditional MCM-GPU storage technology, resulting in low memory access efficiency and performance bottlenecks.

[0004] The present application provides a resource storage optimization method for MCM-GPU, comprising: performing static analysis on the operator-level data flow graph of the computing task in the compilation phase, and extracting the data dependency relationship and life cycle characteristics of each operator; based on the data dependency relationship and life cycle characteristics of each operator, generating a storage optimization scheme, the storage optimization scheme including a data prefetching strategy and a data permutation strategy; In the running stage, the task and data are cooperatively allocated according to the data prefetching strategy and the data replacement strategy, and the cooperative allocation of the task and data is performed in parallel with the computing task.

[0005] According to the resource storage optimization method for MCM-GPU provided by the application, the data dependency relationship and the life cycle characteristics of each operator are extracted, and the method comprises the following steps: A directed acyclic graph of the computing task is constructed, wherein a node represents a computing operator, and an edge represents a data dependency relationship between operators. The usage range of the output tensor of each operator is analyzed, and the life cycle characteristics are determined, wherein the life cycle characteristics comprise the referenced condition in the subsequent computing stage, the time range from generation to the last reference operator, and the data block size and access frequency required by each computing stage.

[0006] According to the resource storage optimization method for MCM-GPU provided by the application, the data prefetching strategy is generated based on the data dependency relationship and the life cycle characteristics of each operator, and the method comprises the following steps: The operators executed in series are divided into consecutive computing stages according to the dependency relationship, and each stage loop is allocated to different GPU modules for execution. During the execution of the current computing stage, the data set required by the next stage is dynamically identified. According to the life cycle characteristics, the survival time window of each data block is calculated to determine the optimal prefetching opportunity. A target GPU module is selected from the idle GPU modules for prefetching the data required by the next stage. The current storage state of the data to be prefetched is judged: if the data is located in the CPU memory and is not cached by other GPU modules, data transmission from the CPU memory to the display memory of the target GPU module is started. When the display memory space of the target GPU module is insufficient, other GPU modules with available display memory are selected as alternative prefetching targets to ensure that the required data has been prefetched and stored in the display memory of the target GPU module before the next stage starts execution.

[0007] According to the resource storage optimization method for MCM-GPU provided by the application, the data replacement strategy is generated based on the data dependency relationship and the life cycle characteristics of each operator, and the method comprises the following steps: In the compiling stage, the reference condition of the output data of each computing task in the subsequent computing task is marked. In the compiling stage, the reference condition of the output data of each computing task in the subsequent computing task is marked. In the running phase, when it is detected that the current task is completed, if the intermediate data is no longer referenced in the subsequent task, the intermediate data is returned from the GPU memory to the CPU memory through the PCIe or CXL interface during the execution of the next task; The data position mapping information is updated in real time, and the latest storage positions of the data blocks are recorded.

[0008] According to the resource storage optimization method for MCM-GPU provided by the application, when it is detected that the current task is completed, if the intermediate data still needs to be used in the subsequent task, it is further judged whether the memory of the GPU module corresponding to the next task is available, and corresponding operations are performed according to the judgment result.

[0009] According to the resource storage optimization method for MCM-GPU provided by the application, the judgment of whether the memory of the GPU module corresponding to the next task is available and the performance of corresponding operations according to the judgment result include: When the memory of the target GPU module is sufficient, the data is migrated to the target GPU module; When the memory of the target GPU module is insufficient, the data is temporarily stored in the current GPU module.

[0010] According to the resource storage optimization method for MCM-GPU provided by the application, the collaborative allocation of tasks and data according to the data prefetching strategy and the data replacement strategy includes: A task-data block association mapping table is established to maintain the corresponding relationship and access characteristics of each computing task and required data block in real time; During the execution of the current computing task, the data prefetching operation of the next task and the data replacement operation of the completed task are triggered in parallel; According to the real-time memory usage rate and computing load state of each GPU module, the selection strategy of the data prefetching target and the replacement target is dynamically adjusted; When it is detected that the memory pressure of the target GPU module exceeds the dynamic threshold, the data storage path is automatically switched.

[0011] According to the resource storage optimization method for MCM-GPU provided by the application, the data storage path includes other GPU modules or CPU memory.

[0012] The application also provides a resource storage optimization device for MCM-GPU, which includes: The extraction module is used for static analysis of the operator-level data flow graph of the computing task in the compilation phase, and the data dependency relationship and life cycle characteristics of each operator in the operator-level data flow graph are extracted; The generating module is configured to generate a storage optimization scheme based on the data dependency relationship and the life cycle characteristics of the operators, the storage optimization scheme including a data prefetching strategy and a data permutation strategy. The allocating module is configured to perform a cooperative allocation of tasks and data according to the data prefetching strategy and the data permutation strategy in a running stage, the cooperative allocation of the tasks and data being performed in parallel with the computing task.

[0013] The present application also provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the resource storage optimization method for MCM-GPU according to any one of the above when executing the program.

[0014] The resource storage optimization method, device, and equipment for MCM-GPU provided by the present application extracts the data dependency relationship and the life cycle characteristics of each operator through static analysis of the operator-level data flow graph of the computing task in the compiling stage, generates a storage optimization scheme based on the data dependency relationship and the life cycle characteristics of the operators, the storage optimization scheme including a data prefetching strategy and a data permutation strategy, and performs a cooperative allocation of tasks and data according to the data prefetching strategy and the data permutation strategy in a running stage, the cooperative allocation of the tasks and data being performed in parallel with the computing task. The data flow graph and operator dependency relationship of the task are analyzed in the compiling stage, the data locality characteristics of the multi-level storage architecture of the MCM-GPU and the concurrency requirement of data access of different modules are combined, and a suitable GPU module is selected for the optimization management of data, so as to reduce the frequent data exchange between the GPU and the CPU and fully exert the performance potential of the MCM-GPU architecture in a super large-scale task. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0016] Figure 1 is a flowchart of the resource storage optimization method for MCM-GPU provided by the embodiments of the present application; Figure 2 is a multi-level storage architecture diagram of the MCM-GPU system provided by the embodiments of the present application; Figure 3 is an operator diagram of the computing task provided by the embodiments of the present application; Figure 4 is a data prefetching strategy diagram provided by the embodiments of the present application; Figure 5 is a data data permutation strategy schematic diagram provided by an embodiment of the present application; Figure 6 is a resource storage optimization device functional structure schematic diagram for MCM-GPU provided by an embodiment of the present application; Figure 7 is a functional structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0018] Figure 1 The flow chart of the resource storage optimization method for MCM-GPU provided by an embodiment of the present application is shown in Figure 1 The resource storage optimization method for MCM-GPU provided by an embodiment of the present application includes: Step 101, performing static analysis on the operator-level data flow graph of the computing task in the compilation stage, and extracting the data dependency relationship and the life cycle characteristics of each operator; Step 102, generating a storage optimization scheme based on the data dependency relationship and the life cycle characteristics of each operator, wherein the storage optimization scheme includes a data prefetch strategy and a data permutation strategy; Step 103, performing task and data collaborative allocation according to the data prefetch strategy and the data permutation strategy in the running stage, wherein the task and data collaborative allocation is performed in parallel with the computing task.

[0019] The multi-level storage architecture for MCM-GPU system is as shown in Figure 2The architecture is composed of multiple GPU chips and a CPU. Inside each GPU chip, multiple compute units (CUs) are integrated, and each CU is connected to a private L1 cache. The L1 cache is used to cache recently accessed tensor data. When a CU requests data, if the required data already exists in the L1 cache, the data will be directly read, reducing access delay. If the L1 cache is not hit, the system starts the address translation mechanism to obtain the physical address of the target data through the TLB in the GPU chip and the page table walker in the CPU. According to the positioning of the physical address, the access path is divided into three cases: if the target data is located in the memory partition of the local GPU module, the access request will first pass through the high-bandwidth local last-level cache (LLC), and if the LLC is not hit, the memory controller (MC) will access the low-bandwidth memory partition. If the data is located in the memory partition of other GPU modules, the request will access the LLC of the module through the high-bandwidth, low-latency inter-chip interconnection network, and if the LLC is not hit, the memory partition of the GPU module will be accessed. If the data is located in the CPU memory, the request needs to be transmitted to the CPU memory controller through the PCIe or CXL bus, and after the data access is completed, the data is returned to the GPU. Due to the limited memory capacity of the MCM-GPU system, large-scale data cannot be completely accommodated, and accessing the CPU main memory will introduce higher latency and bandwidth limitations, which will significantly affect the overall performance of the MCM-GPU. The unified memory management mechanism allows the GPU to swap data pages to the CPU main memory when the video memory is exhausted, and dynamically call back when needed. Although this method expands the available storage space, the data migration is triggered passively, usually when the video memory is insufficient, causing the computing task to stall due to waiting for data loading, introducing additional delay; and the unified memory lacks awareness of the data access pattern of the computing task, which may frequently migrate unnecessary data, causing bandwidth waste. In addition, when the data is located in the CPU main memory, the access needs to pass through the PCIe or CXL bus, which has much lower latency and bandwidth than the intra-package interconnection between GPUs, further exacerbating the performance bottleneck.

[0020] In the MCM-GPU architecture, each GPU module has a corresponding memory partition. Specifically, when the LLC of a certain GPU module is invalidated, the system determines the memory partition where the data is located according to the physical address. Compared with the traditional scheme of accessing data after LLC invalidation, the data is transferred in advance to the memory partition of the GPU module that needs the data or other GPU modules, reducing the waiting time for accessing the remote main memory.

[0021] The method for resource storage optimization for MCM-GPU provided by the embodiment of the present application extracts the data dependency relationship and the life cycle characteristics of each operator by performing static analysis on the operator-level data flow graph of the computing task in the compilation stage; generates a storage optimization scheme based on the data dependency relationship and the life cycle characteristics of each operator, wherein the storage optimization scheme includes a data prefetching strategy and a data permutation strategy; in the running stage, the task and data are collaboratively allocated according to the data prefetching strategy and the data permutation strategy, the collaborative allocation of the task and data is performed in parallel with the computing task, the data flow graph and the operator dependency relationship of the task are analyzed in the compilation stage, the data locality characteristics of the multi-level storage architecture of the MCM-GPU and the concurrency requirement of data access of different modules are combined, and the appropriate GPU module is selected to perform the optimization management of data, so as to reduce the frequent data exchange between the GPU and the CPU, and fully exert the performance potential of the MCM-GPU architecture in the super-large-scale task.

[0022] Based on any of the above embodiments, the data dependency relationship and the life cycle characteristics of each operator are extracted, including: Step 201, constructing a directed acyclic graph of a computing task, wherein a node represents a computing operator, and an edge represents the data dependency relationship between operators; Step 202, analyzing the usage range of the output tensor of each operator to determine the life cycle characteristics, wherein the life cycle characteristics include the referenced situation in the subsequent computing stage, the time range from generation to the last reference operator, and the data block size and access frequency required in each computing stage.

[0023] In the storage optimization method for MCM-GPU, the life cycle characteristics are the key quantitative description of the full cycle behavior of the data block in the computing task, mainly including three core dimensions: first, the reference situation of the operator in the subsequent computing stage is tracked by constructing a data dependency graph to track the frequency, order and dependency strength of each data block called by the subsequent operator, for example, in a deep learning model, the gradient tensor generated by the forward propagation will be referenced by the back propagation operator many times, and this reference mode directly affects the priority of the data that should be retained in the video memory. Second, the effective survival window of the data block is accurately defined from the data generation to the time range of the last reference operator, combined with the pipeline timing information of the computing task, the optimal residence time of the data in the video memory can be calculated, such as some intermediate results are only transmitted between adjacent two computing stages, and the survival period is much shorter than the model parameters that need to be globally shared. Finally, the data block size and access frequency required by each computing stage are predicted through static analysis and historical execution statistics, including the data size scale of each access (such as matrix block) and access intensity (such as the repeated reading of weight in convolution layer), these characteristics together constitute the basis for decision-making of storage resource allocation, to ensure that high-value data is preferentially resident in high-speed storage. The cooperative analysis of the three dimensions enables the system to predict the data heat at the compilation period, providing theoretical support for runtime dynamic optimization.

[0024] In the embodiment of the application, each task of the system at runtime fully utilizes other idle GPU modules to perform data migration operations, so that the computing stage and the data prefetching / data swap-out stage are overlapped in parallel, thereby hiding the delay caused by data transmission. In specific implementation, the operator graph of the entire computing task is analyzed in advance at the compilation period to obtain the dependency relationship between the operators (Ops), the life cycle of the data output by each operator, and the data block size required by each stage. Based on the analysis results, a specific data scheduling strategy is developed.

[0025] Based on any of the above embodiments, the data prefetching strategy is generated based on the data dependency relationship and life cycle characteristics of the operators, including: Step 301, dividing the serially executed operators into consecutive computing stages according to the dependency relationship, and assigning each stage loop to different GPU modules for execution; Step 302, during the execution of the current computing stage, dynamically identifying the data set required by the next stage; Step 303, calculating the survival time window of each data block according to the life cycle characteristics to determine the best prefetching opportunity; Step 304, selecting a target GPU module from the idle GPU modules for prefetching the data required by the next stage; Step 305, judge the current storage state of the data to be prefetched: if the data is located in the CPU memory and has not been cached by other GPU modules, start data transmission from the CPU memory to the target GPU module's video memory; Step 306, when the target GPU module's video memory space is insufficient, select other GPU modules with available video memory as alternative prefetch targets to ensure that the required data has been prefetched and stored in the target GPU module's video memory before the next stage begins execution.

[0026] As shown in Figure 3 In the task allocation stage, operators that are executed in series according to the dependency order are divided into consecutive tasks (denoted as T1, T2, T3…), and the main calculation of each stage is performed by a GPU module. Assuming that there are N GPU modules in the system, a round-robin or designated strategy can be used to select the module for execution in each stage. For example, for three operator tasks in the dependency chain Op1→Op2→Op3, they can be divided into stages T1, T2, T3, and the main calculation of each stage is performed by module 0, module 1, and module N, respectively. By letting different stages be executed by different modules, it is ensured that when a module is calculating, other modules still have the capacity for data management operations.

[0027] In order to implement the data prefetching mechanism, the system will load the data required by the next task from the CPU memory to the video memory of the target GPU module before each task begins. Specifically, as shown in Figure 4 When task Ti is executed by GPU module X, the system schedules another idle GPU module Y to prefetch the data required by stage T(i+1) during the calculation of Ti. If this data has not been accessed by other GPU modules at this time, it is loaded from the CPU memory to the video memory of module Y. If the video memory space of module Y is insufficient, and there are other GPU modules in the system that have available memory, the data is placed in other modules with sufficient video memory first. In this way, when module Y begins to execute the calculation task of task T(i+1), the required data is already prepared, thereby avoiding delays caused by waiting for data loading.

[0028] Based on any of the above embodiments, the data replacement strategy is generated based on the data dependency relationship and the life cycle characteristics of the operators, including: Step 401, in the compilation stage, mark the reference of the output data of each calculation task in the subsequent calculation tasks; Step 402, in the running stage, when it is detected that the current task is completed, perform the following processing on the intermediate data output by it: If the intermediate data is no longer referenced in the subsequent tasks, it is returned from the GPU video memory to the CPU memory through the PCIe or CXL interface during the execution of the next task; If the intermediate data still needs to be used in the subsequent task, further judgment is made on the available memory of the GPU module corresponding to the next task, and corresponding operations are performed according to the judgment result; Step 403, real-time update data position mapping information, record the latest storage position of each data block.

[0029] In the embodiment of the application, the judgment on the available memory of the GPU module corresponding to the next task and the execution of corresponding operations according to the judgment result include: When the target GPU module has sufficient memory, the data is migrated to the target GPU module; When the target GPU module has insufficient memory, the data is temporarily stored in the current GPU module.

[0030] In the embodiment of the application, as shown in Figure 5 For completed computing tasks, the system will perform data replacement operations according to the results of compile-time data flow analysis. If it is analyzed and judged that the data output by a task is not used in the subsequent task, the system will timely return these data from the GPU memory to the CPU memory through PCIe or CXL during the execution of the next task (such as T(i+1)), so as to release the memory resources and improve the resource utilization. For example, when the task Ti is completed, if the intermediate data generated by Ti is not referenced in the subsequent task, the system performs the return operation of the data to avoid occupying valuable memory. If the intermediate data of the Ti task still needs to be used in the subsequent task, the system further judges the available memory of the GPU module corresponding to the target use task (such as T(i+1)): if the memory of the module is sufficient, the data is directly migrated to the module to improve data locality; if the memory of the target module is insufficient, the data is temporarily left in the current GPU module to avoid additional data migration overhead, so as to realize efficient memory management on the premise of ensuring performance.

[0031] Based on any of the above embodiments, the cooperative allocation of tasks and data according to the data prefetching strategy and the data replacement strategy includes: Step 501, establish a task-data block association mapping table, and real-time maintain the corresponding relationship and access characteristics of each computing task and required data block; Step 502, during the execution of the current computing task, trigger the data prefetching operation of the next task and the data replacement operation of the completed task in parallel; Step 503, dynamically adjust the selection strategy of the data prefetching target and the replacement target according to the real-time memory usage rate and computing load state of each GPU module; Step 504, when it is detected that the memory pressure of the target GPU module exceeds the dynamic threshold, automatically switch the data storage path.

[0032] In the embodiment of the present application, the data storage path comprises other GPU modules or CPU memory.

[0033] In the embodiment of the present application, the data life cycle and data locality features are extracted by performing static analysis on the operator-level data flow graph and its dependency relationship of the task in the compilation phase, and the data resident GPU module suitable for the execution of each operator is determined in advance. In the runtime scheduling phase, when the required data of the next computing task is not located in the current GPU module, the system will select the appropriate GPU module for data management based on the analysis result in the compilation phase. If the current GPU memory resource is sufficient, the corresponding data is preferentially prefetched from the CPU to the module, otherwise the data is attempted to be loaded to other GPU modules with available video memory. If all GPU modules have no free memory, the CPU main memory initiates data page transmission to the current GPU module, and the location mapping of the data is dynamically maintained based on the page table information, similar to the traditional design.

[0034] The MCM-GPU-oriented resource storage optimization method provided by the embodiment of the present application can actively avoid high-delay data migration operations, effectively reduce the communication overhead and waiting delay caused by cross-module access or access to the CPU main memory, and improve the overall performance of the system.

[0035] The MCM-GPU-oriented resource storage optimization device provided by the present application will be described below. The MCM-GPU-oriented resource storage optimization device described below can be referred to the MCM-GPU-oriented resource storage optimization method described above.

[0036] Figure 6 The functional structure diagram of the MCM-GPU-oriented resource storage optimization device provided by the embodiment of the present application is shown in FIG. 1, the MCM-GPU-oriented resource storage optimization device provided by the embodiment of the present application comprises: Figure 6 The MCM-GPU-oriented resource storage optimization device provided by the embodiment of the present application comprises: The extraction module 601 is configured to perform static analysis on the operator-level data flow graph of the computing task in the compilation phase, and extract the data dependency relationship and life cycle features of each operator in the operator-level data flow graph. The generation module 602 is configured to generate a storage optimization scheme based on the data dependency relationship and life cycle features of each operator, wherein the storage optimization scheme comprises a data prefetching strategy and a data replacement strategy. The distribution module 603 is configured to, in the running stage, perform cooperative distribution of tasks and data according to the data prefetching strategy and the data permutation strategy, and the cooperative distribution of the tasks and the data is performed in parallel with the computing task.

[0037] The resource storage optimization device for MCM-GPU provided by the embodiment of the application extracts data dependency relationships and life cycle characteristics of each operator by performing static analysis on the operator-level data flow graph of the computing task in the compiling stage; generates a storage optimization scheme based on the data dependency relationships and the life cycle characteristics of each operator, the storage optimization scheme including a data prefetching strategy and a data permutation strategy; and in the running stage, performs cooperative distribution of tasks and data according to the data prefetching strategy and the data permutation strategy, and the cooperative distribution of the tasks and the data is performed in parallel with the computing task.

[0038] Figure 7 An example of a schematic diagram of a physical structure of an electronic device is shown in Figure 7 The electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 can communicate with each other through the communications bus 740. The memory 730 includes a computer program, an operating system, and obtained data, and the processor 710 can call logical instructions in the memory 730 to execute the resource storage optimization method for MCM-GPU, which includes: performing static analysis on an operator-level data flow graph of a computing task to extract data dependency relationships and life cycle characteristics of each operator in the compiling stage; generating a storage optimization scheme based on the data dependency relationships and the life cycle characteristics of each operator, the storage optimization scheme including a data prefetching strategy and a data permutation strategy; and in the running stage, performing cooperative distribution of tasks and data according to the data prefetching strategy and the data permutation strategy, and the cooperative distribution of the tasks and the data is performed in parallel with the computing task.

[0039] In addition, the logic instructions in the memory 730 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the related art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0040] The device embodiments described above are only schematic, wherein the units illustrated as separate components can or can not be physically separate, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0041] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software and the necessary general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the technical solutions described above essentially or the parts that contribute to the related art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0042] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A resource storage optimization method for MCM-GPU, characterized in that, include: During the compilation phase, static analysis is performed on the operator-level data flow graph of the computation task to extract the data dependencies and lifecycle characteristics of each operator; Based on the data dependencies and lifecycle characteristics of each operator, a storage optimization scheme is generated, which includes a data prefetching strategy and a data replacement strategy. During the runtime phase, tasks and data are collaboratively allocated according to the data prefetching strategy and data replacement strategy, and the collaborative allocation of tasks and data is executed in parallel with the computing tasks.

2. The resource storage optimization method for MCM-GPU according to claim 1, characterized in that, The extraction of data dependencies and lifecycle features for each operator includes: Construct a directed acyclic graph of computation tasks, where nodes represent computation operators and edges represent data dependencies between operators; The usage range of each operator's output tensor is analyzed to determine its lifecycle characteristics. These lifecycle characteristics include the operator's reference status in subsequent computation stages, the time range from its generation to the last referenced operator, and the data block size and access frequency required for each computation stage.

3. The resource storage optimization method for MCM-GPU according to claim 1, characterized in that, The data prefetching strategy generated based on the data dependencies and lifecycle characteristics of each operator includes: The operators that are executed serially are divided into consecutive computation stages according to their dependencies, and each stage is cyclically assigned to different GPU modules for execution. During the execution of the current computation phase, dynamically identify the dataset required for the next phase; The lifecycle characteristics are used to calculate the survival time window of each data block and determine the optimal prefetching time. Select the target GPU module from the idle GPU modules to prefetch the data needed for the next stage; Determine the current storage status of the data to be prefetched: If the data is located in CPU memory and is not cached by other GPU modules, then initiate the data transfer from CPU memory to the target GPU module's video memory; When the target GPU module's video memory space is insufficient, another GPU module with available video memory is selected as an alternative prefetch target to ensure that the required data has been prefetched and stored in the target GPU module's video memory before the next stage begins execution.

4. The resource storage optimization method for MCM-GPU according to claim 1, characterized in that, The data replacement strategy generated based on the data dependencies and lifecycle characteristics of each operator includes: During the compilation phase, the references of the output data of each computation task in subsequent computation tasks are marked; During the runtime phase, if the intermediate data is no longer referenced in subsequent tasks after the current task is detected to have been completed, it will be transferred back from the GPU memory to the CPU memory via the PCIe or CXL interface during the execution of the next task. The data location mapping information is updated in real time, and the latest storage location of each data block is recorded.

5. The resource storage optimization method for MCM-GPU according to claim 4, characterized in that, Once the current task is detected as completed, if the intermediate data is still needed in subsequent tasks, the availability of video memory in the GPU module corresponding to the next task is further determined, and corresponding operations are performed based on the determination result.

6. The resource storage optimization method for MCM-GPU according to claim 5, characterized in that, The step of determining the availability of video memory in the GPU module corresponding to the next task and performing corresponding operations based on the determination result includes: When the target GPU module has sufficient video memory, the data is migrated to the target GPU module; When the target GPU module is short of video memory, the data is temporarily stored in the current GPU module.

7. The resource storage optimization method for MCM-GPU according to claim 1, characterized in that, The collaborative allocation of tasks and data based on the data prefetching strategy and data replacement strategy includes: Establish a task-data block association mapping table to maintain the correspondence and access characteristics between each computing task and the required data blocks in real time; During the execution of the current computing task, data prefetching operations for the next task and data replacement operations for completed tasks are triggered in parallel. The selection strategy for data prefetching targets and permutation targets is dynamically adjusted based on the real-time memory usage and computing load status of each GPU module. When the target GPU module's memory pressure exceeds the dynamic threshold, the data storage path is automatically switched.

8. The resource storage optimization method for MCM-GPU according to claim 7, characterized in that, The data storage path includes: other GPU modules or CPU memory.

9. A resource storage optimization device for MCM-GPU, characterized in that, include: The extraction module is used to perform static analysis on the operator-level data flow graph of the computation task during the compilation phase, and extract the data dependencies and life cycle characteristics of each operator in the operator-level data flow graph; The generation module is used to generate a storage optimization scheme based on the data dependency relationship and life cycle characteristics of each operator. The storage optimization scheme includes a data prefetching strategy and a data replacement strategy. The allocation module is used to perform collaborative allocation of tasks and data according to the data prefetching strategy and data replacement strategy during the runtime phase. The collaborative allocation of tasks and data is executed in parallel with the computing tasks.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the resource storage optimization method for MCM-GPU as described in any one of claims 1 to 8.

Images