Method for managing unified virtual memory on basis of page faults in heterogeneous memory system, electronic device for executing same, and recording medium

Abstract

Disclosed are a method for managing a unified virtual memory on the basis of page faults in a heterogeneous memory system, and an electronic device for executing same. The method for managing a unified virtual memory may comprise the steps of: when a page fault occurs in a GPU, receiving, from the GPU, page fault information including a virtual address of a page at which the page fault has occurred; analyzing the received page fault information to generate a first virtual address block on the basis of the virtual address of the page at which the page fault has occurred; generating second virtual address blocks corresponding in number to a predefined number of virtual address blocks to be prefetched, on the basis of the first virtual address block; and migrating the first virtual address block and the second virtual address blocks to a memory of the GPU.

Description

A page fault-based integrated virtual memory management method in a heterogeneous memory system, an electronic device executing the same, and a recording medium

[0001] The following disclosure relates to an integrated virtual memory management method, and more specifically, to an integrated virtual memory management method in a heterogeneous memory system composed of a central processing unit (CPU) and a graphic processing unit (GPU).

[0002] GPUs are hardware with high parallelism that plays a key role in the training and inference processes of artificial neural network-based models, particularly deep learning.

[0003] In the recent deep learning paradigm, there is a tendency for the accuracy of a model to improve as the model size (number of parameters) and the size of the dataset increase, and accordingly, the model size and the size of the dataset are on the rise.

[0004] However, these large-scale models experience a rapid increase in computational load and memory usage during the training and inference processes, leading to problems such as exponentially increasing training time and decreased throughput during the inference process.

[0005] Accordingly, unified virtual memory (UVM) is being actively utilized in computations to overcome the memory capacity limitations of GPUs and to simplify data transfer between the CPU's host memory and the GPU's device memory.

[0006] Unified virtual memory is a technology that enables data access within a single virtual address system by sharing memory space between the CPU and GPU. However, existing unified virtual memory environments still face issues such as handling page faults originating from the GPU, inefficient data prefetching, and memory space shortages, necessitating improvements.

[0007] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0008] The present disclosure can provide a method and apparatus for effectively handling page faults occurring in the GPU in a heterogeneous memory system composed of a CPU and a GPU.

[0009] The present disclosure may provide a method and apparatus for efficiently prefetching data by analyzing the memory access patterns of a GPU.

[0010] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist.

[0011] An integrated virtual memory management method performed by a CPU in a heterogeneous memory system composed of a central processing unit (CPU) and a graphic processing unit (GPU) according to one embodiment may include: receiving page fault information including a virtual address for the page where the page fault occurred from the GPU when a page fault occurs in the GPU; analyzing the received page fault information to create a first virtual address block based on the virtual address of the page where the page fault occurred; creating a second virtual address block equal to a number of predefined virtual address blocks to be prefetched based on the first virtual address block; and migrating the first virtual address block and the second virtual address block to the memory of the GPU.

[0012] The step of generating the second virtual address block can generate the second virtual address block by applying a predefined virtual address block offset based on the virtual address of the first virtual address block.

[0013] The step of generating the second virtual address block above can generate the second virtual address block consecutive to the first virtual address block using a first offset interval determined according to the linear memory access pattern when the memory access pattern of the GPU is linear.

[0014] The first offset interval above may have a fixed value.

[0015] The step of generating the second virtual address block may generate the second virtual address block consecutive to the first virtual address block using a second offset interval determined according to the non-linear memory access pattern when the memory access pattern of the GPU is non-linear.

[0016] The above second offset interval may have a varied value.

[0017] The method may further include a step of verifying whether the virtual address of the first virtual address block and the virtual address of the second virtual address block exist within the virtual address range of the integrated virtual memory.

[0018] The steps of generating the first virtual address block and generating the second virtual address block may prevent the first virtual address block and the second virtual address block from being generated if the virtual address of the first virtual address block and the virtual address of the second virtual address block do not exist within the virtual address range of the integrated virtual memory.

[0019] In the step of migrating to the memory of the GPU, if there is insufficient available space in the memory of the GPU, some pages in the memory of the GPU may be moved to the memory of the CPU, and then the first virtual address block and the second virtual address block may be moved to the memory of the GPU.

[0020] According to one embodiment, instructions stored in a computer-readable recording medium can perform operations of the method of operation of the electronic device when executed by one or more processors.

[0021] An electronic device according to one embodiment includes a central processing unit (CPU); at least one graphic processing unit (GPU); and a memory comprising one or more storage media for storing instructions, wherein when the instructions are executed by the CPU, the electronic device may receive page fault information including a virtual address for the page where the page fault occurred from the GPU when a page fault occurs in the GPU, analyze the received page fault information to create a first virtual address block based on the virtual address of the page where the page fault occurred, create a second virtual address block equal to the number of virtual address blocks to be pre-fetched based on the first virtual address block, and migrate the first virtual address block and the second virtual address block to the memory of the GPU.

[0022] When the above instructions are executed by the CPU, the electronic device may generate the second virtual address block by applying a predefined virtual address block offset based on the virtual address of the first virtual address block.

[0023] When the above instructions are executed by the CPU, the electronic device may generate the second virtual address block consecutive to the first virtual address block using a first offset interval determined according to the linear memory access pattern, when the memory access pattern of the GPU is linear.

[0024] The first offset interval above may have a fixed value.

[0025] When the above instructions are executed by the CPU, the electronic device may generate the second virtual address block consecutive to the first virtual address block using a second offset interval determined according to the non-linear memory access pattern when the memory access pattern of the GPU is non-linear.

[0026] The above second offset interval may have a varied value.

[0027] When the above instructions are executed by the CPU, the electronic device may verify whether the virtual address of the first virtual address block and the virtual address of the second virtual address block exist within the virtual address range of the integrated virtual memory.

[0028] When the above instructions are executed by the CPU, the electronic device may not create the first virtual address block and the second virtual address block if the virtual address of the first virtual address block and the virtual address of the second virtual address block do not exist within the virtual address range of the integrated virtual memory.

[0029] When the above commands are executed by the CPU, the electronic device may move some pages in the GPU's memory to the CPU's memory and then move the first virtual address block and the second virtual address block to the GPU's memory when there is insufficient available space in the GPU's memory.

[0030] According to one embodiment of the present disclosure, in a heterogeneous memory system composed of a CPU and a GPU, performance degradation of the GPU can be prevented by effectively handling page faults occurring in the GPU.

[0031] According to one embodiment of the present disclosure, by performing prefetching based on the memory access pattern of the GPU, the memory latency of the GPU can be reduced and data loading delays can be prevented.

[0032] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0033] FIG. 1 is a diagram illustrating an electronic device for executing a page fault-based integrated virtual memory management method in a heterogeneous memory system according to one embodiment.

[0034] FIG. 2 is a schematic diagram showing a unified virtual memory (UVM) according to one embodiment.

[0035] FIG. 3 is a diagram illustrating the process of handling page faults occurring in the GPU when using integrated virtual memory according to one embodiment.

[0036] FIG. 4 is a diagram illustrating the memory hierarchy of an integrated virtual memory according to one embodiment.

[0037] FIG. 5 is a diagram illustrating a tree-based neighborhood (TBN) prefetching technology according to one embodiment.

[0038] FIG. 6 is a diagram illustrating an extended prefetching technique according to one embodiment.

[0039] FIG. 7 is a diagram illustrating a detailed algorithm of an extended prefetching technique according to one embodiment.

[0040] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0041] In this document, each of the following phrases may include any one of the items listed together in the corresponding phrase, or any combination of A, B, and C, or any combination of all of them. Terms such as "A or B," "at least one of A and B," "at least one of A, B, and C," "at least one of A, B, or C," and "a combination of one or more of A, B, and C" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may also be named the first component.

[0042] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0043] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0044] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0045]

[0046] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0047]

[0048] FIG. 1 is a diagram illustrating an electronic device for executing a page fault-based integrated virtual memory management method in a heterogeneous memory system according to one embodiment.

[0049] Referring to FIG. 1, the electronic device (100) may include one or more processors (110) and a memory (120) that loads or stores a computer program (130) executed by the processors (110). The processors (110) and the memory (120) may be connected to each other via a communication link (e.g., a bus) (140). Optionally, the electronic device (100) may further include a transceiver (150), and the transceiver (150) may be used for data exchange, such as the transmission and / or reception of data between the electronic device (100) and another electronic device. The components included in the electronic device (100) of FIG. 1 are merely examples, and a person skilled in the art to which this disclosure pertains will understand that other general-purpose components may be included in addition to the components shown in FIG. 1.

[0050] The processor (110) can control the overall operation of each component of the electronic device (100). The processor (110) may be implemented as a circuitry (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (110) may include one or more processors. For example, the processor (110) may include a combination of one or more processors such as a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphic processing unit (GPU), a neural processing unit (NPU), a digital signal processor (DSP), an application processor (AP), a communication processor (CP), or any form of processor well known in the art of the present disclosure. Additionally, the processor (110) can perform operations on at least one application or computer program (130) for executing a method / operation according to various embodiments of the present disclosure.

[0051] The memory (120) can store one or more combinations of various data, instructions, and information used by a component (e.g., processor (110)) included in the electronic device (100). The memory (120) may include volatile memory and / or nonvolatile memory.

[0052] A computer program (130) may include one or more actions in which methods / actions according to various embodiments of the present disclosure are implemented, and may be stored in memory (120) in the form of software. Here, the actions may correspond to instructions implemented in the program (130). For example, the program (130) may include instructions such as: when a page fault occurs in the GPU, the CPU receiving page fault information including a virtual address for the page where the page fault occurred from the GPU; the CPU analyzing the received page fault information and creating a first virtual address block based on the virtual address of the page where the page fault occurred; the CPU creating a second virtual address block based on the first virtual address block and a number of predefined virtual address blocks to be prefetched; and the CPU performing an action of migrating the first virtual address block and the second virtual address block to the memory of the GPU.

[0053] When a computer program (130) is loaded into memory (120), the processor (110) can perform methods / operations according to various embodiments of the present disclosure by executing a plurality of operations to implement the program (130).

[0054] A communication link (140) may include a path for transmitting at least one of various data, commands, and information between components included in an electronic device (100). The communication link (140) may be, for example, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, but the types of such buses are merely examples and are not limited to the above examples. For example, in FIG. 1, the bus is represented as a single line for convenience of explanation, but in reality, multiple buses or various types of buses may be included.

[0055]

[0056] FIG. 2 is a schematic diagram showing a unified virtual memory (UVM) according to one embodiment.

[0057] In existing Copy-Then-Execute (CTE) programming models for GPUs, users must explicitly allocate memory and manually perform data transfer between the CPU and GPU. While this approach is functional, it presents problems such as increased programming complexity, inefficient memory utilization, and a heavier burden on the user due to manual data movement.

[0058] To solve this, the present disclosure may utilize an integrated virtual memory (200) that provides a single virtual address space shared by the CPU and the GPU as shown in FIG. 2. The integrated virtual memory (200) serves to map CPU memory and GPU memory into a single virtual address space. Here, CPU memory may include an area allocated to CPU physical memory, and GPU memory may include an area allocated to GPU physical memory. As a result, the user does not need to directly manage data transfer from CPU memory to GPU memory, and the manual page migration and page eviction processes are eliminated, which can significantly reduce the programming burden on the user.

[0059] The integrated virtual memory (200) may apply a demand paging technique to support automatic data migration between the CPU and the GPU. More specifically, when the GPU requests a page that does not exist in its memory, the GPU memory management unit (GMMU) may generate a page fault. When such a page fault occurs, the page fault information may be transmitted to the CPU.

[0060] The CPU can analyze page fault information received from the GPU to identify the pages required by the GPU and migrate the identified pages to GPU memory to support the GPU in performing calculations normally.

[0061] Alternatively, if GPU memory is insufficient, the CPU can analyze the relevant GPU memory space, evict some pages from GPU memory, and move them to CPU memory. This allows the CPU to mitigate GPU memory overflow issues and support the retention of only the data necessary for computation.

[0062]

[0063] FIG. 3 is a diagram illustrating the process of handling page faults occurring in the GPU when using integrated virtual memory according to one embodiment.

[0064] Referring to Fig. 3, when an active thread running on the GPU generates a global memory request (①), the pages included in the global memory request can be searched in the GPU's micro translation lookaside buffer (μTLB).

[0065] If the page included in the corresponding global memory request is not found in the μTLB, a μTLB miss (②) occurs. When such a μTLB miss occurs, the GPU memory management unit can search for the requested page in the page table. If the page is not found in the page table either, it means that the page does not currently exist in GPU memory, so the GPU memory management unit can generate a page fault interrupt.

[0066] As such, page faults occurring in the GPU are handled by an interrupt service routine (ISR), which can be processed in two stages: Top-Half and Bottom-Half. First, the Top-Half ISR responds to the page fault interrupt and checks if there are any actual pending page faults. Subsequently, the Top-Half ISR can determine whether additional page fault processing is required through the Bottom-Half ISR.

[0067] If it is determined that a Bottom-Half ISR is necessary, the Top-Half ISR can disable additional page fault interrupts to prevent sporadic interrupts within the GPU. All page faults are written to the GPU fault buffer, and no new page fault interrupts occur until the GPU fault buffer is managed (③).

[0068] When new page fault interrupts are disabled in this manner, the Top-Half ISR can schedule the Bottom-Half ISR to handle the remaining page faults. Since page fault processing is very costly in terms of latency, the CPU's UVM driver can handle multiple page faults simultaneously by utilizing batch processing instead of processing individual page faults one by one.

[0069] Therefore, page faults stored in the GPU fault buffer can be transferred to the CPU's fault cache (④) for processing in groups called fault batches. This approach has the effect of mitigating the multiple round-trip latency overhead that occurs in PCIe (peripheral component interconnect express) and preventing unnecessary multiple ISR calls in the operating system.

[0070] At this time, the UVM driver of the CPU can process page fault groups in batches until specific conditions are met. For example, the UVM driver can retrieve and process page fault groups until at least one of the following conditions is met: when the number of page faults retrieved from the GPU reaches a preset batch size (e.g., 256 faults), when a preset time limit (e.g., 30ns) is exceeded, or when all page faults in the GPU fault buffer have been retrieved.

[0071] Page faults stored in the fault cache can be widely distributed within the corresponding virtual address space. The UVM driver can group these page faults into logical 2MB blocks (e.g., VABlock, ⑤) to process them in block units rather than individual 4KB pages.

[0072] If page faults are processed in 4KB page units, data movement between the CPU and GPU occurs frequently, which can increase the overall page fault processing latency. Therefore, processing all page faults at once in logical 2MB blocks can reduce data migration overhead.

[0073] When page faults are grouped into logical 2MB blocks, the corresponding pages are unmapped from the CPU page table and migrated to GPU memory to be mapped to the GPU page table (⑥). When the pages are migrated, the UVM driver may request the GMMU to update the page table and issue a page fault replay (⑦).

[0074] Page fault replay eliminates μTLB's wait state and allows prior misses to be re-executed. By default, the UVM driver runs page fault replay once per batch, during which all page faults within the batch can be reprocessed simultaneously.

[0075] When a page fault replay is delivered to the μTLB where the page fault occurred, the previous memory request resumes and the newly migrated pages can be successfully found (⑧).

[0076]

[0077] FIG. 4 is a diagram illustrating the memory hierarchy of an integrated virtual memory according to one embodiment.

[0078] Integrated virtual memory offers the advantage of overcoming GPU memory capacity limitations and improving programming efficiency. However, integrated virtual memory has the potential to cause significant latency when managing data between the CPU and GPU.

[0079] More specifically, GPUs operate using a Single Instruction Multiple Thread (SIMT) execution model. In this model, groups of threads called warps can execute in a locking phase. Therefore, if a page fault occurs in some threads within a warp, execution may be halted for other threads within the same warp until that page fault is fully resolved. Processing such page faults can take up to thousands of clock cycles, which can cause a severe bottleneck in GPU performance.

[0080] In addition, if memory overload occurs, overhead becomes more severe, which can lead to frequent page migrations and page evictions. To mitigate this overhead, the CPU's UVM driver can predict pages likely to be accessed by the GPU in advance and preemptively migrate them to GPU memory.

[0081] Referring to Fig. 4, the integrated virtual memory can manage memory using four conceptual layers: a virtual address space, a virtual address range, a virtual address block (VABlock), and pages.

[0082] A virtual address space is a virtual memory space owned by an individual processor. In a heterogeneous memory system consisting of one CPU and two GPUs, an individual virtual address space is allocated to each processor, so a total of three virtual address spaces can be created.

[0083] Such a virtual address space can be composed of multiple virtual address ranges, and each virtual address range can correspond to a continuous boundary of memory allocation.

[0084] A virtual address block can represent the result of a virtual address range being logically partitioned by the UVM driver. For example, the UVM driver can partition a virtual address range into a logical 2MB VABlock consisting of 512 4KB pages.

[0085] A page represents the smallest unit of memory allocation and can generally have a size of 4KB. These pages can be contained in virtual address blocks.

[0086] Through the hierarchical memory structure of such integrated virtual memory, the UVM driver can perform memory management between the CPU and GPU more efficiently, and in particular, minimize performance degradation caused by page faults by utilizing predictive data migration techniques.

[0087]

[0088] FIG. 5 is a diagram illustrating a tree-based neighborhood (TBN) prefetching technology according to one embodiment.

[0089] The CPU's UVM driver can be designed to migrate multiple consecutive pages within a virtual address block (VABlock) containing the page where the page fault occurred to GPU memory. To this end, each page is grouped into a logical 2MB VABlock, and the VABlock is further divided into virtual address regions of 64KB, so that a binary tree structure based on the 2MB VABlock can be created.

[0090] Referring to Fig. 5, the root node of the binary tree (e.g., ) represents the entire 2MB VABlock, and the CPU's UVM driver can manage memory space in such 2MB VABlock units. The leaf nodes of the binary tree (e.g., ) can represent a 64KB memory region consisting of 16 4KB pages each. Since each leaf node covers a 64KB area of ​​2MB VABlock, the UVM driver can prefetch pages in 64KB units.

[0091] For example, leaf nodes If a page fault occurs on the first page (①) in, the UVM driver can identify a 64KB area containing the page where the page fault occurred. At this time, if even one of the 16 pages in the 4KB area is migrated to GPU memory, the remaining pages can also be prefetched. Therefore, the leaf node The remaining 60KB of data in can be prefetched along with the page where the page fault occurred. Subsequently, the second page (②) and the third page (③) where the page fault occurred can also be prefetched according to the same process as the first page (①).

[0092] At this time, if the total size of GPU-resident pages at each level excluding the leaf level of the binary tree exceeds a predefined threshold (e.g., 51%) of the maximum memory capacity of that level, the UVM driver may prefetch the remaining pages into GPU memory as well.

[0093] For example, leaf nodes , and If the data is prefetched into GPU memory, the UVM driver is the parent node of the corresponding leaf nodes It can detect that more pages reside in GPU memory than a predefined threshold for it. In this case, the UVM driver The remaining leaf nodes with as their parent node Data can also be prefetched into GPU memory.

[0094] Subsequently, when data in the corresponding 64KB area is prefetched in response to the fourth page (④) where a page fault occurred, the UVM driver The parent node of Detecting that more pages reside in GPU memory than a predefined threshold for, with as the parent node Prefetching to GPU memory can be performed for the remaining child nodes.

[0095] Afterwards, the same process as for the fourth page (④) is repeated for the fifth page (⑤) where a page fault occurred, and the node The entire area covered by it can be prefetched into GPU memory.

[0096] Finally, if prefetching is performed on the fifth page (⑤) where the last page fault occurred, the root node Since it is detected that more pages than the predefined threshold for reside in GPU memory, the UVM driver [determines] node The entire area covered by it can be prefetched into GPU memory.

[0097]

[0098] FIG. 6 is a diagram illustrating an extended prefetching technique according to one embodiment.

[0099] Existing prefetching techniques are designed to prefetch remaining pages into GPU memory when the proportion of pages residing in GPU memory within a single virtual address block (VABB) exceeds a predefined threshold. However, since this method limits the scope of prefetching to within a single virtual address block, it may fail to fully utilize the locality of memory access that exists between multiple virtual address blocks.

[0100] To address this, the present disclosure provides an extended prefetching technique that prefetches multiple virtual address blocks at once by considering not only the locality within a virtual address block but also the extended locality existing between multiple virtual address blocks. By applying such an extended prefetching technique, the range of pages prefetched at once can be expanded to maximize the effectiveness of prefetching, and consequently, an improvement in GPU performance can be expected.

[0101] Referring to FIG. 6, the extended prefetch logic provided by the present invention can be started from the moment a page fault occurs when an application running on the GPU requests data that does not exist in GPU memory.

[0102] The CPU can search for the page where the page fault occurred in CPU memory to resolve the page fault. The CPU creates a prefetch entry based on the virtual address of the found page and can insert the first prefetch entry based on the virtual address of the page where the page fault occurred.

[0103] Subsequently, the CPU can generate additional prefetch entries. In this case, the number of additional prefetch entries generated may be equal to the number of pre-configured virtual address blocks.

[0104] According to one embodiment, the CPU can generate a second prefetch entry by applying a pre-set fixed offset to the first prefetch entry. Subsequently, the CPU can generate n prefetch entries by repeatedly applying the same offset as many times as a pre-set number of virtual address blocks (e.g., n).

[0105] According to one embodiment, the CPU analyzes the GPU's past memory access pattern, and if the analysis result indicates that the GPU's past memory access pattern is linear, it may generate a second prefetch entry having consecutive virtual addresses in the first prefetch entry. Similarly, the CPU may generate prefetch entries having consecutive virtual addresses equal to a pre-set number of virtual address blocks (e.g., n). In this case, the virtual address interval of the generated prefetch entries may maintain a constant offset interval. However, the offset interval is determined according to the GPU's past memory access pattern and may differ from the fixed offset mentioned above.

[0106] According to one embodiment, the CPU analyzes the GPU's past memory access pattern, and if the analysis result indicates that the GPU's past memory access pattern is non-linear, it may generate a second prefetch entry having discontinuous virtual addresses in the first prefetch entry. Similarly, the CPU may generate prefetch entries having discontinuous virtual addresses equal to a pre-set number of virtual address blocks (e.g., n). In this case, the virtual address interval of the generated prefetch entries may not be fixed and may be dynamically determined according to the GPU's actual memory access pattern.

[0107] The CPU can insert (①, insert) a prefetch entry containing the virtual addresses of pages retrieved based on the GPU's past memory access patterns into all prefetch entries generated in this way.

[0108] The CPU can create virtual address blocks based on the virtual addresses of the prefetch entries inserted in this way. Integrated virtual memory consists of virtual address blocks, and these virtual address blocks exist logically within virtual memory until they are actually created when actual memory access occurs.

[0109] Therefore, the virtual addresses stored in the prefetch entry enable the integrated virtual memory to recognize logical virtual address blocks, and in the process, multiple virtual address blocks can be created together (②, create).

[0110] At this time, all generated virtual address blocks can be moved to GPU memory at once (③,migrate). As a result, not only the virtual address block where a page fault occurred, but also multiple adjacent virtual address blocks can be prefetched into GPU memory.

[0111] For example, when running an application that exhibits a tendency for memory access locality to extend beyond a single virtual address block to locality between multiple virtual address blocks as shown in Fig. 6, additional page faults can be prevented by applying an extended prefetching technique, and as a result, GPU performance can be improved.

[0112]

[0113] FIG. 7 is a diagram illustrating a detailed algorithm of an extended prefetching technique according to one embodiment.

[0114] The extended prefetching technique proposed in this disclosure can minimize the delay in processing page faults occurring in the GPU and prevent additional page faults during future memory access.

[0115] First, in step (702), the CPU can receive page fault information that occurred in the GPU. At this time, the page fault information may include information about the virtual address of the page where the page fault occurred.

[0116] In step (704), the virtual address of the page where the page fault occurred can be detected by analyzing the received page fault information. At this time, the CPU can generate a subsequent prefetch entry based on the detected virtual address.

[0117] In step (706), the CPU can generate additional prefetch entries by adding a virtual address block offset based on the detected virtual address. At this time, the CPU can generate additional prefetch entries equal to the number of virtual address blocks pre-set as in step (708). Subsequently, the CPU can analyze the GPU's past memory access patterns, calculate the virtual address of an appropriate page for each prefetch entry, and search for and insert the prefetch entry having the corresponding virtual address.

[0118] In step (710), the CPU can determine whether the virtual addresses of each prefetch entry exist within a virtual address range.

[0119] If it exists within the virtual address range, the CPU can create a virtual address block using the corresponding prefetch entry in step (712).

[0120] In contrast, if the virtual address range is exceeded, the CPU may discard the corresponding prefetch entry in step (714) and not perform prefetching.

[0121] In step (716), the CPU can determine whether there is sufficient free space in the GPU memory. If it is determined that there is sufficient free space in the GPU memory, in step (718), a virtual address block can be prefetched into the GPU memory.

[0122] In contrast, if it is determined in step (716) that there is insufficient free space in the GPU memory, in step (720), the CPU may perform prefetching after securing free space in the GPU memory by page eviction of some data existing in the GPU memory. At this time, the evicted pages may be moved to CPU memory.

[0123]

[0124] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0125] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be stored on any type of machine, component, physical device, virtual equipment, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0126] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0127] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0128] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0129] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. In an integrated virtual memory management method performed by a CPU in a heterogeneous memory system composed of a central processing unit (CPU) and a graphic processing unit (GPU), When a page fault occurs in the above GPU, a step of receiving page fault information from the GPU including a virtual address for the page where the page fault occurred; A step of analyzing the received page fault information and generating a first virtual address block based on the virtual address of the page where the page fault occurred; A step of generating a second virtual address block equal to the number of virtual address blocks to be prefetched based on the first virtual address block; and A step of migrating the first virtual address block and the second virtual address block to the memory of the GPU. An integrated virtual memory management method including 2. In Paragraph 1, The step of generating the second virtual address block above is, An integrated virtual memory management method for generating a second virtual address block by applying a predefined virtual address block offset based on the virtual address of the first virtual address block.

3. In Paragraph 1, The step of generating the second virtual address block above is, An integrated virtual memory management method that, when the memory access pattern of the above GPU is linear, generates the second virtual address block consecutive to the first virtual address block using a first offset interval determined according to the linear memory access pattern.

4. In Paragraph 3, The first offset interval above is, Integrated virtual memory management method having a fixed value.

5. In Paragraph 1, The step of generating the second virtual address block above is, An integrated virtual memory management method that, when the memory access pattern of the above GPU is non-linear, generates the second virtual address block consecutive to the first virtual address block using a second offset interval determined according to the non-linear memory access pattern.

6. In Paragraph 5, The above second offset interval is, Integrated virtual memory management method having a variable value.

7. In the first, A step of verifying whether the virtual address of the first virtual address block and the virtual address of the second virtual address block exist within the virtual address range of the integrated virtual memory. An integrated virtual memory management method that further includes 8. In Paragraph 7, The steps of generating the first virtual address block and generating the second virtual address block are: An integrated virtual memory management method that prevents the creation of the first virtual address block and the second virtual address block when the virtual address of the first virtual address block and the virtual address of the second virtual address block do not exist within the virtual address range of the integrated virtual memory.

9. In Paragraph 1, The step of migrating to the above GPU memory is, An integrated virtual memory management method that, when available space in the memory of the GPU is insufficient, moves some pages in the memory of the GPU to the memory of the CPU, and then moves the first virtual address block and the second virtual address block to the memory of the GPU.

10. A computer-readable recording medium storing a computer program that executes the method of paragraph 1.

11. In an electronic device, Central processing unit (CPU); At least one graphic processing unit (GPU); and Memory comprising one or more storage media that store instructions Includes, When the above commands are executed by the CPU, the electronic device, When a page fault occurs in the above GPU, page fault information including a virtual address for the page where the page fault occurred is received from the above GPU, and By analyzing the received page fault information, a first virtual address block is generated based on the virtual address of the page where the page fault occurred, and Based on the first virtual address block mentioned above, a second virtual address block is created for each virtual address block to be pre-fetched according to a predefined number of virtual address blocks, and Migrate the first virtual address block and the second virtual address block to the memory of the GPU, Electronic device.

12. In Paragraph 11, When the above commands are executed by the CPU, the electronic device, Creating the second virtual address block by applying a predefined virtual address block offset based on the virtual address of the first virtual address block. Electronic device.

13. In Paragraph 11, When the above commands are executed by the CPU, the electronic device, When the memory access pattern of the above GPU is linear, the second virtual address block contiguous with the first virtual address block is generated using a first offset interval determined according to the linear memory access pattern. Electronic device.

14. In Paragraph 13, The first offset interval above is, An electronic device having a fixed value.

15. In Paragraph 11, When the above commands are executed by the CPU, the electronic device, If the memory access pattern of the above GPU is non-linear, the second virtual address block contiguous with the first virtual address block is generated using a second offset interval determined according to the non-linear memory access pattern. Electronic device.

16. In Paragraph 15, The above second offset interval is, An electronic device having a fluctuating value.

17. In Paragraph 11, When the above commands are executed by the CPU, the electronic device, Verifying whether the virtual address of the first virtual address block and the virtual address of the second virtual address block exist within the virtual address range of the integrated virtual memory. Electronic device.

18. In Paragraph 17, When the above commands are executed by the CPU, the electronic device, If the virtual address of the first virtual address block and the virtual address of the second virtual address block do not exist within the virtual address range of the integrated virtual memory, the first virtual address block and the second virtual address block are not created. Electronic device.

19. In Paragraph 11, When the above commands are executed by the CPU, the electronic device, If there is insufficient available space in the memory of the GPU, some pages in the memory of the GPU are moved to the memory of the CPU, and then the first virtual address block and the second virtual address block are moved to the memory of the GPU. Electronic device.

Images