Operation method of computing cluster, operation method of computing device and electronic device

Abstract

The invention discloses an operation method of a computing cluster, an operation method of a computing device and an electronic device. An operation method of a computing cluster includes: generating, by a first node, an instruction for storing data of the first node at a virtual address in a virtual memory; determining, by the first node, whether a cache block corresponding to the virtual address is stored in a cache of the first node according to the virtual address; generating, in the first node, a target cache block including the data based on the determined step indicating that the cache block corresponding to the virtual address is not stored in the cache of the first node; and updating, by the second node, a cache block stored in a cache of the second node, where the cache block in the cache of the second node corresponds to the virtual address, and where the step of updating is based on the target cache block.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2024-0165062, filed on November 19, 2024, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to the operation methods of computing clusters, computing devices, and electronic devices. Background Technology

[0003] Global Shared Memory (GSM) is a technology that provides virtual memory shared by a group of nodes. In practice, GSM can be distributed memory, but individual applications can treat it as local memory. That is, the global nature of the memory can be transparent to the application. Participating nodes can access the GSM using generic load / store instructions. Participating nodes can be independent computing devices that make up an electronic device (e.g., each with its own operating system). For example, when the electronic device is a cluster system, the nodes can be individual computers that make up the cluster system. Summary of the Invention

[0004] The present invention is provided in a simplified form to introduce the choice of concepts further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a method of operating a computing cluster, the computing cluster including a first node and a second node, the first node and the second node sharing virtual memory, the method comprising: the first node generating an instruction for storing data of the first node at a virtual address in the virtual memory; the first node determining, based on the virtual address of the instruction, whether a cache block corresponding to the virtual address of the instruction is stored in the cache of the first node; based on the determined step indicating that the cache block corresponding to the virtual address is not stored in the cache of the first node, generating a target cache block including the data in the first node, wherein the target cache block is stored in the cache of the first node; and the second node updating a cache block stored in the cache of the second node, wherein the cache block in the cache of the second node corresponds to a virtual address, and wherein the updating step is based on the target cache block.

[0006] The target cache block may include: a data field for storing the data; a global field for indicating whether the location of the cache block exists outside the first node, wherein the update steps of the cache block in the cache of the second node may be based on the global field; and a vector field for indicating the location of the data within the data field.

[0007] The steps for generating the target cache block may include: storing the data in a data field; setting bit values ​​corresponding to the location where the data is stored in the data field; and setting bit values ​​for a global field.

[0008] The steps for generating the target cache can be based on determining that an empty storage space exists in the cache of the first node, and that the target cache block can be stored in the empty storage space.

[0009] The steps for generating the target cache block may include: based on the determination that the empty storage space does not exist in the cache of the first node, removing one of the cache blocks stored in the cache of the first node, and storing the generated target cache block in the space where one of the cache blocks has been removed.

[0010] The steps of updating the cache block in the cache of the second node based on the target cache block may include: updating only the data included in the target cache block.

[0011] The step of updating a cache block in the cache of a second node based on a target cache block may include: determining the location to be updated based on a vector field indicating the location where the data is stored within the data field of the target cache block, and updating the data to be updated in the data field of the cache block with the data to be updated.

[0012] The operation method may further include: in response to the execution of a target application configured to perform sequential writes by the first node, storing sequentially generated data sequentially in a target cache block; and in response to the data fields of the target cache block being filled with sequentially generated data, updating the cache block in the cache of the second node based on the target cache block.

[0013] A non-transitory computer-readable storage medium stores instructions that, when executed by a processor, cause the processor to perform any of the methods.

[0014] In another general aspect, a method of operating a computing device includes: receiving or generating instructions for storing data of the computing device at a virtual address in a virtual memory shared by the computing device and a target computing device; determining, based on the virtual address, whether a cache block corresponding to the virtual address is stored in the cache of the computing device; based on the determined instructions indicating that the cache block corresponding to the virtual address is not stored in the cache of a first node, generating a target cache block including the data, and storing the target cache block in the cache of the computing device; and updating a cache block stored in the cache of the target computing device, the cache block in the cache of the target computing device corresponding to the virtual address, based on the target cache block.

[0015] In another general aspect, an electronic device includes: a first node and a second node configured to share a virtual memory, wherein the first node is configured to: generate instructions for storing data of the first node at a virtual address in the virtual memory; determine, based on the virtual address of the instructions, whether a cache block corresponding to the virtual address is stored in the cache of the first node; based on the determined step indicating that the cache block corresponding to the virtual address is not stored in the cache of the first node, generate a target cache block including the data in the first node, wherein the target cache block is stored in the cache of the first node; and provide the target cache block to the second node to ensure that the second node updates the cache block in its cache, the cache of the second node corresponding to the virtual address.

[0016] The target cache block may include: a data field for storing the data; a global field indicating whether the location of the cache block exists outside the first node, wherein the step of updating the cache block in the cache of the second node may correspond to a virtual address, and wherein the update step may be performed according to the target cache block; and a vector field indicating the location of the data stored within the data field.

[0017] The first node can be configured to: store the data in a data field; set a bit value corresponding to the location of the data storage within the data field; and set a bit value for a global field.

[0018] The first node can be configured to generate a target cache block in the empty storage space in response to the existence of an empty storage space in the first node's cache.

[0019] The first node can be configured to: in response to the absence of empty storage space in the first node's cache, remove one of the cache blocks stored in the first node's cache, and generate a target cache, which is stored in the space where one of the cache blocks stored in the cache has been removed.

[0020] The second node can be configured to update the cache block in the second node's cache using only the data included in the target cache block.

[0021] The second node can be configured to: determine the location in the data field of the cache block in the second node's cache that will be updated using the data, based on a vector field indicating the location in the data field of the target cache block where the data is stored.

[0022] The first node can be configured to: in response to the execution of a target application configured to perform sequential writes, sequentially store sequentially generated data, including the data, into a target cache block; and in response to the data fields of the target cache block being filled with sequentially generated data, update the cache block in the cache of the second node based on the target cache block.

[0023] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0024] Figure 1 An example configuration of an electronic device according to one or more embodiments is shown.

[0025] Figure 2 An example of a Global Shared Memory (GSM) according to one or more embodiments is shown.

[0026] Figure 3 An example of retrieving a cache block according to one or more embodiments is shown.

[0027] Figure 4 An example of false sharing according to one or more embodiments is shown.

[0028] Figure 5 Examples of modified bit vectors according to one or more embodiments are shown.

[0029] Figure 6 An example method of operation of an electronic device according to one or more embodiments is shown.

[0030] Figure 7 An example cache structure of an electronic device according to one or more embodiments is shown.

[0031] Figure 8 An example cache block is shown according to one or more embodiments.

[0032] Figure 9 An example target cache block is shown according to one or more embodiments.

[0033] Figure 10 An example of updating a cache block based on a target application is shown according to one or more embodiments.

[0034] Figure 11 Example operation of an electronic device according to one or more embodiments is shown.

[0035] Figure 12 An example method of operation of an electronic device according to one or more embodiments is shown.

[0036] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same or similar reference numerals will be understood to denote the same or similar elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0037] The following detailed description is provided to assist the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become clear upon understanding the disclosure of this application. For example, the order of operations described herein is merely illustrative and is not limited to those set forth herein, but may be changed as will become clear upon understanding the disclosure of this application, except for operations that must occur in a specific order. Furthermore, for the sake of clarity and conciseness, descriptions of features known upon understanding the disclosure of this application may be omitted.

[0038] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples provided herein are merely to illustrate some of the many possible ways in which the methods, apparatus, and / or systems described herein will become clear upon understanding the disclosure of this application.

[0039] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more. As a non-limiting example, the terms “comprising,” “including,” and “having” specify the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0040] Throughout this specification, when a component or element is described as being "connected to," "joined to," or "attached to" another component or element, it may be directly "connected to," "joined to," or "attached to" that other component or element, or there may reasonably be one or more other components or elements in between. When a component or element is described as being "directly connected to," "directly joined to," or "directly attached to" another component or element, there may be no other elements in between. Similarly, expressions such as "between" and "immediately between," and "adjacent to" and "closely adjacent to" may also be interpreted as described above.

[0041] Although terms such as “first,” “second,” and “third,” or A, B, (a), (b), etc., may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Each of these terms is not used to define, for example, the nature, order, or sequence of the corresponding component, assembly, region, layer, or part, but only to distinguish the corresponding component, assembly, region, layer, or part from other components, other assemblies, other regions, other layers, or other parts. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part mentioned in the examples may also be referred to as a second component, second assembly, second region, second layer, or second part.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on the understanding of the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art and in the disclosure of this application, and shall not be interpreted in an idealized or overly formal sense. The use of the term “may” herein with respect to examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature; however, all examples are not limited thereto.

[0043] Figure 1 An example of the configuration of an electronic device 100 (e.g., a computing cluster) according to one or more embodiments is shown.

[0044] Electronic device 100 may include nodes 110 (node ​​0), 120 (node ​​1), and 130 (node ​​2). Each of nodes 110, 120, and 130 may be an independent computing device (e.g., a node with its own operating system, applications, etc.). Nodes 110, 120, and 130 may communicate with each other, for example, using a switching fabric. For example, electronic device 100 may be a cluster system, and each of nodes 110, 120, and 130 may be a server / node. In addition... Figure 1 In addition to the components shown, the electronic device 100 may also include other general-purpose components.

[0045] The following description of the component applies to other nodes.

[0046] Node 0 110 may include processor 111, memory 113, and accelerator 115. Processor 111, memory 113, and accelerator 115 can communicate with each other via bus, on-chip network (NoC), peripheral component interconnect fast (PCIe), etc. In addition... Figure 1 In addition to the components shown, node 0 110 may also include other general components.

[0047] Processor 111 performs general functions for controlling node 0 110. Processor 111 typically controls node 0 110 by executing an operating system and programs and / or instructions stored in memory 113. Processor 111 may be implemented as a central processing unit (CPU), graphics processing unit (GPU), application processor (AP), etc., included in node 0 110. However, the examples are not limited to this.

[0048] Memory 113 may be hardware used to store processed data and data to be processed in node 0 110. Furthermore, memory 113 may store applications, drivers, etc., that will be driven by node 0 110. Memory 113 may include volatile memory (e.g., dynamic random access memory (DRAM)) and / or non-volatile memory.

[0049] Node 0 110 may include an accelerator 115 for operation. Accelerator 115 can handle tasks that, due to the characteristics of the task, can be handled more efficiently by a separate dedicated processor (such as accelerator 115) than by a general-purpose processor (such as processor 111). In this case, one or more processing elements (PEs) included in accelerator 115 may be used. Accelerator 115 may correspond to, for example, a neural processor (NPU), tensor processor (TPU), digital signal processor (DSP), GPU, neural engine, etc., that performs operations according to a neural network.

[0050] Nodes 110, 120, and 130 can share virtual memory. For example, nodes 110, 120, and 130 share virtual memory based on Global Shared Memory (GSM) technology. Because nodes 110, 120, and 130 can share virtual memory, all processes on the nodes can use the same address space. The shared virtual memory can be referred to as GSM. Nodes 110, 120, and 130 can exchange data with each other via virtual memory (e.g., through reads from and writes to GSM). In practice, virtual memory can be memory distributed across each node, but applications on any node can interface with virtual memory as if it were local memory. Therefore, typical load and store instructions can be used to access virtual memory. For example, each node can also access data stored in memory included in other nodes according to memory tiers.

[0051] However, cache consistency between nodes cannot be guaranteed using GSM technology.

[0052] Next, we will describe the virtual memory shared by a group of nodes.

[0053] Figure 2 An example of a GSM according to one or more embodiments is shown.

[0054] Reference Figure 2 The diagram shows nodes 0 (210), 1 (220), and 2 (230). Nodes 210, 220, and 230 may share virtual memory (e.g., GSM).

[0055] Virtual memory 200 (e.g., GSM) may represent a virtual memory space shared by nodes 210, 220, and 230 that share virtual memory 200. In practice, virtual memory 200 may be physically distributed across multiple nodes. For example, portions of the physical memory of each node 210, 220, and 230 may be allocated to virtual memory 200.

[0056] When an application running on a particular node uses virtual memory 200, the application may use: (i) memory allocated to virtual memory 200 in the particular node; (ii) memory allocated to virtual memory 200 in other nodes; and (iii) private local memory of the particular node. For example, when an application running on node 0 210 uses virtual memory 200, the application may use memory allocated to virtual memory 200 in node 0 210, memory allocated to virtual memory 200 in other nodes (e.g., node 1 220 and node 2 230), and private local memory of node 0 210. A node's private local memory may be memory in the node's physical memory that is not allocated to virtual memory.

[0057] Figure 3 An example of retrieving a cache block according to one or more embodiments is shown.

[0058] Figure 3 Nodes 0 (300) and 1 (310) are shown. It is assumed that nodes 0 (300) and 1 (310) are electronic devices sharing virtual memory (e.g., ...). Figure 1 The electronic device 100) is a node. Systems using virtual memory (e.g., computing clusters) can use write-back caching techniques to store data. Write-back techniques may include direct writes of data to the cache, and writes of data to non-cached storage devices / memory may be delayed. Write-back caching techniques may be used in conjunction with write-allocating caching techniques. Write-allocating caching techniques may involve retrieving cache blocks (e.g., cache lines) containing data to be stored in the local cache, and subsequently storing the cache blocks.

[0059] For example, refer to Figure 3 The instruction for storing data Y at virtual address A can be generated in core 301 of node 0 300 (e.g., ST A, R1(Y)). Node 0 300 can check whether the cache block corresponding to virtual address A has already been stored in cache 303 of node 0. A write hit occurs when the cache block corresponding to virtual address A has been stored in cache 303. A write miss occurs when the cache block corresponding to virtual address A has not been stored in cache 303 when the instruction for storing data Y is executed.

[0060] When a write miss occurs ( Figure 3When a cache miss occurs (as indicated by a cache miss), node 0 300 can read the cache block corresponding to virtual address A from node 1 310, which is currently storing the cache block corresponding to virtual address A. For example, node 0 300 can retrieve the cache block corresponding to virtual address A from node 1 310 and store the retrieved cache block in cache 303 of node 0.

[0061] When the cache block corresponding to virtual address A is retrieved back into cache 303, node 0 300 can update the data of the retrieved (and / or currently locally stored) cache block corresponding to virtual address A from data X to data Y of the previously mentioned storage instruction of core 301.

[0062] Subsequently, since the cache block corresponding to virtual address A has been retrieved back to cache 303, when an instruction (e.g., ST A, R2(Z)) for storing data Z at virtual address A is generated in core 301 of node 0 300, a write hit can occur and data Z can be stored without having to retrieve the cache block again.

[0063] A node that retrieves cache blocks from memory included in another node may be called a cache node or a first node. A node that provides cache blocks may be called a home node or a second node. Depending on the situation (e.g., which data is being processed and which operation is being performed), each node included in an electronic device may be either a cache node or a home node.

[0064] Next, we will describe false sharing. False sharing can occur when sharing virtual memory.

[0065] Figure 4 An example of pseudo-sharing according to one or more embodiments is shown.

[0066] Figure 4 CPU 0 400 and CPU 1 420 are shown. Assume that thread 0 of CPU 0 400 and thread 1 of CPU 1 420 access different data contained in the same cache block 440 of shared virtual memory 450. For example, thread 0 can access the first data in cache block 440 (checked mode), and thread 1 can access the second data in cache block 440 (diagonal mode).

[0067] Cache block 440 can be retrieved into cache 410 of CPU 0 400 and cache 430 of CPU 1 420. CPU 0 400 can update the first data in its copy of cache block 440, and CPU 1 420 can update the second data in its copy of cache block 440.

[0068] (For example, when a cache block is written to cache 410) the second data in the copy of cache block 440 stored in cache 410 (such a cache block has been updated by thread 0) can be past data. Similarly, (for example, when a cache block is written to cache 430) the first data in the copy of cache block 430 (such a cache block has been updated by thread 1) can be past data.

[0069] The copies of cache block 440 in cache 410 and the versions / copies of cache block 440 in cache 430, as well as the copies / copies of cache block 440 in shared virtual memory 450, may all correspond to the same cache block, but may each contain different data, and therefore, cache consistency may not be guaranteed.

[0070] Furthermore, when (i) cache block 440 stored in shared virtual memory 450 is updated based on a copy of a cache block stored in cache 410 (such as that updated by thread 0), and then (ii) cache block 440 stored in shared virtual memory 450 is updated based on a copy of a cache block stored in cache 430 (such as that updated by thread 1), the data updated by thread 0 can be changed back to past data (the original first data of the original cache block 440 in shared virtual memory 450). In this case, false sharing can occur.

[0071] The aforementioned lack of cache consistency and false sharing can occur in electronic devices (e.g., cluster systems) that include multiple nodes.

[0072] For example, a lack of cache coherence and false sharing can occur in an electronic device when node 0 and another node (e.g., node 2) access different data in a copy / version of the same cache block stored in the memory of node 1.

[0073] Ensuring cache coherence through hardware architecture can have limitations. Therefore, methods that ensure cache coherence through software can be used. For example, in timestep symmetric multiprocessing (TSMP), cache coherence can be ensured by updating the master node's cache blocks to the cache blocks of the cache nodes at defined intervals. However, cache coherence may not be guaranteed within the same cycle in TSMP (e.g., one or more cycles may be required to restore cache coherence).

[0074] Next, we will describe a method for updating only a portion of the data in a cache block in GSM.

[0075] Figure 5 Examples of modified bit vectors (modvec) according to one or more embodiments are shown. Here, "modified" means modifying the data represented by the bit vector, rather than modifying the bit vector itself.

[0076] In GSM networks used by a group of nodes in an electronic device (e.g., a computing cluster composed of nodes) to share virtual memory, a modified bit vector can be used to prevent the aforementioned false sharing. The modified bit vector can be a vector that manages, on a word-by-word basis, whether multiple data entries (corresponding to bits in the bit vector) in a cache block have been updated.

[0077] For example, suppose node 0 500 retrieves the cache block corresponding to virtual address A from node 1 510, and node 2 520 retrieves the same cache block corresponding to virtual address A from node 1 510. The cache block corresponding to virtual address A may include data (e.g., 0x12345678: a number representing / including the corresponding word (or word pair) in the cache block).

[0078] Node 0 500 can modify the data in its cache block to 0x9abc5678. Only a portion of the data in the cache block is modified (this portion now stores two leading words "9abc"). Node 2 520 can modify only a portion of the data in its cache block to 0x1234def0 (this portion now stores two trailing words "def0"). Each node updates the data in its cache block on a word-by-word basis, not on a cache block-by-cache-block basis.

[0079] Node 0 500 can update its modified bit vector from 0b0000 to 0b1100 to indicate that the data at positions 1 to 4 in the cache block has been updated.

[0080] The cache block of node 1510, which is the master node in this example, can be updated based on the updated cache block of node 0500, and the update can be based on the modified bit vector of node 1510. Specifically, based on the modified bit vector (e.g., 0b1100), the data in the copy of the cache block in node 1510 located at positions 1 to 4 (based on the positions of "1"s in the modified bit vector of node 1) is updated; however, in this example, the entire cache block of node 1510 is not updated. For example, the data in the copy of the cache block in node 1510 can be updated from 0x12345678 to 0x9abc5678.

[0081] Utilize Figure 5As shown in the timing diagram, node 2 520 can update its modified bit vector from 0b0000 to 0b0011 to indicate that the data at positions 5 to 8 of its cache block has been updated.

[0082] The cache block of node 1 (the master node in this example) can be updated based on the updated cache block of node 2 (520). Based on the modified bit vector (e.g., 0b0011), the data at positions 5 to 8 of the data in the cache block of node 1 (510) can be updated using the data in the cache block of node 2 corresponding to the "1" in the modified bit vector of node 2, without updating the entire cache block of node 1 (510). For example, the data in the cache block of node 1 (510) can be updated from 0x9abc5678 to 0x9abcdef0.

[0083] In a GSM where a group of electronic devices share virtual memory, the aforementioned false sharing can be prevented by using a modified bit vector.

[0084] Furthermore, write-allocated caching can incur significant overhead in GSM. For a store instruction to be executed using write-allocated caching, the entire cache block corresponding to virtual address A may need to be fetched to execute the store instruction. A typical store instruction updates one word (e.g., 4 bytes), and the typical cache block size can be 16 words (e.g., 64 bytes).

[0085] Write allocation caching techniques can incur significant overhead because updating a single word requires retrieving an entire cache block. Therefore, even in general-purpose systems different from GSM with shared virtual memory, memory buffers and write buffers have been used to mitigate this problem.

[0086] GSM systems can incur greater write-allocated caching overhead than existing non-GSM systems. This is because in GSM systems, cache blocks that need to be retrieved when executing store commands can be stored on other nodes within the GSM system. Furthermore, when the target application is a high-performance computing (HPC) or artificial intelligence (AI) application, large amounts of data can be updated sequentially and immediately. In such cases, entire cache blocks can be retrieved while incurring significant overhead, but the entire cache block can also be completely rewritten, resulting in unnecessary data retrieval. Therefore, to optimize GSM performance, when using write-allocated caching techniques, it is necessary to minimize the retrieval of entire cache blocks.

[0087] The following describes a method for minimizing the retrieval of the entire cache block by using a modified bit vector in a GSM shared by a set of nodes of shared virtual memory.

[0088] Figure 6 Examples of operating methods of an electronic device according to one or more embodiments are shown.

[0089] Operations 610 to 650 can be performed by electronic device 100.

[0090] In operation 610, the first node can generate storage instructions for storing the data of the first node.

[0091] The data in the first node can be the result of performing a specific operation at the core of the first node. The storage instruction can be an instruction used to store data corresponding to a cache block, which corresponds to a virtual address where the first node is storing its data. The first node can be the node storing the data and can be referred to as a cache node.

[0092] In operation 620, the first node can determine whether the cache block corresponding to the virtual address exists in the first node.

[0093] The following reference Figure 7 Describes a method for the first node to determine whether the cache block corresponding to the virtual address exists in the first node.

[0094] When it is determined in operation 620 that the cache block corresponding to the virtual address exists in the first node, operation 650 (storing data) can be executed directly. The first node can store the data in the cache block corresponding to the virtual address, and when the first node updates the cache block corresponding to the virtual address, the first node can update the vector field and the global field. See below. Figure 8 and Figure 9 Describe the methods for updating vector fields and global fields.

[0095] When it is determined in operation 620 that the cache block corresponding to the virtual address does not exist in the first node, operation 630 can be executed.

[0096] In operation 630, the first node can determine whether there is empty storage space in the first node's cache.

[0097] When there is free storage space in the first node's cache, the first node can perform operation 650. For example, the first node can generate a new target cache block in the free storage space. The first node can store data in the generated target cache block, and update the corresponding vector fields and global fields when the data is stored in the new target cache block.

[0098] When there is no free storage space in the cache of the first node, the first node can perform operation 640.

[0099] In operation 640, the first node may evict one of the cache blocks stored in the first node's cache. Specifically, the first node may evict one of the cache blocks according to a strategy used for evictment. For example, the first node may remove the oldest cache block from the cache blocks stored in the first node's cache. For example, the first node may remove the least recently used cache block from the cache blocks. The method of removing the least recently used cache block is a non-limiting example.

[0100] The first node can generate a new target cache block at the location of the removed cache block in its cache. The first node can store data in the generated target cache block and update vector fields and global fields as the data is stored.

[0101] Data can be stored in operation 650.

[0102] When data is stored in a cache block, the first node can update fields related to the data's storage. See below for reference. Figure 8 Describes fields related to data storage.

[0103] The cache block of the second node can be updated based on the target cache block (in the first node) where the data is stored. According to conventional techniques, the second node can be called the master node because it is the node from which the cache block is retrieved from the first node. The cache block being updated on the second node can be the cache block corresponding to a virtual address. In the cache block of the second node, only the data corresponding to one or more words / locations of the data stored (by the first node) in the target cache block can be updated. See below for reference. Figure 9 Describe the update of the second node.

[0104] Based on the above operations, in a GSM (Gateway Management System) traversed by a group of nodes sharing virtual memory, the overhead of retrieving a cache block from a second node can be omitted by generating a target cache block in the first node. In a GSM traversed by a group of nodes sharing virtual memory, when updating the cache block of a second node based on the target cache block, only the data corresponding to the data stored in the target cache block can be updated to the other node (e.g., the master node), thereby reducing unnecessary rewrites.

[0105] Figure 7 An example of a cache structure for an electronic device according to one or more embodiments is shown.

[0106] Reference Figure 7 This illustrates a four-way set-associative cache 700, one of various cache structures that can be used with the methods described herein for minimizing cache block retrieval. The four-way set-associative cache 700 manages the cache by dividing it into four sets. The following description can be applied to various other types of cache structures. Figure 7 The cache structure shown can be the cache structure of the first node, and other nodes can have similar cache structures.

[0107] The first node can generate a store instruction for storing its own data. The data can be executed into the cache block corresponding to the virtual address 710 of the store instruction. The first node can use virtual address 710 to determine whether the cache block corresponding to virtual address 710 is stored in the first node's cache.

[0108] Assuming virtual address 710 is a 32-bit address, the first node can determine whether the cache block corresponding to virtual address 710 is stored in the first node's cache by using the 8-bit index portion and the 22-bit tag portion of virtual address 710. (See reference...) Figure 7 Although only the valid field (V), tag field, and data field are shown in the cache blocks stored in four cache sets, cache blocks may include a variety of other fields. For example, each cache block may include a corresponding vector field (e.g., based on...). Figure 5 (The modified bit vector), each vector field has bits indicating the position of each updated data entry in the corresponding data field.

[0109] A write miss is determined when it is found that no cache block in set-associative cache 700 matches the 8-bit index and 22-bit tag of virtual address 710. When a write miss occurs, the following steps can be performed: Figure 6 Operation 630.

[0110] A write hit is determined when a cache block matching the 8-bit index and 22-bit tag of virtual address 710 is found. Upon a write hit, execution can proceed. Figure 6 Operation 650.

[0111] As referenced above Figure 5 The description describes how modified bit vectors can be used in GSM to prevent false sharing. The modified bit vectors can correspond to vector fields. These modified bit vectors can be used to manage target cache blocks. By using modified bit vectors to prevent false sharing, cache blocks can be managed without changing the cache structure.

[0112] Figure 7 The AND gates and equality checkers ("=" components) shown perform their normal functions and are used to determine whether a cache hit has occurred, and when a hit occurs, combine the data fields from the four caches (via the multiplexer) into a final data.

[0113] The following describes a method for managing target cache blocks using a modified bit vector.

[0114] Figure 8 An example of a cache block 800 according to one or more embodiments is shown.

[0115] Cache block 800 can be stored in the cache of the first node. Cache block 800 may include valid field 850, vector field 840, global field 830, tag field 820, and data field 810.

[0116] The valid field 850 may include a valid bit. The valid bit indicates whether the entire cache block 800 is valid or invalid. When the valid field 850 is 1, it indicates that the entire cache block 800 is valid. For example, when cache block 800 is retrieved from the second node back to the first node, the valid field 850 may be 1. When the valid field 850 is 0, it indicates that at least a portion of cache block 800 is invalid.

[0117] Vector field 840 can be referenced above. Figure 5 The modified bit vector is described. The number of bits in vector field 840 corresponds to the number of words (W) in data field 810. The bits in vector field 840 indicate the corresponding position in data field 810 where the target data is stored. Vector field 840 can also indicate the position in data field 810 where valid data is stored. For example, when vector field 840 includes 0001000, it indicates that only the fourth word in data field 810 is valid.

[0118] The global field 830 of each cache block can indicate whether the corresponding cache block 800 corresponds to a virtual memory (e.g., GSM) region. For example, the global field 830 can indicate whether the location of the cache block to be updated based on cache block 800 (e.g., the cache block corresponding to the virtual address) exists outside the first node.

[0119] The tag (T) field 820 of a cache block indicates the location of the cache block in the cache.

[0120] The data field 810 of the cache block can store the data of the cache block. As a non-limiting example, the data field 810 can store data in units of words. As a non-limiting example, the data field 810 can store 16 words. For the sake of description only, it is assumed here that it can store 8 words.

[0121] Since vector field 840 can be used to indicate the validity status of each word included in the corresponding data field 810, the number of bits in vector field 840 can correspond to the number of words in data field 810. For example, when the third word in data field 810 is updated, the third bit in vector field 840 can be updated to 1.

[0122] Figure 9 An example of a target cache block 900 according to one or more embodiments is shown.

[0123] According to the example, a storage instruction with a virtual address as the target of storage can be generated to store data of the first node in a cache block corresponding to the virtual address. For example, the storage instruction could be an instruction to store the result of a specific operation in a virtual address mapped to a third word of the cache block. The first node can determine whether the cache block corresponding to the virtual address is stored in the first node's cache. When the cache block is determined not to be stored in the first node's cache, the target cache block 900 can be generated by the first node and stored in the first node's cache. When generating the target cache block 900, the first node can store data in a data field 910, set the bit value in a vector field 940 corresponding to the location where the data is stored in the data field 910, and set the bit value of a global field 930. For example, the first node can store data in a third word of the data field 910 according to the storage instruction, set the bit value of the vector field 940 corresponding to the third word to 1, and set the bit value to 1 when the target cache block 900 corresponds to a virtual memory region, thereby generating the target cache block 900.

[0124] Based on the example, a storage instruction can be generated to store the data of the first node in a cache block corresponding to the virtual address of the storage instruction. For example, the storage instruction could be an instruction for storing the result of a specific operation that maps to a third word of the corresponding cache block. The first node can determine whether the cache block corresponding to the virtual address is stored in the first node's cache. When the cache block is stored in the first node's cache, the first node can update the cache block by storing data in the data field of the cache block, setting the bit value corresponding to the location of the data field where the data is stored, and setting the bit value of a global field. For example, the first node can update the cache block by: storing data in the third word of the data field according to the storage instruction, setting the bit value of the vector field corresponding to the third word to 1, and setting the bit value to 1 when the cache block corresponds to a virtual memory region.

[0125] According to the example, the cache block of the second node can be updated based on the target cache block 900. The cache block of the second node may correspond to a virtual address and will need to be retrieved back to the first node using conventional techniques (e.g., because the conventional first node would not be able to generate its own new cache block). The second node can receive the target cache block 900 from the first node. The second node can update the cache block based on the data field 910 and the vector field 940 of the target cache block 900. The second node can determine the location in the data field of the cache block to be updated based on the vector field 940, which indicates the location in the data field 910 of the target cache block 900 where data is stored. The second node can update the data at the determined location with the data in the corresponding data field 910.

[0126] Figure 10 An example of updating a cache block based on a target application is shown according to one or more embodiments.

[0127] For example, HPC applications (such as AI applications) can perform sequential writes. Sequential access to data can occur based on sequential writes, and therefore, prefetching data (e.g., cache blocks) can be easier. However, when the corresponding sequential write is performed, the retrieved cache blocks can be overwritten. For example, unnecessary data sending could be considered by retrieving cache blocks that include data that may not be used.

[0128] Based on the above reference Figures 6 to 9 The described method allows data to be stored without retrieving cache blocks from another node (e.g., by generating new cache blocks).

[0129] In one example, it can be assumed that the first node performs the target application that executes sequential writes. The first node can determine whether a cache block for storing sequentially generated data is stored in the first node's cache. When a cache block is stored in the first node's cache (e.g., as copied from another node), the first node can sequentially store (S) the sequentially generated data in the cache block. When the data field of the corresponding cache block is filled with sequentially generated data, the first node can send the cache block to the second node. The second node can update its cache block based on the received cache block.

[0130] In one example, it can be assumed that the first node performs the target application's sequential writes. During a sequential write, the first node can determine whether the cache block used to store the sequentially generated data is stored in the first node's cache or whether it is available in the first node's cache. If the cache block is not stored in the first node's cache, the first node can generate the target cache block. (Refer to the above.) Figure 6 This describes a method for generating a target cache block. The first node can sequentially store the sequentially generated data in the target cache block. When the data fields of the target cache block are filled with the sequentially generated data, the first node can send the target cache block to the second node. The second node can update its own cache block based on the received target cache block.

[0131] For example, sequentially generated data can be sequentially stored in the data field of target cache block 1000. Figure 10 As shown, the vector fields can be updated sequentially as the sequentially generated data is stored sequentially in the target cache block 1000. When the data fields are filled with data, the target cache block 1010 can be sent to the second node.

[0132] Figure 11 Examples of operation of an electronic device according to one or more embodiments are shown.

[0133] Reference Figure 11 The diagram shows nodes 0 (1110), 1 (1120), and 2 (1130). It can be assumed that nodes 0 (1110), 1 (11120), and 2 (1130) are nodes of electronic devices sharing virtual memory.

[0134] In the core of node 0 1110, instructions for storing data (e.g., 0x9abcxxxx) can be generated. These instructions can be implemented using a cache block corresponding to virtual address A. When generating instructions, the cache block corresponding to virtual address A can reside in node 1 1120.

[0135] By using Figure 6 In this method, node 0 1110 can store the cache block corresponding to virtual address A in node 0 1110 without retrieving the cache block from node 1 1120. For example, a target cache block can be generated, and data (e.g., 0x9abcxxxx) can be stored in it.

[0136] The target cache block may include a vector field (e.g., 1100). Based on the vector field, it can be confirmed that only the "9abc" of the data (e.g., 0x9abcxxxx) is stored in the target cache block. For example, "xxxx" could be an invalid value that is a remnant in the generation of the cache block.

[0137] Node 0 1110 can write the target cache block of Node 0 1110 to Node 1 1120. Specifically, Node 0 1110 can send the target cache block, which includes a vector field (e.g., 1100) and data (e.g., 0x9abcxxxx), to Node 1 1120.

[0138] Node 1 1120 can update the version of the cache block corresponding to virtual address A based on the received target cache block. For example, Node 1 1120 can update the first four bits of the data field of the cache block corresponding to virtual address A to "9abc" based on the vector field and data.

[0139] In the core of node 2 1130, instructions can be generated to store data (e.g., 0xxxxxdef0) in a cache block corresponding to virtual address A. When this instruction is generated, the cache block corresponding to virtual address A may exist in node 1 1120.

[0140] By using Figure 6 In this method, node 2 1130 can store the cache block corresponding to virtual address A in node 2 1130 without retrieving the cache block from node 1 1120. For example, a target cache block can be generated, and data (e.g., 0xxxxxdef0) can be stored in it.

[0141] The target cache block may include a vector field (e.g., 0011). Based on the vector field, it can be confirmed that only data (e.g., 0xxxxxdef0) with the value "def0" is stored in the target cache block. For example, "xxxx" can be an invalid value.

[0142] Node 2 1130 can write target cache blocks from Node 2 1130 to Node 1 1120. Node 2 1130 can send target cache blocks, including vector fields (e.g., 0011) and data (e.g., 0xxxxdef0), to Node 1 11120.

[0143] Node 1 1120 can update the cache block corresponding to virtual address A based on the received target cache block. For example, Node 1 1120 can update the last four bits of the data field of the cache block corresponding to virtual address A to "def0" based on the vector field and data.

[0144] Therefore, the cache block corresponding to virtual address A in node 1120 can store data (e.g., 0x9abcdef0) in the data field.

[0145] Figure 12 Examples of operating methods of an electronic device according to one or more embodiments are shown.

[0146] Figure 12 Operations 1200 to 1220 shown can be performed by Figure 1 The electronic device 100 performs the operation.

[0147] In operation 1200, the electronic device can determine, according to instructions for storing data, whether a cache block corresponding to a virtual address is stored in the cache of a first node included in the electronic device, wherein the virtual address is used to store data of the first node.

[0148] In operation 1210, when the cache block corresponding to the virtual address is not stored in the cache of the first node, the electronic device may generate a target cache block including data in the first node.

[0149] In operation 1220, the electronic device can update the cache block for the second node based on the target cache block. The second node shares virtual memory with the first node and stores the cache block corresponding to the virtual address.

[0150] Reference above Figures 1 to 11 Detailed description of operations 1200 to 1220.

[0151] Regarding Figures 1 to 12The described computing devices, electronic devices, processors, memories, displays, information output systems and hardware, storage devices, and other devices, apparatuses, units, modules, and components are implemented or represent hardware components through hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements, such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For the sake of brevity, the singular terms "processor" or "computer" are used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may implement a single hardware component or two or more hardware components. The hardware components may have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0152] Figures 1 to 12The methods for performing the operations described in this application are executed by computing hardware (e.g., by one or more processors or a computer), which is implemented as described above to execute instructions or software for performing the operations performed by the methods described in this application. For example, a single operation or two or more operations may be executed by a single processor or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation or two or more operations.

[0153] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers to operate as a machine or special-purpose computer to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include higher-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding description herein, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0154] Instructions or software used to control computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, may be recorded, stored, or fixed on or in one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-RLTH, BD -RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), flash memory, card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other means configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers so that one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0155] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

[0156] Therefore, in addition to the above disclosure, the scope of the disclosure may also be defined by the claims and their equivalents, and all variations within the scope of the claims and their equivalents shall be interpreted as included in the disclosure.

Claims

1. A method for operating a computing cluster, the computing cluster comprising a first node and a second node, the first node and the second node sharing a virtual memory, the method comprising: The first node generates instructions to store the data of the first node at a virtual address in the virtual memory; The first node determines whether the cache block corresponding to the virtual address of the instruction is stored in the cache of the first node based on the virtual address of the instruction. Based on the determined step indication that the cache block corresponding to the virtual address is not stored in the cache of the first node, a target cache block including the data is generated in the first node, wherein the target cache block is stored in the cache of the first node; and The cache block stored in the cache of the second node is updated by the second node, wherein the cache block in the cache of the second node corresponds to a virtual address, and wherein the update step is based on the target cache block.

2. The operating method as described in claim 1, wherein, The target cache block includes: Data field, which stores the data; A global field indicates whether the location of the cache block corresponding to the virtual address exists outside the first node, wherein the update steps of the cache block in the second node's cache are based on the global field; and The vector field indicates the location within the data field where the data is stored.

3. The operation method as described in claim 2, wherein, The steps for generating the target cache block include: The data is stored in a data field; Set the bit value corresponding to the location where the data is stored in the data field; and Sets the bit value of a global field.

4. The operating method as described in claim 1, wherein, The step of generating the target cache block is based on determining that an empty storage space exists in the cache of the first node, and wherein the target cache block is stored in the empty storage space.

5. The operating method as described in claim 1, wherein, The steps for generating the target cache block include: Based on the determination that the empty storage space does not exist in the cache of the first node, one of the cache blocks stored in the cache of the first node is removed, and the generated target cache block is stored in the space where the one of the cache blocks stored in the cache of the first node has been removed.

6. The operating method as described in claim 1, wherein, The steps for updating the cache block in the second node's cache based on the target cache block include: The cache block in the second node's cache is updated using only the data included in the target cache block.

7. The operating method as described in claim 6, wherein, The steps for updating the cache block in the second node's cache based on the target cache block include: Based on a vector field indicating the location where the data is stored within the data field of the target cache block, the location to be updated is determined, and the data to be updated in the data field of the cache block is updated to the data described above.

8. The operating method as described in claim 1, further comprising: In response to the first node executing the target application configured to perform sequential writes, the sequentially generated data is stored sequentially in the target cache block; as well as In response to the data fields of the target cache block being populated with sequentially generated data, the cache blocks in the cache of the second node are updated based on the target cache block.

9. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the operating method as described in any one of claims 1 to 8.

10. A method of operating a computing device, the method comprising: The computing device receives or generates instructions for storing the computing device's data at a virtual address in a virtual memory shared by the computing device and the target computing device; Determine whether the cache block corresponding to the virtual address is stored in the cache of the computing device based on the virtual address; Based on the determined step indication that the cache block corresponding to the virtual address is not stored in the cache of the first node, a target cache block including the data is generated, and the target cache block is stored in the cache of the computing device. as well as Update the cache block stored in the cache of the target computing device based on the target cache block, where the cache block in the cache of the target computing device corresponds to a virtual address.

11. An electronic device comprising: The first and second nodes are configured to share virtual memory. The first node is configured as follows: Generate instructions to store the data of the first node at a virtual address in the virtual memory; Based on the virtual address of the instruction, determine whether the cache block corresponding to the virtual address is stored in the cache of the first node; Based on the determined step indication that the cache block corresponding to the virtual address is not stored in the cache of the first node, a target cache block including the data is generated in the first node, wherein the target cache block is stored in the cache of the first node; and The target cache block is provided to the second node to ensure that the second node updates the cache block in its own cache, and the cache block of the second node corresponds to the virtual address.

12. The electronic device of claim 11, wherein, The target cache block includes: Data field, which stores the data; A global field indicating whether the location of the cache block corresponding to the virtual address exists outside the first node, wherein the cache block in the cache of the second node corresponds to the virtual address, and wherein the update step is performed based on the target cache block; and The vector field indicates the location within the data field where the data is stored.

13. The electronic device of claim 12, wherein, The first node is configured as follows: The data is stored in a data field; Set the bit value corresponding to the location where the data is stored in the data field; and Sets the bit value of a global field.

14. The electronic device of claim 11, wherein, The first node is configured as follows: In response to the existence of empty storage space in the cache of the first node, the target cache block is generated in the empty storage space.

15. The electronic device of claim 11, wherein, The first node is configured as follows: In response to the absence of empty storage space in the cache of the first node, one of the cache blocks stored in the cache of the first node is removed, and a target cache block is generated and stored in the space of the cache block that has been removed from the cache of the first node.

16. The electronic device of claim 11, wherein, The second node is configured to update the cache block in the second node's cache only using the data included in the target cache block.

17. The electronic device of claim 16, wherein, The second node is configured as follows: Based on a vector field indicating the location in the data field of the target cache block where the data is stored, the location in the data field of the cache block in the second node's cache that will be updated using the data is determined.

18. The electronic device of claim 11, wherein, The first node is configured as follows: In response to the first node executing the target application configured to perform sequential writes, the sequentially generated data, including the data, is sequentially stored into the target cache block; as well as In response to the data fields of the target cache block being filled with sequentially generated data, the cache blocks in the cache of the second node are updated based on the target cache block.

Images