Storage partitions for processing units

Abstract

A non-transitory machine-readable storage medium (500) containing instructions that, when executed, cause a system (100; 600) to: Dividing a common memory address space of a common memory (110, 602) among a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) into a plurality of memory partitions (108-1, 108-2, 108-3), wherein each processing unit (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units has exclusive ownership of a respective memory partition (108-1, 108-2, 108-3) of the plurality of memory partitions (108-1, 108-2, 108-3) of the common memory address space, such that a respective memory partition (108-1, 108-2, 108-3) is not subject to access contention by a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2); Performing data operations (106-1, 106-2, 106-3) in local memories (112-1, 112-2, 112-3) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) by the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2), wherein the data operations (106-1, 106-2, 106-3) serve to update a common data structure (108, 604) in the common memory address space; Forwarding a first data operation (106-1) by a first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2) based on a determination by the first processing unit (102-1, 606-1) that the first data operation (106-1) is to be applied to data in a memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2); and Applying the first data operation (106-1) by the second processing unit (102-2, 606-2), which comprises writing data of the first data operation (106-1) to the memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2) using a non-atomic operation.

Description

background

[0001] A memory can store data shared by multiple processing units. Processing units can refer to machine-readable instructions or hardware processors. Processing units can include, for example, threads. A multithreaded system can execute multiple threads in parallel. Hardware processors can be microprocessors, cores of multi-core processors, microcontrollers, digital signal processors, or any other type of processing circuitry.

[0002] US 2011 / 0 296 437 A1 discloses a lock-free communication capability that enables lock-free communication between cores of a multi-core processor. The lock-free communication between a first core and a second core of a multi-core processor is provided using a message queue mechanism. A first bitmap contains a plurality of bit positions associated with the respective messages of the message queue and is configured for use by the first core to indicate the availability of the respective queued message data.A second bitmap contains a plurality of bit positions associated with the respective messages of the message queue and is configured for use by the second core to acknowledge the availability of the respective queued message data and to indicate receipt of the respective queued message data.

[0003] US 2015 / 0 149 737 A1 relates to methods and systems for reading or writing to a resource, such as a memory shared among multiple computing devices.

[0004] US 2016 / 0 352 663 A1 discloses the prioritization of lock assignments, wherein it is determined whether a received first message is related to one or more messages in the queue, and then a priority value is assigned to the received message, which indicates an order of access of the message to a resource lock.

[0005] It is an object of the present invention to provide a storage medium, a computer system, and a method in which a common data structure can be accessed in parallel by multiple processing units. This object is achieved by a non-transitory machine-readable storage medium according to claim 1, a computer system according to claim 11, and a computer-implemented method according to claim 19. Brief description of the drawings

[0006] Some embodiments of the present disclosure are described with reference to the following figures. Fig. is a block diagram of a system with processing units according to some examples. Fig. is a block diagram of a processing unit according to some examples. Fig. is a flowchart of a method according to some examples. Fig. is a block diagram of multiple processing units including a multi-stage queuing arrangement, according to some examples. Fig. is a block diagram of the storage medium that stores machine-readable instructions according to some examples. Fig. is a block diagram of a system according to some examples. Fig. is a flowchart of a method according to some examples.

[0007] In the drawings, identical reference numbers indicate similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated to clarify the example shown. Furthermore, the drawings contain examples and / or embodiments consistent with the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description

[0008] In this disclosure, the use of the term "a," "an," or "the" includes the plural forms unless the context clearly indicates otherwise. Likewise, the term "comprises," "including," "comprises," "includes," "has," or "have," when used in this disclosure, specifies the presence of the specified elements but does not preclude the presence or addition of other elements.

[0009] When computations on shared data in memory are distributed among multiple processing units, techniques can be provided to protect data integrity during multi-stage update sequences. An example of a multi-stage update sequence involves parallel processing units concurrently reading data values ​​from the shared data, applying an operator to the data values ​​read by the processing units, and then writing back the modified data values ​​produced by the operator. In such cases, for a given data value at a given memory location, the latest results produced by a particular processing unit would overwrite all previous concurrent results from other processing units.

[0010] In some examples, locks (such as mutexes) may be used to prevent data corruption. In other examples, atomic operations supported by processors may be used. As further examples, atomic operations of a network structure may be used to ensure that the result of an operator of a first processing unit is visible before another operator is initiated on the same data.

[0011] Locks and atomic operations can cause significantly greater latency than non-atomic operations. This can slow down the execution of programs in a system.

[0012] In accordance with some implementations of the present disclosure, a shared memory is partitioned among multiple processing units such that the shared memory (or more precisely, a shared memory address space of the shared memory) is divided into individual memory partitions associated with the respective individual processing units. Data to be written to the memory partitions is manipulated by the respective owners of the memory partitions.

[0013] Each processing unit is assigned exclusive ownership of a specific memory partition of the shared memory. This enables single-writer access to the respective memory partitions, as multiple processing units do not compete for access to the same memory partition. A processing unit "owns" or has exclusive ownership of a memory partition if, among the multiple processing units, the processing unit has the exclusive right to write to the memory partition. In other words, among the multiple processing units, only one processing unit has the right to write to the memory partition - the remaining processing units of the multiple processing units do not have the right to write to the memory partition. It should be noted that a program (e.g.,an operating system or firmware) that has higher privileges than the processing units may be able to write to the shared memory address space.

[0014] The memory partitions belonging to the respective processing units can be considered “atomic” memory partitions because they can be updated by multiple processing units without contention.

[0015] Due to the use of atomic memory partitions, a common data structure (stored in the atomic memory partitions) can be accessed in parallel by multiple processing units, avoiding the use of atomic operations or locks to update the common data structure containing data to be updated by data operations of multiple processing units.

[0016] In some examples, a first processing unit forwards a first data operation to a second processing unit based on a determination by the first processing unit that the first data operation should be applied to data for a storage partition associated with the second processing unit. The second processing unit applies the first data operation, which includes writing data of the first data operation to the storage partition associated with the second processing unit using a non-atomic operation (and without using locks).

[0017] Fig. shows an example system 100 that includes multiple processing units 102-1, 102-2, and 102-3. Although in Fig. While three processing units are depicted, in other examples, there may be fewer or more than three processing units. In some examples, processing units 102-1 through 102-3 include threads executing in system 100. In other examples, processing units 102-1 through 102-3 may also include hardware processors or other types of processing units.

[0018] System 100 may include one or more compute nodes. A "compute node" may refer to one computer or a collection of multiple computers.

[0019] Each processing unit processes a corresponding input data collection to apply a corresponding operator that applies a data analysis operation to the corresponding input data collection. The first processing unit 102-1 may apply a data operation 106-1 that processes data in an input data collection 104-1, the second processing unit 102-2 may apply a data operation 106-2 that processes data in an input data collection 104-2, and the third processing unit 102-3 may apply a data operation 106-3 that processes data in an input data collection 104-3. An "input data collection" comprises a collection of data records (one record or multiple records). The input data collections 104-1 through 104-3 may be received from units within the system 100 or from external sources.

[0020] The data operations performed by processing units 102-1 to 102-3 can be applied to data stored in the respective local memories 112-1 to 112-3. Local memories 112-1 to 112-3 are, for example, caches that can be accessed at higher access speeds than shared memory 110. A data operation can be applied to data retrieved from a local memory, and / or the result data generated by the data operation can be written to the local memory.

[0021] In some examples, a data operation 106-1 to 106-3 may include a data accumulation operation (e.g., a count operation to count a set of occurrences of an attribute value), a summation function or other type of mathematical aggregation function (e.g., to calculate an average, a median, a maximum, a minimum, etc.), or any other type of data operation.

[0022] As in Fig. As shown, processing units 102-1 through 102-3 may update the respective portions of a common data structure 108 with the values ​​generated by data operations 106-1 through 106-3. Examples of common data structure 108 include a histogram, a table, a file system, and so on.

[0023] The shared data structure 108 is stored in a shared memory 110. The shared memory 110 may be implemented with a collection of storage devices (one or more storage devices), such as dynamic random access memory (DRAM), static random access memory (SRAM), flash memory devices, disk-based storage devices, and so on.

[0024] The shared data structure 108 contains entries stored in a shared memory address space associated with the shared memory 110. A memory address space includes memory locations defined by corresponding memory addresses.

[0025] The shared memory address space in which the shared data structure 108 is stored is divided into multiple partitions 108-1, 108-2, and 108-3, for example, if there are three processing units 102-1, 102-2, and 102-3. If there are fewer or more processing units, there may be fewer or more partitions.

[0026] Each processing unit 102-i has a corresponding partition 108-i (i equals 1 through 3). In some examples, partitions 108-1 through 108-3 are nested.

[0027] The nesting of partitions 108-1, 108-2 and 108-3 refers to alternating segments of partitions 108-1, 108-2 and 108-3, such that multiple segments of a particular partition are separated by segments of other partitions.

[0028] In other examples, partitions 108-1, 108-2, and 108-3 are not nested.

[0029] As illustrated by arrows 114, processing units 102-1 to 102-3 may communicate with each other, e.g., via interprocess communication, via queues, or via any other type of interface. For example, first processing unit 102-1 may send a message to second processing unit 102-2, where the message may include information related to a data operation to cause second processing unit 102-2 to apply the data operation. As used herein, a "message" may refer to an information item that can transfer certain information between processing units. For example, if first processing unit 102-1 determines that a first data operation to be applied to data (e.g., a data set that is part of input data collection 104-1) should be stored in a memory partition assigned to another processing unit (e.g.,the second processing unit 102-2), the first processing unit 102-1 sends a message to the second processing unit 102-2, the message containing information about the first data operation to be performed. The message causes the second processing unit 102-2 to perform the first data operation, which includes writing the result data generated by the first data operation to the memory partition assigned to the second processing unit 102-2.

[0030] In some examples, the partitioning of data across partitions 108-1 through 108-3 of the common data structure 108 may be based on hash values ​​calculated based on attributes of records in the input data collections 104-1 through 104-3.

[0031] Fig. shows an example in which a processing unit 102 (e.g. one of the processing units 102-1 to 102-3 of Fig. ) is to process an input data set 212 (e.g. from one of the input data collections 104-1 to 104-3).

[0032] The input data set 212 has multiple attributes a1, a2,..., an. In this example, attribute a2 is assumed to be a key. A "key" can include a collection of attributes (a single attribute or multiple attributes) in a data set. The key can be used in some examples to distribute data sets across partitions of the shared data structure 108.

[0033] As in Fig. As shown, the value of the key of the input data set 212 is supplied to an input of a hash function 214. In some examples, the hash function 214 may be a cryptographic hash function, such as a Secure Hash Algorithm (SHA) function in some examples.

[0034] The hash function 214 generates a hash value 216 based on the value of the key (e.g., the value of attribute a2). The hash value 216 can be used to determine which of the partitions 108-1 through 108-3 the input data set 212 belongs to. This determination would allow the processing unit 102 to identify the partition's owner processing unit to which a resulting value generated based on a data operation applied to the input data set 212 would be written.

[0035] Processing unit 102 includes an owner determination engine 202 for determining the owner processing unit associated with input data set 212. As used herein, an "engine" may refer to a hardware processing circuit, which may include any one or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuit. Alternatively, an "engine" may refer to a combination of hardware processing circuitry and machine-readable instructions (software and / or firmware) executable on the hardware processing circuitry.

[0036] For example, the owner determination engine 202 may include a map in which a lookup value points to a memory section that provides a value for the target. Another example is that the owner determination engine 202 may include a decoder that decodes the hash value for a target processing unit. As further examples, the owner determination engine 202 may include a lookup engine or a translator.

[0037] In further examples, the ownership determination engine 202 may calculate Hash(KeyV)%Num_P, where Hash() represents the hash function 214, KeyV represents a value of a key in an input data set, % represents a modulus operation, and Num_P represents the number of processing units in the system 100. The operation Hash(KeyV)%Num_P generates an integer that is one of a plurality of values ​​related to one of the plurality of partitions owned by the respective processing units. In the example of Fig. With three processing units, the operation Hash(KeyV)%Num_P generates one of three integer values ​​(e.g., 1, 2, or 3) that refer to one of the three partitions 108-1 to 108-3. Thus, the operation Hash(KeyV)%Num_P generates an output value that identifies its own processing unit (e.g., an output value of 1 identifies the first processing unit 102-1, an output value of 2 identifies the second processing unit 102-2, and an output value of 3 identifies the third processing unit 102-3).

[0038] Processing unit 102 further includes a message passing engine 204 configured to send messages to other processing units. For example, if the owner determination engine 202 indicates to processing unit 204 that processing unit X owns the input data record 212 (where X is one of several integer values), then the message passing engine 204 sends a message to processing unit X.

[0039] In some examples, the message includes information describing the data operation to be performed (e.g., data accumulation, data aggregation, etc.), data on which the data operation is to be performed, and possibly other information. The data on which the data operation is to be performed may be identified by specifying a memory location (e.g., with a memory address) in a memory in which the data is stored. The data to which the message refers may include the input data set 212 or, alternatively or additionally, the result data generated by the data operation applied by the processing unit 102 to the input data set 212.

[0040] In other examples, a message does not contain any information describing the data operation. Rather, different data operations can be associated with different operation queues, and the messages for different data operations can be sent to the respective operation queues. A target processing unit can determine which data operations to apply based on which queue a message is retrieved from.

[0041] Processing unit 102 further includes a notification system 206. In some examples, notification module 206 may send a notification to an owner processing unit when message forwarding module 204 has placed a threshold number of messages in a queue associated with the owner processing unit (see below). In other examples, notification module 206 may be omitted.

[0042] In some examples, hash value 216 is associated with one of a plurality of sets 208-1 through 208-M. The possible hash values ​​generated by hash function 214 correspond to the number of sets in a cache 210 (which may be a software cache or a hardware cache), which is an example of local memory (e.g., 112-1, 112-2, or 112-3) used by a processing unit. In some examples, the key values ​​are evenly distributed among the total number of sets to better utilize the cache.

[0043] Each set 208-j (j = 1 to M) represents an amount of data that can fit into an access unit, e.g., a cache line 210-1 to 210-M of cache 210. A cache line is a unit of data transfer between cache 210 and shared memory 110. More generally, a local memory access unit refers to an amount of data transferred as a unit between local memory and shared memory 110.

[0044] In the example of Fig. It is assumed that hash value 216 is associated with set 208-1. Different hash values ​​generated by hash function 214 would be associated with different sets.

[0045] In some examples, partitioning into groups corresponding to access units such as cache lines can prevent multiple processing units from sharing discrete access to a single data unit.

[0046] Additionally, hash-based partitioning can more evenly distribute the workload for a target access pattern, such as an access pattern of the shared data structure 108, across the processing units. For example, the number of partitions defined by hash-based partitioning is set to exceed the arrival rate of data operations divided by the operation latency (the latency of a processing unit in performing a data operation).

[0047] Fig. is a flowchart of a process involving a source processing unit 302 and a destination processing unit 304, according to some examples. The "source" processing unit is the processing unit at which a data operation is initiated. The "destination" processing unit is the processing unit to which a data operation is transferred under certain conditions.

[0048] Source processing unit 302 receives (at 306) an indication that a data operation should be initiated. For example, source processing unit 302 receives input data to which the data operation should be applied. Alternatively, source processing unit 302 may have received a command from another source, where the command specifies the data operation to be performed.

[0049] The owner determination engine 202 in the source processing unit 302 determines (at 308) which processing unit is the owner processing unit for the input data to which the data operation is to be applied.

[0050] If the owner processing unit is the source processing unit 302 (“self” branch of 308), then the source processing unit 302 applies (at 310) the data operation to the input data and stores (at 312) the data resulting from the data operation in a local memory of the source processing unit 302.

[0051] If the owner processing unit is the destination processing unit 304 (“Other” branch of 308), then the messaging system 204 sends (at 314) a message containing information about the data operation to the destination processing unit 304. In response to the source processing unit 302 forwarding the data operation to the destination processing unit 304 (by sending the message), the source processing unit 302 disassociates itself from the data operation so that the source processing unit 302 can continue with another data operation.

[0052] In response to the message, the target processing unit 304 applies (at 316) the data operation and stores (at 318) the data resulting from the data operation applied by the target processing unit 304 in a local memory of the target processing unit 304.

[0053] Filtering accesses by a specific processing unit also improves data locality, thus increasing cache utilization if present. Furthermore, this method extends the supported native atomic operations to any sequence that can be expressed as a single-threaded software algorithm.

[0054] In some examples, each processing unit (e.g., 102-1 through 102-3) is connected to a queue (or more generally, a queue array) to allow the processing units to communicate with each other (e.g., to forward messages between each other).

[0055] In some examples, the queuing arrangement used may be a multi-level queuing arrangement that includes multiple levels of queues. Fig. is a block diagram of a multi-level queuing arrangement 400 associated with the second processing unit 102-2. Note that each of the first processing unit 102-1 and the third processing unit 102-3 may also be connected to its respective multi-level queuing arrangement.

[0056] In some examples, multi-level queuing arrangement 400 includes a first level of work queues 402-1 and 402-2 and a second level of notification queue 404. Work queue 402-1 is used by first processing unit 102-1, and work queue 402-2 is used by third processing unit 102-3. In other examples, the first level may include more work queues if there are a larger number of processing units.

[0057] To transmit corresponding data operations to the second processing unit 102-2, the first processing unit 102-1 may send messages (408) to the work queue 402-1. Similarly, the third processing unit 102-3 may send messages (410) to the work queue 402-2 to transmit corresponding data operations to the second processing unit 102-2.

[0058] In some examples, multiple messages may accumulate in a work queue (402-1 or 402-2) until a certain number of messages are queued. In response to the threshold number of messages in the queue, a source processing unit may send a message to the message queue 404 in the second level. The message queue 404 may be shared by multiple processing units, e.g., 102-1 and 102-3 in Fig. . The first processing unit 102-1 (the source processing unit) sends a notification 412 to the notification queue 404 in response to the first processing unit 102-1 having sent a number R of messages 408 to the work queue 402-1, where R ≥ 1. If R > 1, the messages in the work queue 402-1 are bundled or batched for processing by the second processing unit 102-2 (the destination processing unit) to amortize the cost of transmitting or owning messages to the destination processing unit. Similarly, the third processing unit 102-3 sends a notification 414 to the notification queue 404 in response to the third processing unit 102-3 having sent a number R of messages 410 to the work queue 402-2.

[0059] The notification 412 or 414 indicates which work queue 402-1 or 402-2 of the source processing unit contains pending messages. In response to receiving the notification, the second processing unit 102-2 retrieves the respective messages from the work queue of the source processing unit and performs the corresponding data operations on the respective messages.

[0060] Since the second processing unit 102-2 owns the partition 108-2 of the shared data structure 108 to which the result data of the data operations identified in the messages retrieved from the worklist are to be added, the second processing unit 102-2 can perform updates to the partition 108-2 without atomic operations.

[0061] Also note that no atomic operations are used to write messages to a 402-1 or 402-2 work queue, because each source processing unit writes messages to a different work queue in the destination processing unit.

[0062] However, since the notification queue 404 is shared by multiple source processing units, a notification is written to the notification queue using an atomic operation.

[0063] Fig. is a block diagram of a non-transitory machine-readable or computer-readable storage medium 500 that stores machine-readable instructions that, when executed, cause a system (e.g., 100 in Fig. ) to perform various tasks.

[0064] The machine-readable instructions include instructions 502 for partitioning a shared memory to define a shared memory address space of a shared memory (e.g., 110 in Fig. ) among a variety of processing units (e.g. 102-1 to 102-3 in Fig. ) into a plurality of memory partitions, each memory partition being connected to a respective processing unit.

[0065] In some examples, each processing unit of the plurality of processing units has exclusive ownership of a respective memory partition of the shared memory address space such that the respective memory partition is not subject to access contention by multiple processing units.

[0066] The machine-readable instructions include instructions 504 for performing data operations to perform data operations in local memories (e.g., 112-1 to 112-3 in Fig. in Fig. ) of the plurality of processing units, wherein the data operations are used to update a common data structure (e.g. 108 in Fig. ) in the shared memory address space.

[0067] In some examples, each respective local memory has an access unit (e.g., a cache line) that is transferred as a unit between the respective local memory and the shared memory, wherein the respective local memory is divided into a plurality of sets each corresponding to a different access unit, and wherein a data operation in the respective local memory performed by a respective processing unit affects data in one set of the plurality of sets.

[0068] The machine-readable instructions include data operation forwarding instructions 506 for forwarding, by a first processing unit, a first data operation to a second processing unit based on a determination by the first processing unit that the first data operation is to be applied to data for a storage partition associated with the second processing unit.

[0069] In some examples, in response to the first processing unit forwarding the first data operation from the first processing unit to the second processing unit, the first processing unit disconnects from the first data operation to allow the first processing unit to continue with another data operation.

[0070] The machine-readable instructions include data operation application instructions 508 for applying, by the second processing unit, the first data operation, which includes writing data of the first data operation to the memory partition associated with the second processing unit using a non-atomic operation.

[0071] In some examples, the first data operation applied by the second processing unit includes writing data of the first data operation to the memory partition associated with the second processing unit by combining the data of the first data operation with data in the memory partition associated with the second processing unit (e.g., by summing or other data aggregation).

[0072] Fig. is a block diagram of a system 600 according to some examples. The system 600 includes a shared memory 602 for storing a common data structure 604.

[0073] The system 600 includes a plurality of processing units (including a first processing unit 606-1 and a second processing unit 606-2) that can access the common data structure 604 in the shared memory 602. A common memory address space of the shared memory 602 is divided among the plurality of processing units into a plurality of memory partitions, with each memory partition belonging to a respective processing unit.

[0074] The first processing unit 606-1 determines that a first data operation should be applied to data for a storage partition connected to the second processing unit 606-2. Based on the determination, the first processing unit 606-1 forwards the first data operation to the second processing unit 606-2 (at 608).

[0075] The second processing unit 606-2 performs the first data operation forwarded from the first processing unit 606-1, wherein the first data operation comprises writing (at 610) data of the first data operation to the memory partition associated with the second processing unit 606-2 using a non-atomic operation.

[0076] Fig. is a flowchart of a method 700 according to some examples of the present disclosure.

[0077] Process 700 includes partitioning (at 702) a shared memory storing a common data structure into a plurality of memory partitions. This partitioning may be performed by an operating system or another entity within a system.

[0078] Process 700 includes assigning (at 704) exclusive ownership of memory partitions to corresponding processing units, where the processing units are to access the shared data structure in the shared memory. Ownership assignment may be performed by the operating system or another entity in the system.

[0079] Process 700 includes receiving (at 706) first data to which a first data operation is to be applied by a first processing unit of the plurality of processing units. The first data operation may be initiated based on receipt of input data or receipt of a command from a source.

[0080] Process 700 includes determining (at 708) by the first processing unit whether to write the first data to a storage partition connected to the second processing unit. The determination may be based, for example, on applying a hash function to a key of the first data.

[0081] Process 700 includes, in response to determining that the first data is to be written to a memory partition associated with the second processing unit, sending (at 710) a message containing information about the first data operation from the first processing unit to the second processing unit. The message may, for example, be sent to a queue associated with the second processing unit.

[0082] After sending the message, process 700 includes disconnecting (at 712) the first processing unit from the first data operation. By decoupling from the first data operation, the first processing unit can perform other data operations.

[0083] The process 700 includes, in response to the message, performing (at 714) the first data operation by the second processing unit, which includes writing data of the first data operation to the memory partition associated with the second processing unit using a non-atomic operation.

[0084] A storage medium (e.g. 500 in Fig.) may include any one or a combination of the following elements: a semiconductor memory device such as DRAM or SRAM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; a magnetic disk such as a hard disk, floppy disk, and removable disk; other magnetic medium including tape; an optical medium such as a compact disk (CD) or digital video disk (DVD); or another type of storage device. It should be noted that the instructions described above may be provided on a single computer- or machine-readable storage medium, or alternatively, on multiple computer- or machine-readable storage media distributed throughout a large system, possibly with multiple nodes.Such computer- or machine-readable storage medium or media are considered part of an article (or article of manufacture). An article or article of manufacture may refer to each manufactured individual component or multiple components. The storage medium or media may be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded over a network for execution.

[0085] In the foregoing description, numerous details are set forth to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details described above. The appended claims are intended to cover such modifications and variations.

Claims

[1] A non-transitory machine-readable storage medium (500) containing instructions that, when executed, cause a system (100; 600) to: Dividing a common memory address space of a common memory (110, 602) among a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) into a plurality of memory partitions (108-1, 108-2, 108-3), wherein each processing unit (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units has exclusive ownership of a respective memory partition (108-1, 108-2, 108-3) of the plurality of memory partitions (108-1, 108-2, 108-3) of the common memory address space, such that a respective memory partition (108-1, 108-2, 108-3) is not subject to access contention by a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2); Performing data operations (106-1, 106-2, 106-3) in local memories (112-1, 112-2, 112-3) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) by the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2), wherein the data operations (106-1, 106-2, 106-3) serve to update a common data structure (108, 604) in the common memory address space; Forwarding a first data operation (106-1) by a first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2) based on a determination by the first processing unit (102-1, 606-1) that the first data operation (106-1) is to be applied to data in a memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2); and Applying the first data operation (106-1) by the second processing unit (102-2, 606-2), which comprises writing data of the first data operation (106-1) to the memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2) using a non-atomic operation. [2] The non-transitory machine-readable storage medium (500) of claim 1, wherein the instructions, when executed, cause the system (100; 600): in response to the forwarding of the first data operation (106-1) from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2), decouple the first processing unit (102-1, 606-1) from the first data operation (106-1) so that the first processing unit (102-1, 606-1) can continue another data operation. [3] The non-transitory machine-readable storage medium (500) of claim 1, wherein each respective local memory (112-1, 112-2, 112-3) of the local memories (112-1, 112-2, 112-3) has an access unit that is transferred as a unit between the respective local memory (112-1, 112-2, 112-3) and the shared memory (110, 602), wherein the respective local memory (112-1, 112-2, 112-3) is divided into a plurality of sets (208) each corresponding to a different access unit, and wherein a data operation (106-1, 106-2, 106-3) in the respective local memory (112-1, 112-2, 112-3) performed by a respective processing unit (102-1, 102-2, 102-3, 606-1, 606-2) is performed on data in one set (208-j) of the plurality of sets (208). [4] The non-transitory machine-readable storage medium (500) of claim 3, wherein the local memories (112-1, 112-2, 112-3) comprise cache memories (210) and the access unit is a cache line. [5] The non-transitory machine-readable storage medium (500) of claim 1, wherein the instructions, when executed, cause the system (100; 600): Determining, by the first processing unit (102-1, 606-1), that a second data operation (106-2) should be applied to data for another memory partition (108-1) belonging exclusively to the first processing unit (102-1, 606-1); and in response to determining that the second data operation (106-2) is to be applied to data for the further memory partition (108-1) belonging exclusively to the first processing unit (102-1, 606-1), performing the second data operation (106-2) by the first processing unit (102-1, 606-1). [6] The non-transitory machine-readable storage medium (500) of claim 1, wherein the first data operation (106-1) comprising writing data of the first data operation (106-1) to the memory partition (108-2) owned exclusively by the second processing unit (102-2, 606-2) comprises combining the data of the first data operation (106-1) with data in the memory partition (108-2) owned exclusively by the second processing unit (102-2, 606-2). [7] The non-transitory machine-readable storage medium (500) of claim 1, wherein forwarding the first data operation (106-1) from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2) comprises sending a message from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2), the message containing information about the first data operation (106-1). [8] The non-transitory machine-readable storage medium (500) of claim 7, wherein the message further includes information about data to which the first data operation (106-1) is to be applied. [9] The non-transitory machine-readable storage medium (500) of claim 7, wherein the second processing unit (102-2, 606-2) is connected to a queue (402-1, 402-2), and wherein sending the message to the second processing unit (102-2, 606-2) comprises writing the message to the queue (402-1, 402-2). [10] The non-transitory machine-readable storage medium (500) of claim 9, wherein the instructions, when executed, cause the system (100; 600) to: accumulate multiple messages in the queue (402-1, 402-2); and Sending a batch of multiple messages for processing the respective data operations (106-1, 106-2, 106-3) by the second processing unit (102-2, 106-2). [11] A computer system (100; 600) comprising: a common memory (110, 602) for storing a common data structure (108, 604); and one or more processors executing a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) comprising a first processing unit (102-1) and a second processing unit (102-2), wherein each processing unit (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) has exclusive ownership of a corresponding memory partition (108-1, 108-2, 108-3) of the plurality of memory partitions (108-1, 108-2, 108-3) of the shared memory address space, such that a respective memory partition (108-1, 108-2, 108-3) is not subject to access contention by multiple processing units (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2), wherein the first processing unit (102-1, 606-1) has the task of: in response to a decision that a first data operation (106-1) is to be applied to data for a memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2), forward the first data operation (106-1) to the second processing unit (102-2, 606-2), and wherein the second processing unit (102-2, 606-2) is to perform the first data operation (106-1) forwarded by the first processing unit (102-1, 606-1), wherein the first data operation (106-1) comprises writing data of the first data operation (106-1) to the memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2) using a non-atomic operation. [12] The computer system (100; 600) of claim 11, wherein the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) comprises a plurality of threads. [13] The computer system (100; 600) of claim 11, wherein the first processing unit (102-1, 606-1) is to: determine that a second data operation (106-2) should be applied to data for another memory partition (108-1) belonging exclusively to the first processing unit (102-1, 606-1), and in response to determining that the second data operation (106-2) is to be applied to data for the further memory partition (108-1) belonging exclusively to the first processing unit (102-1, 606-1), performing the second data operation (106-2) by the first processing unit (102-1, 606-1). [14] The computer system (100; 600) of claim 11, wherein forwarding the first data operation (106-1) from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2) comprises sending a message from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2), the message containing information about the first data operation (106-1). [15] The computer system (100; 600) of claim 14 further comprises a queue (402-2) connected to the second processing unit (102-2, 606-2), wherein forwarding the first data operation (106-1) from the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2) comprises sending the message to the queue (402-2) connected to the second processing unit (102-2, 606-2). [16] The computer system (100; 600) of claim 15, wherein the queue (402-2) is for accumulating a plurality of messages, and the queue (402-2) is for transferring a batch of the plurality of messages for processing of the respective data operations (106-1, 106-2, 106-3) by the second processing unit (102-2, 606-2). [17] The computer system (100; 600) of claim 16, wherein the queue (402-1, 402-2, 404) is a first queue (402-1, 402-2) in a first level of a multi-level queuing arrangement, further comprising a second queue (404) in a second level, wherein the first processing unit (102-1, 606-1) is to send a notification (412) to the second queue (404) when an amount of the multiple messages collected in the first queue (402-1, 402-2) exceeds a threshold. [18] The computer system (100; 600) of claim 17, wherein the second processing unit (102-2, 606-2) is responsive to the notification (412) sent to the second queue (404) to retrieve the plurality of messages from the first queue (402-1, 402-2). [19] A computer-implemented method comprising: Partitioning, by a computer, a common memory (110, 602) storing a common data structure (108, 604) into a plurality of memory partitions (108-1, 108-2, 108-3); Assigning exclusive ownership of memory partitions (108-1, 108-2, 108-3) of the plurality of memory partitions (108-1, 108-2, 108-3) to a plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2), wherein the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) access the common data structure (108, 604) in the common memory (110, 602), wherein each processing unit (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2) has exclusive ownership of a corresponding memory partition (108-1, 108-2, 108-3) of the plurality of memory partitions (108-1, 108-2, 108-3) of the shared memory address space, such that a respective memory partition (108-1, 108-2, 108-3) is not subject to access contention by multiple processing units (102-1, 102-2, 102-3, 606-1, 606-2) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2); Receiving first data to which a first data operation (106-1) is to be applied by a first processing unit (102-1, 606-1) of the plurality of processing units (102-1, 102-2, 102-3, 606-1, 606-2); in response to determining that the first data is to be written to a memory partition (108-2) belonging exclusively to the second processing unit (102-2, 606-2), sending a message containing information about the first data operation (106-1) by the first processing unit (102-1, 606-1) to the second processing unit (102-2, 606-2); after sending the message, disconnecting the first processing unit (102-1, 606-1) from the first data operation (106-1); and in response to receiving the message, performing the first data operation (106-1) by the second processing unit (102-2, 606-2), which comprises writing data of the first data operation (106-1) to the memory partition (108-2) owned exclusively by the second processing unit (102-2, 606-2) using a non-atomic operation.

Images