APPARATUS, METHOD AND APPARATUS FOR IMPLEMENTING SLICE COORDINATION, AND, NON-TRANSIENTIAL COMPUTER-READABLE MEDIUM

Description

1 / 32 “APPARATUS, METHOD AND APPARATUS FOR IMPLEMENTING SLICE COORDINATION, AND COMPUTER-READABLE NON-TRANSIENTIAL MEANS” CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims priority from the pending non-provisional US patent application No. 18 / 184,381, filed March 15, 2023, and assigned to the assignee thereof, and expressly incorporated herein by reference as if fully set forth below and for all applicable purposes. TECHNICAL FIELD

[0002] This disclosure refers, in general, to the field of coordination and, in particular, to slice coordination. BACKGROUND

[0003] Most information processing and communication systems have multiple nodes in a distributed system. The multiple nodes include multiple message and data producers and multiple message and data consumers. In general, some level of coordination and control of the multiple nodes is necessary to maintain data consistency. That is, as messages and data are updated, it is important to maintain consistency among the multiple nodes. An example of a distributed system is a computing system with multiple processing cores or slices that need to be coordinated. Maintaining data consistency among the multiple processing cores or slices through an efficient coordination scheme is desirable in distributed information processing systems. Petition 870250078697, dated 03 / 09 / 2025, page 9 / 89 2 / 32 SUMMARY

[0004] The following description presents a simplified summary of one or more aspects of this disclosure to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated attributes of the disclosure, and is not intended to identify key or critical elements of all aspects of the disclosure, nor to delineate the scope of any or all aspects of the disclosure. Its sole purpose is to present some concepts of one or more aspects of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0005] In one aspect, disclosure provides an apparatus for performing local coordination. Consequently, an apparatus includes a plurality of slices, wherein each slice of the plurality of slices is configured for distributed information processing; and a plurality of dedicated data buses, wherein each slice of the plurality of slices is coupled to one of the plurality of dedicated data buses and each slice of the plurality of slices is configured for local coordination for distributed information processing.

[0006] In one example, each slice of the plurality of slices includes a unit of memory. In one example, each slice of the plurality of slices is a unit of processing. In one example, the device additionally includes a plurality of current workload batches. In one example, each of the plurality of current workload batches is stored in the memory unit of each slice of the plurality of slices. Petition 870250078697, dated 03 / 09 / 2025, page 10 / 89 3 / 32

[0007] In one example, each of the plurality of current workload batches is different from another batch among the plurality of current workload batches. In one example, at least two of the plurality of current workload batches are the same. In one example, the appliance additionally includes a plurality of external memory units configured to store a plurality of future workload batches, where each of the plurality of slices is associated with one of the plurality of future workload batches.

[0008] In one example, the device additionally includes a plurality of data producers, wherein each of the plurality of data producers is configured to execute a write request on each of the plurality of slices. In another example, the device additionally includes a plurality of data consumers, wherein each of the plurality of data consumers is configured to execute a read request on each of the plurality of slices.

[0009] Another aspect of the disclosure provides a method for implementing slice coordination, wherein the method includes executing a write request with local coordination on the first batch of a plurality of batches in each slice of a plurality of slices; setting a local event tag on the first batch of the plurality of batches in each slice of the plurality of slices; monitoring the write request on the first batch of the plurality of batches in each slice of the plurality of slices; executing a read request with local coordination on the first batch of the plurality of batches in each slice of the plurality of slices after the Petition 870250078697, dated 03 / 09 / 2025, page 11 / 89 4 / 32 monitoring of the write request has been completed; and monitoring of the read request in the first batch of the plurality of batches in each slice of the plurality of slices.

[0010] In one example, the write request on each slice of the plurality of slices is executed asynchronously with respect to other slices within the plurality of slices. In another example, the method additionally includes each slice of the plurality of slices that provides a local output coordination signal to indicate the completion of the execution of a batch on each slice of the plurality of slices.

[0011] In one example, write request monitoring is performed until all previous write requests in a route have been completed. In one example, write request monitoring is performed until a route has a verified local event tag receipt. In one example, write request monitoring is performed until the write request in the first batch of multiple batches has been completed. In one example, write request monitoring is performed until two or more of the following actions occur: a) all previous write requests in a route are completed; b) the route has a verified local event tag receipt; c) the write request in the first batch of multiple batches is completed.

[0012] In one example, the read request in each slice of the plurality of slices is executed asynchronously with respect to other slices within the plurality of slices. In one example, the monitoring of the read request is performed until all read requests have been completed. Petition 870250078697, dated 03 / 09 / 2025, page 12 / 89 5 / 32 previous requests in a route have been completed. In one example, read request monitoring is performed until a route has a verified local event tag receipt. In another example, read request monitoring is performed until the read request in the first batch of multiple batches has been completed. In another example, read request monitoring is performed until two or more of the following actions occur: a) all previous read requests in a route are completed; b) the route has a verified local event tag receipt; c) the read request in the first batch of multiple batches is completed.

[0013] Another aspect of the disclosure provides an apparatus for implementing slice coordination, wherein the apparatus includes means for executing a write request with local coordination on the first batch of a plurality of batches in each slice of a plurality of slices; means for setting a local event tag on the first batch of the plurality of batches in each slice of the plurality of slices; means for monitoring the write request on the first batch of the plurality of batches in each slice of the plurality of slices; means for executing a read request with local coordination on the first batch of the plurality of batches in each slice of the plurality of slices after the monitoring of the write request has been completed; and means for monitoring the read request on the first batch of the plurality of batches in each slice of the plurality of slices.

[0014] In one example, the means to execute the write request are configured to execute from Petition 870250078697, dated 03 / 09 / 2025, p. 13 / 89 6 / 32 Asynchronously with respect to other slices within the plurality of slices, and the means to execute the read request are configured to execute asynchronously with respect to other slices within the plurality of slices. In one example, the means to monitor the write request are configured to monitor until one or more of the following actions occur: a) all previous write requests in a path are completed; b) the path has a verified local event tag receipt; c) the write request in the first batch of the plurality of batches is completed.In one example, the means to monitor the read request are configured to perform monitoring until one or more of the following actions occur: a) all previous read requests on a route are completed; b) the route has a verified receipt of the local event tag; c) the read request in the first batch of the plurality of batches is completed.

[0015] Another aspect of the disclosure provides a non-transient, computer-readable medium that stores computer-executable code, operable on a device that includes at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement slice coordination, wherein the computer-executable code includes instructions for making a computer execute a write request with local coordination on the first batch of a plurality of batches in each slice of a plurality of slices; instructions for making the computer set a local event tag on the first batch of the plurality of batches in Petition 870250078697, dated 03 / 09 / 2025, page 14 / 89 7 / 32 each slice of the plurality of slices; instructions to have the computer monitor the write request in the first batch of the plurality of batches in each slice of the plurality of slices; instructions to have the computer execute a read request with local coordination in the first batch of the plurality of batches in each slice of the plurality of slices after the monitoring of the write request has been completed; and instructions to have the computer monitor the read request in the first batch of the plurality of batches in each slice of the plurality of slices.

[0016] In one example, the nontransient, machine-readable medium additionally includes instructions to make the computer execute the write request asynchronously with respect to other slices among the plurality of slices and to execute the read request asynchronously with respect to other slices among the plurality of slices. In one example, the nontransient, machine-readable medium additionally includes instructions to make the computer monitor the write request until one or more of the following actions occur: a) all previous write requests in a path are completed; b) the path has a verified receipt of the local event tag; c) the write request in the first batch of the plurality of batches is completed.In one example, the non-transient, computer-readable medium additionally includes instructions for the computer to monitor the read request until one or more of the following actions occur: a) all previous read requests in a path are completed; b) the path has a verified receipt of the data. Petition 870250078697, dated 03 / 09 / 2025, page 15 / 89 8 / 32 local event tag; c) the reading request in the first batch of multiple batches is completed.

[0017] These and other aspects of the present disclosure will become more fully understood by reviewing the detailed description presented below. Other aspects, attributes, and implementations of the present disclosure will become apparent to those skilled in the art by reviewing the subsequent description of specific and exemplary implementations of the present invention in conjunction with the accompanying figures. Although the attributes of the present invention may be discussed in relation to certain implementations and figures below, all implementations of the present invention may include one or more of the advantageous attributes discussed in the present invention. In other words, although one or more implementations may be discussed as having certain advantageous attributes, one or more of such attributes may also be used in accordance with the various implementations of the disclosure discussed in the present invention.Similarly, although exemplary implementations may be discussed below as device, system, or method implementations, it should be understood that such exemplary implementations can be implemented in various devices, systems, and methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 illustrates a first example information processing system with a plurality of slices.

[0019] Figure 2 illustrates a second example information processing system with a plurality Petition 870250078697, dated 03 / 09 / 2025, page 16 / 89 9 / 32 of slices and global coordination.

[0020] Figure 3 illustrates an example distributed information processing system with a plurality of slices.

[0021] Figure 4 illustrates an example flow diagram for performing local coordination in a distributed information processing system. DETAILED DESCRIPTION

[0022] The detailed description set forth below in connection with the accompanying drawings is intended to serve as a description of various configurations and is not intended to represent the only configurations in which the concepts described in the present invention can be practiced. The detailed description includes specific details with the aim of providing a complete understanding of various concepts. However, it will be evident to those skilled in the art that these concepts can be practiced without these specific details. In some cases, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0023] Although, for the sake of simplicity of explanation, the methodologies are shown and described as a series of acts, it should be understood and recognized that the methodologies are not limited by the order of acts, since some acts may, according to one or more aspects, occur in different orders and / or simultaneously with other acts in relation to those shown and described in the present invention. For example, those skilled in the art will understand and recognize that a methodology could alternatively be represented as a series of interrelated states or events, such as Petition 870250078697, dated 03 / 09 / 2025, page 17 / 89 10 / 32 in a state diagram. Furthermore, not all illustrated actions may be necessary to implement a methodology according to one or more aspects.

[0024] A distributed information processing system, for example, a multi-slice computing system, requires a coordination scheme among a plurality of nodes to ensure data coherence. In one example, a slice is a processing core or a subset of a computing system. For example, each slice may have at least one processing unit. In one example, data coherence implies uniformity of shared data among a plurality of nodes, such as slices and associated memory units. In one example, the associated memory units may form a memory hierarchy with local memory dedicated to each slice, global memory shared among all slices, and other memory units with varying degrees of shared access (e.g., a level 2 (L2) cache).

[0025] In one example, the coordination scheme can operate on a global basis, that is, on all slices, as a global coordination or global synchronization. In another example, the coordination scheme can operate on a local basis, that is, on each slice individually, as a local coordination or local synchronization. That is, the coordination scheme can be a global coordination or a local coordination. For example, a coordination node can be one among a plurality of nodes that enforces data consistency among other nodes accessing the same data. In one example, the coordination node can enforce data consistency by controlling a coordination signal or by executing Petition 870250078697, dated 03 / 09 / 2025, p. 18 / 89 11 / 32 state transitions that can be accessible among some nodes of the plurality of nodes. In one example, slice coordination is local coordination.

[0026] Figure 1 illustrates a first example information processing system 100 with a plurality of slices. In one example, a slice is a processing unit or a core. In one example, the first information processing system 100 may be a graphics processing unit (GPU) or a central processing unit (CPU). In one example, Figure 1 includes a first slice 110, a second slice 120, a third slice 130, and an nth slice 140. In general, the first information processing system 100 may have an arbitrary number of slices. The plurality of slices may have a plurality of memory units. The plurality of slices may be interconnected by a common data bus (not shown) to transport messages and data among the plurality of slices.In the example in Figure 1, the quantity N of slices is illustrated. A person skilled in the art would know that N is any quantity expressed as a whole number.

[0027] Figure 1 also illustrates a plurality of workload batches 150, such as a first batch 151, a second batch 152, an nth batch 153, etc. In one example, a workload batch or a batch is a subset of a workload. In one example, the batch is a sequence of operations that can be executed by a slice. In the example in Figure 1, the quantity N of batches is illustrated. A person skilled in the art would know that N is any whole number. Petition 870250078697, dated 03 / 09 / 2025, page 19 / 89 12 / 32

[0028] In one example, in the first information processing system 100, an input can be self-generated. In one example, the first information processing system 100 includes a plurality of data producers and a plurality of data consumers. For example, the first information processing system 100 can execute the workload with a plurality of batches. In one example, the data producer is a processing unit with a software algorithm configured to generate data. In one example, the data producer executes a write request (that is, a request to write data to a memory unit). In one example, the data consumer is a processing unit with a software algorithm configured to read data. In one example, the data producer executes a read request (that is, a request to read data from a memory unit).

[0029] In one example, a first data consumer operating on a current batch (e.g., batch N) may need to read data produced from a previous batch (e.g., batch N-1). In one example, the first information processing system 100 may require a coordination scheme to maintain data consistency and avoid a write-after-read risk. In one example, the coordination scheme might be global coordination.

[0030] In one example, an event tag can be part of the sequence of operations as a message or a discrete token to facilitate coordination among the plurality of nodes in the distributed information processing system. For example, if the coordination scheme is coordination Petition 870250078697, dated 03 / 09 / 2025, page 20 / 89 13 / 32 global, a global event tag can be used as a coordination message or token. For example, if the coordination scheme is local coordination, a local event tag can be used as a coordination message.

[0031] In one example, global coordination can insert a global event tag into the sequence of operations to ensure that all data producers have written data to a target cache, buffered memory, or main memory before any data consumer initiates a read operation in the sequence of operations.

[0032] In one example, the global coordination performed by a data producer can execute a sequence of write requests as follows: {batch write request0 -> global event tag -> batch write request1}.

[0033] In one example, the global coordination performed by a data consumer might execute a sequence of read requests as follows: {read request in batch0 -> global event tag -> read request in batch1}.

[0034] In one example, when a coordination node (e.g., L2 cache memory) receives the global event tag, it may push back all subsequent requests from a common path until the following conditions are met: 1. All previous requests on the common path are being completed. 2. The global event tag is received in other routes within the same slice, and requests in those other routes are completed. Petition 870250078697, dated 03 / 09 / 2025, p. 21 / 89 14 / 32 3. The global event tag is received in other paths in other slices, and the requests in those other paths are completed.

[0035] In one example, requests in a path are completed when all write data is visible in the cache memory and all read requests receive returned data.

[0036] In one example, for the first 100 information processing systems, a data path can be partitioned into a plurality of tracks or slices. For example, different slices can operate in different batches. In one example, the coordination of the first 100 information processing systems can stall all slices and introduce a significant impact on performance. For example, a slice that is ahead must wait for a slice that is behind to complete (i.e., a large bubble is introduced in the execution). In one example, for a large-scale information processing system, performance overhead can increase.

[0037] Figure 2 illustrates a second example information processing system 200 with a plurality of slices and global coordination. In one example, the plurality of slices includes a first slice 210, a second slice 220, a third slice 230, an nth slice 240, etc. In the example in Figure 2, the quantity N of slices is illustrated. A person skilled in the art would know that N is any integer quantity. In general, the second information processing system 200 can have an arbitrary number of slices. The plurality of slices can be interconnected by a common data bus 201 to Petition 870250078697, dated 03 / 09 / 2025, page 22 / 89 15 / 32 transport messages and data across multiple slices.

[0038] Figure 2 also illustrates a plurality of current workload batches 250 which includes a first current batch 251, a second current batch 252, a third current batch 253, and an nth current batch 254. In the example in Figure 2, the quantity N of current batches is illustrated. A person skilled in the art would know that N is any whole number.

[0039] For example, the first slice 210 is operating with the first current batch 251, the second slice 220 is operating with the second current batch 252, the third slice 230 is operating with the third current batch 253, the nth slice 240 is operating with the nth current batch 254. Furthermore, Figure 2 also illustrates a plurality of future workload batches 260, such as a first future batch 261, a second future batch 262, a third future batch 263, an nth future batch 264, etc. In the example in Figure 2, the quantity N of future batches is illustrated. A technician in the field would know that N is any integer quantity.

[0040] In one example, the plurality of future workload batches 260 is synchronously introduced into the plurality of slices via the common data bus 201 after the plurality of current workload batches 250 has been completely executed. In one example, the synchronously inserted expression implies that a single state transition or a single trigger results in all slices of the plurality of slices having provided an updated input.

[0041] In Figure 2, each slice of the plurality of slices provides an output coordination signal to indicate the completion of a batch execution in the slice. For example, Petition 870250078697, dated 03 / 09 / 2025, p. 23 / 89 16 / 32 The first slice 210 provides a first output synchronization signal 211, the second slice 220 provides a second output synchronization signal 221, the third slice 230 provides a third output synchronization signal 231, the nth slice 240 provides an nth output synchronization signal 241, etc. In one example, the first output synchronization signal 211, the second output synchronization signal 221, the third output synchronization signal 231, the nth output synchronization signal 241, etc. are provided to a coordination module 270. In the example in Figure 2, the quantity N of output synchronization signals is illustrated. A technician in the field would know that N is any integer quantity.

[0042] In one example, coordination module 270 generates a global coordination signal 271 with a global status that depends on the first output synchronization signal 211, the second output synchronization signal 221, the third output synchronization signal 231, the nth output synchronization signal 241, etc. In one example, the global status of the global coordination signal 271 transitions from a first global state to a second global state when each status of the first output synchronization signal 211, the second output synchronization signal 221, the third output synchronization signal 231, the nth output synchronization signal 241, etc., has transitioned to an active state. For example, the first global state indicates a non-synchronized global state. For example, the second global state indicates a synchronized global state. That is, a transition from the first global state to the second global state is a transition from being non-synchronized. Petition 870250078697, dated 03 / 09 / 2025, page 24 / 89 17 / 32 to be synchronized for the plurality of slices.

[0043] In one example, each transition of the global status of the global coordination signal 271 occurs synchronously. That is, each slice can operate with global coordination. In one example, the first slice 210, the second slice 220, the third slice 230, the nth slice 240, etc. operate synchronously with respect to batch execution. That is, global coordination ensures data consistency across all slices, but with greater execution latency due to the intrinsic need for global coordination among all slices.

[0044] In one example, global coordination signal 271 is carried to common data bus 201. In one example, global coordination signal 271 governs a transition from the current workload batch plurality 250 to the future workload batch plurality 260 as input to the slice plurality. For example, the transition from the current workload batch plurality 250 to the future workload batch plurality 260 operates synchronously.

[0045] Figure 3 illustrates an example 300 distributed information processing system with a plurality of slices. In an example, a slice is a processing unit or a processing core. In an example, the 300 distributed information processing system could be a graphics processing unit (GPU) or a central processing unit (CPU). In an example, the plurality of slices includes a first slice 310, a second slice 320, a third slice 330, an nth slice 340, etc. In general, the processing system Petition 870250078697, dated 03 / 09 / 2025, page 25 / 89 18 / 32 of distributed information 300 can have an arbitrary number of slices. The plurality of slices can include a plurality of memory units. In one example, each slice includes one memory unit. In another example, each slice of a plurality of slices can be coupled to one or more memory units of a plurality of memory units, where the plurality of memory units is shared among the plurality of slices.

[0046] Each slice of the plurality of slices uses one of a plurality of dedicated data buses to transport messages and data. For example, the first slice 310 might have a first dedicated data bus 301, the second slice 320 might have a second dedicated data bus 302, the third slice 330 might have a third dedicated data bus 303, the nth slice 340 might have an nth dedicated data bus 304, etc. In one example, the distributed information processing system 300 might have a common data bus (not shown) that interconnects the plurality of slices together. In the example in Figure 3, the quantity N of slices and the quantity N of dedicated data buses are illustrated. A person skilled in the art would know that N is any integer quantity.

[0047] Figure 3 also illustrates a plurality of current workload batches 350 which includes a first current batch 351, a second current batch 352, a third current batch 353, an nth current batch 354, etc. In one example, the first slice 310 is operating with the first current batch 351, the second slice 320 is operating with the second current batch 352, the third slice 330 is operating with the third current batch 354, etc. Petition 870250078697, dated 03 / 09 / 2025, page 26 / 89 19 / 32 current batch 353, the nth slice 340 is operating with the nth current batch 354. In the example in Figure 3, the quantity N of current batches is illustrated. A technician in the field would know that N is any integer quantity. In one example, the plurality of current workload batches 350 is stored in a plurality of memory units.

[0048] Furthermore, Figure 3 also illustrates a plurality of future workload batches 360, such as a first future batch 361, a second future batch 362, a third future batch 363, an nth future batch 364, etc. For example, each future workload batch of the plurality of future workload batches 360 is asynchronously introduced into each slice of the plurality of slices via the plurality of dedicated data buses after each current workload batch of the plurality of current workload batches 350 has been individually executed. In one example, the expression asynchronously inserted implies that multiple independent state transitions or multiple triggers result in each slice of the plurality of slices having an updated input independently and asynchronously provided. In the example in Figure 3, the quantity N of future batches is illustrated. A technician in the field would know that N is any integer quantity.In one example, the plurality of future workload batches (360) is stored in a plurality of external memory units. In another example, the plurality of external memory units is not the same as the plurality of memory units that stores the plurality of current workload batches (350).

[0049] In Figure 3, each slice of the plurality of Petition 870250078697, dated 03 / 09 / 2025, p. 27 / 89 20 / 32 slices provide a local output coordination signal to indicate the completion of a batch execution in the slice. For example, the first slice 310 provides a first local output synchronization signal 311, the second slice 320 provides a second local output synchronization signal 321, the third slice 330 provides a third local output synchronization signal 331, the nth slice 340 provides an nth local output synchronization signal 341, etc. In the example in Figure 3, the quantity N of local output synchronization signals is illustrated. A technician in the field would know that N is any integer quantity.

[0050] In one example, the first local output synchronization signal 311 is provided to the first dedicated data bus 301, the second local output synchronization signal 321 is provided to the second dedicated data bus 302, the third signal of Local output synchronization 331 is provided to the third dedicated data bus 303, the nth local output synchronization signal 341 is provided to the nth dedicated data bus 304, etc. [00 51] In one example, the first local output synchronization signal 311, the second local output synchronization signal 321, the third local output synchronization signal 331, the nth local output synchronization signal 341 are all asynchronous with respect to each other. In one example, the first local output synchronization signal 311 has a first local status, The second local output synchronization signal 321 has a second local status, the third local output synchronization signal 331 has a third local status, the nth signal Petition 870250078697, dated 03 / 09 / 2025, page 28 / 89 21 / 32 of local output synchronization 341 has an nth local status, etc. In the example in Figure 3, the quantity N of local status is illustrated. A technician in the field would know that N is any integer quantity.

[0052] In one example, the transition of the first local status from an unsynchronized state to a synchronized state when the first slice 310 completes the execution of a batch in the first slice 310. In one example, the transition of the second local status from an unsynchronized state to a synchronized state when the second slice 320 completes the execution of a batch in the second slice 320. In one example, the transition of the third local status from an unsynchronized state to a synchronized state when the third slice 330 completes the execution of a batch in the third slice 330. In one example, the transition of the nth local status from an unsynchronized state to a synchronized state when the nth slice 340 completes the execution of a batch in the nth slice 340.

[0053] In one example, each transition from each local status to each local output synchronization signal occurs asynchronously. That is, each slice can operate with local coordination. That is, each slice can complete the execution of a batch in its slice and operate with local coordination, rather than global coordination. In one example, the first slice 310, the second slice 320, the third slice 330, the nth slice 340, etc. operate asynchronously with respect to batch execution. In one example, a slice X operates in its batch (i.e., batch X) asynchronously with respect to another slice (e.g., slice Y) that operates in its batch (i.e., batch Y). In this example, slice X and slice Y are two different slices in the plurality of slices. Petition 870250078697, dated 03 / 09 / 2025, page 29 / 89 22 / 32

[0054] In one example, operating each slice of the plurality of slices with a local output coordination signal improves the timing performance of the distributed information processing system 300 by reducing execution latency. That is, timing performance can be improved by ensuring data coherence in a slice, rather than across all slices. In one example, execution latency can be reduced by employing local coordination in each slice, rather than global coordination across all slices.

[0055] In one example, local coordination, rather than global coordination, is feasible when processing applications (e.g., graphics processing) primarily perform local processing (e.g., pixel-level) with local memory access, rather than global memory access. For example, data producers and data consumers in a given slice can be independent of data producers and data consumers in other slices. That is, batch execution for each slice can proceed asynchronously and independently of batch executions for other slices, which reduces timing performance overhead.

[0056] In one example, data producers and data consumers in each slice require local coordination and data coherence only in that slice. In one example, the timing distortion in batch execution in one slice is much smaller than in multiple slices, resulting in better timing performance with the use of local coordination instead of global coordination.

[0057] In one example, the local coordination can insert a local event tag in the sequence of operations to Petition 870250078697, dated 03 / 09 / 2025, page 30 / 89 23 / 32 ensure that all data producers have data written to a slice memory, a buffer memory, or main memory before any data consumer starts a read operation in the sequence of operations.

[0058] In one example, the local coordination performed by a data producer can execute a sequence of write requests as follows: {batch recording request0 -> local event tag -> batch recording request1}.

[0059] In one example, the local coordination performed by a data consumer might execute a sequence of read requests as follows: {Read batch request 0 -> local event tag -> Read batch request 1}.

[0060] In one example, when a local coordination node (e.g., a slice memory) receives the local event tag, it can push back all subsequent requests for the same path until the following conditions are met: 1. All previous requests on the same route are being completed. 2. The local event tag is being received on other routes within the same slice, and requests on those other routes are being completed.

[0061] In one example, requests on the same path are completed when all write data is visible in the slice memory and all read requests receive returned data. In one example, local coordination for a distributed information processing system can be used in many scenarios, Petition 870250078697, dated 03 / 09 / 2025, page 31 / 89 24 / 32 rendering for textures, secondary slice memory pass-through, local data coherence between computing kernels, machine learning, etc.

[0062] Figure 4 illustrates an example flow diagram 400 for performing local coordination in a distributed information processing system. In block 410, a workload is decomposed into a plurality of batches in an information processing system. In one example, the plurality of batches includes a first batch, a second batch, a third batch, etc., up to an nth batch. For example, the plurality of batches is arranged to be executed in sequential order. In one example, the plurality of batches is stored in a global memory of the information processing system.

[0063] In block 420, a first batch of a plurality of batches is loaded from global memory to local memory for each slice of a plurality of slices in a distributed information processing system. In one example, the first batch from global memory is loaded into each local memory using a dedicated data bus for each slice.

[0064] In block 430, the first batch of the plurality of batches is processed in each slice of the plurality of slices. In one example, each slice includes a plurality of data producers. In one example, each slice includes a plurality of data consumers. In one example, each data producer of the plurality of data producers provides data to one or more data consumers of the plurality of data consumers. In one example, the processing step may include operations such as, but not necessarily. Petition 870250078697, dated 03 / 09 / 2025, page 32 / 89 25 / 32 limiting to mathematical operations, sorting operations, storage operations, retrieval operations, logical operations, symbolic operations, etc.

[0065] In block 440, a write request is executed with local coordination on the first batch of the plurality of batches in each slice of the plurality of slices. In one example, the write request is executed by a data producer from a plurality of data producers. In one example, the write request in each slice of the plurality of slices is executed asynchronously with respect to other slices within the plurality of slices.

[0066] In block 450, a local event tag is defined in the first batch of the plurality of batches in each slice of the plurality of slices. In one example, each slice of the plurality of slices provides a local exit coordination signal to indicate the completion of the execution of a batch in the slice.

[0067] In block 460, the write request is monitored in the first batch of the plurality of batches in each slice of the plurality of slices. In one example, monitoring is performed until all previous write requests in a route have been completed. In one example, monitoring is performed until the route has a verified local event tag receipt. In one example, monitoring is performed until the write request in the first batch of the plurality of batches has been completed. In one example, monitoring is performed until two or more of the following actions occur: a) all previous write requests in a route are completed; b) the route has a verified local event tag receipt; c) the write request Petition 870250078697, dated 03 / 09 / 2025, page 33 / 89 26 / 32 recording in the first batch of the plurality of batches is completed. In one example, monitoring in each slice of the plurality of slices is performed asynchronously with respect to other slices within the plurality of slices.

[0068] In block 470, a read request is executed with local coordination on the first batch of the plurality of batches in each slice of the plurality of slices after the write request monitoring has been completed. In one example, the read request is executed by a data consumer from the plurality of data consumers. In one example, each data consumer from the plurality of data consumers receives data from one or more data producers from the plurality of data producers. In one example, the read request in each slice of the plurality of slices is performed asynchronously with respect to other slices within the plurality of slices.

[0069] In block 480, the read request is monitored in the first batch of the plurality of batches in each slice of the plurality of slices. In one example, monitoring is performed until all previous read requests in a route have been completed. In one example, monitoring is performed until the route has a verified local event tag receipt. In one example, monitoring is performed until the read request in the first batch of the plurality of batches has been completed. In one example, monitoring is performed until two or more of the following actions occur: a) all previous read requests in a route are completed; b) the route has a verified local event tag receipt; c) the read request Petition 870250078697, dated 03 / 09 / 2025, page 34 / 89 27 / 32 in the first batch of multiple batches to be completed.

[0070] In block 490, a second batch is loaded from global memory to the local memory of each slice in the plurality of slices in the distributed information processing system after read request monitoring has been completed. In one example, the second batch from global memory is loaded into each local memory using the dedicated data bus for each slice.

[0071] In one aspect, one or more of the steps in Figure 4 may be executed by one or more processors which may include hardware, software, firmware, etc. In one aspect, one or more of the steps in Figure 4 may be executed by one or more processors which may include hardware, software, firmware, etc. The one or more processors, for example, may be used to execute software or firmware necessary to perform the steps in the flow diagram of Figure 4. The term software should be interpreted broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable elements, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description language or otherwise.

[0072] Software may reside on a computer-readable medium. A computer-readable medium may be a non-transient computer-readable medium. A non-transient computer-readable medium includes, by way of Petition 870250078697, dated 03 / 09 / 2025, page 35 / 89 28 / 32 For example, a magnetic storage device (e.g., hard disk, floppy disk, magnetic tape), an optical disc (e.g., a compact disc (CD) or a digital versatile disc (DVD)), a smart card, a flash memory device (e.g., a card, a stick, or a key drive), random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), a register, a removable disk, and any other suitable means for storing software and / or instructions that can be accessed and read by a computer.Computer-readable media may also include, by way of example, a carrier wave, a transmission line, and any other suitable medium for transmitting software and / or instructions that can be accessed and read by a computer. Computer-readable media may reside within a processing system, be external to the processing system, or be distributed among multiple entities that include the processing system. Computer-readable media may be incorporated into a computer program product. By way of example, a computer program product may include computer-readable media in packaging materials. Computer-readable media may include software or firmware. Those skilled in the art will recognize the best ways to implement the described functionalities presented throughout this disclosure, depending on the particular application and the... Petition 870250078697, dated 03 / 09 / 2025, page 36 / 89 29 / 32 general design constraints imposed on the overall system.

[0073] Any set of circuits included in the processor(s) is merely provided as an example, and other means for performing the functions described may be included in the various aspects of the present disclosure, including, but not limited to, instructions stored in computer-readable media, or any other suitable apparatus or means described in the present invention, and with the use, for example, of the processes and / or algorithms described in the present invention in relation to the example flow diagram.

[0074] In the present disclosure, the word exemplifier is used to mean serving as an example, an instance, or an illustration. Any implementations or aspects described in the present invention as exemplifiers should not necessarily be interpreted as preferential or advantageous in relation to other aspects of the disclosure. Similarly, the term aspects does not require that all aspects of the disclosure include the attribute, advantage, or mode of operation discussed. The term coupled is used in the present invention to mean a direct or indirect coupling between two objects. For example, if an object A physically touches an object B, and object B touches an object C, then objects A and C can still be considered coupled to each other even if they do not directly physically touch each other.The terms circuit and circuit assembly are used broadly and are intended to include, without limitation, both hardware implementations of electrical devices and conductors that, when connected and configured, enable the performance of the functions described in this disclosure. Petition 870250078697, dated 03 / 09 / 2025, page 37 / 89 30 / 32 with regard to the type of electronic circuits, as well as software implementations of information and instructions that, when executed by a processor, enable the performance of the functions described in this disclosure.

[0075] One or more of the components, steps, attributes and / or functions illustrated in the figures may be rearranged and / or combined into a single component, step, attribute or function, or incorporated into multiple components, steps or functions. Additional elements, components, steps and / or functions may also be added without deviating from the innovative attributes disclosed in the present invention. The apparatus, devices and / or components illustrated in the present invention may be configured to perform one or more of the methods, attributes or steps described in the present invention. The innovative algorithms described in the present invention may also be efficiently implemented in software and / or incorporated into hardware.

[0076] It should be understood that the specific order or hierarchy of steps in the disclosed methods is an illustration of exemplary processes. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the methods may be rearranged. The appended method claims present elements of the various steps in a sample order and are not intended to be limited to the specific order or hierarchy presented, unless specifically indicated in this disclosure.

[0077] The above description is provided to enable any person skilled in the art to practice the various aspects described in the present invention. Various modifications thereof Petition 870250078697, dated 03 / 09 / 2025, p. 38 / 89 31 / 32 aspects will be readily apparent to those skilled in the art, and the generic principles defined in the present invention can be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown in the present invention, but should be considered to be in harmony with the complete scope and consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean one and only one, unless specifically stated otherwise, but rather one or more. Unless specifically stated otherwise, the term "any" refers to one or more. A phrase referring to at least one of a list of items refers to any combination of those items, including isolated members. As an example, the phrase "at least one of: a, b or c" is intended to encompass a; b; c; aeb; aec; bec; and a, be c.All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or may become known to those skilled in the art are expressly incorporated into the present invention by reference and are intended to be covered by the claims. Furthermore, nothing disclosed in this invention is intended to be exclusive to the public, regardless of whether such disclosure is explicitly mentioned in the claims. No element of a claim shall be interpreted in accordance with the provisions of Title 35 of the USC Code § 112, sixth paragraph, unless the element is expressly mentioned using the expression "means to" or, in the case of a method claim, the element is mentioned using "the". Petition 870250078697, dated 03 / 09 / 2025, p. 39 / 89 32 / 32 use of the expression "stage of".

[0078] A person skilled in the art would understand that various attributes from different modalities can be combined or modified and still be within the spirit and scope of the present disclosure. Petition 870250078697, dated 03 / 09 / 2025, p. 40 / 89

Claims

1 / 7 CLAIMS 1. Apparatus characterized by comprising: a plurality of slices, wherein each slice of the plurality of slices is configured for distributed information processing; and a plurality of dedicated data buses, wherein each slice of the plurality of slices is coupled to one of the plurality of dedicated data buses and each slice of the plurality of slices is configured for local coordination for distributed information processing.

2. Device according to claim 1, characterized in that each slice of the plurality of slices includes a memory unit.

3. Apparatus, according to claim 2, characterized in that each slice of the plurality of slices is a processing unit.

4. Apparatus, according to claim 2, characterized by further comprising a plurality of current workload batches.

5. Device according to claim 4, characterized in that each of the plurality of current workload batches is stored in the memory unit of each slice of the plurality of slices.

6. Apparatus according to claim 5, characterized in that each of the plurality of current workload batches is different from another of the plurality of current workload batches.

7. Apparatus, according to claim 5, characterized in that at least two out of the plurality of current workload batches are equal. Petition 870250078697, dated 03 / 09 / 2025, pp. 74 / 89 2 / 7 8. Apparatus, according to claim 2, characterized by further comprising a plurality of external memory units configured to store a plurality of future workload batches, wherein each of the plurality of slices is associated with one of the plurality of future workload batches.

9. Apparatus, according to claim 1, characterized by further comprising a plurality of data producers, wherein each of the plurality of data producers is configured to execute a write request on each of the plurality of slices.

10. Device according to claim 1, characterized by further comprising a plurality of data consumers, wherein each of the plurality of data consumers is configured to execute a read request on each of the plurality of slices.

11. Method for implementing slice coordination characterized by comprising: executing a write request with local coordination on the first batch of a plurality of batches in each slice of a plurality of slices; setting a local event tag on the first batch of the plurality of batches in each slice of the plurality of slices; monitoring the write request on the first batch of the plurality of batches in each slice of the plurality of slices; executing a read request with local coordination on the first batch of the plurality of batches in each slice of the plurality of slices after monitoring of the write request has been completed; and monitoring the read request on the first batch of the plurality of batches in each slice of the plurality of slices.

12. Method, according to claim 11, characterized in that the recording request in each slice of the plurality of slices is executed asynchronously with respect to other slices within the plurality of slices.

13. A method according to claim 12, characterized by further comprising each slice of the plurality of slices that provides a local output coordination signal to indicate the completion of the execution of a batch in each slice of the plurality of slices.

14. Method, according to claim 13, characterized in that the monitoring of the recording request is carried out until all previous recording requests in a path have been completed.

15. Method, according to claim 13, characterized in that the monitoring of the recording request is carried out until a route has a verified receipt of the local event tag.

16. Method, according to claim 13, characterized in that the monitoring of the recording request is carried out until the recording request in the first batch of the plurality of batches has been completed.

17. Method, according to claim 13, characterized in that the recording request monitoring is performed until two or more of the following actions occur: a) all previous recording requests in a route are completed; b) the route has a verified receipt of the event tag Petition 870250078697, dated 03 / 09 / 2025, page 76 / 89 4 / 7 local; c) the recording request in the first batch of the plurality of batches is completed.

18. Method, according to claim 11, characterized in that the read request in each slice of the plurality of slices is executed asynchronously with respect to other slices within the plurality of slices.

19. Method, according to claim 18, characterized in that the monitoring of the reading request is carried out until all previous reading requests on a route have been completed.

20. Method, according to claim 18, characterized in that the monitoring of the reading request is carried out until a route has a verified receipt of the local event tag.

21. Method, according to claim 18, characterized in that the monitoring of the reading request is carried out until the reading request in the first batch of the plurality of batches has been completed.

22. Method, according to claim 18, characterized in that the monitoring of the reading request is carried out until two or more of the following actions occur: a) all previous reading requests in a route are completed; b) the route has a verified receipt of the local event tag; c) the reading request in the first batch of the plurality of batches is completed.

23. Apparatus for implementing slice coordination characterized by comprising: means for executing a write request with local coordination in a first batch of a plurality of batches in each slice of a plurality of slices; Petition 870250078697, dated 03 / 09 / 2025, p. 77 / 89 5 / 7 means for setting a local event tag in the first batch of the plurality of batches in each slice of the plurality of slices; means for monitoring the write request in the first batch of the plurality of batches in each slice of the plurality of slices; means for executing a read request with local coordination in the first batch of the plurality of batches in each slice of the plurality of slices after the monitoring of the write request has been completed; and means for monitoring the read request in the first batch of the plurality of batches in each slice of the plurality of slices.

24. Device according to claim 23, characterized in that the means for executing the write request are configured to execute asynchronously with respect to other slices among a plurality of slices, and in that the means for executing the read request are configured to execute asynchronously with respect to other slices among a plurality of slices.

25. Device according to claim 24, characterized in that the means for monitoring the recording request are configured to perform monitoring until one or more of the following actions occur: a) all previous recording requests in a route are completed; b) the route has a verified receipt of the local event tag; c) the recording request in the first batch of the plurality of batches is completed.

26. Device, according to claim 24, characterized in that the means for monitoring the reading request are configured to perform monitoring until one or more of the following actions occur: a) all previous reading requests on a route are completed; b) the route has a verified receipt of the local event tag; c) the reading request in the first batch of the plurality of batches is completed.

27. A non-transient, computer-readable medium characterized by storing operable computer-executable code on a device comprising at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement slice coordination, in which the computer-executable code comprises: instructions for making a computer execute a write request with local coordination on the first batch of a plurality of batches in each slice of a plurality of slices; instructions for making the computer set a local event tag on the first batch of the plurality of batches in each slice of the plurality of slices; instructions for making the computer monitor the write request on the first batch of the plurality of batches in each slice of the plurality of slices;Instructions for having the computer execute a read request with local coordination on the first batch of multiple batches in each slice of multiple slices after the write request monitoring has been completed; and instructions for having the computer monitor the read request on the first batch of multiple batches in each slice of multiple slices.

28. A non-transient, computer-readable medium according to claim 27, characterized by further comprising instructions for causing the computer to execute the write request asynchronously with respect to other slices among a plurality of slices and to execute the read request asynchronously with respect to other slices among a plurality of slices.

29. A non-transient, computer-readable medium according to claim 28, characterized by further comprising instructions for causing the computer to monitor the write request until one or more of the following actions occur: a) all previous write requests in a path are completed; b) the path has a verified receipt of the local event tag; c) the write request in the first batch of the plurality of batches is completed.

30. Non-transient computer-readable medium according to claim 28, characterized by further comprising instructions for causing the computer to monitor the read request until one or more of the following actions occur: a) all previous read requests in a path are completed; b) the path has a verified receipt of the local event tag; c) the read request in the first batch of the plurality of batches is completed. Petition 870250078697, dated 03 / 09 / 2025, pp. 80 / 89

Images