Shared memory interconnect device and arbitration method and arbitration circuit

Abstract

This disclosure provides a shared memory interconnect device, arbitration method, and arbitration circuit, relating to the field of integrated circuit interconnect technology. The implementation scheme is as follows: a hierarchical interconnect network includes an outer routing network and an inner switching network. The outer routing network includes a multi-level switching network composed of multiple butterfly switching units, used to route data stream packets from input ports to multiple subgroups of the inner switching network. The inner switching network includes multiple crossbar switching modules for performing local non-blocking switching. An arbitration module maintains the weights of each input port and determines the arbitration result based on the weights between conflicting ports. When arbitration fails, the weight of the corresponding port is incremented by a preset increment. When arbitration succeeds, the weight of the corresponding port is restored to its initial value. The arbitration result of each port is determined by cascading the arbitration status between conflicting ports, which can significantly reduce hardware area overhead while ensuring local bandwidth and low latency.

Description

Technical Field

[0001] This disclosure relates to integrated circuit interconnect technology, and more particularly to shared memory interconnect devices, arbitration methods, and arbitration circuits. Background Technology

[0002] In the field of AI chip design, shared memory, as a key component, plays a crucial role in data interaction within the processing unit (PE). Traditional shared memory interconnect architectures primarily employ either a pure crossbar architecture or a pure Benes network architecture. While the pure crossbar architecture offers advantages such as low latency and high bandwidth, its hardware complexity increases quadratically with the number of ports, resulting in significant area overhead, high power consumption, and complex metal wiring, making it difficult to implement in large-scale AI chips. The pure Benes network architecture, while offering good scalability, requires multiple tiers of switches, resulting in high end-to-end latency and complex routing calculations, making it difficult to meet the demands of low-latency, intensive access to shared memory in AI computing. Summary of the Invention

[0003] This disclosure provides a shared memory interconnect device and an arbitration method and arbitration circuit.

[0004] According to one aspect of this disclosure, a shared memory interconnect device is provided, the interconnect device being configured to handle data communication between N input ports and N output ports. The interconnect device includes: a hierarchical interconnect network comprising an outer routing network and an inner switching network. The outer routing network includes a multi-level switching network composed of multiple butterfly switching units for routing data stream packets from the N input ports to multiple subgroups of the inner switching network. The number of switching levels in the multi-level switching network is less than 2×log2(N)-1. The inner switching network contains multiple M×M cross-connect switching modules. The input of the module is coupled to the output of the corresponding subgroup of the outer routing network to perform local non-blocking switching, where N is a positive integer and M is a positive integer less than N; the arbitration module is used to arbitrate concurrent access requests from the N input ports. The arbitration module is configured to: maintain the weight of each input port, determine the arbitration result based on the weight of each port when there is a conflict, increment the weight of the corresponding port by a preset increment when the arbitration fails, and restore the weight of the corresponding port to the initial value when the arbitration succeeds. The arbitration result of each port is determined by cascading the arbitration status between the conflicting ports.

[0005] In some embodiments, the multi-level switching network is a truncated Benes network, which is formed by splicing the first L stages and the last L stages of the complete Benes network, where L satisfies 1≤L≤log2(N)-1.

[0006] In some embodiments, the product of the number of the plurality of subgroups and the size M of the cross switch module is equal to N.

[0007] In some embodiments, the arbitration module is distributed and integrated into each cross-connect switching module of the inner switching network.

[0008] In some embodiments, the arbitration module further includes a conflict detection unit, which is used to traverse and compare the target memory identifiers of each input port, and determine the ports with the same target memory identifier as ports with conflict.

[0009] In some embodiments, the cascaded arbitration status includes an arbitration passed status, an arbitration failed status, and a pending status. The arbitration status of each port is passed up and down the cascaded status checker until the arbitration status of all ports is determined to be either arbitration passed or arbitration failed.

[0010] In some embodiments, each level of the status checker is configured to: if any of the ports that conflict with the current port and have a higher weight have an arbitration status of arbitration passed, then determine the arbitration status of the current port as arbitration failed; if all the ports that conflict with the current port and have a higher weight have an arbitration status of arbitration failed, then determine the arbitration status of the current port as arbitration passed; otherwise, maintain the arbitration status of the current port as pending and pass it to the next level status checker.

[0011] In some embodiments, the arbitration module is further configured to determine the arbitration result of the current port as failed when the subsequent stage indicates through a blocking signal that the request of the current port cannot be processed.

[0012] In some embodiments, at least one of the initial value of the weight and the preset increment is a configurable value.

[0013] In some embodiments, the N output ports are respectively connected to N storage banks of a shared static random access memory, and the device is used for shared memory access interconnection within the processing unit of an artificial intelligence accelerator chip.

[0014] According to another aspect of this disclosure, a shared memory interconnect arbitration method is provided, applied to the shared memory interconnect device as described above. The method includes: acquiring access requests and their target memory identifiers for each input port during the current arbitration cycle; performing a traversal comparison of the target memory identifiers for each input port; identifying ports with the same target memory identifier as conflicting ports; determining the initial state of the port, including: obtaining the port's weight and determining the initial arbitration state of the current port, wherein the initial arbitration state is one of arbitration passed, arbitration failed, or pending; and transmitting the arbitration state of each port level by level through a multi-level cascaded state checker, wherein each level of the state checker is configured to: if the current... If any port with a conflicting port and a higher weight has an arbitration status of "arbitration passed," then the arbitration status of the current port is determined to be "arbitration failed." If all ports with a conflicting port and a higher weight have arbitration statuses of "arbitration failed," then the arbitration status of the current port is determined to be "arbitration passed." Otherwise, the pending status is passed to the next level state checker. The port with an arbitration status of "arbitration passed" at the last level state checker is obtained as the final arbitration result. Based on the final arbitration result, the port status is updated, including: for ports with arbitration failed, the weight is increased by a preset increment; and for ports with arbitration passed, the weight is restored to the initial value for use in the next arbitration cycle.

[0015] In some embodiments, determining the initial state of a port further includes: when a downstream stage indicates that a request for the current port cannot be processed via a blocking signal, determining the initial arbitration state of the current port as arbitration failed.

[0016] In some embodiments, at least one of the initial value of the weight and the preset increment is a configurable value.

[0017] In some embodiments, determining the initial arbitration state of the current port includes: for ports with conflict, comparing the weight of the current port with the weights of other ports with conflict, and determining the initial arbitration state of the current port in conjunction with the subsequent ready signal.

[0018] According to another aspect of this disclosure, a shared memory interconnect arbitration circuit is provided, applied to the shared memory interconnect device as described above. The arbitration circuit includes: a conflict detection circuit, used to acquire access requests and their target memory identifiers for each input port in the current arbitration cycle, perform traversal comparison of the target memory identifiers for each input port, and determine ports with the same target memory identifier as ports with a conflict; an initial state determination circuit, used to obtain the weight of the port and determine the initial arbitration state of the current port, wherein the initial arbitration state is one of arbitration passed, arbitration failed, or pending; and a state transmission circuit, composed of multiple cascaded state checkers, each state checker being configured to: if the current port... If any port with a conflicting arbitration status and a higher weight has an arbitration status of "arbitration passed", then the arbitration status of the current port is determined to be "arbitration failed". If all ports with conflicting arbitration status and higher weights have an arbitration status of "arbitration failed", then the arbitration status of the current port is determined to be "arbitration passed". Otherwise, the pending status is passed to the next level status checker. An arbitration result management circuit is used to obtain the port with an arbitration status of "arbitration passed" at the last level status checker as the final arbitration result. Based on the final arbitration result, the port status is updated, including: for ports with an arbitration failure, the weight is increased by a preset increment; for ports with an arbitration success, the weight is restored to the initial value for use in the next arbitration cycle.

[0019] According to one or more embodiments of this disclosure, it is possible to ensure local bandwidth and low latency while reducing hardware area overhead.

[0020] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0021] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0022] Figure 1 A schematic diagram of the overall architecture of a shared memory interconnect device according to an exemplary embodiment of the present disclosure is shown.

[0023] Figure 2 A logic block diagram of an arbitration module according to an exemplary embodiment of the present disclosure is shown.

[0024] Figure 3 A schematic diagram of the transfer logic of a cascaded state transfer circuit according to an exemplary embodiment of the present disclosure is shown.

[0025] Figure 4 A flowchart of a shared memory interconnect arbitration method according to an exemplary embodiment of the present disclosure is shown.

[0026] Figure 5 A schematic diagram of a shared memory interconnect arbitration circuit according to an exemplary embodiment of the present disclosure is shown. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0029] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0030] Figure 1 A schematic diagram of the overall architecture of a shared memory interconnect device 100 according to an embodiment of the present disclosure is shown.

[0031] like Figure 1 As shown, the shared memory interconnect device 100 can be configured to handle data communication between N input ports and N output ports. The shared memory interconnect device 100 may include a hierarchical interconnect network and an arbitration module. Figure 1As shown, the hierarchical interconnection network may include an outer routing network 110 and an inner switching network 120. The outer routing network 110 may include a multi-level switching network composed of multiple butterfly switching units. The number of switching levels in the multi-level switching network may be less than 2×log2(N)-1, where N is a positive integer, and is used to route the data streams from N input ports to multiple subgroups in the inner switching network 120.

[0032] The inner switching network 120 may contain multiple M×M crossbar switching modules. The input of each crossbar switching module is coupled to the output of the corresponding subgroup of the outer routing network 110 to perform local non-blocking switching. M is a positive integer less than N.

[0033] Return to reference Figure 1 The inner switching network 120 may include crossbar switching modules 120-1, 120-2, 120-3...120-K. Although only an example of K=4 is shown in the figure, it is understood that the above is merely an example and this disclosure is not limited thereto.

[0034] The arbitration module can be used to arbitrate concurrent access requests from N input ports. The arbitration module can be configured to: maintain the weights of each input port; determine the arbitration result based on the weights of each port when there is a conflict; increment the weight of the corresponding port by a preset increment when arbitration fails; and restore the weight of the corresponding port to its initial value when arbitration succeeds. The arbitration result of each port is determined by cascading the arbitration status among the conflicting ports.

[0035] The apparatus provided by the embodiments of this disclosure can significantly reduce hardware area overhead while ensuring local bandwidth and low latency, effectively solving the wiring complexity of large-scale pure cross switches and the routing delay problem of pure Benes networks.

[0036] The following will describe in detail non-limiting embodiments of this disclosure.

[0037] According to some embodiments, a multi-stage switching network can be a truncated Benes network, constructed by splicing the first L stages and the last L stages of a complete Benes network, where L satisfies 1 ≤ L ≤ log2(N)-1. For example, refer to... Figure 1As shown, the outer routing network 110 may include a front-end routing network 111 and a back-end routing network 112. By truncating redundant intermediate stages in the Benes network, the number of switching stages is significantly reduced while retaining the outer routing distribution capability, decreasing it from 2×log2(N)-1 stages in the complete Benes network to 2L stages. This effectively reduces the end-to-end transmission latency of the outer routing network, while also reducing the number of switching units and lowering hardware area overhead. It is understood that although an example of L=2 is shown in the figure, the above is merely an example, and this disclosure is not limited thereto.

[0038] According to some embodiments, the product of the number of subgroups and the size M of the crossbar switching module can be equal to N. By constraining the product of the number of subgroups and M to be equal to the total number of ports N, the entire hierarchical interconnection network is ensured to have complete connectivity, that is, each input port can reach any crossbar switching module in the inner layer through the outer routing network, thereby ensuring logically achieving N×N non-blocking interconnection.

[0039] It is understandable that the number of subgroups can be denoted as K, then K×M=N. For example, for a network with N=32, if a cross-connect switching module with M=8 is selected, then K=4, meaning the outer routing network divides the 32 inputs into 4 subgroups, each containing 8 paths, corresponding to 8×8 cross-connect switching modules. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0040] According to some embodiments, the arbitration module can be located at the inner switching network. According to some embodiments, the arbitration module (not shown) can, for example, be distributed and integrated into each cross-connect switching module of the inner switching network. By integrating intelligent arbitration logic at each cross-connect point, multiple concurrent requests can be dynamically processed, ensuring that high-priority requests are processed promptly. Because the control logic employs a distributed arbitration mechanism, the bottleneck problem of a central arbitrator is avoided.

[0041] In some embodiments, the arbitration module may include a conflict detection unit. The conflict detection unit can be used to perform a traversal comparison of the target bank IDs of each input port, identifying ports with the same target bank ID as conflicting ports. According to this embodiment, by performing a full port traversal comparison of the target bank IDs (bank IDs) by the conflict detection unit, all port pairs with resource contention can be accurately identified within one cycle, providing precise conflict information for subsequent weight comparison and arbitration status determination.

[0042] It is understood that the target memory identifier can be the SRAM bank number pointed to by the requesting port. For example, the implementation of the traversal comparison can be to compare the bank ID of each port with the bank IDs of all other ports in parallel, generating a conflict hotbit, where each bit corresponds to a port, and a high bit indicates that the corresponding port and the current port point to the same memory bank, i.e., there is an access conflict. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0043] In some embodiments, the cascading arbitration status can include an arbitration passed status, an arbitration failed status, and a pending status. The arbitration status of each port is passed step-by-step along the cascading status checker until the arbitration status of all ports is determined to be either arbitration passed or arbitration failed. According to such an embodiment, by introducing the "pending" intermediate state, the arbitration logic can propagate status information step-by-step in a pipeline manner. Ports with determined results can immediately provide information to the next level, while ports with undetermined results continue to be passed in the pending state, thereby realizing parallel chain arbitration in hardware and significantly reducing arbitration latency.

[0044] Understandably, in hardware implementation, the three states can be encoded using 2 bits, for example, 2'b01 indicates arbitration passed, 2'b10 indicates arbitration failed, and 2'b00 indicates pending. At the final-level state checker, there is no 2'b00 (pending) state; the arbitration status of all ports has been determined as either passed or failed. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0045] In some embodiments, each level of state checker can be configured to: if any one of the ports that conflict with the current port and have a higher weight has an arbitration status of "arbitration passed," then the arbitration status of the current port is determined to be "arbitration failed"; if all the ports that conflict with the current port and have a higher weight have an arbitration status of "arbitration failed," then the arbitration status of the current port is determined to be "arbitration passed"; otherwise, the arbitration status of the current port is maintained as "pending" and passed to the next level state checker. Through this embodiment, each state checker implements efficient distributed arbitration decision-making with a concise set of priority inference logic: when a conflicting port with a higher weight than itself has passed arbitration, the current port must fail and can therefore be directly determined to have failed arbitration; when all conflicting ports with higher weights have been determined to have failed arbitration, the current port has no stronger competitors and can be directly determined to have passed arbitration.

[0046] It is understood that each level of the state checker is implemented with parallel hardware logic, and the list of "higher-weight conflicting ports" to be checked is determined by the conflict hot bits and the weight order. When the number of ports is N, there are N-1 cascaded state checker levels. Additionally and / or alternatively, multiple state checkers can be merged within a single pipeline stage to further reduce propagation latency. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0047] In some embodiments, the arbitration module can also be configured to determine the arbitration result of the current port as failed when the subsequent stage indicates through a blocking signal that the request of the current port cannot be processed. According to such an embodiment, by introducing the back pressure signal of the subsequent stage into the arbitration judgment logic in advance, unnecessary weight comparison calculations are avoided for requests that are known to be unacceptable by the subsequent stage, saving power consumption of the arbitration logic and preventing incorrect updates to the port weights.

[0048] For example, the subsequent blocking signal can be the inverted ready signal from the crossbar switch module, i.e., ready low (0) indicates subsequent blocking. In this case, the arbitration result of the current port can be directly set to arbitration failed without triggering the conflict comparison process. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0049] In some embodiments, at least one of the initial value of the weight and the preset increment can be a configurable value. According to such embodiments, by designing the initial weight value and the increment step size as configurable parameters, the arbitration strategy can be dynamically adjusted according to specific application load characteristics and service quality requirements without modifying the hardware design. For example, in scenarios with strict requirements on access latency to certain ports, the increment step size can be appropriately increased to make the weight increase faster, thereby enabling the port to obtain arbitration priority more quickly after waiting, effectively alleviating the "starvation" problem of specific ports.

[0050] It is understood that the initial value and preset increment can be written by software at runtime through configuration registers. For example, the initial value can be set to 0, and the increment step can be set to different values ​​such as 1, 2, or 4 to accommodate different fairness requirements. It is understood that the above are merely examples, and this disclosure is not limited thereto.

[0051] In some embodiments, the arbitration module can also be configured to support multiple configurable priorities. Ports with the same priority determine the arbitration result through weight comparison, with requests from higher-priority ports taking precedence over those from lower-priority ports in the arbitration. According to this embodiment, by introducing a multi-priority mechanism, static priority management is superimposed on dynamic weight arbitration, enabling the system to assign different priorities to different types of access requests. This ensures that the service quality of high-priority services is not affected by competition from low-priority ports, meeting the differentiated service needs of multiple computing engines concurrently accessing shared memory in AI accelerators.

[0052] For example, multi-priority arbitration can be implemented as follows: First, all requesting ports are grouped according to priority. When a port in a high-priority group enters arbitration, ports in a low-priority group are considered non-existent. Ports in a low-priority group only participate in arbitration after the ports in the high-priority group have no requests or have completed all arbitrations. Ports within the same priority group then compete according to dynamic weighting rules. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0053] Figure 2 A logic block diagram of an arbitration module according to an embodiment of the present disclosure is shown. (Reference) Figure 2 The arbitration module may include a conflict detection unit 210, a weight comparison unit 220, a cascaded state transmission unit 230, and a weight management unit 240.

[0054] The conflict detection unit 210 is used to traverse and compare the target memory identifiers of each input port, identify ports with the same target memory identifier as conflicting ports, and generate a conflict hotbit. Each high bit indicates that the corresponding port and the current port point to the same memory, i.e., an access conflict exists. For example, the conflict detection unit 210 can issue a conflict enable signal `conflict en`. For example, the conflict detection unit 210 can traverse the bank ID of each port and compare it with the corresponding bank ID of other ports. If there is an equality, it is considered that this port conflicts with the corresponding port, and the corresponding bit value of the corresponding port conflict hotbit is pulled high. The weight comparison unit 220 compares the weights of conflicting ports and determines the initial arbitration status of each port in conjunction with the subsequent ready signal. The initial arbitration status is one of arbitration passed, arbitration failed, or pending. When the subsequent stage indicates that the request of the current port cannot be processed through a blocking signal, the weight comparison unit 220 directly determines the initial arbitration status of the current port as arbitration failed without performing a weight comparison, thereby saving arbitration resources. For example, the weight comparison unit 220 can compare the weight of the current port with the weight of the port corresponding to the port where the port cfl hotbit is high, and determine whether the subsequent stage is blocked by the ready signal; if ready is 0, that is, the subsequent stage blocks the current port, then output state 2'b10, indicating that the arbitration failed; if ready is 1, if the weight of the current port is the highest, then output state 2'b01, indicating that the arbitration passed, otherwise output state 2'b00, indicating that it is pending, and at the same time output n bits of the greaters hotbit signal, where the higher bits of the greaters hotbit signal indicate that the weight of the corresponding port is greater than the current one, and n is the number of ports.

[0055] The cascaded status transmission unit 230 consists of multiple cascaded status checkers, used to transmit the arbitration status of each port level by level until the arbitration status of all ports is determined to be either successful or unsuccessful. The weight management unit 240 is used to maintain the weight of each input port based on the final arbitration result. When arbitration fails, the weight of the corresponding port is incremented by a preset increment; when arbitration succeeds, the weight of the corresponding port is restored to its initial value. At least one of the initial value and the preset increment of the weight is a configurable value, which can be written by the software at runtime through a configuration register to adapt to fairness requirements under different workloads. Additionally and / or alternatively, the arbitration module can also support multiple configurable priorities. Requests from high-priority ports take precedence over low-priority ports in weight comparison, and ports with the same priority further determine the arbitration status through weight comparison.

[0056] Understandable, Figure 2 The logic units of the arbitration module can be applied to the shared memory interconnect apparatus or shared memory interconnect arbitration method described in various embodiments of this disclosure, and the details thereof will not be repeated here.

[0057] In some embodiments, the N output ports can be connected to N memory banks of a shared static random access memory (SRAM), and the device can be used for shared memory access interconnection within the processing unit of an AI accelerator chip. According to this embodiment, directly interfacing the interconnection device with multiple memory banks of the shared SRAM and applying it to an AI accelerator chip can fully leverage the advantages of this solution—low area and low latency—supporting high-concurrency access to shared memory by multiple computing engines within the processing unit, effectively improving the efficiency of data transfer in AI inference and training tasks, and reducing the constraint of memory access bottlenecks on overall computing power.

[0058] Figure 3 A schematic diagram of the transfer logic of a cascaded state transfer circuit (bypass detect) according to an embodiment of the present disclosure is shown. Figure 3 Taking four ports (ports 0 to 3) as an example, the collision hot bits, weight comparison stages, and state transmission processes of each level of the state checker are illustrated. Figure 3 As shown, as a specific non-limiting example, a cascaded state transfer circuit can be composed of n-1 cascaded state checkers. Exemplarily, the arbitration status of each port is encoded using 2 bits: 2'b01 indicates arbitration passed, 2'b10 indicates arbitration failed, and 2'b00 indicates pending.

[0059] For example, each level of the status checker can be configured as follows: if any port with a higher weight that conflicts with the current port has an arbitration success status (2'b01), then the arbitration status of the current port is determined to be arbitration failure (2'b10) and output to the next level; if all ports with higher weights that conflict with the current port have an arbitration failure status (2'b10), then the arbitration status of the current port is determined to be arbitration success (2'b01) and output to the next level; otherwise, the pending status (2'b00) is maintained and passed to the next level. At the last level of the status checker, there is no pending status, and the port with an arbitration status of 2'b01 is taken as the final arbitration success output result.

[0060] It is understood that the aforementioned cascading mechanism enables the arbitration logic to complete arbitration of all ports in parallel hardware mode, without waiting for the centralized arbitrator to make individual decisions, thus completing the entire arbitration process within a very short clock cycle. It is understood that the above is merely an example, and this disclosure is not limited thereto.

[0061] Figure 4 A flowchart of a shared memory interconnect arbitration method 400 according to an embodiment of the present disclosure is shown.

[0062] In step 401, the access requests and their target storage identifiers for each input port in the current arbitration cycle are obtained. The target storage identifiers of each input port are traversed and compared, and ports with the same target storage identifier are identified as ports with conflict.

[0063] In step 402, the initial state of the port is determined, including: obtaining the port's weight and determining the initial arbitration state of the current port, which is one of arbitration passed, arbitration failed, or pending. When the subsequent stage indicates through a blocking signal that the request of the current port cannot be processed, the initial arbitration state of the current port is directly determined as arbitration failed.

[0064] At step 403, the arbitration status of each port is passed level by level through a multi-level cascaded status checker. Each level of status checker is configured as follows: if any port with a higher weight that conflicts with the current port has an arbitration status of "arbitration passed", then the arbitration status of the current port is determined to be "arbitration failed"; if all ports with a higher weight that conflict with the current port have an arbitration status of "arbitration failed", then the arbitration status of the current port is determined to be "arbitration passed"; otherwise, the pending status is passed to the next level status checker.

[0065] At step 404, the port whose arbitration status is "arbitration passed" at the last-level status checker is obtained as the final arbitration result.

[0066] At step 405, the port status is updated based on the final arbitration result: for ports that fail arbitration, the weight is incremented by a preset increment; for ports that pass arbitration, the weight is restored to the initial value for use in the next arbitration cycle.

[0067] In some embodiments, determining the initial state of a port further includes: when a downstream stage indicates that a request for the current port cannot be processed via a blocking signal, determining the initial arbitration state of the current port as arbitration failed.

[0068] In some embodiments, at least one of the initial value of the weight and the preset increment is a configurable value.

[0069] In some embodiments, determining the initial arbitration state of the current port includes: for ports with conflict, comparing the weight of the current port with the weights of other ports with conflict, and determining the initial arbitration state of the current port in conjunction with the subsequent ready signal.

[0070] In some embodiments, determining the initial state of a port may further include: arbitrating based on the priority of each input port, with requests from high-priority ports taking precedence over those from low-priority ports in weight comparison, and determining the arbitration state between ports of the same priority through weight comparison, thereby meeting the differentiated access priority requirements of different computing engines in the AI ​​accelerator.

[0071] Now for reference Figure 5 This describes a logical schematic diagram of a shared memory interconnect arbitration circuit according to an embodiment of the present disclosure. The shared memory interconnect arbitration circuit 500 may include a conflict detection circuit 510, configured to acquire access requests from each input port during the current arbitration cycle and their target memory identifiers, perform a traversal comparison of the target memory identifiers of each input port, and determine ports with the same target memory identifier as conflicting ports; an initial state determination circuit 520, configured to obtain the weight of the port and determine the initial arbitration state of the current port, wherein the initial arbitration state is one of arbitration passed, arbitration failed, or pending; and a state transfer circuit 530, composed of multiple cascaded state checkers, each state checker being configured to: if a conflict exists with the current port and a port with a higher weight exists... If any arbitration status is successful, the current port's arbitration status is determined to be unsuccessful; if the arbitration status of ports that conflict with the current port and have higher weights are all unsuccessful, the current port's arbitration status is determined to be successful; otherwise, the pending status is passed to the next level status checker; and the arbitration result management circuit 540 is used to obtain the port with an arbitration status of successful at the last level status checker as the final arbitration result; and based on the final arbitration result, update the port status, including: for ports that failed arbitration, incrementing the weight by a preset increment; for ports that passed arbitration, restoring the weight to the initial value for use in the next arbitration cycle.

[0072] It is understood that the various modifications and benefits described in connection with the apparatus embodiments in this disclosure are equally applicable to the corresponding method and circuit embodiments, and will not be repeated here.

[0073] It is understood that the various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and the associated blocks or states may be executed in other suitable orders. For example, the described blocks or states may be executed in a non-specifically disclosed order, or multiple blocks or states may be combined in a single block or state. Exemplary blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the disclosed exemplary embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the disclosed exemplary embodiments.

[0074] In this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are described and illustrated as independent operations, one or more individual operations may be performed concurrently, and these operations are not required to be performed in the order shown. Structures and functionalities presented as independent components in the example configuration may be implemented as combined structures or components. Similarly, structures and functionalities presented as individual components may be implemented as independent components. These and other variations, modifications, additions, and improvements are all within the scope of this document.

[0075] As used herein, “or” is inclusive rather than exclusive unless explicitly stated or indicated by context. Furthermore, “and” is both common and individual unless explicitly stated or indicated by context. Moreover, multiple instances may be provided for the resources, operations, or structures described herein as a single example. Furthermore, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are illustrated within the context of a particular illustrative configuration. The allocation of other functionalities is conceivable and may fall within the scope of various embodiments of this disclosure. Generally, structures and functionalities presented as independent resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functionalities presented as individual resources may be implemented as independent resources. These and other variations, modifications, additions, and improvements are all within the scope of embodiments of this disclosure. Therefore, this specification and accompanying drawings should be viewed in an illustrative rather than restrictive sense.

[0076] The terms “comprising” or “including” are used to indicate the presence of a subsequently stated feature, but do not preclude the addition of other features. Conditional language, in particular, such as “may,” “can,” or “may,” unless specifically stated or otherwise understood in the context of use, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, such conditional language generally does not imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments must include logic that, with or without user input or prompting, determines whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment.

Claims

1. A shared memory interconnect device, characterized in that, The interconnection device is configured to handle data communication between N input ports and N output ports, and the interconnection device includes: A hierarchical interconnection network includes an outer routing network and an inner switching network. The outer routing network includes a multi-level switching network composed of multiple butterfly switching units, used to route data stream packets from the N input ports to multiple subgroups of the inner switching network. The number of switching levels in the multi-level switching network is less than 2×log2(N)-1. The inner switching network contains multiple M×M cross-connect switching modules. The input of each cross-connect switching module is coupled to the output of the corresponding subgroup of the outer routing network to perform local non-blocking switching. Here, N is a positive integer and M is a positive integer less than N. An arbitration module is used to arbitrate concurrent access requests from the N input ports. The arbitration module is configured to: maintain the weight of each input port, determine the arbitration result based on the weight of each port when there is a conflict, increment the weight of the corresponding port by a preset increment when the arbitration fails, and restore the weight of the corresponding port to the initial value when the arbitration succeeds. The arbitration result of each port is determined by cascading the arbitration status between the conflicting ports.

2. The apparatus according to claim 1, characterized in that, The multi-level switching network is a truncated Benes network, which is formed by splicing the first L levels and the last L levels of the complete Benes network, where L satisfies 1≤L≤log2(N)-1.

3. The apparatus according to claim 1 or 2, characterized in that, The product of the number of the multiple subgroups and the size M of the cross switch module is equal to N.

4. The apparatus according to claim 1 or 2, characterized in that, The arbitration module is distributed and integrated into each cross-connect switching module of the inner switching network.

5. The apparatus according to claim 1 or 2, characterized in that, The arbitration module also includes a conflict detection unit, which is used to traverse and compare the target memory identifiers of each input port, and determine the ports with the same target memory identifier as ports with conflict.

6. The apparatus according to claim 1 or 2, characterized in that, The cascaded arbitration status includes arbitration passed, arbitration failed, and pending status. The arbitration status of each port is passed up and down the cascaded status checker until the arbitration status of all ports is determined to be arbitration passed or arbitration failed.

7. The apparatus according to claim 6, characterized in that, Each level of status checker is configured to: if any of the ports that conflict with the current port and have a higher weight have an arbitration status of arbitration passed, then the arbitration status of the current port is determined to be arbitration failed; If the arbitration status of ports that conflict with the current port and have a higher weight is all "arbitration failed", then the arbitration status of the current port will be determined as "arbitration passed". Otherwise, the arbitration status of the current port remains pending and is passed to the next level status checker.

8. The apparatus according to claim 1 or 2, characterized in that, The arbitration module is also configured to determine the arbitration result of the current port as failed when the subsequent stage indicates through a blocking signal that the request of the current port cannot be processed.

9. The apparatus according to claim 1 or 2, characterized in that, At least one of the initial value of the weight and the preset increment is a configurable value.

10. The apparatus according to claim 1 or 2, characterized in that, The N output ports are respectively connected to N storage banks of a shared static random access memory, and the device is used for shared memory access interconnection within the processing unit of an artificial intelligence accelerator chip.

11. A shared memory interconnect arbitration method, applied to the shared memory interconnect device as described in claim 1, characterized in that, The method includes: Obtain the access requests and their target storage identifiers for each input port during the current arbitration cycle. Iterate through and compare the target storage identifiers of each input port, and identify ports with the same target storage identifier as ports with conflicting information. Determining the initial state of a port includes: obtaining the weight of the port and determining the initial arbitration state of the current port, wherein the initial arbitration state is one of arbitration passed, arbitration failed, or pending. The arbitration status of each port is passed through a multi-level cascaded status checker. Each level of the status checker is configured as follows: if any port with a higher weight that conflicts with the current port has an arbitration status of "arbitration passed", then the arbitration status of the current port is determined to be "arbitration failed"; if all ports with a higher weight that conflict with the current port have an arbitration status of "arbitration failed", then the arbitration status of the current port is determined to be "arbitration passed"; otherwise, the pending status is passed to the next level status checker. The port whose arbitration status is "arbitration passed" at the last-level status checker is obtained as the final arbitration result; and Based on the final arbitration result, the port status is updated, including: for ports that fail arbitration, the weight is increased by a preset increment; for ports that pass arbitration, the weight is restored to the initial value for use in the next arbitration cycle.

12. The method according to claim 11, characterized in that, Determining the initial state of a port also includes: when a downstream stage indicates that a request on the current port cannot be processed via a blocking signal, the initial arbitration state of the current port is determined to be arbitration failed.

13. The method according to claim 11 or 12, characterized in that, At least one of the initial value of the weight and the preset increment is a configurable value.

14. The method according to claim 11 or 12, characterized in that, Determining the initial arbitration state of the current port includes: for ports with conflict, comparing the weight of the current port with the weights of other ports with conflict, and combining the subsequent ready signal to determine the initial arbitration state of the current port.

15. A shared memory interconnect arbitration circuit, applied to the shared memory interconnect device as described in claim 1, characterized in that, The arbitration circuit includes: The conflict detection circuit is used to obtain the access requests and their target memory identifiers of each input port in the current arbitration cycle, traverse and compare the target memory identifiers of each input port, and determine the ports with the same target memory identifier as ports with conflicts. An initial state determination circuit is used to obtain the weight of the port and determine the initial arbitration state of the current port, wherein the initial arbitration state is one of arbitration passed, arbitration failed, or pending. The state passing circuit consists of multiple cascaded state checkers. Each state checker is configured as follows: if any port with a higher weight that conflicts with the current port has an arbitration success status, then the arbitration status of the current port is determined to be arbitration failure; if all ports with higher weights that conflict with the current port have arbitration failure statuses, then the arbitration status of the current port is determined to be arbitration success; otherwise, the pending state is passed to the next level state checker; and The arbitration result management circuit is used to obtain the port with the arbitration status of "arbitration passed" at the last-level status checker as the final arbitration result; and to update the port status based on the final arbitration result, including: for ports that failed arbitration, increasing the weight by a preset increment; for ports that passed arbitration, restoring the weight to the initial value for use in the next arbitration cycle.

Images