Distributed flow control hardware, distributed flow control system, and artificial intelligence processor

Abstract

The application discloses a kind of distributed flow control hardware, distributed flow control system and AI processor.The distributed flow control hardware includes throughput control module, transaction request buffer and constant speed control module, including anti-fluctuation buffer in throughput control module;Throughput control module is used to decide in each clock cycle, the transaction request received in real time is sent to transaction request buffer or is sent to anti-fluctuation buffer and is temporarily stored, or, directly from anti-fluctuation buffer one transaction request stored is selected and sent to transaction request buffer;Constant speed control module is used to obtain transaction request from transaction request buffer in sequence, and according to the clock cycle interval corresponding to each transaction request respectively decided, each transaction request is sent out in sequence.The embodiment of the application solves the problem of traffic burst effectively by innovative distributed flow control design, and brings better peak capacity and business flow balance for AI processor.

Description

Technical Field

[0001] This invention relates to the field of flow control technology, and in particular to a distributed flow control hardware, a distributed flow control system, and an artificial intelligence (AI) processor. Background Technology

[0002] The modern AI processor architecture design trend is towards multiple computing cores. The working characteristics of neural networks also determine that if one or more AI processors want to simulate the way the human brain works, they must move towards an architecture that uses massive computing cores for parallel processing.

[0003] With the continuous advancement of semiconductor technology, the number of computing cores that an AI processor can accommodate is growing exponentially. Some design directions even involve dividing existing computing cores into smaller, parallelizable microcores. When these numerous computing cores work in parallel, they often need to interact with other computing cores, access shared data cache units, or exclusively access data caches. At the same time, the computing tasks deployed on each computing core are either homogeneous or heterogeneous.

[0004] In the process of developing this invention, the inventors discovered that in existing multi-core parallel operation modes, AI processors generate architecture-level bursts of data traffic. These bursts are more severe than those in traditional processors, and their duration is extremely short, making them difficult to effectively capture and statistically analyze. The impact of these issues on AI processor performance manifests as: drastic fluctuations in real-time performance during business operations, a lack of stable benchmarks for performance adjustments, and difficulty in achieving further optimization and improvement. Furthermore, when users run the same business flow, system performance is easily affected by factors such as network conditions, device thermal effects, and scheduling randomness, resulting in significant random variations, making optimization difficult and preventing the smooth release of the AI ​​processor's peak performance. Summary of the Invention

[0005] This invention provides a distributed flow control hardware, a distributed flow control system, and an AI processor, which effectively solves the problem of traffic bursts through innovative distributed flow control design, bringing better peak capacity and business flow balance to the AI ​​processor.

[0006] According to one aspect of the present invention, a distributed flow control hardware is provided, including a throughput control module, a transaction request buffer, and a rate control module. The throughput control module includes an anti-fluctuation buffer, and the transaction request buffer is connected to both the throughput control module and the rate control module.

[0007] The throughput control module is used to decide, in each clock cycle, whether to send the real-time received transaction request to the transaction request buffer or to the anti-ripple buffer for temporary storage, or to directly select a temporary transaction request from the anti-ripple buffer and send it to the transaction request buffer.

[0008] The constant speed control module is used to sequentially retrieve transaction requests from the transaction request buffer and send each transaction request out sequentially according to the determined clock cycle sending interval corresponding to each transaction request.

[0009] According to another aspect of the present invention, a distributed flow control system is also provided, including two distributed flow control hardware as described in any one of the embodiments of the present invention. One of the two distributed flow control hardware is used for distributed flow control of read transaction requests, and the other of the two distributed flow control hardware is used for distributed flow control of write transaction requests.

[0010] According to another aspect of the present invention, an AI processor is also provided, including: multiple data service initiators and multiple data service providers, wherein each data service initiator establishes a communication connection with each data service provider through an on-chip network on the AI ​​processor;

[0011] Specifically, at the business exit of each data service provider, a distributed flow control system as described in any one of the embodiments of the present invention is deployed.

[0012] The technical solution of this invention provides a novel distributed flow control hardware, including a throughput control module and a rate control module. The throughput control module can, in each clock cycle, decide whether to send real-time received transaction requests to a transaction request buffer or to an anti-fluctuation buffer for temporary storage, or directly select a temporarily stored transaction request from the anti-fluctuation buffer and send it to the transaction request buffer. The rate control module can sequentially retrieve transaction requests from the transaction request buffer and, according to the determined clock cycle sending interval corresponding to each transaction request, send each transaction request outward sequentially. Through the joint cooperation between the throughput control module and the rate control module, the distribution characteristics of data service flows can be fundamentally changed, effectively controlling the data flow distribution characteristics at confluence bottlenecks and minimizing the occurrence of self-similar bursty service characteristics. This achieves the technical effect of effectively solving the problem of traffic bursts, bringing better peak capacity and service flow balance to AI processors.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the structure of a distributed flow control hardware according to an embodiment of the present invention;

[0016] Figure 2 This is a schematic diagram of the structure of a throughput control module applicable to an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of another throughput control module applicable to embodiments of the present invention;

[0018] Figure 4 This is a schematic diagram of a working state machine applicable to the transaction request router in the throughput control module of this invention.

[0019] Figure 5 This is a schematic diagram of the structure of a constant speed control module applicable to an embodiment of the present invention;

[0020] Figure 6 This is a schematic diagram of another constant speed control module applicable to the embodiments of the present invention;

[0021] Figure 7 This is a schematic diagram illustrating the discretization effect of transaction requests with different numbers within a time window of a set size, as applied in an embodiment of the present invention.

[0022] Figure 8 This is an architecture diagram of a distributed flow control system provided according to an embodiment of the present invention;

[0023] Figure 9 This is a schematic diagram of another distributed flow control system provided according to an embodiment of the present invention;

[0024] Figure 10 This is a schematic diagram of a working state machine applicable to the constant-speed scheduling unit in the distributed flow control system of this invention.

[0025] Figure 11 This is a structural diagram of an AI processor provided according to an embodiment of the present invention;

[0026] Figure 12 This is a simulation verification table for single-port read bandwidth flow control based on the embodiments of the present invention;

[0027] Figure 13 This is a performance verification table of actual simulated bandwidth under the configuration of a distributed flow control system when 6 data service initiators are running in parallel, based on an embodiment of the present invention.

[0028] Figure 14 This is a schematic diagram illustrating the impact of parallel data throughput flow control on balance and peak values ​​when performing parallel data throughput flow control on 80 computing cores based on an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Figure 1 This is a schematic diagram of a distributed flow control hardware provided in an embodiment of the present invention. This embodiment can be applied to the situation where a sudden surge in data traffic is generated by a data service initiator, and precise flow control is performed on the aforementioned sudden data traffic.

[0032] like Figure 1 As shown, the distributed flow control hardware may include: a throughput control module 110, a transaction request buffer 120, and a constant-rate control module 130. The throughput control module 110 includes an anti-fluctuation buffer 1101. The transaction request buffer 120 is connected to both the throughput control module 110 and the constant-rate control module 130.

[0033] The throughput control module 110 is used to decide, in each clock cycle, whether to send the real-time received transaction request to the transaction request buffer 120 or to the anti-fluctuation buffer 1101 for temporary storage, or to directly select a temporary transaction request from the anti-fluctuation buffer 1101 and send it to the transaction request buffer 120.

[0034] In this embodiment, the distributed flow control hardware can be connected to a data service initiator (MST) at one end and to a communication network at the other end. The data service initiator can be understood as the initiator of a transaction request (also called a Transactions or TRS). For example, the data service initiator can be a computing core in a multi-core AI processor. The communication network can be understood as a transmission channel for transaction requests, used to send each transaction request from multiple data service initiators to multiple matching receivers (also called data service providers or SLVs). Continuing the previous example, for this multi-core AI processor scenario, the aforementioned communication network can be a NOC (Network-on-Chip) on the multi-core AI processor.

[0035] Furthermore, a transaction request can be understood as a logical request sent by the data service initiator to the matching receiver to initiate a read or write access operation, thereby starting a complete transmission transaction. Generally, this transaction request can include read transaction requests and write transaction requests.

[0036] In this system, after receiving the transaction request sent in real time by the connected data service initiator through the internal throughput control module 110, the distributed flow control hardware does not directly send the transaction request outward. Instead, it first controls the flow rate within the distributed flow control hardware.

[0037] Specifically, the throughput control module 110 triggers a judgment logic once per clock cycle. Under this judgment logic, it may choose to send the real-time received transaction requests to the transaction request buffer 120 so that the constant-rate control module 130 can passively obtain transaction requests from the transaction request buffer 120 and send them; or, it may choose to temporarily store the real-time received transaction requests in the anti-fluctuation buffer 1101 within the module and obtain a temporarily stored transaction request from the anti-fluctuation buffer 1101 to send to the transaction request buffer 120; or, it may choose to only temporarily store the real-time received transaction requests and not send any transaction requests to the transaction request buffer 120 within this clock cycle.

[0038] It is understandable that by implementing the method of sending or not sending transaction requests to the transaction request buffer 120 in each clock cycle, the size of the traffic data sent by the data service initiator can be effectively controlled.

[0039] Furthermore, the technical solution of this invention creatively introduces an anti-fluctuation buffer 1101, which can effectively combat random fluctuations from upstream and downstream (i.e., the upstream data service initiator and the downstream communication network). The fluctuations from upstream are mainly the suddenness of transaction requests sent by the data service initiator, while the fluctuations from downstream are mainly random system backpressure, preventing the distributed flow control hardware from sending transaction requests outwards.

[0040] By adding an anti-fluctuation buffer 1101 to the distributed flow control hardware, backpressure can be absorbed within the distributed flow control hardware instead of the entire transmission line when downstream backpressure occurs. This ensures that the data service initiator can utilize the transmission link at full bandwidth without interrupting the incomplete pipeline from the data service initiator to the distributed flow control hardware. Furthermore, the incomplete pipeline of the data service initiator will not be continuously interrupted by the direct transmission of random backpressure from downstream of the distributed flow control hardware, preventing the data service initiator from constantly operating in a state of random stuttering.

[0041] The transaction request buffer 120 is used to buffer transaction requests from the throughput control module 110 to the rate control module 130. Its main function is to isolate the active / passive switching between the two processing stages executed by the throughput control module 110 and the rate control module 130, respectively. That is, the processing stage executed by the throughput control module 110 actively chooses whether to send transaction requests to the transaction request buffer 120, while the processing stage executed by the rate control module 130 passively retrieves transaction requests from the transaction request buffer 120 and sends the transaction requests externally at an appropriate time.

[0042] In this embodiment, the cache capacity of the transaction request cache 120 can be adjusted according to the size of the system resources. For example, the cache capacity can be set to be greater than the maximum burst length of a transaction request.

[0043] The constant speed control module 130 is used to sequentially obtain transaction requests from the transaction request buffer 120 and send each transaction request out sequentially according to the determined clock cycle sending interval corresponding to each transaction request.

[0044] In this embodiment, the main implementation method of the constant speed control module 130 is to determine a matching clock cycle transmission interval for each transaction request, and send each transaction request sequentially according to the aforementioned clock cycle transmission interval. The goal of the above implementation method is to effectively shape the traffic data sent by the throughput control module 110.

[0045] The inventors discovered in their research that as the number of computing cores in AI processors gradually increases, the number of data streams also increases. Therefore, to achieve stable system-level flow control, the relationship between multiple data streams becomes crucial. Practical research shows that system bursts are unavoidable without flow shaping. Therefore, this invention proposes for the first time a flow shaping design based on a constant-speed control module 130. By shaping the data streams, the original distribution characteristics of the data streams can be altered to avoid bursts in single data streams and multiple parallel data services.

[0046] It should be noted that in the distributed flow control hardware of this invention embodiment, the throughput control module 110 may include multiple rate control units, which are used to perform independent flow control on different types of transaction requests according to their respective independent flow control parameters, so as to achieve the effect of distributed flow control.

[0047] In this embodiment, Figure 1 The arrow pointing from the throughput control module 110 to the transaction request cache 120 describes the action of the throughput control module 110 actively sending a transaction request to the transaction request cache 120. The arrow pointing from the rate control module 130 to the transaction request cache 120 describes the action of the rate control module 130 actively retrieving a transaction request from the transaction request cache 120.

[0048] The technical solution of this invention provides a novel distributed flow control hardware, including a throughput control module and a rate control module. The throughput control module can, in each clock cycle, decide whether to send real-time received transaction requests to a transaction request buffer or to an anti-fluctuation buffer for temporary storage, or directly select a temporarily stored transaction request from the anti-fluctuation buffer and send it to the transaction request buffer. The rate control module can sequentially retrieve transaction requests from the transaction request buffer and, according to the determined clock cycle sending interval corresponding to each transaction request, send each transaction request outward sequentially. Through the joint cooperation between the throughput control module and the rate control module, the distribution characteristics of data service flows can be fundamentally changed, effectively controlling the data flow distribution characteristics at confluence bottlenecks and minimizing the occurrence of self-similar bursty service characteristics. This achieves the technical effect of effectively solving the problem of traffic bursts, bringing better peak capacity and service flow balance to AI processors.

[0049] Furthermore, in Figure 2 The diagram shows a schematic representation of a throughput control module applicable to an embodiment of the present invention. Figure 2As shown, the throughput control module specifically includes multiple rate control units 210, a transaction request router 220, and multiple anti-fluctuation buffer units 230 corresponding to each rate control unit 210. Each anti-fluctuation buffer unit 230 forms the aforementioned anti-fluctuation buffer. The transaction request router 220 is connected to each rate control unit 210 and each anti-fluctuation buffer unit 230 respectively.

[0050] The multiple rate control units 210 include: rate control unit 1, rate control unit 2, ..., rate control unit N, and the multiple anti-fluctuation buffer units 230 include: anti-fluctuation buffer unit 1, anti-fluctuation buffer unit 2, ..., anti-fluctuation buffer unit N.

[0051] Wherein: Each rate control unit 210 is used to determine in real time whether the transaction request of the control type of this unit is allowed or not allowed to be sent, based on the sending ratio of the transaction request of the control type of this unit within the preset time window configured in the unit register 2101 and the number of transaction requests of the control type of this unit to be sent in the current time window, and send a matching allow or disallow send signal.

[0052] One of the rate control units 210 includes one rate control unit 2101. Specifically, rate control unit 1 includes unit register 1, rate control unit 2 includes unit register 2, ..., and rate control unit N includes unit register N.

[0053] In this embodiment, each rate control unit 210 is configured to receive a specific type of transaction request (i.e., Figure 2 Each type of transaction request (TRS) performs flow control independently. The types of these transaction requests can include: register read / write transaction requests, data cache read / write transaction requests, and configuration space read / write transaction requests, etc.

[0054] Optionally, the rate control unit 210 can identify the type of a transaction request by using a status identifier carried in the transaction request. Alternatively, it can also identify the type of a transaction request based on the address range where the request address of a transaction request is located.

[0055] Each rate control unit 210 has one or more unit registers 2101. Within each unit register 2101, the transmission ratio of transaction requests of the corresponding control type within a preset time window is configured. This preset time window can be understood as a unit time window for traffic statistics and traffic shaping. This time window can be preset or pre-determined according to the actual hardware environment, for example, it can be 16 clock cycles or 32 clock cycles. The control type of this unit can be understood as the type of transaction request for which a rate control unit 210 can perform traffic control.

[0056] Understandably, the time window cannot be configured too large. If the window is too large, changes cannot be detected in time, and the system cannot keep up with the speed of changes in the business flow, causing adjustments to degrade system performance. At the same time, the time window cannot be configured too small. If the window is too small, there is no fine adjustment step size, and the space for discretization is also lost. Therefore, those skilled in the art need to scientifically select the time window based on actual needs.

[0057] In this embodiment, a status identifier parsing component and a status identifier broadcasting component can be set in the throughput control module. After the status identifier in the input transaction request A is parsed by the status identifier parsing component, the status identifier is broadcast to each rate control unit 210 via the status identifier broadcasting component. Based on the unit register 2101 set in each rate control unit 210, it is identified whether the type of transaction request A is the pre-configured unit control type in that rate control unit 210.

[0058] Alternatively, an address extraction component and an address broadcasting module can be set in the throughput control module. After the address extraction component identifies the request address of the input transaction request B, the address broadcasting module broadcasts the request address to each rate control unit 210. Based on the unit register 2101 set in each rate control unit 210, it is identified whether the type of transaction request B is the control type of this unit pre-configured in each rate control unit 210.

[0059] In this embodiment, since the request types of the real-time received transaction requests can be distributed to each rate control unit 210 in real time, and combined with the current time window of the current system time and the types of transaction requests sent to the transaction request buffer by the transaction request router 220 in real time, each rate control unit 210 can count the number of transaction requests to be sent that match the control type of its unit within the current time window through one or more internal unit registers 2101, and compare the number of requests to be sent with the sending ratio of the transaction requests of its unit control type within its own preset time window. If it is found that the number of requests to be sent of a specific type of transaction request does not exceed the sending ratio allocated to the transaction request of that type, the rate control unit 210 will decide the allowed sending status of the transaction requests of its unit control type, and then send an allowed sending signal (on) to the transaction request router 220; otherwise, it will decide the disallowed sending status of the transaction requests of its unit control type, and then send a disallowed sending signal (off) to the transaction request router 220.

[0060] Each anti-fluctuation buffer unit 230 is used to temporarily store transaction requests of a specified type that are not currently allowed to be sent.

[0061] In this embodiment, a one-to-one correspondence can be established between each rate control unit 210 and each anti-fluctuation buffer unit 230. Furthermore, each anti-fluctuation buffer unit 230 can be used to cache transaction requests of the same control type controlled by the rate control unit 210 that matches that anti-fluctuation buffer unit 230. That is, when a type of transaction request is no longer allowed to be sent within the current time window, if one or more transaction requests of that type are received again within the current time window, these one or more transaction requests will be temporarily stored in the matching anti-fluctuation buffer unit 230.

[0062] Alternatively, the number of rate control units can be set to one more than the number of anti-fluctuation buffer units. Figure 2 (Not shown in the diagram), and in an additional rate control unit, important types of transaction requests are configured. The rate control unit corresponding to this type of important transaction request will continuously send an on signal to the transaction request router 220. That is to say, whenever an important type of transaction request is received, this transaction request will no longer be subject to flow control, or temporarily stored in the corresponding anti-fluctuation buffer unit, but will be directly sent to the transaction request buffer via the transaction request router 220 so that the rate control module can send it out as soon as possible.

[0063] The transaction request router 220 is used to decide, in each clock cycle, whether to send the real-time received transaction request to the transaction request buffer or to the anti-ripple buffer for temporary storage, or to directly select a temporarily stored transaction request from the anti-ripple buffer and send it to the transaction request buffer, based on the allow or disallow signal sent in real time by each rate control unit 210.

[0064] In this embodiment, a state machine system with preset logic can be configured in the transaction request router 220. The state machine system can, in each clock cycle, determine whether to send the real-time received transaction request to the transaction request buffer, or send the real-time received transaction request to the anti-fluctuation buffer for temporary storage, or directly select a temporarily stored transaction request from the anti-fluctuation buffer and send it to the transaction request buffer, based on the real-time allow or disallow transmission signals sent by each rate control unit 210, whether there is a transaction request to be sent in the current clock cycle, and the buffering status of each anti-fluctuation buffer unit 230 for transaction requests.

[0065] Based on the above embodiments, the transaction request router can also be used for:

[0066] After sending a transaction request to the transaction request buffer, a completion signal (done) is sent back to the rate control unit that matches the type of the transaction request, so that the rate control unit can update the number of transaction requests of the type controlled by this unit to be sent within the current time window.

[0067] Understandably, the rate control unit is mainly used to control the number (or proportion) of each type of transaction request sent in each time window. Transaction requests for flow control are temporarily stored and sent in subsequent time windows, while transaction requests that are allowed to be sent are sent to the transaction request buffer via the transaction request router, so that the rate control module can send the transaction request out at an appropriate time in the current time window.

[0068] Accordingly, whenever the transaction request router sends a transaction request C to the transaction request buffer, it indicates that the transaction request C is a transaction request to be sent in the current time window. Then, the transaction request router will generate a sending completion signal and feed the sending completion signal back to the unit register in the rate control unit that matches the type x of the transaction request C, so as to update the number of transaction requests of type x to be sent in the current time window in the unit register.

[0069] Furthermore, in Figure 3 The diagram shows a structural schematic of another throughput control module applicable to an embodiment of the present invention. Figure 3As shown, the throughput control module also includes an address extraction component 310 and a first address broadcasting component 320, with the address extraction component 310 connected to the first address broadcasting component 320 and the transaction request router, respectively.

[0070] Address extraction component 310 is used to extract the request address (i.e., TRS address) from the real-time received transaction request (i.e., TRS), and send the request address to the first address broadcast component 320 while forwarding the transaction request to the transaction request router.

[0071] The first address broadcasting unit 320 is used to broadcast the received request address to each rate control unit, wherein the transaction requests of different request address ranges have different types.

[0072] In this embodiment, considering that the type of a transaction request can be clearly determined by the range of request addresses of a transaction request, it is considered to introduce an address extraction component 310 and a first address broadcasting component 320 to distribute the request addresses of the real-time received transaction requests to each rate control unit (i.e., rate control unit 1, rate control unit 2, ..., rate control unit N, rate control unit N+1), so that each rate control unit can determine whether the type of the transaction request matches the control type of its own unit based on the request address, and accordingly decide whether to send an allow signal or a disallow signal to the transaction request router.

[0073] exist Figure 3 In this configuration, the total number of rate control units exceeds the total number of anti-fluctuation buffer units by one. This is because the control type corresponding to the additional rate control unit (N+1) is an important transaction request type within a specific address range. Accordingly, the rate control unit (N+1) only needs to continuously send a transmission permission signal to the transaction request router. When the transaction request router receives such a transaction request, it will not temporarily store the request but will instead send it directly to the transaction request buffer.

[0074] In an optional implementation of this embodiment, the transaction request router can specifically be used for:

[0075] If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type is currently sending a transmit permission signal, and the target anti-fluctuation buffer unit corresponding to the target rate control unit is currently empty, then the target transaction request is sent to the transaction request buffer.

[0076] If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type sends a signal to allow transmission, and the target anti-fluctuation buffer unit corresponding to the target rate control unit is not empty, then a transaction request is sequentially obtained from the target anti-fluctuation buffer unit and sent to the transaction request buffer, and the target transaction request is temporarily stored in the target anti-fluctuation buffer unit.

[0077] If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type is currently sending a signal that it is not allowed to send, and the alternative transaction requests in other rate control units polled meet the sending conditions, then the alternative transaction requests are sent to the transaction request buffer, and the target transaction request is temporarily stored in the target anti-fluctuation buffer unit.

[0078] If a target transaction request of the target type is received in the current clock cycle, and the target rate control unit corresponding to the target type is currently sending a signal that it is not allowed to send and has not polled for a candidate transaction request that meets the sending conditions, then the target transaction request will only be temporarily stored in the target anti-fluctuation buffer unit.

[0079] If no transaction request is received in the current clock cycle, and alternative transaction requests in other rate control units meet the sending conditions, then the alternative transaction request is sent to the transaction request buffer.

[0080] If no transaction request is received in the current clock cycle, and no alternative transaction request that meets the sending conditions is found in the poll, then no operation is performed.

[0081] In the above implementation, a state machine mode for the transaction request router is defined. Through the above state settings, the transaction request router can make a clear execution action in each clock cycle. Using the anti-fluctuation buffer unit as a temporary storage medium, each transaction request is provided to the transaction request buffer in an orderly manner according to the preset flow control target for preparation for sending.

[0082] Furthermore, in Figure 4 The diagram illustrates a working state machine applicable to the transaction request router in the throughput control module of an embodiment of the present invention. Figure 4 As shown, the transaction request router relies on, for example, Figure 2 or Figure 3 The hardware architecture of the throughput control module is based on the built-in state machine logic. It achieves flow control of TRS by switching between eight states, ST01-ST08.

[0083] The workflow of a transaction request router can be mainly divided into the following types:

[0084] 1. ST01->ST03->ST01

[0085] In the current clock cycle, execution begins from ST01. If the current input.TRS is available, and the rate control unit (RC-X) corresponding to that input.TRS currently sends a transmit enable signal (RC-X.on), and the anti-ripple buffer unit (FRB-X) corresponding to the RC-X does not currently store any TRS (FRB-X.candidate TRS not available), then execution directly proceeds to ST03. That is, the input.TRS is directly forwarded to the transaction request buffer (TB module), and then the process returns to ST01. Here, FRB-X.candidate TRS can be understood as the next TRS to be sent in FRB-X.

[0086] 2. ST01->ST02->ST04->ST07->ST05->ST01

[0087] In the current clock cycle, execution begins at ST01. If the rate control unit (RC-X) corresponding to the current input TRS currently sends a transmit enable signal (RC-X.on) and the anti-ripple buffer unit (FRB-X) corresponding to RC-X stores at least one temporary TRS (FRB-X.candidate TRS available), then the process first proceeds to ST02 to forward the temporary TRS in FRB-X to the transaction request buffer. Afterwards, if it is determined that FRB-X is not empty, the process proceeds to ST04 to retrieve the first TRS in the FRB-X queue. Furthermore, if it is detected that the current input.TRS is in an available state, the process flows to ST07 to push the input.TRS into the tail of the FRB-X queue. When it is detected that RC-X is currently sending RC-X.on, the process flows to ST05 to fill the first TRS in the FRB-X queue into the candidate TRS position in FRB-X, update to obtain a new FRB-X.candidate TRS, and return to ST01.

[0088] 3. ST01->ST07->ST06->ST01

[0089] In the current clock cycle, execution begins at ST01. If the rate control unit (RC-X) corresponding to the current input.TRS is currently sending a disallowed signal (RC-X.off), and the current input.TRS is available, then the process proceeds to ST07. The input.TRS is pushed to the tail of the FRB-X queue. Since RC-X is currently sending RC-X.off, the process directly proceeds to ST06. Following a pre-defined polling mechanism, a new valid anti-ripple buffer unit is found among all candidate TRS (valid FRB.candidate TRS) of valid anti-ripple buffer units that temporarily store TRS. The current X is updated using the candidate unit's identifier (id) information, and the updated RC-X is checked to see if it sends RC-X.on. If not, the process returns directly to ST01 and continues to wait for or process the next transaction request.

[0090] 4. ST01->ST07->ST06->ST01->Workflow x

[0091] The state transition process from ST01 to ST07 to ST06 to ST01 is partly the same as that in workflow 3 above, and will not be repeated here. Furthermore, if in state ST01, it is determined that the updated RC-X is sent as RC-X.on, then based on this RC-X.on, the subsequent state machine transitions continue.

[0092] 5. ST01->ST08->ST01

[0093] If no new input.TRS is found in the current clock cycle (i.e., in the input.TRS not available state), the process flows to ST08, where a search is performed on the candidate TRS (FRB-X.candidate TRS) of all anti-ripple buffer units according to the preset polling mechanism. If the queue of an anti-ripple buffer unit is not empty but not filled with FRB.candidate TRS, the corresponding TRS filling operation needs to be performed. After the polling is completed, if no FRB-X with candidate TRS is found, i.e., FRB-X.candidate TRS not available, the process returns directly to ST01 and continues to wait for or process the next transaction request.

[0094] 6. ST01->ST08->ST02->ST04->ST05->ST01

[0095] If there is no new input.TRS in the current clock cycle (i.e., it is in the input.TRS not available state), the process will proceed to ST08, where the process will search for candidate TRS (FRB. candidate TRS) in all anti-ripple buffer units according to the preset polling mechanism (ronudrobin). If the RC-X corresponding to FRB-X is finally polled and sent as RC-X.on, and the FRB-X. candidate TRS is available, the process will proceed to ST02 and continue with the subsequent operations.

[0096] In fact, through this mechanism, the various embodiments of the present invention achieve the greatest possible flow control and smooth control of upstream and downstream data flow.

[0097] Furthermore, in Figure 5 The diagram shows a structural schematic of a constant speed control module applicable to an embodiment of the present invention. Figure 5 As shown, the constant speed control module specifically includes a transaction request scheduling unit 510, multiple interval control units 520, and a constant speed scheduling unit 530; the transaction request scheduling unit 510 is connected to the transaction request buffer and the constant speed scheduling unit 530 respectively, and the constant speed scheduling unit 530 is connected to each interval control unit 520 respectively.

[0098] The multiple interval control units 520 include: interval control unit 1, interval control unit 2, ..., interval control unit N.

[0099] The transaction request scheduling unit 510 is used to prefetch a new transaction request from the transaction request buffer after determining that the fixed-speed scheduling unit 530 has completed sending a transaction request, and send the cached transaction request stored in the unit to the fixed-speed scheduling unit 530.

[0100] In this embodiment, the transaction request scheduling unit 510 is directly interconnected with the transaction request cache and retrieves each transaction request sequentially from the transaction request cache by actively pulling them. Specifically, the fixed-rate scheduling unit 530 can actively send a sending completion response signal, such as an issue done signal, to the transaction request scheduling unit 510 after actually completing the outward transmission of each transaction request. When the transaction request scheduling unit 510 receives the issue done signal, it determines that the fixed-rate scheduling unit 530 has completed the transmission of a transaction request.

[0101] At this time, the transaction request scheduling unit 510 directly sends a cached transaction request (TRS) currently cached within its unit to the fixed-rate scheduling unit 530, and simultaneously prefetches a new transaction request from the transaction request cache to continue caching within the unit, so as to improve the sending efficiency of continuous transaction requests.

[0102] Each interval control unit 520 is used to calculate the target clock cycle transmission interval of the current cached transaction request based on the information configured in the unit register 5201 when it is determined that the type of the current cached transaction request is consistent with the control type of this unit, and then send it to the fixed speed scheduling unit 530.

[0103] One of the interval control units 520 includes a unit register 5201, that is, the interval control unit 1 includes unit register 1, the interval control unit 2 includes unit register 2, ..., the interval control unit N includes unit register N.

[0104] In this embodiment, the number of interval control units 520 matches the number of rate control units. Furthermore, each interval control unit 520 also has a unit control type. It is used to determine the target clock cycle transmission interval (i.e., ...) between the current cached transaction request and the most recently sent transaction request of the unit control type, based on information configured in the internal unit register 5201, such as the number of clock cycles contained in the time window, the interface clock frequency of the transaction request output interface, and a preset flow control bandwidth target. Figure 5 The `trs_interval_set` parameter in the `trs_interval_set` parameter allows for the synchronization of determining which of the pending transaction requests within the current time window the current cached transaction request belongs to. Figure 5 (not shown in the image), and sends the above information to the speed control unit 530.

[0105] The constant-speed scheduling unit 530 is used to determine the transmission timing that matches the current cached transaction request based on the transmission interval of the received target clock cycle, and to send the current cached transaction request outward at the transmission timing.

[0106] In an optional implementation of this embodiment, after receiving the target clock cycle transmission interval, the constant speed scheduling unit 530 can combine the number of transaction requests that have been successfully sent in the current time window with the amount of debt caused by the back pressure of the distributed flow control hardware in the actual network environment, and finally decide on a future clock cycle. After the current clock cycle reaches this decision clock cycle, the current cached transaction request will be sent out.

[0107] Furthermore, in Figure 6 The diagram shows a structural schematic of another constant speed control module applicable to an embodiment of the present invention. Figure 6As shown, the constant speed control module also includes a second address broadcasting unit 610, a transaction request counting unit 620, a debt statistics unit 630, and a transaction request output interface 640. The second address broadcasting unit 610 is connected to the transaction request scheduling unit, each interval control unit, the transaction request counting unit 620, and the debt statistics unit 630, respectively. The transaction request output interface 640 is connected to the constant speed scheduling unit, the transaction request counting unit 620, and the debt statistics unit 630, respectively.

[0108] The transaction request scheduling unit is also used to send the request address of the current cached transaction request to the second address broadcast unit 610.

[0109] The second address broadcasting unit 610 is used to broadcast the received request address to each interval control unit, wherein the transaction requests of different request address ranges have different types.

[0110] The transaction request output interface 640 is used to receive transaction requests sent by the fixed-speed scheduling unit at each sending time, and to send each transaction request outward.

[0111] Transaction request counting unit 620 is used to perform statistical calculations on each transaction request successfully sent by transaction request output interface 640, and send the calculated actual clock cycle interval (i.e., Figure 6 The trs_interval_cnt in the data is sent to each interval control unit.

[0112] Specifically, the transaction request counting unit 620 is a statistical unit that monitors the successful transmission of TRS by the transaction request output interface 640. It feeds back the actual transmission interval between TRSs, expressed as clock cycles, to the constant-speed scheduling unit via each interval control unit, so that the constant-speed scheduling unit can make decisions on the timing of transmission. For example, whenever the transaction request output interface 640 successfully transmits a transaction request, it generates a corresponding "issued done" signal, which is provided to the transaction request counting unit 620 to statistically analyze the timing value of the successfully transmitted transaction request.

[0113] The debt statistics unit 630 is used to monitor the backpressure situation of the transaction request output interface 640 and to count the debt count caused by the backpressure (i.e., Figure 6 The trs_debt_cnt in the interval control unit sends the debt count value to each interval control unit.

[0114] Specifically, the debt statistics unit 630 is a unit that monitors the back pressure of the transaction request output interface 640 and records the back pressure causing TRS transmission debt. It calculates the debt status (i.e., the count of outstanding transaction requests, or debt count) using the flow control bandwidth target set in its internal register, and feeds this debt count back to the rate scheduling unit via each interval control unit for decision-making on transmission timing. For example, whenever the transaction request output interface 640 detects back pressure, it generates a corresponding back pressure signal and provides it to the debt statistics unit 630 for debt count calculation.

[0115] Each interval control unit is further used to send the calculated target clock cycle transmission interval of the current cached transaction request, as well as the received actual clock cycle transmission interval and debt count value, to the constant speed scheduling unit.

[0116] The constant-speed scheduling unit is further used to determine the transmission timing that matches the current cached transaction request based on the received target clock cycle transmission interval, actual clock cycle transmission interval, and debt count value.

[0117] Based on the above embodiments, each interval control unit can be specifically used for:

[0118] The target clock cycle sending interval for the current cached transaction request is calculated based on the request number of the current cached transaction request within the current time window, the number of clock cycles contained in a time window configured in the unit register, the interface clock frequency of the transaction request output interface, and the preset flow control bandwidth target.

[0119] Alternatively, it can be based on the formula:

[0120]

[0121] Calculate the target clock cycle sending interval trs_interval_set for the current cached transaction request.

[0122] Where trsIdx is the request number of the current cached transaction request within the current time window, reg.calWin is the number of clock cycles contained in a time window configured in the unit register, for example, reg.calWin can be a value such as 16 or 32. if.freg is the interface clock frequency of the transaction request output interface, reg.bwRatio is the preset flow control bandwidth target (for example, it can be a specific bandwidth value, or bandwidth utilization, etc.), and trunc(.) is the truncation function to zero.

[0123] In a specific example, taking reg.calWin=32 as an example, in Figure 7The diagram illustrates the discretization effect of transaction requests with different numbers within a time window of a set size, as described in an embodiment of the present invention. Figure 7 As shown, the time window size is fixed at 32, and trsNum is the total number of transaction requests to be sent within a time window. Figure 7 The black areas represent the time slots for sending transaction requests, while the white areas represent the idle time slots where no transaction requests are sent.

[0124] Based on the above embodiments, the constant speed scheduling unit can specifically be used for:

[0125] After obtaining the current cache transaction request, if it is detected that the actual clock cycle sending interval is greater than or equal to the target clock cycle sending interval, the current clock cycle is directly determined as the sending time that matches the current cache transaction request.

[0126] After obtaining the current cached transaction request, if it is detected that the actual clock cycle sending interval is less than the target clock cycle sending interval, the debt count value is greater than 0, and the actual clock cycle sending interval is greater than the preset condition relaxation interval, then the current clock cycle is directly determined as the sending time matching the current cached transaction request. The condition relaxation interval is calculated based on the target clock cycle sending interval. For example, the condition relaxation interval can be 1 / 2 or 1 / 3 of the target clock cycle sending interval, etc., and this embodiment does not impose any limitations on this.

[0127] After obtaining the current cached transaction request, if it is detected that the actual clock cycle sending interval is less than the target clock cycle sending interval and the debt count value is equal to 0, then wait for the next clock cycle to continue the judgment.

[0128] Figure 8 This is an architecture diagram of a distributed flow control system provided according to an embodiment of the present invention. Figure 8 As shown, the distributed flow control system includes two distributed flow control hardware components as described in any one of the embodiments of the present invention. One of the two distributed flow control hardware components, namely the first distributed flow control hardware 810, is used to handle read transaction requests ( Figure 8 Distributed flow control is performed on the read TRS in the middle. The other distributed flow control hardware in the two distributed flow control hardware, namely the second distributed flow control hardware 820, is used to control write transaction requests ( Figure 8 Distributed flow control is performed using TRS (Telematics Streaming System).

[0129] As mentioned earlier, one end of the distributed flow control hardware is connected to the data service initiator, and the other end is connected to the communication network. It is used to perform flow control on the transaction requests from the data service initiator before sending them to the data service provider via the communication network. Accordingly, the data service initiator may send both read and write transaction requests. To achieve more granular flow control, the embodiments of this invention consider performing independent flow control on read and write transaction requests respectively.

[0130] Furthermore, a distributed flow control system (DFCU) consisting of two distributed flow control hardware components can be constructed. This distributed flow control system is used to receive read transaction requests and write transaction requests sent by data service initiators through different data interfaces, and to control the read transaction requests and write transaction requests through two independent distributed flow control hardware components.

[0131] It is understandable that the two distributed flow control hardware devices in a distributed flow control system may not interact with each other and may work independently. Alternatively, a coordinating unit may establish communication connections with the two distributed flow control hardware devices respectively. In this way, when read or write transaction requests cannot fully utilize the bandwidth allocated to them, the freed-up bandwidth can be used to send another type of transaction request, thereby achieving joint control of read and write traffic.

[0132] Correspondingly, in Figure 9 The diagram shows a schematic representation of another distributed flow control system provided by an embodiment of the present invention. Figure 9 As shown, the distributed flow control system also includes a read / write coordinating and scheduling unit 910, which is connected to two distributed flow control hardware devices (i.e., the first distributed flow control hardware and the second distributed flow control hardware).

[0133] The read / write coordination and scheduling unit 910 is used to perform coordinated flow control on read transaction requests and write transaction requests.

[0134] In a specific example, when the first distributed flow control hardware needs more bandwidth to send read TRS, it can send a bandwidth request (or, more specifically, an additional transmission permission request) to the read / write coordinating scheduling unit 910, for example, an "ask" signal. The read / write coordinating scheduling unit 910 then sends a signal indicating whether the bandwidth request is permitted (e.g., a "permit" signal) based on the current bandwidth usage of the second distributed flow control hardware. Simultaneously, when the first distributed flow control hardware does not need to request bandwidth from the read / write coordinating scheduling unit 910, it continuously sends an idle signal, for example, an "idle" signal, to the write coordinating scheduling unit 910.

[0135] Specifically, when the read / write coordinating and scheduling unit 910 allows bandwidth requests, it can send a successful request signal (e.g., a permit OK signal) to the first distributed flow control hardware. When the read / write coordinating and scheduling unit 910 does not allow bandwidth requests, it can send a failed request signal (e.g., a permit No signal) to the first distributed flow control hardware.

[0136] Based on the above embodiments, the read / write coordinating scheduling unit 910 is connected to the constant-speed scheduling unit (not shown in the figure) in two distributed flow control hardware respectively.

[0137] Accordingly, the constant speed scheduling unit can be further used for:

[0138] After obtaining the current cache transaction request, if it is detected that the actual clock cycle sending interval is less than the target clock cycle sending interval and the debt count value is equal to 0, an application request for additional sending permission is sent to the read-write coordinating scheduling unit. If an additional sending permission is obtained, the current clock cycle is directly determined as the sending time that matches the current cache transaction request; otherwise, the process waits for the next clock cycle to continue the judgment.

[0139] The read / write coordinating scheduling unit 910 is specifically used to detect the bandwidth utilization of another distributed flow control hardware when it receives an application request for additional transmission permission from a fixed-rate scheduling unit in a distributed flow control hardware. If it is determined that the other distributed flow control hardware still has spare bandwidth, a successful application response is generated as feedback result for the application request.

[0140] Specifically, in Figure 10 The diagram illustrates a working state machine applicable to the constant-speed scheduling unit in the distributed flow control system of this invention. Figure 10 As shown, this constant-speed scheduling unit is mainly used to implement flow shaping. The flow shaping workflow is mainly divided into the following types:

[0141] 1. ST01->ST02->ST03->ST01

[0142] In the current clock cycle, execution starts from ST01. If a current cache transaction request (newTRS) is received, the process flows to ST02 to perform a comparison operation of the sending interval. When it is determined that the sending interval of the actual clock cycle is greater than or equal to the sending interval of the target clock cycle corresponding to the current cache transaction request, that is, trs_interval_cnt>=trs_interval_set, the process flows to ST03 to perform the sending operation for the current cache transaction request, and then directly returns to ST01.

[0143] 2. ST01->ST02->ST04->ST03->ST01

[0144] At the current clock cycle, start execution from ST01. If a current cache transaction request (newTRS) is received, then transfer to ST02 and perform the comparison operation of the sending interval. When it is determined that trs_interval_cnt < trs_interval_set and the debt count value is greater than 0, that is, trs_debt_cnt > 0, then transfer to ST04 and perform the debt repayment operation. If it is detected in the ST04 state that the actual clock cycle sending interval is greater than or equal to the preset condition relaxation interval ( Figure 10 in which the condition relaxation interval is set to half of the target clock cycle sending interval), that is, trs_interval_cnt >= trs_interval_set / 2, then transfer to ST03, perform the sending operation for the current cache transaction request, and then directly return to ST01.

[0145] 3. ST01 -> ST02 -> ST04 -> ST05 -> ST03 -> ST01

[0146] At the current clock cycle, start execution from ST01. If a current cache transaction request (newTRS) is received, then transfer to ST02 and perform the comparison operation of the sending interval. When it is determined that trs_interval_cnt < trs_interval_set and trs_debt_cnt > 0, then transfer to ST04 and perform the debt repayment operation. If it is detected in the ST04 state that trs_interval_cnt < trs_interval_set / 2, then transfer to ST05 and perform the application request for an additional sending permission to the read-write coordination scheduling unit. After the read-write coordination scheduling unit feedbacks permit OK, transfer to ST03 to continue execution.

[0147] 4. ST01 -> ST02 -> ST04 -> ST05 -> ST02

[0148] In the current clock cycle, start execution from ST01. If a current cache transaction request (newTRS) is received, then transfer to ST02 to perform the comparison operation of the transmission interval. When it is determined that trs_interval_cnt < trs_interval_set and trs_debt_cnt > 0, then transfer to ST04 to perform the debt repayment operation. If it is detected that trs_interval_cnt < trs_interval_set / 2 in the ST04 state, then transfer to ST05 to send an application request for an additional transmission permit to the read-write co-management scheduling unit. After the read-write co-management scheduling unit feedbacks permit No, transfer to ST02 and re-execute all the review processes.

[0149] 5. ST01 -> ST02 -> ST05 -> ST03 -> ST01

[0150] In the current clock cycle, start execution from ST01. If a current cache transaction request (newTRS) is received, then transfer to ST02 to perform the comparison operation of the transmission interval. When it is determined that trs_interval_cnt < trs_interval_set and trs_debt_cnt = 0, then continue execution by transferring to ST05.

[0151] 6. ST01 -> ST01

[0152] In the current clock cycle, start execution from ST01. If the current cache transaction request (no newTRS) is not received, then wait in place at ST01.

[0153] Further, Figure 11 shows a structural diagram of an AI processor according to an embodiment of the present invention. As Figure 11 shown, the AI processor includes a plurality of data service initiators 1101 and a plurality of data service providers 1104. Each data service initiator 1101 establishes a communication connection with each data service provider 1104 through the on-chip network 1103 on the AI processor.

[0154] Among them, the plurality of data service initiators 1101 include: MST1, MST2,..., MSTx, and the plurality of data service providers 1104 include: SLV1, SLV2,..., SLVy. Among them, both x and y are integer values, and they can be the same or different.

[0155] Among them, at the service export of each data service initiator 1101, a distributed flow control system 1102 as described in any one of the embodiments of the present invention is respectively deployed.

[0156] Among them, a distributed flow control system 1102 is deployed at the service exit of a data service initiator 1101. DFCU1 is deployed at the service exit of MST1, DFCU2 is deployed at the service exit of MST2, ..., and DFCUx is deployed at the service exit of MSTx.

[0157] In this embodiment of the invention, the data service initiator 1101 can be a computing core in an AI processor, used to achieve data throughput. The data service provider 1104 can be various types of memory in the AI ​​processor, such as registers or caches. Any two data service initiators 1101 and data service providers 1104 can be interconnected via an on-chip network 1103. The on-chip network 1103 includes multiple service flow convergence points (in... Figure 11 SAP is often abbreviated as SAP in Chinese, and data service provider 1103 can also be understood as a special type of SAP.

[0158] In this embodiment, considering that the primary objective of flow control is to influence the characteristic distribution of data services, it is necessary to place the DFCU as close as possible to the data transmission source to prevent data retention between the source and the DFCU deployment point from causing bursts in data services at the next level, thus affecting the flow control effect. Simultaneously, placing the DFCU close to the transmission end avoids large buffer requirements, allowing the DFCU to directly interact with the MST and directly control the MST's active data transmission periods. Therefore, in this embodiment, each DFCU is deployed close to the MST's exit location to minimize the distribution of undesirable data services mixed into the AI ​​processor.

[0159] By scientifically applying the technical solutions of the various embodiments of this invention, a variety of methods for optimizing hardware architecture can be provided, as follows:

[0160] 1. For a DFCU on a port corresponding to an MST, the actual flow control requirements can be achieved by setting the number of clock cycles and the flow control bandwidth target within a time window in the corresponding register of the DFCU.

[0161] 2. For DFCUs on multiple MSTs, the total flow control bandwidth target can be evenly distributed across all participating MSTs, or it can be proportionally configured across different MSTs as needed. When different MSTs occupy different proportions of the flow control bandwidth target, traffic ratio control between MSTs can be achieved. Furthermore, for DFCUs on multiple MSTs, the same flow control ratio can be applied to all traffic flows of a certain type, achieving a stable and balanced flow control effect.

[0162] 3. Due to their inherent characteristics, read and write operations exhibit significant performance differences, especially when system-level chip-level data is stored using hash-based access methods. In practical engineering, to balance the read / write ratio and maximize the computing power of the cores, traffic control is needed to adjust the read / write ratio to meet actual requirements.

[0163] 4. Traffic debt compensation

[0164] Ideally, data services would be perfectly deployed according to the predetermined traffic volume and shaping process. However, in real-world scenarios, this is impossible. First, the architecture design cannot allow each MST to have an independent data channel to its destination. Typically, the architecture is designed to share as many interconnect resources as possible among all MSTs. Therefore, random backpressure on the interconnect network is an inevitable scenario. The technical solution of this invention addresses real-world problems by statistically analyzing traffic debt and providing limited compensation based on the transmission interval. This implementation method ensures, as far as possible, that the flow control target is achieved within the flow control time window (i.e., the time window mentioned above), despite source-end bursts and random backpressure on the interconnect.

[0165] 5. Read / write channel traffic reciprocity mechanism

[0166] In real-world scenarios, an MST typically has at least one read port and one write port. DFCU supports information exchange between adjacent read and write ports. Within the overall bandwidth target of one MST, read-write reciprocity can be enabled. The benefit of this feature is that when read or write traffic doesn't fully utilize the bandwidth of its data channel, the reduced number of TRS (Traffic Responsibility Requests) within a flow control window can be lent to another channel that needs to repay traffic debt. Debt repayment opportunities appearing in the current flow control window do not accumulate; they are reset to zero in the next flow control window. This ensures limited traffic reciprocity between channels within adjacent flow control windows, without excessively impacting the overall service ratio and traffic burst characteristics.

[0167] To further explain the beneficial effects of the various embodiments of the present invention, the inventors conducted a simulation on an AI processor with 80 MSTs accessing 64 SLVs in parallel, setting the number of clock cycles contained in a time window to 32, and obtained the actual flow control effect presented by the real simulation data under the condition of distributed DFCU operation.

[0168] Specifically, in Figure 12 A simulation verification table for single-port read bandwidth flow control based on an embodiment of the present invention is shown. That is, in Figure 12 The diagram illustrates a flow control effect that is infinitely close to the theoretical upper limit of the physical port. For example... Figure 2As shown, after configuring the set flow control target (bandwidth utilization of the read port) by using the distributed flow control system, the read port can perfectly achieve the configuration expectation within the configuration range of 100%-1%, with high accuracy and the flow control granularity fully meets the business adjustment needs.

[0169] exist Figure 13 The table shows the performance verification of actual simulated bandwidth under the configuration of a distributed flow control system when six data service initiators operate in parallel, based on an embodiment of the present invention. Figure 13 As shown, for a multi-port transmission scenario with 6 parallel MSTs, the DFCU distributed flow control mechanism was verified to achieve a synergistic optimization effect on system bandwidth peak and service balance.

[0170] In this context, each MST, acting as an independent data service initiator, did not reach the peak bandwidth limit of the hardware architecture. Furthermore, the DFCU configuration value is the target bandwidth quota for the flow control module, the actual simulated bandwidth is the real bandwidth value measured by the system, and SIP_PTP is the peak-to-peak value of data services between computing cores (the difference between the maximum and minimum values), used to characterize the service balance among computing cores; a smaller value indicates better balance.

[0171] The test results show that when the DFCU flow control mechanism is enabled, it can accurately and uniformly manage the peak bandwidth of each MST according to the configured ratio, achieving on-demand allocation of system bandwidth peaks. Simultaneously, this flow control mechanism can significantly improve service balance in multi-MST parallel scenarios. Specifically, when the DFCU configuration value is 37.50%, SIP_PTP is 22.52, indicating a service range difference of 22GB / s between computing cores; when the DFCU configuration value is adjusted to 34.38%, SIP_PTP drops sharply to 0.17, essentially eliminating the service range difference between computing cores, achieving a dual-objective optimization of system bandwidth peak control and improved service balance.

[0172] This embodiment demonstrates that, using only the DFCU distributed flow control technology, precise control of bandwidth peaks and a significant improvement in service balance can be achieved simultaneously in scenarios involving multiple MSTs in parallel and multi-port transmission. Furthermore, by combining other optimization methods, the system performance can still be further improved.

[0173] Furthermore, in Figure 14 The diagram illustrates the impact of parallel data throughput flow control on balance and peak performance when performing parallel data throughput flow control on 80 computing cores based on an embodiment of the present invention. Figure 14As shown, the horizontal axis represents the flow control percentage, i.e., the bandwidth target ratio set in the DFCU. The left vertical axis is the total bandwidth (totalBw) (unit: GB / s), representing the total service throughput of 80 computing cores outputting in parallel. The right vertical axis is bwPtp (unit: GB / s, abbreviated as ptp), representing the peak-to-peak service throughput between computing cores, i.e., the difference between the maximum and minimum throughput, used to characterize the service balance index within the system. Figure 14 The horizontal dashed line (bw-Target) in the figure represents the system's preset ideal bandwidth target value.

[0174] In the high flow control percentage range, the total system bandwidth remains high due to hardware limitations. As the flow control percentage decreases to the wide box area, the traffic shaping mechanism can effectively reduce the service range while maintaining the peak total bandwidth. When the flow control percentage drops to the inflection point area shown in the narrow box (approximately 0.46875), the system enters the coordinated optimization zone of flow control and traffic shaping, where PTP drops sharply but the total bandwidth begins to decline.

[0175] In summary, the configuration points corresponding to the narrow box achieve an optimal trade-off between service balance and bandwidth utilization, and can be used as the baseline configuration parameters for the system. The above experiments verify the effectiveness and practicality of the flow control mechanisms in the various embodiments of the present invention.

[0176] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0177] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A distributed flow control hardware, characterized in that, It includes a throughput control module, a transaction request buffer, and a rate control module. The throughput control module includes an anti-fluctuation buffer. The transaction request buffer is connected to both the throughput control module and the rate control module. The throughput control module is used to decide, in each clock cycle, whether to send the real-time received transaction request to the transaction request buffer or to the anti-ripple buffer for temporary storage, or to directly select a temporary transaction request from the anti-ripple buffer and send it to the transaction request buffer. The constant speed control module is used to sequentially retrieve transaction requests from the transaction request buffer and send each transaction request out sequentially according to the determined clock cycle sending interval corresponding to each transaction request.

2. The distributed flow control hardware according to claim 1, characterized in that, The throughput control module specifically includes multiple rate control units, a transaction request router, and multiple anti-fluctuation buffer units corresponding to each rate control unit. These anti-fluctuation buffer units form an anti-fluctuation buffer. The transaction request router is connected to each rate control unit and each anti-fluctuation buffer unit, wherein: Each rate control unit is used to determine in real time whether the transaction request of its control type is allowed or not allowed to be sent, based on the proportion of the transaction requests of its control type sent within the preset time window configured in the unit register and the number of transaction requests of its control type to be sent within the current time window, and to send a matching allow or disallow signal. Each anti-fluctuation buffer unit is used to temporarily store transaction requests of a specified type that are not currently allowed to be sent; The transaction request router is used to determine, in each clock cycle, whether to send the real-time received transaction request to the transaction request buffer or to the anti-ripple buffer for temporary storage, or to directly select a temporarily stored transaction request from the anti-ripple buffer and send it to the transaction request buffer.

3. The distributed flow control hardware according to claim 2, characterized in that, The transaction request router is also used for: After sending a transaction request to the transaction request buffer, a completion signal is sent back to the rate control unit that matches the type of the transaction request, so that the rate control unit can update the number of transaction requests of the type controlled by this unit to be sent within the current time window.

4. The distributed flow control hardware according to claim 2, characterized in that, The throughput control module also includes an address extraction component and a first address broadcasting component, wherein the address extraction component is connected to the first address broadcasting component and the transaction request router, respectively; The address extraction component is used to extract the request address from the real-time received transaction request, and then send the request address to the first address broadcast component while forwarding the transaction request to the transaction request router. The first address broadcasting unit is used to broadcast the received request address to each rate control unit, wherein the transaction requests for different request address ranges have different types.

5. The distributed flow control hardware according to any one of claims 2-4, characterized in that, Transaction request router, specifically used for: If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type is currently sending a transmit permission signal, and the target anti-fluctuation buffer unit corresponding to the target rate control unit is currently empty, then the target transaction request is sent to the transaction request buffer. If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type sends a signal to allow transmission, and the target anti-fluctuation buffer unit corresponding to the target rate control unit is not empty, then a transaction request is sequentially obtained from the target anti-fluctuation buffer unit and sent to the transaction request buffer, and the target transaction request is temporarily stored in the target anti-fluctuation buffer unit. If a target transaction request of the target type is received in the current clock cycle, the target rate control unit corresponding to the target type is currently sending a signal that it is not allowed to send, and the alternative transaction requests in other rate control units polled meet the sending conditions, then the alternative transaction requests are sent to the transaction request buffer, and the target transaction request is temporarily stored in the target anti-fluctuation buffer unit. If a target transaction request of the target type is received in the current clock cycle, and the target rate control unit corresponding to the target type is currently sending a signal that it is not allowed to send and has not polled for a candidate transaction request that meets the sending conditions, then the target transaction request will only be temporarily stored in the target anti-fluctuation buffer unit. If no transaction request is received in the current clock cycle, and alternative transaction requests in other rate control units meet the sending conditions, then the alternative transaction request is sent to the transaction request buffer. If no transaction request is received in the current clock cycle, and no alternative transaction request that meets the sending conditions is found in the poll, then no operation is performed.

6. The distributed flow control hardware according to claim 1, characterized in that, The constant speed control module specifically includes a transaction request scheduling unit, multiple interval control units, and a constant speed scheduling unit; the transaction request scheduling unit is connected to the transaction request buffer and the constant speed scheduling unit respectively, and the constant speed scheduling unit is connected to each interval control unit respectively; The transaction request scheduling unit is used to prefetch a new transaction request from the transaction request buffer after determining that the fixed-rate scheduling unit has completed sending a transaction request, and send the cached transaction requests stored in the unit to the fixed-rate scheduling unit. Each interval control unit is used to calculate the target clock cycle transmission interval of the current cached transaction request based on the information configured in the unit register when it is determined that the type of the current cached transaction request is consistent with the control type of this unit, and then send it to the fixed speed scheduling unit. The fixed-speed scheduling unit is used to determine the transmission timing that matches the current cached transaction request based on the transmission interval of the received target clock cycle, and to send the current cached transaction request outward at the transmission timing.

7. The distributed flow control hardware according to claim 6, characterized in that, The constant speed control module also includes a second address broadcast unit, a transaction request counting unit, a debt statistics unit, and a transaction request output interface; the second address broadcast unit is connected to the transaction request scheduling unit, each interval control unit, the transaction request counting unit, and the debt statistics unit respectively, and the transaction request output interface is connected to the constant speed scheduling unit, the transaction request counting unit, and the debt statistics unit respectively. The transaction request scheduling unit is also used to send the request address of the current cached transaction request to the second address broadcast unit; The second address broadcasting unit is used to broadcast the received request address to each interval control unit, wherein the transaction requests of different request address ranges have different types; The transaction request output interface is used to receive transaction requests sent by the fixed-speed scheduling unit at each sending time, and to send each transaction request outward; The transaction request counting unit is used to perform statistical calculations on each transaction request that was successfully sent by the transaction request output interface, and send the calculated actual clock cycle sending interval to each interval control unit. The debt statistics unit is used to monitor the backpressure of the transaction request output interface, count the debt caused by the backpressure, and send the debt count to each interval control unit. Each interval control unit is further used to send the calculated target clock cycle transmission interval of the current cached transaction request, as well as the received actual clock cycle transmission interval and debt count value, to the constant speed scheduling unit. The constant-speed scheduling unit is further used to determine the transmission timing that matches the current cached transaction request based on the received target clock cycle transmission interval, actual clock cycle transmission interval, and debt count value.

8. The distributed flow control hardware according to claim 6, characterized in that, Each interval control unit is specifically used for: The target clock cycle sending interval for the current cached transaction request is calculated based on the request number of the current cached transaction request within the current time window, the number of clock cycles contained in a time window configured in the unit register, the interface clock frequency of the transaction request output interface, and the preset flow control bandwidth target.

9. The distributed flow control hardware according to claim 7, characterized in that, The constant speed scheduling unit is specifically used for: After obtaining the current cache transaction request, if it is detected that the actual clock cycle sending interval is greater than or equal to the target clock cycle sending interval, the current clock cycle is directly determined as the sending time that matches the current cache transaction request. After obtaining the current cache transaction request, if it is detected that the actual clock cycle sending interval is less than the target clock cycle sending interval, the debt count value is greater than 0, and the actual clock cycle sending interval is greater than the preset condition relaxation interval, then the current clock cycle is directly determined as the sending time that matches the current cache transaction request. The condition relaxation interval is calculated based on the target clock cycle sending interval. After obtaining the current cached transaction request, if it is detected that the actual clock cycle sending interval is less than the target clock cycle sending interval and the debt count value is equal to 0, then wait for the next clock cycle to continue the judgment.

10. A distributed flow control system, characterized in that, It includes two distributed flow control hardware devices as described in any one of claims 1-9, one of the two distributed flow control hardware devices being used for distributed flow control of read transaction requests, and the other of the two distributed flow control hardware devices being used for distributed flow control of write transaction requests.

11. The distributed flow control system according to claim 10, characterized in that, The distributed flow control system also includes a read / write coordinating and scheduling unit, which is connected to two distributed flow control hardware devices respectively. The read / write coordination and scheduling unit is used to perform coordinated flow control on read and write transaction requests.

12. The distributed flow control system according to claim 11, characterized in that, The read / write coordinating and scheduling unit is connected to the constant-speed scheduling unit in each of the two distributed flow control hardware components. The fixed-speed scheduling unit is further used to: after obtaining the current cache transaction request, if it detects that the actual clock cycle sending interval is less than the target clock cycle sending interval and the debt count value is equal to 0, send an application request for additional sending permission to the read-write coordinating scheduling unit, and when the additional sending permission is obtained, directly determine the current clock cycle as the sending time that matches the current cache transaction request; otherwise, wait for the next clock cycle to continue to make judgments. The read / write coordinating scheduling unit is specifically used to detect the bandwidth utilization of another distributed flow control hardware when it receives an application request for additional transmission permission from the fixed-rate scheduling unit in one distributed flow control hardware. If it is determined that the other distributed flow control hardware still has spare bandwidth, a successful application response is generated as feedback result for the application request.

13. An artificial intelligence processor, characterized in that, include: Multiple data service initiators and multiple data service providers, with each data service initiator establishing a communication connection with each data service provider through the on-chip network on the artificial intelligence processor; Specifically, at the business exit of each data service provider, a distributed flow control system as described in any one of claims 10-12 is deployed.

Images