A dynamic bandwidth allocation system in a chip stacking system

Abstract

The application relates to the field of digital information transmission and discloses a dynamic bandwidth allocation system in a chip stacking system, which comprises an intention capturing and coding unit, an out-of-band intention signaling bus and a context fusion arbitrator. The unit captures service intention and codes the intention into an intention code, which is sent to the arbitrator through the bus. The arbitrator arbitrates based on the intention code and queue state, and reversely sends an intention selective back pressure signal to the unit through the bus when congestion occurs. The application establishes an intention-aware closed-loop flow control, so that the arbitrator can distinguish transmission urgency, guarantees key transmission low delay such as stoppage avoidance, maximizes the throughput of batch prefetching, and solves the contradiction between delay and throughput.

Description

Technical Field

[0001] This invention relates to a dynamic bandwidth allocation system in a chip stacking system, belonging to the field of digital information transmission technology. Background Technology

[0002] In current chip stacking systems, in the architecture of vertically integrated computing dies and high-bandwidth memory, the high-density interconnect channels between dies carry massive amounts of digital information transmission. Physical bandwidth resources are limited and costly. The effective arbitration and allocation of these resources is a key factor determining the overall system performance. Such inter-die switches or arbitrators mainly rely on preset static Quality of Service (QoS) tags in the packet header, such as high, medium, and low priority, to adjudicate the physical bandwidth usage of different transmission requests. This is a conventional technical method in the industry for differentiating service levels in the field of digital information transmission. However, this arbitration method that relies on static tags has inherent limitations. The arbitrator is context-blind to the transmission request. When a computing unit initiates a transmission, it knows the true business intent of the transmission, such as whether the request is to prevent an immediate halt in the computing pipeline or to prefetch non-urgent background data. When this business intent is encapsulated and reduced to a few static QoS tags, its original dynamic urgency information is lost.

[0003] Arbitration methods based on static QoS tags create a technical constraint between the goals of low latency guarantee and high throughput utilization in the transmission system. Attempts to solve the problem by adding more and more refined static QoS levels cannot fundamentally solve the problem. Its essence is still based on static inference, which cannot reflect the true intention of the computing unit's dynamic changes during operation in real time. In view of the limitations of the static QoS mechanism, some research attempts to break out of the framework of traditional wired interconnection and explore other dynamic connection methods. For example, Chinese invention patent with authorization announcement number CN105393353B discloses a 3D stacked device, chip and communication method, which attempts to configure a first antenna array and a second antenna array between the computing unit and the storage unit, and adjust the transmission parameters such as the phase of the antenna by adjusting the adjustment unit to realize wireless dynamic connection between the core and the storage sub-unit. However, this method based on antenna array and radio frequency communication, which aims to solve the flexibility problem of dynamic connection and resource allocation, introduces a new and complex radio frequency physical layer, which faces great challenges in terms of signal isolation, power consumption and implementation complexity in high-density stacking environments.

[0004] Therefore, the technical problem to be solved by this invention is how to provide an arbitration allocation method for digital information transmission, so that the arbitration decision can be dynamically adjusted according to the real-time business intent of the transmission request, while ensuring low latency for high-urgency transmission and improving the overall utilization of interconnection bandwidth. Summary of the Invention

[0005] To address the problems mentioned in the background section, the technical solution of this invention is as follows:

[0006] A dynamic bandwidth allocation system in a chip stacking system, the system being applied in a stacked structure containing compute dies and memory dies, the compute dies having inter-die interconnect interfaces connected to the memory dies, comprising:

[0007] The intent capture and encoding unit, located in the computing die, is used to capture the business intent associated with a data transmission request when the computing die initiates such a request; and to encode the business intent into an intent code.

[0008] An out-of-band intent signaling bus is configured at the inter-die interconnection interface. The bus includes a forward path and a reverse path. The bus is used to: send intent codes from the intent capture and encoding unit to the context fusion arbitrator using the forward path;

[0009] The context fusion arbitrator, located at the inter-die interconnection interface, has at least one queue inside. The arbitrator is used to receive data packets corresponding to data transmission requests; receive intent codes via the out-of-band intent signaling bus; and dynamically arbitrate the transmission of data packets based on the intent codes and the status of at least one queue. The context fusion arbitrator is also used to monitor the congestion status of the inter-die interconnection interface; when the congestion status exceeds a preset threshold, it sends an intent-selective backpressure signal to the intent capture and encoding unit through the reverse path.

[0010] The intent capture and encoding unit is also used to respond to intent selective backpressure signals, pausing or reducing the transmission of data transmission requests for specific service intents at the source end of the computation die.

[0011] Preferably, the business intent characterizes the urgency of the data transmission request, and the urgency includes at least: pause avoidance intent and batch prefetch intent; the context fusion arbitrator includes arbitration logic, which is used to: when the intent code indicates a pause avoidance intent, assign the corresponding data packet the highest arbitration priority, and transmit it before the data packet with the batch prefetch intent.

[0012] Preferably, the network interface controller for computing Die includes multiple virtual intent queues, which are used to queue data packets of different service intent levels. The intent capture and encoding unit is also used to: route data packets to the corresponding queues in the multiple virtual intent queues for queuing according to the intent code. The network interface controller includes a source arbitrator, which is used to select data packets from the virtual intent queues with higher priorities based on the priorities of the virtual intent queues and submit them to the context fusion arbitrator.

[0013] Preferably, the intent capture and encoding unit is also used to locally associate the intent code with the data packets that have not yet been sent in the internal data packet buffer managed by the intent capture and encoding unit; receive intent invalidation signaling from the computation die; and in response to intent invalidation signaling, clear or mark the data packets associated with intent invalidation signaling in the internal data packet buffer as invalid before the data packets are sent to the context fusion arbitrator.

[0014] Preferably, the intent capture and encoding unit is further configured to: encapsulate a copy of the intent code as an in-band intent code into the data packet; the context fusion arbitrator is further configured to extract the in-band intent code from the data packet and compare whether the in-band intent code is consistent with the intent code received from the out-of-band intent signaling bus; if the two are inconsistent, a security mode is triggered, and the arbitrator ignores the intent code received from the out-of-band intent signaling bus in the security mode and performs arbitration based on a preset static arbitration rule.

[0015] Preferably, it includes: an intent sequence monitor, connected to an out-of-band intent signaling bus, the monitor being used to classify and count intent codes transmitted on the out-of-band intent signaling bus within a sliding time window, generating a congestion prediction factor. The generation is at least in part based on the count value of the batch prefetching intent. The count value of the intention to avoid pauses A quantitative relationship between them, defined as: .

[0016] Preferably, the context fusion arbitrator is also used when the congestion prediction factor... When the preset threshold is exceeded, a selective back pressure signal is triggered.

[0017] Preferably, it includes: an intent signaling integrity monitor, which is connected to the out-of-band intent signaling bus and is used to monitor the out-of-band intent signaling bus and detect signaling fault conditions that indicate physical failure of the bus; when a signaling fault condition is detected, a fail-safe preemption signal is sent to the context fusion arbitrator; the context fusion arbitrator is also used to respond to the fail-safe preemption signal, switch to a safe mode, and in the safe mode, ignore the intent code and arbitrate based on a preset static arbitration rule.

[0018] Preferably, the signaling failure conditions include at least one of the following: the monitor receives an intent code defined as illegal encoding from the out-of-band intent signaling bus, and the signal level of the out-of-band intent signaling bus does not change within a preset timeout threshold.

[0019] Preferably, it includes: an intent profiling analyzer, which communicates with a context fusion arbitrator. The intent profiling analyzer is used to make authorization decisions on data packets based on the context fusion arbitrator, and to classify and statistically analyze the computational die from different sources and the authorization events of different service intents. Based on the classification and statistics, it identifies pathological sources whose transmission behavior patterns exceed a preset profiling threshold. It generates a management alarm, which indicates that the pathological source has been identified.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] 1. By setting up an out-of-band intent signaling bus separate from data packet transmission, the context fusion arbitrator at the Die interconnection interface obtains the service intent code before the data packet arrives. The arbitrator makes fusion decisions based on this intent code and the internal queue status, allowing the arbitrator's arbitration logic to switch between two operating states: when a pause avoidance intent occurs, the transmission is prioritized to ensure low latency on the critical path; when there is no pause avoidance intent, transmissions such as batch prefetching are prioritized to fully utilize the physical bandwidth of the interconnection channel. This method of separating control signaling and data transmission on the physical path and integrating them with the arbitration logic enables the transmission system to achieve a balance between ensuring low latency on critical paths and improving overall throughput, avoiding the bandwidth waste inherent in the traditional static reservation method.

[0022] 2. By utilizing the reverse path of the out-of-band intent signaling bus and combining it with the congestion monitoring function of the arbitrator, an intent-selective closed-loop backpressure system is established. When the arbitrator detects that the congestion exceeds the threshold, the backpressure signal sent is associated with the service intent. The intent capture and encoding unit at the source responds to this specific signal and temporarily stores or limits the transmission request of the specific intent at the source of data transmission. This ensures that when congestion occurs in the interconnection channel, the flow of low service intent is accurately controlled, while the transmission path of high service intent remains unobstructed. This achieves fine-grained management of the transmission flow and avoids the one-size-fits-all transmission blocking of traditional flow control mechanisms.

[0023] 3. The generated intent code is not only used for arbitration between downstream dies, but can also be reused at the source by the network interface controller (NIC) of the computing die. The NIC has multiple virtual queues corresponding to different service intent levels. When generating the intent code, the intent capture unit distributes the data packets to the corresponding virtual queues according to the intent code. The source arbitrator inside the NIC prioritizes selecting data packets from the high-priority virtual queue. By moving the intent awareness logic forward to the source queuing stage, the transmission request of high intent does not need to wait for the transmission request of low intent to complete, avoiding queue head blocking at the NIC source and ensuring that the priority of service intent is maintained throughout the entire transmission path from the computing core to the inter-die interface. Attached Figure Description

[0024] Figure 1 This is a system architecture diagram for sensing dynamic arbitration and closed-loop back pressure, as intended by the present invention.

[0025] Figure 2 This is a comparative diagram illustrating the transmission intent profiles of the pathological and normal sources in this invention.

[0026] Figure 3 This is a sequence diagram of the dynamic priority arbitration based on out-of-band intent signaling in this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, a further detailed description will be provided in conjunction with the accompanying drawings and specific embodiments; it should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0028] This invention provides a dynamic bandwidth allocation system in a chip stacking system. Architecturally, it mainly includes an intent capture and encoding unit located in the computing die, a context fusion arbitrator located at the inter-die interconnection interface, and an out-of-band intent signaling bus for connecting the unit and the arbitrator. The capture and encoding unit captures the service intent of a data transmission request and encodes it into an intent code when the request is initiated. The context fusion arbitrator dynamically arbitrates the transmission of data packets based on the received intent code and the internal queue state. The out-of-band intent signaling bus also transmits congestion control signals in reverse, forming an intent-aware closed-loop flow control system. In a specific implementation, the capture and encoding unit is located in the computing die and can be functionally associated with the task scheduler or instruction issuing unit within the computing die. When the computing die initiates a data transmission request, it captures the service intent associated with the request. The service intent characterizes the urgency of the data transmission request and can be at least divided into pause avoidance intent and batch prefetch intent. The former indicates that failure to execute the transmission will cause a pause in the computing pipeline, indicating high urgency, while the latter corresponds to non-urgent background data loading, prioritizing throughput. Prioritizing the target, after capturing the intent, the service intent is encoded into an intent code. This intent code can be a digital signal with a specific bit width. For example, 2 bits can be used, where 11 can be defined as pause avoidance and 10 can be defined as batch prefetching. To achieve intent-aware queuing at the source end, multiple virtual intent queues can be set up inside the network interface controller (NIC) that computes the die. These queues are used to queue data packets of different service intent levels. After generating the intent code, the intent capture and encoding unit also routes the corresponding data packet to one of the multiple virtual intent queues according to the service intent indicated by the intent code. Taking pause avoidance and batch prefetching as examples, the NIC can set up high-priority virtual queues and low-priority virtual queues. Packets with intent code 11 enter the high-priority virtual queue, and packets with intent code 10 enter the low-priority virtual queue. Correspondingly, the NIC also includes a source-end arbitrator. The source-end arbitrator arbitrates based on the priority of the virtual intent queues, prioritizing the selection of data packets from the higher-priority virtual intent queues and submitting them to the downstream context fusion arbitrator. This setting avoids head-of-queue blocking caused by high intent requests at the source end due to low intent requests.

[0029] To handle rapid changes in the source computing state, the intent capture and encoding unit is also used to locally associate unsent data packets in its managed internal data packet buffer with intent codes; it is also used to receive intent invalidation signaling from the computing die, indicating that a previously initiated service intent has now become invalid; in response to intent invalidation signaling, before sending data packets to the context fusion arbitrator, it searches the internal data packet buffer and clears or marks data packets associated with intent invalidation signaling as invalid, avoiding invalid data packets occupying inter-die transmission bandwidth; to enhance system reliability, the intent capture and encoding unit can also be used to use a copy of the intent code as an in-band intent... The image code is encapsulated in the data packet header or metadata field; in addition, the intent capture and encoding unit is also used to receive and respond to the intent selective backpressure signal from the context fusion arbitrator. When this signal is received, the transmission of data transmission requests for specific service intents is paused or reduced at the source end of the computing die, such as pausing the request for batch prefetching intents without affecting the request for pause avoidance intents; the out-of-band intent signaling bus is configured at the inter-die interconnection interface and can be physically a set of dedicated signal lines separate from the main data path carrying data packets; the bus includes a forward path and a reverse path; the forward path is used to transmit the intent code generated by the intent capture and encoding unit to the computing die. The context fusion arbitrator is sent to the inter-die interconnection interface; the reverse path is used to send the intent-selective backpressure signal generated by the context fusion arbitrator back to the intent capture and encoding unit, forming a closed-loop control; the context fusion arbitrator is located at the inter-die interconnection interface and has at least one queue inside to buffer data packets corresponding to data transmission requests received from the computing die; the arbitrator also receives the intent code corresponding to the data packet through the forward path of the out-of-band intent signaling bus; the core function of the arbitrator is to dynamically arbitrate the transmission of data packets based on the intent code and the state of at least one queue; the arbitrator includes arbitration logic, which can be implemented as a... Group priority coding rules, for example, when the arbitrator receives an intent code indicating a pause avoidance intent, the arbitration logic assigns the corresponding data packet the highest arbitration priority, prioritizing the transmission of data packets with batch prefetch intents. When there is no pause avoidance intent request, it prioritizes the transmission of batch prefetch intents to maximize bandwidth utilization. The context fusion arbitrator is used to monitor the congestion status of the inter-die interconnection interface by monitoring the occupancy level or water level of its internal queue. When the congestion status exceeds a preset threshold, such as when the queue occupancy rate exceeds 80%, the arbitrator sends an intent selective backpressure signal to the intent capture and encoding unit through the reverse path of the out-of-band intent signaling bus.

[0030] To address faults in the out-of-band signaling bus itself, the context fusion arbitrator also extracts in-band intent codes from data packets and compares them with intent codes received from the out-of-band intent signaling bus. If they do not match, it indicates a potential fault in the out-of-band signaling path, and the arbitrator triggers a safety mode. In safety mode, the arbitrator ignores intent codes received from the out-of-band intent signaling bus and arbitrates based on preset static arbitration rules, using traditional weighted round-robin or strict priority methods, ensuring continued operation even during signaling failures. For congestion control, an intent sequence monitor is included, connected to the out-of-band intent signaling bus. The monitor classifies and counts intent codes transmitted on the out-of-band intent signaling bus within a sliding time window, generating a congestion prediction factor to characterize future congestion tendencies, based on the count values ​​of batch prefetched intents. The count value of the intention to avoid pauses Defined by a quantitative relationship between them, such as Context fusion arbitrator is used when congestion prediction factors are in use. When the preset threshold is exceeded, the current queue occupancy rate is not yet at the congestion threshold, triggering a selective backpressure signal in advance to achieve predictive flow control. Another method to ensure system reliability includes an intent signaling integrity monitor connected to the out-of-band intent signaling bus. This monitor monitors the out-of-band intent signaling bus and detects signaling fault conditions indicating physical bus failure. When a signaling fault condition is detected, a fail-safe preemption signal is sent to the context fusion arbitrator. The signaling fault condition specifically includes at least one of the following: the monitor receives an intent code defined as illegal in the encoding scheme from the out-of-band intent signaling bus, or the signal level of the out-of-band intent signaling bus does not change within a preset timeout threshold, i.e., the signal is stuck. Context fusion arbitrator... Arbitrator response failure - security preemption signal, switch to security mode. In security mode, intent codes are ignored and arbitration is performed based on preset static arbitration rules. To achieve system-level transmission behavior monitoring, the system may also include an intent profiling analyzer, which communicates with the context fusion arbitrator. The intent profiling analyzer is used to make authorization decisions for data packets based on the context fusion arbitrator, and to classify and statistically analyze authorization events for computing dies from different sources and for different service intents. Based on the classification statistics, the analyzer can identify pathological sources whose transmission behavior patterns exceed preset profiling thresholds, such as an abnormally high proportion of pause avoidance requests initiated by a computing die. The analyzer generates management alerts, which indicate that pathological sources have been identified, and the alerts can be reported to a higher-level system manager.

[0031] Example 1: This example demonstrates the specific operation of a high-dynamic-load scenario. The chip stack system is deployed in a high-performance computing environment. The computing die carries a computing core that is highly sensitive to latency, while simultaneously running a high-bandwidth data prefetching task. In the specific operating conditions, the data prefetching task continuously initiates data transmission requests for batch prefetching intents to the inter-die interconnect interface. Inside the network interface controller of the computing die, the intent capture and encoding unit captures this intent and generates the corresponding intent code. According to the intent virtual queue mechanism, the data packets are routed to the low-priority intent virtual queue for queuing. The downstream context fusion arbitrator receives these batch prefetched intent codes via the out-of-band intent signaling bus. Currently, there is no high-urgency transmission, and the arbitration logic prioritizes serving the batch prefetching requests. The high-bandwidth transmission of the inter-die interconnect channel is fully occupied, and it is in a high-throughput operating state. At a certain moment, the computing core urgently initiates a read request with a very small amount of data due to an impending pipeline stoppage. The intent capture and encoding unit captures this stoppage avoidance intent and generates the corresponding intent code. Due to the source arbitrator and intent Through the coordination of virtual queues, the pause avoidance request packet is sent to a high-priority virtual queue, bypassing the large number of batch prefetch packets queuing in the low-priority virtual queue. It obtains the highest transmission priority at the source end of the network interface controller, and the pause avoidance intent code is sent non-blockingly to the out-of-band intent signaling bus. Upon receiving the pause avoidance intent code, the context fusion arbitrator triggers its arbitration logic. Even if the internal queue is occupied by batch prefetch packets, the arbitrator suspends the transmission of all batch prefetch intents and instead grants full access to the inter-die interconnect interface to the pause avoidance request. High-urgency packets are given priority to pass through the interconnect channel, allowing the computing core to obtain data promptly and avoiding pipeline stalls. After the pause avoidance transmission is completed, the context fusion arbitrator automatically resumes service for batch prefetch transmission. The inter-die interconnect channel is used to achieve high-throughput data loading. This process demonstrates that, through the coordination of out-of-band intent signaling and the source-end intent virtual queue, the goals of low latency guarantee for the critical path and high bandwidth throughput for background tasks—two mutually constraining objectives in traditional static QoS arbitration—are achieved without relying on static bandwidth reservations.

[0032] Example 2: This example constructs a digital logic simulation test platform to simulate a chip stack system containing compute dies and memory dies. The interconnect interfaces between dies have a fixed theoretical peak bandwidth. The platform has a built-in hybrid traffic generator to simulate high dynamic load scenarios, continuously generating high-bandwidth background traffic for batch prefetching intentions, and injecting critical traffic of small data volumes and high urgency for pause avoidance intentions at random intervals. The core monitoring indicators of the experiment are the P99 tail latency of pause avoidance requests (in ns) and the total bandwidth utilization of the interconnect channels between dies (in %). For comparative analysis, three test groups are set up: Control Group 1 uses the existing static Qo... The S-arbitration method statically reserves 20% of the interconnect bandwidth for high-priority queues and uses a single FIFO source queue. The control group 2 uses the context fusion arbitrator of the present invention, but its network interface controller for calculating the die still uses a traditional single FIFO queue and lacks a source intent virtual queue mechanism. The sample group of the present invention adopts the complete technical solution of the present invention, including an intent capture and encoding unit, an out-of-band intent signaling bus, a context fusion arbitrator, and an intent virtual queue set at the source end of the network interface controller. Under the same hybrid traffic model, all three configurations run for one million simulation clock cycles. The key performance index data obtained are shown in Table 1.

[0033] Table 1: Performance Comparison Test Data under Three Arbitration Methods

[0034]

[0035] Table 1 shows that in control group 1, i.e., the static QS method, a critical packet P99 latency of 12.3ns was achieved by sacrificing 20% ​​of fixed bandwidth, but the total bandwidth utilization rate was only 79.8% at most. This is attributed to the fact that the bandwidth reserved for high priority cannot be reused for batch prefetching traffic when idle, verifying the constraint relationship between latency and throughput in the background technology. In control group 2, which lacks a source-end virtual queue, the total bandwidth utilization rate improved to 91.2%, but its P99 latency deteriorated sharply to 85.7ns. The reason is that although the downstream arbitrator has intent-aware capabilities, the high-urgency pauses prevent packets from being blocked in the single FIFO at the source end of the network interface controller. Non-urgent batch prefetching packets are blocked for a long time, resulting in queue head blocking, which causes the intent-aware arbitration mechanism to fail. The sample group of this invention simultaneously achieves a low latency of 11.9ns and a high bandwidth utilization rate of 97.4%. The low latency of 11.9ns is comparable to 12.3ns of the static reservation method, proving its ability to guarantee critical packets, and the high utilization rate of 97.4% is close to the theoretical peak.

[0036] Example 3: This example combines Figures 1 to 3 A description of a dynamic bandwidth allocation system in a chip stacking system, such as... Figure 1As shown, it begins with a data transmission request, which is received by the intent capture and encoding unit. This unit captures the service intent and encodes it into an intent code. Subsequently, the unit forwards the intent code to the context fusion arbitrator, the intent sequence monitor, and the intent virtual queue. Simultaneously, it performs packet routing, guiding the packets to the intent virtual queue. The intent virtual queue implements source-end intent-aware queuing to prevent head-of-queue blocking and submits the packets to the context fusion arbitrator. Meanwhile, the intent sequence monitor generates a congestion prediction factor based on the received intent code, implementing forward flow control, and sends the congestion prediction factor to the context fusion arbitrator. The context fusion arbitrator performs dynamic arbitration based on the packets from the virtual queue, the forward intent code from the encoding unit, and the congestion prediction factor from the monitor, using the intent code and queue status. After arbitration, the arbitrated data transmission is sent to the memory die (MemoryDie) as the data transmission destination. In addition, this process also demonstrates a closed-loop control, in which the context fusion arbitrator can send a reverse intent selective backpressure signal to the intent capture and encoding unit.

[0037] like Figure 2 As shown, the Y-axis represents the transmission percentage from 0 to 100, while the X-axis lists three service intentions: pause avoidance, batch prefetching, and result write-back. The chart data shows that for pause avoidance, the proportion of pathological sources is significantly higher than that of normal sources, while for batch prefetching and result write-back, the proportion of pathological sources is lower than that of normal sources. Figure 3 As shown, it involves five participants: a computation die, an intent capture and encoding unit, an out-of-band intent signaling bus, a context fusion arbitrator, and a memory die. The process begins with the computation die initiating a data transmission request to the intent capture and encoding unit. This unit then captures the service intent for batch prefetching / pause avoidance and encodes it into an intent code. Subsequently, this unit transmits the intent code out-of-band via the out-of-band intent signaling bus in parallel and sends the data packet to the context fusion arbitrator. After receiving the transmitted intent code from the bus, the context fusion arbitrator makes a decision based on the intent code and queue status. If it is a pause avoidance intent, it is given the highest priority and the data packet is transmitted immediately. If it is a batch prefetch intent, it is processed with normal priority and transmitted in queue order. The data packet is finally transmitted to the memory die, and the memory die returns a data response to the computation die.

[0038] Example 4: This example describes the collaborative operation mechanism when facing asynchronous changes in source state and physical channel interference. During operation, the computing unit PE_A of the computing die initiates a batch prefetch request. The intent capture and encoding unit generates intent code 10 and sends it through the out-of-band intent signaling bus, encapsulating a copy of the in-band intent code 10 in the corresponding data packet header. While the data packet is still queuing in the internal data packet buffer of the network interface controller (NIC) waiting to be sent, the upper-layer scheduler of PE_A preempts the service intent of the batch prefetch request, and the computing die sends an intent invalidation signaling to the intent capture and encoding unit accordingly. The intent capture and encoding unit responds to this signaling, searches its internal data packet buffer, locates the data packet associated with the invalid intent that has not yet been sent, and clears it from the buffer to avoid the invalid data packet from subsequently occupying the bandwidth of the interconnection interface between dies.

[0039] In another parallel operation, the computing unit PE_B initiates a pause avoidance request. The intent capture and encoding unit generates intent code 11 and sends it through the out-of-band intent signaling bus, encapsulating the in-band intent code 11 in the packet header. During transmission through the out-of-band intent signaling bus, the intent code 11 undergoes bit flipping due to transient physical interference, and the downstream context fusion arbitrator incorrectly receives it as 10. When the pause avoidance data packet arrives at the arbitrator, the arbitrator extracts the correct in-band intent code 11, protected by the error correction mechanism, from the data packet header. The comparison logic inside the arbitrator compares the received out-of-band intent code 10 with the extracted in-band intent code 11. If they are inconsistent, the arbitrator is triggered to switch to a safe mode. In this mode, the arbitrator ignores all information from the out-of-band intent signaling bus and automatically degrades to operation based on preset static arbitration rules.

[0040] Example 5: This example describes the calibration process for determining key control thresholds in a dynamic bandwidth allocation system. It is executed in a digital logic simulation platform, simulating a stacked chip system containing compute dies and memory dies, with interconnect interfaces between dies having a theoretical peak bandwidth of 100 GB / s. The platform incorporates a hybrid traffic generator to generate pause avoidance traffic and bulk prefetch traffic, monitoring the P99 tail latency of critical transmissions and total bandwidth utilization. The congestion threshold of the context fusion arbitrator and the prediction threshold of the intent sequence monitor are used: predictive flow control is disabled, and the prediction threshold is set to 100%. Bulk prefetch background traffic is injected into the platform, with the injection rate starting from 50 GB / s. The flow rate was monitored up to 95 GB / s, in 5 GB / s increments, with P99 tail latency monitored under different congestion threshold settings (from 50% to 95%, in 5% increments). When the congestion threshold was set to 80%, the P99 tail latency for pause avoidance requests remained below 15 ns when the total throughput reached 90 GB / s (90% utilization). When the throughput exceeded 92 GB / s, the latency began to increase rapidly, so 80% was selected as the congestion threshold. With the congestion threshold fixed at 80%, a batch prefetch traffic pulse with a peak flow rate of 95 GB / s was superimposed on a 70 GB / s batch prefetch background flow, lasting for 1000 cycles. Predictive flow control was enabled, and the predicted threshold was scanned. The threshold is set from 50% to 90%, in increments of 5%. When the prediction threshold is set to 70%, the number of events where the instantaneous physical queue occupancy rate caused by traffic pulses exceeds the 80% congestion threshold is reduced by 92% compared to when prediction flow control is disabled, and the impact on total throughput is less than 2.1%. Therefore, 70% is determined as the prediction threshold.

[0041] The timeout threshold for the intent signaling integrity monitor was calibrated. A traffic model representing a high-load mixed operating condition was run in the simulation platform, including 80% batch prefetch traffic and 10% pause avoidance traffic. Under this condition, the out-of-band intent signaling bus was monitored for 1 billion clock cycles over a long period, and the maximum time interval between two signal level transitions was counted. The observed value was... One clock cycle; provides a response in case of signal jamming, avoids misjudgment under normal signal fluctuations, and has a timeout threshold. Set as 200 clock cycles, used as the upper limit of the timeout counter in the monitor, calibrates the profiling threshold of the intent profiling analyzer: A set of benchmark programs representing computationally intensive and normal communication behavior are run in the simulation platform. Within a large statistical window of 1 million clock cycles, the intent profiling analyzer statistically analyzes the proportion of authorizations obtained from pause avoidance intents initiated by each computational die source to the total number of authorizations for that source. Under all normal operating conditions, the mean of this proportion is observed to be 8.5%, with a standard deviation of 2.1%, and the highest observed proportion... The threshold was 14.8%; to identify pathological sources with abnormal transmission behavior patterns, the profiling threshold was set. Set to twice the maximum observed value, that is This value is used as the comparison threshold for triggering pathological source management alarms within the analyzer.

[0042] Example 6: This example describes a standardized configuration procedure executed by the dynamic bandwidth allocation system during initial deployment or application load changes. This procedure ensures logical coordination between the context fusion arbitrator and the intent capture and encoding unit. It defines a system-wide mapping relationship between service intents and intent codes, using 3-bit encoding: 001 defines a pause avoidance intent, 010 defines a batch prefetch intent, and 100 defines a result write-back intent. Based on specific application goals, configurable arbitration logic is set within the context fusion arbitrator. If the goal is latency priority, the arbitration logic is set to strict priority encoding; upon receiving intent code 001, the highest arbitration priority is immediately granted to that source. If there is no 001 request, the 100 request is served first, and the 010 request is served last. If the goal is throughput priority, the arbitration logic is set to different rules; upon receiving 00... The highest priority is still granted when 0x1 is not requested, but when there is no 001 request, the bulk prefetch request of 010 is given priority to fill the interconnect bandwidth. The standardized configuration procedure also includes the definition and response configuration of the intention selective backpressure signal. A set of backpressure signal codes is defined and sent by the reverse path of the out-of-band intention signaling bus. In this embodiment, signal 0x1 is defined as first-level backpressure and signal 0x2 is defined as second-level backpressure. Correspondingly, the intention capture and encoding unit in the upstream computing die sets the response logic as follows: when responding to the intention selective backpressure signal and receiving 0x1, the source end suspends the transmission of all data transmission requests with the service intention of bulk prefetch. When receiving 0x2, the source end simultaneously suspends the transmission of requests for bulk prefetch intention and result write-back intention. Through this configuration process, the dynamic arbitration and closed-loop flow control behavior is deterministic and verifiable during deployment and can be matched with specific upper-layer application requirements.

[0043] Example 7: This example describes an offline feature analysis and configuration procedure required before deployment of the dynamic bandwidth allocation system. This procedure is used to calibrate the internal mapping logic of the intent capture and encoding unit. By performing static program flow analysis or instruction set simulation on the core computing tasks carried on the target computing die, specific instruction types or memory access patterns that are strongly correlated with pipeline pauses are identified. For example, all target registers whose load instructions within the next 5 clock cycles are relied upon are identified as high-urgency instructions. The opcodes or specific triggers of these identified instructions... Bus transaction characteristics are programmed and embedded in the internal lookup table of the intent capture and encoding unit, and uniquely mapped to pause avoidance intent codes such as 11. All memory access requests initiated by the hardware data prefetcher are uniformly mapped to batch prefetch intent codes such as 10, while regular computation result write-back is mapped to result write-back intent codes such as 01. When the instruction issuing unit for computation Die is about to issue a marked load instruction, the intent capture and encoding unit is triggered. Before sending the data transmission request to the queue of the network interface controller, the corresponding pause avoidance intent code has been generated and is ready to be sent to the out-of-band intent signaling bus.

[0044] The internal arbitration logic of the context fusion arbitrator is configured as a two-stage decision-making system. The first stage is a preemptive priority decision-making process. The logic unit directly monitors the out-of-band intent signaling bus. Once it receives the pause avoidance intent code 11, it preempts the arbitration right, grants the inter-die interconnection interface to the request, and bypasses all other decision logic. The second stage is fusion arbitration, which is activated when intent code 11 is not present. The arbitration logic performs weighted arbitration based on the intent code and queue status. Taking the weighted deficit polling logic as an example, the arbitrator sets different weights for different intent codes. For example, the weight of result write-back intent 01 is set to 8, and the weight of batch prefetch intent 10 is set to 2. The queuing time or data packet age of each intent's corresponding queue is used as the deficit amount, thereby ensuring that result write-back takes precedence over batch prefetching while avoiding batch prefetch traffic. Prolonged starvation; in addition, the triggering logic for selective backpressure is also clearly configured; the arbitrator is set to monitor two independent queue levels: one is a dedicated queue level for batch prefetching intentions, and the other is the overall queue level within the arbitrator; the backpressure logic is set as follows: when the level of the batch prefetching queue exceeds its independent preset threshold, such as 70%; when the overall queue level exceeds its congestion threshold, such as 80% as specified in Example 4; when the proportion of the batch prefetching queue occupancy to the overall queue occupancy exceeds the contribution threshold, such as 60%, the arbitrator triggers the sending of a first-level backpressure signal 0x1 to suspend the batch prefetching data transmission request from the source end; when the overall queue level exceeds a higher emergency threshold, such as 95%, the arbitrator triggers the sending of a second-level backpressure signal 0x2, which is used to suspend requests for both batch prefetching and result write-back intentions.

[0045] Example 8: Under high load conditions, multiple computing dies simultaneously initiate massive batch prefetch requests to the inter-die interconnect interface, causing the overall queue level of the downstream context fusion arbitrator to continuously rise. When the congestion monitor detects that the overall queue level exceeds the calibrated 80% congestion threshold and determines that the batch prefetch traffic is the main source of congestion, the context fusion arbitrator broadcasts a level 1 backpressure signal 0x1 to all upstream intent capture and encoding units through the reverse path of the out-of-band intent signaling bus. All upstream computing dies' intent capture and encoding units receive this 0x1 signal and, according to the configured response logic, suspend sending all new batch prefetches at their respective source ends. The data packet of intent, while the request for result write-back and pause avoidance intent is not affected by this backpressure signal; during this congestion and Level 1 backpressure state, the computing core PE_X urgently initiates a pause avoidance request due to computing dependency; the backpressure mechanism is intent-selective, and the intent capture and encoding unit of PE_X determines that the 0x1 backpressure signal is not applicable to pause avoidance intent, the request is normally encoded and sent out without blocking through the high-priority channel of the source intent virtual queue; the downstream context fusion arbitrator receives the pause avoidance intent code 11, performs a preemptive priority decision, grants the Die interconnect interface to the request, and completes the transmission before all batch prefetch packets still in the queue.

[0046] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A dynamic bandwidth allocation system in a chip stacking system, the system being applied in a stacked structure comprising compute dies and memory dies, the compute dies having inter-die interconnect interfaces connected to the memory dies, characterized in that, include: The intent capture and encoding unit, located in the computing die, is used to capture the business intent associated with a data transmission request when the computing die initiates such a request. Encode business intent into intent codes; An out-of-band intent signaling bus is configured at the inter-die interconnection interface. The bus includes a forward path and a reverse path. The bus is used to: send intent codes from the intent capture and encoding unit to the context fusion arbitrator using the forward path; The context fusion arbitrator, located at the inter-die interconnection interface, has at least one queue inside. The arbitrator is used to receive data packets corresponding to data transmission requests; receive intent codes via the out-of-band intent signaling bus; and dynamically arbitrate the transmission of data packets based on the intent codes and the status of at least one queue. The context fusion arbitrator is also used to monitor the congestion status of the inter-die interconnection interface; when the congestion status exceeds a preset threshold, it sends an intent-selective backpressure signal to the intent capture and encoding unit through the reverse path. The intent capture and encoding unit is also used to respond to the intent selective backpressure signal, at the source end of the computation die, to pause or reduce the transmission of data transmission requests for bulk prefetch intents; Among them, the business intent represents the urgency of the data transmission request, and the urgency includes at least: pause avoidance intent and batch prefetch intent; the context fusion arbitrator includes arbitration logic, which is used to: when the intent code indicates a pause avoidance intent, assign the corresponding data packet the highest arbitration priority, and transmit it before the data packet with the batch prefetch intent. The network interface controller for computing dies includes multiple virtual intent queues, which are used to queue data packets of different service intent levels. The intent capture and encoding unit is also used to route data packets to the corresponding queues among the multiple virtual intent queues for queuing according to the intent code. The network interface controller includes a source arbitrator, which is used to select data packets from the virtual intent queues with higher priorities based on the priorities of the virtual intent queues and submit them to the context fusion arbitrator.

2. The dynamic bandwidth allocation system in a chip stacking system according to claim 1, characterized in that, The intent capture and encoding unit is also used to locally associate intent codes with unsent data packets in the internal data packet buffer managed by the intent capture and encoding unit; receive intent invalidation signaling from the computation die; and, in response to intent invalidation signaling, clear or mark the data packets associated with intent invalidation signaling in the internal data packet buffer as invalid before the data packets are sent to the context fusion arbitrator.

3. The dynamic bandwidth allocation system in a chip stacking system according to claim 1, characterized in that, The intent capture and encoding unit is also used to: encapsulate a copy of the intent code as an in-band intent code into the data packet; the context fusion arbitrator is also used to extract the in-band intent code from the data packet and compare whether the in-band intent code is consistent with the intent code received from the out-of-band intent signaling bus; if the two are inconsistent, a safe mode is triggered, and the arbitrator ignores the intent code received from the out-of-band intent signaling bus in the safe mode and performs arbitration based on the preset static arbitration rules.

4. The dynamic bandwidth allocation system in a chip stacking system according to claim 1, characterized in that, include: An intent sequence monitor, connected to the out-of-band intent signaling bus, classifies and counts intent codes transmitted on the out-of-band intent signaling bus within a sliding time window, generating a congestion prediction factor. The generation is at least in part based on the count value of the batch prefetching intent. The count value of the intention to avoid pauses A quantitative relationship between them, defined as: .

5. The dynamic bandwidth allocation system in a chip stacking system according to claim 4, characterized in that, The context fusion arbitrator is also used when congestion prediction factors... When the preset threshold is exceeded, a selective backpressure signal is triggered.

6. The dynamic bandwidth allocation system in a chip stacking system according to claim 1, characterized in that, include: The intent signaling integrity monitor is connected to the out-of-band intent signaling bus and is used to monitor the out-of-band intent signaling bus and detect signaling fault conditions that indicate physical failure of the bus. When a signaling failure condition is detected, a fail-safe preemption signal is sent to the context fusion arbitrator; The context fusion arbitrator is also used to respond to failure-safe preemption signals, switch to safe mode, and in safe mode, ignore intent codes and arbitrate based on preset static arbitration rules.

7. The dynamic bandwidth allocation system in a chip stacking system according to claim 6, characterized in that, Signaling failure conditions include at least one of the following: the monitor receives an intent code defined as illegal encoding from the out-of-band intent signaling bus, or the signal level of the out-of-band intent signaling bus does not change within a preset timeout threshold.

8. The dynamic bandwidth allocation system in a chip stacking system according to claim 1, characterized in that, include: The intent profiling analyzer communicates with the context fusion arbitrator. The intent profiling analyzer is used to make authorization decisions for data packets based on the context fusion arbitrator, and to classify and statistically analyze the computational die from different sources and authorization events for different business intents. Based on classification statistics, pathological sources whose transmission behavior patterns exceed the preset profile threshold are identified; Generate an administrative alert that indicates the source of the problem has been identified.

Images