Computing device, data processing method and computing system

Abstract

The present application provides a computing device, a data processing method, and a computing system. The computing device may include: one or more stream processor clusters, each stream processor cluster is configured to perform computing processing to generate a data stream; an on-chip network circuit is configured to send the data stream to a peer communication circuit; and the peer communication circuit is configured to determine whether to suppress sending a set of successful responses to the data stream to the on-chip network circuit in response to a congestion state of a peer communication link between the peer communication circuit and a second computing device, wherein each stream processor cluster is also configured to release the current data to be transmitted in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the data stream from the on-chip network circuit.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a computing device, a data processing method, and a computing system. Background Art

[0002] In modern high-performance computing, some workloads require data exchange between two or more Graphic Processing Units (GPUs) located in the same server. The usual approach is to first copy the data from the GPU (and the associated storage module) to the host's fixed shared memory through the Central Processing Unit (CPU) and the Peripheral Component Interconnect (PCI) (e.g., PCIe). Then, the data is copied from the host's fixed shared memory to the target GPU (and the associated storage module) through the CPU and the bus (e.g., PCI). The data needs to be copied twice before reaching the destination.

[0003] To speed up processing, GPU peer-to-peer (P2P) communication technology (peer-to-peer technology) has emerged, in which data can be copied from a source GPU (associated storage module) to another GPU (associated storage module) without the need to temporarily store the data in the host memory first. If two GPUs are connected to the same bus, GPU peer-to-peer communication technology allows access to their corresponding memory without CPU involvement, thus halving the number of copy operations required to perform the same task.

[0004] However, since the amount of data generated by the GPU is large and the communication bandwidth is large, the peer-to-peer communication link between GPUs may be congested. Therefore, how to process the data stream generated at the GPU when the peer-to-peer communication link is congested so that the entire system can run normally is an urgent problem to be solved. Summary of the invention

[0005] According to one aspect of the present application, a computing device is provided, comprising: one or more stream processor clusters, each stream processor cluster being configured to perform computing processing to generate a data stream; an on-chip network circuit being configured to send the data stream to a peer communication circuit; and the peer communication circuit being configured to determine whether to suppress sending a set of successful responses for the data stream to the on-chip network circuit in response to a congestion state of a peer communication link between the peer communication circuit and a second computing device, wherein each stream processor cluster is further configured to release the current data to be transmitted in the data stream in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the data stream from the on-chip network circuit.

[0006] According to an embodiment of the present application, each stream processor cluster is also configured to perform the computational processing to generate a second data stream, wherein the on-chip network circuit is further configured to: send the second data stream to a storage device, and send a second successful response set for the second data stream from the storage device to the corresponding stream processor cluster, wherein each stream processor cluster also releases the current data to be transmitted in the second data stream in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the second data stream from the on-chip network circuit.

[0007] According to an embodiment of the present application, the peer communication circuit includes a buffer controller and a buffer, wherein the congestion state of the peer communication link is associated with the remaining buffer space of the buffer.

[0008] According to an embodiment of the present application, the buffer controller is configured to compare the remaining buffer space of the buffer with one or more thresholds to determine the congestion state of the peer communication link.

[0009] According to an embodiment of the present application, the buffer controller is configured to: determine that the congestion state is not congested when the remaining buffer space is greater than a maximum threshold value among the one or more threshold values; determine that the congestion state is severely congested when the remaining buffer space is less than or equal to a minimum threshold value among the one or more threshold values; and determine that the congestion state is generally congested when the remaining buffer space is greater than the minimum threshold value and less than or equal to the maximum threshold value.

[0010] According to an embodiment of the present application, the buffer controller is configured to: in response to the congestion state being severe congestion, send a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit or prohibit sending a success response corresponding to the current data to be transmitted to the on-chip network circuit.

[0011] According to an embodiment of the present application, the buffer controller is also configured to: determine the sending frequency of the successful response set for the data flow to the on-chip network circuit based on the size relationship between the remaining buffer space and one or more thresholds, wherein the sending frequency is positively correlated with the remaining buffer space.

[0012] According to an embodiment of the present application, the buffer controller is further configured to detect the remaining buffer space of the buffer included in the peer communication circuit whenever a predetermined amount of data to be transmitted is received; or to detect the remaining buffer space periodically or regularly.

[0013] According to an embodiment of the present application, each stream processor cluster includes multiple computing units connected to the on-chip network circuit via a cluster bus interface, and each computing unit corresponds to a data stream, wherein each of the multiple computing units includes a load temporary storage module, and one or more computing units of the multiple computing units correspond to a to-do queue module, wherein the load temporary storage module of each computing unit is configured to send new to-be-transmitted data to the corresponding to-do queue module when there is remaining queue space in the corresponding to-do queue module, so as to be included in the data stream corresponding to the computing unit and thus be sent.

[0014] According to an embodiment of the present application, the to-do queue module is configured to: in response to receiving a successful response from the on-chip network circuit for current to-be-transmitted data in a data stream corresponding to the computing unit, release the current to-be-transmitted data in the corresponding to-be-transmitted queue module.

[0015] According to an embodiment of the present application, the corresponding to-be-transmitted queue module includes each to-be-transmitted data in the corresponding data stream in a corresponding data request, and sends it to the on-chip network circuit via the cluster bus interface.

[0016] According to an embodiment of the present application, the computing device includes a graphics processing unit, a tensor processing unit, or a general-purpose graphics processing unit.

[0017] According to another aspect of the present application, a data processing method is also provided, which is performed by a computing device, and the method includes: each stream processor cluster in one or more stream processor clusters performs computing processing to generate a data stream; sending the data stream to a peer communication circuit via an on-chip network circuit; the peer communication circuit determines, in response to a congestion state of a peer communication link between the peer communication circuit and a second computing device, whether to suppress sending a set of successful responses for the data stream to the on-chip network circuit; and each stream processor cluster releases the current data to be transmitted in the data stream in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the data stream from the on-chip network circuit.

[0018] According to an embodiment of the present application, each stream processor cluster performs the computing processing and also generates a second data stream, and the method also includes: the on-chip network circuit sends the second data stream to a storage device; and each stream processor cluster releases the current data to be transmitted in the second data stream in the stream processor cluster in response to receiving a successful response from the on-chip network circuit for the current data to be transmitted in the second data stream.

[0019] According to an embodiment of the present application, the congestion status of the peer communication link is associated with the remaining buffer space of the peer communication circuit.

[0020] According to an embodiment of the present application, the method further includes: detecting, by the peer communication circuit, a remaining buffer space, and comparing the remaining buffer space with one or more thresholds to determine the congestion state of the peer communication link.

[0021] According to an embodiment of the present application, determining the congestion state includes: when the remaining buffer space is greater than a maximum threshold value among the one or more threshold values, determining that the congestion state is not congested; when the remaining buffer space is less than or equal to a minimum threshold value among the one or more threshold values, determining that the congestion state is severely congested; and when the remaining buffer space is greater than the minimum threshold value and less than or equal to the maximum threshold value, determining that the congestion state is a moderate degree of congestion.

[0022] According to an embodiment of the present application, the method also includes: in response to the congestion state being severe congestion, the peer communication circuit sends a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit or prohibits sending a success response corresponding to the current data to be transmitted to the on-chip network circuit.

[0023] According to an embodiment of the present application, the method also includes: the peer communication circuit determines the sending frequency of the successful response set for the data stream to the on-chip network circuit based on the size relationship between the remaining buffer space and one or more thresholds, wherein the sending frequency is positively correlated with the remaining buffer space.

[0024] According to another aspect of the present application, a computing system is further provided, comprising a plurality of computing devices connected via a bus, wherein each computing device in the plurality of computing devices is the computing device described above.

[0025] According to the scheme of the embodiment of the present application, the computing device can determine whether to suppress the transmission of data from the stream processor cluster to the peer communication circuit based on the remaining buffer space of the buffer in the peer communication circuit (which can be associated with congestion), thereby ensuring that the on-chip network circuit can normally send the data to be transmitted from the stream processor cluster or other stream processor clusters to the storage device, thereby improving the data processing efficiency of the computing device and ensuring the execution of other tasks when congestion occurs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0027] Figure 1 A schematic diagram of a computing system including multiple computing devices according to an embodiment of the present application is shown.

[0028] Figure 2 A schematic diagram of another computing system including multiple computing devices according to an embodiment of the present application is shown.

[0029] Figure 3 Shows Figure 2 The stream processor cluster in the computing device shown includes an example structure of multiple computing units.

[0030] Figure 4 A schematic flow chart of a method for controlling a data flow according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] As mentioned earlier, the peer-to-peer communication link between GPUs may be congested. In this case, if one GPU still sends a data stream to another GPU, all communication resources may be occupied by the peer-to-peer communication, which may affect the execution of other tasks except the peer-to-peer communication task.

[0033] Figure 1 A schematic diagram of a computing system including multiple computing devices according to an embodiment of the present application is shown. The computing device may be a hardware accelerator such as a graphics processing unit (GPU), a tensor processing unit (TPU) or a general graphics processing unit (GPGPU), and may be implemented as a chip. In the embodiment of the present application, the computing device is a GPU as an example for explanation.

[0034] like Figure 1 As shown, multiple GPUs (10-1, 10-2, ... 10-N, collectively referred to as 10) are connected to each other via a bus 20 (e.g., PCIe) for peer-to-peer communication (P2P communication). Therefore, in this application, the communication link between each two GPUs 10 is also referred to as a peer-to-peer (P2P) communication link.

[0035] Each GPU 10 may include one or more stream processor clusters (SPCs) 102, a network on chip (NOC) circuit 104, and a peer communication circuit 106. In addition, the GPU 10 may also include a storage device 108 (e.g., high-bandwidth memory (HBM)), or may also include or be communicatively connected to a storage device 108 for interacting with the CPU (e.g., L1 / L2 cache, etc.).

[0036] Optionally, each stream processor cluster 102 may include multiple computing units (CUs) (not shown), and these computing units may be connected to the on-chip network circuit 104. In addition, when there are multiple stream processor clusters, each of the multiple stream processor clusters 102 may be connected to the on-chip network circuit 104 to utilize the communication resources of the on-chip network circuit 104 to send data streams to peer communication circuits via the on-chip network circuit 104, and / or send data streams to the storage device 108.

[0037] In the present application, a data stream may refer to a plurality of data generated at each stream processor cluster 102, which includes continuously generated new data accompanying the ongoing computational processing at the stream processor cluster 102 for transmission to the on-chip network circuit 104. Since each stream processor cluster 102 may process different tasks, in addition to the data stream transmitted to another GPU via the on-chip network circuit 104 and the peer communication circuit 106, a second data stream DS2 may be generated and transmitted to the storage device 108 (e.g., L2 / HBM) via the on-chip network circuit 104 for further processing.

[0038] Although Figure 1 It is shown that each stream processor cluster corresponds to a data stream DS1 (going to the line 106 for peer communication, i.e., the first data stream) and a second data stream DS2 (going to the storage device 108), but it may not be limited to this in practice. For example, each stream processor cluster may correspond to two or more data streams and two or more second data streams, or the data generated by two or more stream processor clusters 102 may jointly form a data stream and a second data stream. Since the data in each data stream is data to be transmitted from the computing device to other locations (e.g., other computing devices) via a peer communication link, it is also called data to be transmitted. The data to be transmitted in each data stream can be transmitted between the components in a corresponding computing device.

[0039] The on-chip network circuit 104 may be located between two components (eg, Figure 1 The stream processor cluster 102 and the peer communication circuit 106 and the stream processor cluster 102 and the storage device 108 shown in the figure are used to provide communication resources for data exchange, that is, to build a circuit structure similar to a "computer network" at the hardware level to exchange data between the various components, which has the advantages of short channel distance and fast data transmission rate.

[0040] The peer communication circuit 106 may be a module for managing peer communication between computing devices, which may be used to manage data to be transmitted via a peer communication link, for example, may include a buffer so that data to be transmitted via peer communication may be placed in the buffer before being sent.

[0041] exist Figure 1 In the peer communication process in which the first GPU 10-1 sends a data stream to the second GPU 10-2, one of the stream processor clusters 102 may perform computational processing to generate a data stream DS1, and send it to the peer communication circuit 106 via the on-chip network circuit 104, and then the peer communication circuit 106 may send the data stream to the second GPU via the peer communication link. In the present application, the peer communication link between two GPUs may be equivalent to a peer communication link between a peer communication circuit in one GPU and another GPU or its peer communication circuit.

[0042] Generally, the internal bandwidth of a stream processor cluster is much higher than the peer communication link, and there may be multiple stream processor clusters. Since the bandwidth of the peer communication link is relatively low, congestion of the peer communication link is likely to occur. This congestion phenomenon will cause the buffer in the peer communication circuit to quickly reach saturation, thereby back-pressing the network on chip (NOC) circuit 104, and this back-pressure will further back-press the cluster bus interface (CBI) and each computing unit (CU) of the stream processor cluster.

[0043] As described above, in addition to the data stream sent to another GPU via the on-chip network circuit 104 and the peer communication link 106, each stream processor cluster 102 may also generate a second data stream DS2 that is sent to the storage device 108 (e.g., L2 / HBM) via the on-chip network circuit 104 for further processing. Therefore, if the network on chip (NOC) circuit 104, the cluster bus interface (CBI) and each computing unit (CU) are back-pressured due to congestion, even if there is no problem with the communication link from the computing unit inside the stream processor cluster 102 to the storage device 108, the data of the computing unit inside the stream processor cluster 102 cannot be sent to the storage device 108, thereby affecting the data processing performance of the system.

[0044] Therefore, an embodiment of the present application proposes an improved computing device for this situation, which can determine whether to suppress data transmission from each stream processor cluster to the peer communication circuit based on the congestion status of the peer communication link, thereby ensuring that the on-chip network circuit can normally send data from each stream processor cluster to the storage device.

[0045] Figure 2A schematic diagram of another computing system including multiple computing devices according to an embodiment of the present application is shown. Figure 2 Computing systems and Figure 1 The overall architecture of the computing system is similar to that of the present invention, but more details are shown to better illustrate the solution of the present application.

[0046] like Figure 2 As shown, similarly, each computing device (e.g., GPU) 200 may include a stream processor cluster (SPC) 202, a network on chip (NOC) circuit 204, and a peer communication circuit 206. In addition, the computing device may further include a storage device 208 (e.g., high bandwidth memory (HBM)), or may further include or be communicatively connected to a storage device 208 for interacting with the CPU (e.g., L1 / L2 cache, etc.). Figure 2 Only one stream processor cluster 202 is shown in the figure, but it should be understood that there may be multiple stream processor clusters 202 , and these stream processor clusters 202 may share the on-chip network circuit 204 .

[0047] Each stream processor cluster (SPC) 202 may be configured to perform computational processing to generate a data stream DS1, and the on-chip network circuit (NOC) 204 may be configured to send the data stream DS1 to the peer communication circuit 206. The data stream DS1 sent from each stream processor cluster (SPC) 202 to the peer communication circuit 206 via the on-chip network circuit 204 may be understood as data sent one after another in sequence. Figure 2 In the figure, each stream processor cluster (SPC) 202 is shown to correspond to one data stream DS1 to the peer communication circuit 106, but it should be understood that each stream processor cluster (SPC) 202 can correspond to multiple data streams to the peer communication circuit 106, and the operation for each data stream in various embodiments is similar. Each stream processor cluster (SPC) 202 can temporarily store or save various to-be-transmitted data that have been sent to the peer communication circuit 206 until receiving an instruction to release the to-be-transmitted data.

[0048] The peer-to-peer communication circuit (P2P) 206 may be configured to determine whether to suppress sending a set of successful responses (Resp1.) for the data stream DS1 to the on-chip network circuit 204 in response to the congestion state (e.g., severe congestion, moderate congestion, or no congestion) of the peer-to-peer communication link between the peer-to-peer communication circuit 206 and the second computing device. For example, the peer-to-peer communication circuit 206 may return a successful response to the stream processor cluster 202 that sent the data to be transmitted via the on-chip network circuit 204 each time a data to be transmitted is successfully written to its buffer, so multiple successful responses may be returned for multiple data to be transmitted of the data stream. In addition, the stream processor cluster 202 may release the current data to be transmitted in the stream processor cluster 202 in response to receiving a successful response (indication as described above) for the current data to be transmitted from the on-chip network circuit 204. For example, if a data to be transmitted is sent from the stream processor cluster 202, in some cases, the stream processor cluster 202 may send subsequent data to be transmitted after sending the data to be transmitted, regardless of whether a successful response is received, but the data to be transmitted may be released only in response to a successful response to the data to be transmitted (the embodiments of the present disclosure are mainly described by taking this as an example); or in other cases, the stream processor cluster 202 may send the next data to be transmitted and may release the data to be transmitted only after receiving a successful response to the data to be transmitted (that is, the operation of sending the next data to be transmitted and the operation of releasing the current data to be transmitted are performed in association). On the contrary, if the stream processor cluster 202 does not receive a successful response to the data to be transmitted, the data to be transmitted may not be released first (or the next data to be transmitted may be optionally not sent) until the successful response is received. If the stream processor cluster 202 does not release the placement space, the placement space of the stream processor cluster 202 will be full, so the new data to be transmitted in the data stream generated by the stream processor cluster 202 relative to the data to be transmitted that has not been released in the current placement space will not be able to continue to be placed in the stream processor cluster 202, so that the stream processor cluster 202 cannot send these new data to be transmitted via the on-chip network circuit 204; and after the stream processor cluster 202 releases at least a part of the placement space and thus has free placement space, the new data to be transmitted in the data stream generated by the stream processor cluster 202 relative to the data to be transmitted that has not been released in the current placement space can be placed in the free placement space, so that these new data to be transmitted can be sent to the peer communication circuit 206 via the on-chip network circuit 204.

[0049] Ideally, for example, when the peer communication link is not congested, the peer communication circuit 206 does not suppress the sending of a successful response, that is, when a data to be transmitted in the data stream from a stream processor cluster 202 is written into the buffer, the on-chip network circuit 204 will return the reception indication information for the data to be transmitted to the stream processor cluster 202, such as an acknowledgement, and it can be based on a request-response mode, that is, the data to be transmitted can be included in the data request and sent to the peer communication circuit 206, and the peer communication circuit 206 returns a successful response. When the stream processor cluster 202 receives a successful response to each data to be transmitted, it will normally release each data to be transmitted in the stream processor cluster, so that new data to be transmitted can continue to be sent from the stream processor cluster to the on-chip network circuit 204.

[0050] In addition, in the case of congestion in the peer communication link, the peer communication circuit 206 may not send a successful response to the current data to be transmitted in the data stream from the stream processor cluster 202 or reduce the frequency of sending a successful response to achieve the suppression of the successful response set for the data stream, so that the stream processor cluster 202 can suspend the release of the corresponding data to be transmitted, but because the stream processor cluster 202 may still be performing computing processing and thus continuously generating data to be transmitted in the data stream, when the placement space at the stream processor cluster 202 is full, the new data to be transmitted in the data stream relative to the unreleased data to be transmitted cannot continue to be placed at the stream processor cluster 202, so these new data to be transmitted cannot be sent or the sending frequency is reduced. Therefore, the on-chip network circuit 204 can have sufficient communication resources for the communication between the stream processor cluster 202 or other stream processor clusters 202 and the storage device 208.

[0051] Optionally, each stream processor cluster 202 may also be configured to perform computational processing to generate a second data stream DS2 to the storage device 208. Figure 2 , one second data stream DS2 corresponding to each stream processor cluster (SPC) 202 is shown, but it should be understood that each stream processor cluster (SPC) 202 may correspond to multiple second data streams, and the operations for each data stream in various embodiments are similar.

[0052] The on-chip network circuit 204 can also be configured to: send the second data stream to the storage device 208, and send the second successful response set for the second data stream from the storage device 208 to the corresponding stream processor cluster. For example, the storage device 208 can return a successful response to the stream processor cluster 202 that sent the data to be transmitted via the on-chip network circuit 204 every time a data to be transmitted is successfully written, so multiple successful responses can be returned for multiple data to be transmitted from the second data stream. Similarly, each stream processor cluster 202 also releases the current data to be transmitted in the second data stream in the stream processor cluster 202 in response to receiving a successful response to the current data to be transmitted in the second data stream from the on-chip network circuit 204.

[0053] In some embodiments, the peer communication circuit 206 may include a buffer controller 206-1 and a buffer 206-2. If the data to be transmitted in the buffer 206-2 cannot be sent at its normal frequency due to congestion in the peer communication circuit relative to when there is no congestion, but the data to be transmitted from the stream processor cluster 202 is still written to the buffer 206-2 at its normal frequency, the data to be transmitted in the buffer 206-2 will accumulate, resulting in the remaining buffer space (for example, Figure 2 Therefore, the congestion state of the peer communication circuit can also be reflected by the remaining buffer space of the buffer 206-2 in the peer communication circuit 206.

[0054] For example, the buffer controller 206-1 may be configured to detect the remaining buffer space of the buffer 206-2 (eg, Figure 2 The remaining buffer space of the buffer 206-2 is compared with one or more thresholds to determine the congestion state of the peer communication link between the peer communication circuit 206 and the second computing device. The numerical form of the remaining buffer space of the buffer may include the actual numerical value of the remaining buffer space, or the percentage or ratio of the actual numerical value of the remaining buffer space relative to the numerical value of the total buffer space of the buffer, and the numerical form of each corresponding threshold is also defined according to the numerical form of the remaining buffer space.

[0055] Optionally, different congestion states may result in different remaining buffer spaces of the buffer 206-2, so that the buffer controller 206-1 may perform different control operations in response to different remaining buffer spaces, and therefore, the remaining buffer space of the buffer 206-2 needs to be constantly detected during the operation of the computing device. For example, the buffer controller 206-1 may be configured to detect the remaining buffer space of the buffer whenever a predetermined amount of data to be transmitted (which may belong to different data streams) is received (e.g., detect once in response to each data request, or detect once in response to two or more data requests); or, detect the remaining buffer space periodically or regularly (e.g., at multiple predetermined time points).

[0056] For example, when the remaining buffer space is greater than the maximum threshold value among the one or more threshold values, that is, there is enough remaining buffer space in the buffer 206-2, the buffer controller 206-1 may determine that the congestion state of the peer communication link is not congested. For another example, when the remaining buffer space is less than or equal to the minimum threshold value among the one or more threshold values, that is, the remaining buffer space is relatively small, the buffer controller 206-1 may determine that the congestion state of the peer communication link is severely congested, because due to the existence of the congestion phenomenon, the buffer 206-2 has no time to send the data from the on-chip network circuit 204, so that these data may be accumulated at the buffer 206-2, so that if no further control is performed, it is very likely to cause back pressure on the on-chip network circuit 204 as described above. For another example, when the remaining buffer space is greater than the minimum threshold value and less than or equal to the maximum threshold value, that is, the remaining buffer space has not yet reached the stage where adverse consequences will occur, but certain measures may need to be taken to avoid subsequent adverse consequences, the buffer controller 206-1 determines that the congestion state of the peer communication link is generally congested.

[0057] In the case where the number of thresholds is 1, the maximum threshold and the minimum threshold are the same. Therefore, when the remaining buffer space is less than or equal to the same threshold, that is, the congestion state is severe congestion, the buffer controller 206-1 can send a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit 204 or prohibit sending a success response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit 204, so that when the placement space of the stream processor cluster is full, the new data to be transmitted in the data stream cannot continue to be placed at the stream processor cluster and cannot be sent to the on-chip network circuit 204 until a success response is received. When the remaining buffer space is greater than the same threshold, that is, the congestion state is not congested, the buffer controller 206-1 can normally send a success response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit 204, so that the stream processor cluster 202 can normally release the data to be transmitted, thereby freeing up free placement space for placing new data to be transmitted, and thus can continue to send subsequent data to be transmitted in the data stream.

[0058] When the number of thresholds is greater than 1, and when the remaining buffer space is greater than the minimum threshold among the multiple thresholds, the congestion state of the peer communication link is determined to be moderately congested or non-congested (non-congested corresponds to the case where the remaining buffer space is greater than the maximum threshold among the multiple thresholds). In addition, the buffer controller may further perform different control methods based on the size relationship between the remaining buffer space and these thresholds.

[0059] For example, in response to the congestion state being non-congested, the buffer controller 206-1 can normally save the data to be transmitted into the buffer based on the data request including the data to be transmitted from the on-chip network circuit 204, and return a successful response for the data to be transmitted (i.e., the current data to be transmitted) to the on-chip network circuit 204. In addition, the on-chip network circuit 204 can also normally send the data to be transmitted of the second data stream to the second computing device via the peer communication link, that is, perform a normal data transmission process.

[0060] For another example, in response to the congestion state being severe congestion, the buffer controller 206-1 may send a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit 204 or prohibit sending a successful response to the current data to be transmitted to the on-chip network circuit 204. As described above, sending the next data to be transmitted of the current data to be transmitted in the data stream to the on-chip network circuit 204 is based on the successful response to the current data to be transmitted, so if no successful response to the current data to be transmitted is received (or a failure response is received), the stream processor cluster 202 suspends releasing the current data to be transmitted, although it may continue to send a data request including the next data to be transmitted in the data stream to the on-chip network circuit 204 (the operations of the stream processor cluster 202 sending the next data request and releasing the current data to be transmitted may be independent), but after the placement space of the stream processor cluster 202 is full, the stream processor cluster 202 can no longer store new data to be transmitted, so after all the data to be transmitted currently placed by the stream processor cluster 202 are sent, no data request including new data to be transmitted is sent to the on-chip network circuit 204.

[0061] In addition, optionally, the buffer controller 206-1 can also be configured to determine the sending frequency of sending a set of successful responses to the data stream to the on-chip network circuit 204 based on the size relationship between the remaining buffer space and the one or more thresholds, wherein the sending frequency is positively correlated with the remaining buffer space. That is, the larger the remaining buffer space, the higher the sending frequency of successful responses. Optionally, the sending frequency here can be defined relative to the successful response to the data to be transmitted for multiple data streams, or it can be defined relative to the successful response to the data to be transmitted for each data stream, with the latter being taken as an example below).

[0062] That is to say, under normal congestion conditions, the buffer controller may continue to return to the on-chip network circuit 204 various successful responses for each written data to be transmitted, but the sending frequencies of these successful responses may be different, for example, the 1st to 5th successful responses may be sent at a first frequency, and the 6th to 9th successful responses may be sent at a second frequency. In this way, the sending frequency of subsequent data to be transmitted in the data stream sent from the corresponding stream processor cluster 202 to the buffer 206-2 of the peer communication circuit based on the successful response can also be reduced (because the execution frequency of the release operation of the data to be transmitted at the stream processor cluster 202 can be reduced, that is, the release speed can be reduced). Different space intervals (defined by the one or more thresholds) where the remaining buffer space is located correspond to different sending frequencies.

[0063] For example, the buffer controller 206-1 may set a reference transmission frequency for successful responses, and then for all data streams or a set of successful responses for data to be transmitted in each data stream, under normal circumstances (for example, when the remaining buffer space is greater than a maximum threshold), the reference transmission frequency may be used for transmission, so that the stream processor cluster 202 may also basically transmit the data to be transmitted in the corresponding data stream to the peer communication circuit 206 according to the transmission frequency. Then, according to the size relationship between the remaining buffer space and each threshold, the successful response is sent relative to the reference transmission frequency F.

[0064] For example, the correspondence between the interval where the remaining buffer space is located and the different sending frequencies can be predetermined. For example, one or more thresholds can be expressed as a percentage relative to the total buffer space, and can include 1 / 2, 1 / 4, 1 / 8 and 1 / 16. When the remaining buffer space is between 1 / 2 and 1 / 4, each successful response can be sent at a sending frequency of 0.5F; when the remaining buffer space is between 1 / 4 and 1 / 8, each successful response can be sent at a sending frequency of 0.25F; when the remaining buffer space is between 1 / 8 and 1 / 16, each successful response can be sent at a sending frequency of 0.125F; in addition, when the remaining buffer space is less than or equal to 1 / 16, the sending of successful responses can be suspended; when the remaining buffer space is greater than 1 / 2, each successful response can be sent at the base sending frequency F.

[0065] Since the congestion state of the peer communication link may be changing, after the congestion is alleviated, the buffer controller may detect that the remaining buffer space changes from a smaller value to a larger value, so that the corresponding buffer controller can return a success response more quickly, so that the corresponding data to be transmitted of the stream processor cluster can be released more quickly. Therefore, when the placement space of the stream processor cluster is full, the free placement space vacated based on the released data to be transmitted can be used more quickly to store new data to be transmitted, so that the new data to be transmitted can be sent and stored in the buffer for sending to the peer second computing device via the peer communication link.

[0066] Therefore, through Figure 2 The computing device shown can determine whether to suppress data transmission from the stream processor cluster to the peer communication circuit based on the remaining buffer space of the buffer in the peer communication circuit (which can be associated with congestion), thereby ensuring that the on-chip network circuit can normally send the data to be transmitted from the stream processor cluster or other stream processor clusters to the storage device, thereby improving the data processing efficiency of the computing device and ensuring the execution of other tasks when congestion occurs.

[0067] Figure 3 Shows Figure 2The stream processor cluster in the computing device shown includes an example structure of multiple computing units. Figure 2 Other components in the computing device are also included Figure 3 Shown in.

[0068] like Figure 3 As shown, each stream processor cluster 302 of the computing device 300 may include multiple computing units (CUs), and these computing units may be connected to the on-chip network circuit 304 via a cluster bus interface (CBI). Each computing unit CU in each stream processor cluster 302 may be a processing core (Core), and may include a load temporary storage module 310 and a to-do queue module 320. The load temporary storage module 310 and the to-do queue module 320 may also be formed by hardware. Figure 3 In , each computing unit CU can correspond to a data stream. Figure 3 It is schematically shown that each computing unit CU corresponds to a to-do queue module 320, but it should be understood that one or more computing units CU can share a to-do queue module 320, that is, the to-be-transmitted data generated by these computing units CU can be placed in the same to-do queue module for sending.

[0069] like Figure 3 The load temporary storage module 310 of each computing unit CU shown can be used to temporarily store the data to be transmitted generated by the computing unit CU performing computing processing, and is configured to send the data to be transmitted generated at the computing unit to the corresponding to-be-transmitted queue module when there is remaining queue space in the corresponding to-be-transmitted queue module 320 so as to be included in a data stream corresponding to the computing unit, thereby being further transmitted.

[0070] For example, the to-do queue module 320 corresponding to each computing unit can be configured to release the current to-be-transmitted data in the to-do queue module in response to receiving a successful response from the on-chip network circuit 304 for the current to-be-transmitted data in a data stream corresponding to the computing unit, wherein the placement space of the released current to-be-transmitted data in the to-do queue module is used to place the next to-be-transmitted data after the released current to-be-transmitted data, for example, shifting one placement space in sequence. Alternatively, the placement space of the released current to-be-transmitted data in the to-do queue module is used to place new to-be-transmitted data relative to the to-be-transmitted data that has been placed in the to-be-transmitted queue module.

[0071] Optionally, the storage space (queue space) corresponding to each queue position in the to-be-transmitted queue module 320 corresponding to each computing unit can be used to place a to-be-transmitted data in a data stream corresponding to the computing unit, and the to-be-transmitted queue module 320 can include each to-be-transmitted data in the data stream in a corresponding data request and send it to the on-chip network circuit via the cluster bus interface CBI.

[0072] For example, the to-be-transmitted data in the to-be-transmitted queue module 320 corresponding to each computing unit CU can accommodate a predetermined number of to-be-transmitted data at most, and each to-be-transmitted data occupies a queue space of the to-be-transmitted queue module 320 and corresponds to a queue position (for example, it can be identified by a position identifier). Each to-be-transmitted data can be placed and released according to a first-in-first-out (FIFO) mechanism. The to-be-transmitted data in the to-be-transmitted queue module 320 can be released in sequence according to the successful response received from the on-chip network circuit 204. After the released to-be-transmitted data is released, new queue space can be freed up, so that new to-be-transmitted data from the load temporary storage module 310 can be allowed to be written into the to-be-transmitted queue module 320.

[0073] For example, when the data to be transmitted at the first queue position of the to-be-transmitted queue module 320 (the earliest data in the current queue in terms of time) is sent to the on-chip network circuit 204 and then to the peer communication circuit 306, if the peer communication circuit 306 returns a successful response to the first data to be transmitted, the to-be-transmitted queue module 320 can release the first data to be transmitted, so that the data to be transmitted at the second queue position (the second earliest data in the current queue in terms of time) can be placed at the queue position previously occupied by the first data to be transmitted, and the data to be transmitted at each queue position in the to-be-transmitted queue module 320 moves forward one queue position in turn. Assuming that there is still remaining queue space in the to-be-transmitted queue module 320 when the first data to be transmitted is sent, the load temporary storage module 310 can write the new data to be transmitted into the to-be-transmitted queue module 320 in turn according to its internally set logic until there is no remaining queue space. Since the process of the load storage module 310 writing data into the to-do queue module 320 is asynchronous with the process of the peer communication circuit 304 transmitting the data to the second computing device, if there is still queue space left in the to-do queue module 320, the load storage module 310 can write data into the to-do queue module 320 even if the to-do queue module 320 does not receive a successful response (i.e., the queue space is not released).

[0074] Assuming that the pending queue module 320 is full when the first data to be transmitted is sent, after the data to be transmitted at the first queue position is released in response to the corresponding successful response, the load temporary storage module 310 can write new data to be transmitted to the pending queue module 320 as the data to be transmitted at the last queue position of the pending queue module 320 (the latest data in the current queue in terms of time). Correspondingly, if no successful response has been received for the first data to be transmitted, no data to be transmitted can be released at present, so the pending queue module 320 has no queue space for the load temporary storage module 310 to write new data to be transmitted. This is similar for each computing unit, so there will not be a large amount of data to be transmitted that needs to be sent via the on-chip network circuit, so the resources of the on-chip network circuit 204 can be used to send the data to be transmitted of the second data stream from another computing unit to the storage device 208.

[0075] In addition, the multiple computing units of each stream processor cluster 302 are connected to the on-chip network circuit 304 via the cluster bus interface, and their respective data to be transmitted can be sent to the on-chip network circuit in series via the cluster bus interface based on the corresponding to-do queue module as a data stream or a second data stream from the stream processor cluster 302, and sent to the peer communication circuit or storage device accordingly. In addition, the peer communication circuit can also return to the stream processor cluster 302 serially or in parallel respective success responses or failure responses for the respective data to be transmitted from the multiple computing units. Optionally, each data request can include the address information of the computing unit as the data source, so that the response returned by the peer communication circuit can be sent to the corresponding computing unit.

[0076] Therefore, by Figure 3As described above, in each computing unit of the stream processor cluster, a queue mechanism is used and based on whether a successful response is received, it is determined whether to release the queue space in the queue, so that when the queue is full, whether the new data to be transmitted written into the queue can continue to be sent to the on-chip network circuit, so that it is convenient to control whether to continue to write the data to be transmitted to the buffer in the peer communication circuit. In this way, when no successful response is received due to congestion or a successful response with a reduced sending frequency is received, the corresponding data to be transmitted in the queue may not be released or the execution frequency of the release operation may be reduced, so that when the queue no longer has free space, no new data to be transmitted is received, so it is possible to stop continuing to send the data to be transmitted to the peer communication circuit or reduce the frequency of sending the data to be transmitted, so that it can be ensured that the buffer in the peer communication circuit does not store too much data to be transmitted, so that the back pressure phenomenon described above can be avoided, and it can be ensured that the on-chip network circuit can normally send the data from the stream processor cluster to the storage device, thereby improving the data processing efficiency of the computing device and ensuring the execution of other tasks when congestion occurs.

[0077] In addition, as previously described, by suppressing data transmission from the on-chip network circuit to the peer communication circuit, the data transmission link from the on-chip network circuit to the storage device can be guaranteed to operate normally. Therefore, in this case, the to-be-transmitted queue module corresponding to the computing unit that generates the data to be transmitted to the storage device can continue to receive successful responses at a normal frequency and can be continuously released, and the load temporary storage module 310 can continue to write new data to be transmitted to the to-be-transmitted queue module, and the new data to be transmitted can also be continuously sent to the storage device via the on-chip network circuit.

[0078] According to another aspect of the present application, a method for controlling data flow is also provided.

[0079] Figure 4 FIG. 1 shows a flow chart of a method for controlling a data flow according to an embodiment of the present application. The control method may be Figure 2 and Figure 3 The computing device is used to execute.

[0080] For example, Figure 4 As shown, in step S410, each of the one or more stream processor clusters of the computing device performs computing processing to generate a data stream.

[0081] For example, as described above, the computing device may include multiple stream processor clusters, and each stream processor cluster may perform computing processing and generate corresponding data streams (eg, for sending to the Figure 2In addition, each stream processor cluster can also generate a corresponding second data stream (for example, for sending to Figure 2 Storage device 208 shown).

[0082] In step S420, the data stream is sent to the peer communication circuit via the on-chip network circuit.

[0083] For example, the computing device may include an on-chip network circuit, and all stream processor clusters included in the computing device are connected to the on-chip network circuit, so the communication resources of the on-chip network circuit are shared by all stream processor clusters. The number of data streams and second data streams generated by each stream processor cluster can be one or more, and can be sent via the on-chip network circuit.

[0084] In step S430, the peer communication circuit (P2P) determines whether to suppress sending a successful response set for the data flow to the on-chip network circuit in response to a congestion state of a peer communication link between the peer communication circuit and a second computing device.

[0085] For example, the computing device and the second computing device may be peers, and the data streams generated by each stream processor cluster of the computing device are intended to be sent to the peer communication circuit of the computing device to be sent to the second computing device via the peer communication circuit.

[0086] In step S440, each stream processor cluster releases the current data to be transmitted in the stream processor cluster in response to receiving a successful response from the on-chip network circuit for the current data to be transmitted in the data stream.

[0087] For example, each stream processor cluster will generate multiple data to be transmitted as the computing process is executed, and these data to be transmitted will be sent to the on-chip network circuit in turn, and then sent to the peer communication circuit. If the peer communication circuit returns a successful response to the current data to be transmitted that is currently written and is received by the stream processor cluster, the stream processor cluster can release the current data to be transmitted, and the new data to be transmitted can be placed in the placement space vacated based on the released current data to be transmitted for sending. On the contrary, if no successful response is received for the current data to be transmitted, the current data to be transmitted will not be released temporarily, so when the stream processor cluster is full, the new data to be transmitted cannot be placed at the stream processor cluster, so the stream processor cluster cannot continue to send new (additional) data to be transmitted to the on-chip network circuit until a successful response is received.

[0088] Optionally, as described above, each stream processor cluster can also perform computing processing to generate a second data stream, so the second data stream can also be sent to the storage device by the on-chip network circuit, and each stream processor cluster releases the current data to be transmitted in the second data stream in the stream processor cluster in response to receiving a successful response from the on-chip network circuit for the current data to be transmitted in the second data stream.

[0089] For example, a successful response for each data to be transmitted in the second data stream of each stream processor cluster can come from a storage device and be returned to the stream processor cluster via the on-chip network circuit, so that the stream processor cluster can release the current data to be transmitted in response to receiving a successful response to the current data to be transmitted, and new data to be transmitted can be placed in the placement space vacated based on the released current data to be transmitted for sending.

[0090] If the data to be transmitted in the buffer cannot be sent at its normal frequency relative to when there is no congestion due to congestion in the peer communication circuit, but the data to be transmitted from the stream processor cluster is still written to the buffer at its normal frequency, the data to be transmitted in the buffer will accumulate, causing the remaining buffer space to gradually decrease. Therefore, the congestion state of the peer communication circuit can also be reflected by the remaining buffer space of the buffer in the peer communication circuit. When it is necessary to detect the congestion state of the peer communication link to control the sending operation of the data to be transmitted from the stream processor cluster, the method can also include: detecting the remaining buffer space by the peer communication circuit, and comparing the remaining buffer space with one or more thresholds to determine the congestion state of the peer communication link.

[0091] For example, when the remaining buffer space is greater than the maximum threshold among the one or more thresholds, the congestion state of the peer communication link is determined to be non-congested; when the remaining buffer space is less than or equal to the minimum threshold among the one or more thresholds, the congestion state of the peer communication link is determined to be severely congested; and when the remaining buffer space is greater than the minimum threshold and less than or equal to the maximum threshold, the congestion state of the peer communication link is determined to be moderately congested.

[0092] Optionally, in response to the congestion state of the peer communication link being severe congestion, the peer communication circuit sends a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit, or prohibits sending a success response corresponding to the current data to be transmitted to the on-chip network circuit. Optionally, in response to the congestion state of the peer communication link being moderate congestion, the peer communication circuit determines the frequency of sending a set of successful responses to the data stream to the on-chip network circuit based on the size relationship between the remaining buffer space and the one or more thresholds, wherein the sending frequency is positively correlated with the remaining buffer space.

[0093] refer to Figure 4 More details on the operations in each step of the described method can be found in the previous references Figure 2 and Figure 3 The detailed description of the content is omitted here.

[0094] Therefore, through Figure 4 The method executed by the computing device shown can determine whether to suppress data transmission from the stream processor cluster to the peer communication circuit based on the remaining buffer space of the buffer in the peer communication circuit (which can be associated with congestion), thereby ensuring that the on-chip network circuit can normally send the data to be transmitted from the stream processor cluster or other stream processor clusters to the storage device, thereby improving the data processing efficiency of the computing device and ensuring the execution of other tasks when congestion occurs.

[0095] According to another aspect of the present application, a computing system is also provided. The computing system may be as follows Figure 1 The illustrated computing system includes multiple computing devices connected via a bus.

[0096] Each computing device in the computing system according to the embodiment of the present application may be as described above with reference to Figure 2 and Figure 3 Describe the computing device.

[0097] For example, each computing device may include one or more stream processor clusters (SPCs), a network on chip (NOC) circuit, and a peer communication circuit. Each stream processor cluster (SPC) may be configured to perform computing processing to generate a data stream. The network on chip circuit (NOC) may be configured to send the data stream to the peer communication circuit. The peer communication circuit (P2P) may be configured to determine whether to suppress sending a successful response set for the data stream to the network on chip circuit in response to the congestion state of the peer communication link between the peer communication circuit and the second computing device. Each stream processor cluster may also be configured to release the current data to be transmitted in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the corresponding data stream from the network on chip circuit.

[0098] More details on the various components in the computing device can be found in the previous reference Figure 2 and Figure 3 The detailed description of the content is omitted here.

[0099] Therefore, through this computing system, it is possible to determine whether to suppress data transmission from each stream processor cluster to the peer communication circuit based on the remaining buffer space of the buffer in the peer communication circuit (which can be associated with congestion), thereby ensuring that the on-chip network circuit can normally send the data to be transmitted from the stream processor cluster or other stream processor clusters to the storage device, thereby improving the data processing efficiency of the computing device and ensuring the execution of other tasks when congestion occurs.

[0100] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the methods and devices according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains at least one executable instruction for realizing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, or the various modules mentioned, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] In the present application, the expressions “first”, “second”, etc. are merely used to distinguish various elements but do not refer to any order, importance, priority, etc.

[0102] The embodiments of the present application described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present application, and such modifications should fall within the scope of the present application.

Claims

1. A computing device, comprising: one or more stream processor clusters, each stream processor cluster configured to perform computational processing to generate a data stream and a second data stream; an on-chip network circuit configured to send the data stream to a peer communication circuit, and send the second data stream to a storage device, and send a second set of successful responses from the storage device for the second data stream to a corresponding stream processor cluster; as well as The peer communication circuit is configured to determine whether to suppress sending a set of successful responses for the data flow to the on-chip network circuit in response to a congestion status of a peer communication link between the peer communication circuit and a second computing device, Each stream processor cluster is further configured to release the current data to be transmitted in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the data stream from the on-chip network circuit.

2. The computing device according to claim 1, in, Each stream processor cluster also releases the current data to be transmitted in the second data flow in the stream processor cluster in response to receiving a successful response to the current data to be transmitted in the second data flow from the on-chip network circuit.

3. The computing device of claim 1, wherein: The peer-to-peer communication circuit includes a buffer controller and a buffer, The congestion status of the peer communication link is associated with the remaining buffer space of the buffer.

4. The computing device of claim 3, wherein: The buffer controller is configured to compare the remaining buffer space of the buffer with one or more thresholds to determine the congestion status of the peer communication link.

5. The computing device of claim 4, wherein: The buffer controller is configured to: In a case where the remaining buffer space is greater than a maximum threshold value among the one or more threshold values, determining that the congestion state is non-congested; When the remaining buffer space is less than or equal to a minimum threshold value among the one or more threshold values, determining that the congestion state is severe congestion; as well as When the remaining buffer space is greater than the minimum threshold and less than or equal to the maximum threshold, it is determined that the congestion state is a moderate degree of congestion.

6. The computing device according to claim 4 or 5, wherein: The buffer controller is configured to: In response to the congestion state being severe congestion, a failure response corresponding to the current data to be transmitted in the data stream is sent to the on-chip network circuit, or a success response corresponding to the current data to be transmitted is prohibited from being sent to the on-chip network circuit.

7. The computing device of claim 3, wherein: The buffer controller is further configured to: determining a sending frequency of sending the successful response set for the data flow to the on-chip network circuit based on a size relationship between the remaining buffer space and one or more thresholds, The sending frequency is positively correlated with the remaining buffer space.

8. The computing device of claim 3, wherein: The buffer controller is further configured to: detect the remaining buffer space of the buffer included in the peer communication circuit whenever a predetermined amount of data to be transmitted is received; or to detect the remaining buffer space periodically or regularly.

9. The computing device of claim 1, wherein: Each stream processor cluster includes a plurality of computing units connected to the on-chip network circuit via a cluster bus interface, and each computing unit corresponds to a data stream, Each of the plurality of computing units includes a load temporary storage module, and one or more computing units of the plurality of computing units correspond to a to-do queue module. Among them, the load temporary storage module of each computing unit is configured to send new data to be transmitted to the corresponding to-be-transmitted queue module when there is remaining queue space in the corresponding to-be-transmitted queue module, so as to be included in the data flow corresponding to the computing unit and thus be sent.

10. The computing device of claim 9, wherein: The corresponding to-do queue module is configured as follows: In response to receiving a successful response from the on-chip network circuit to the current data to be transmitted in a data stream corresponding to the computing unit, the current data to be transmitted in the corresponding to-be-transmitted queue module is released.

11. The computing device of claim 9, wherein: The corresponding to-do queue module includes each to-be-transmitted data in a corresponding data request and sends the request to the on-chip network circuit via the cluster bus interface.

12. The computing device of claim 1, wherein: The computing device includes a graphics processing unit, a tensor processing unit, or a general-purpose graphics processing unit.

13. A data processing method, executed by a computing device, the method comprising: performing computational processing by each of the one or more stream processor clusters to generate a data stream and a second data stream; sending the data stream to a peer communication circuit via an on-chip network circuit, and sending the second data stream to a storage device via the on-chip network circuit and sending a second set of successful responses from the storage device for the second data stream to a corresponding stream processor cluster; determining, by the peer communication circuit in response to a congestion status of a peer communication link between the peer communication circuit and a second computing device, whether to suppress sending a set of successful responses for the data flow to the on-chip network circuit; as well as In response to receiving a successful response to the current data to be transmitted in the data flow from the on-chip network circuit, each stream processor cluster releases the current data to be transmitted in the stream processor cluster.

14. The method according to claim 13, further comprising: In response to receiving a successful response to the current data to be transmitted in the second data flow from the on-chip network circuit, each stream processor cluster releases the current data to be transmitted in the second data flow in the stream processor cluster.

15. The method according to claim 13, wherein: The congestion status of the peer to peer communication link is associated with a remaining buffer space of the peer to peer communication circuit.

16. The method according to claim 15, further comprising: Remaining buffer space is detected by the peer communication circuit and compared to one or more thresholds to determine the congestion status of the peer communication link.

17. The method according to claim 16, wherein: Determining the congestion state includes: In a case where the remaining buffer space is greater than a maximum threshold value among the one or more threshold values, determining that the congestion state is non-congested; In a case where the remaining buffer space is less than or equal to a minimum threshold value among the one or more threshold values, determining that the congestion state is severe congestion; and When the remaining buffer space is greater than the minimum threshold and less than or equal to the maximum threshold, it is determined that the congestion state is a moderate degree of congestion.

18. The method according to claim 16 or 17, further comprising: In response to the congestion state being severe congestion, the peer communication circuit sends a failure response corresponding to the current data to be transmitted in the data stream to the on-chip network circuit or prohibits sending a success response corresponding to the current data to be transmitted to the on-chip network circuit.

19. The method according to claim 15, further comprising: The peer communication circuit determines, based on a size relationship between the remaining buffer space and one or more thresholds, a frequency of sending the successful response set for the data flow to the on-chip network circuit. The sending frequency is positively correlated with the remaining buffer space.

20. A computing system comprising a plurality of computing devices connected via a bus, wherein each computing device of the plurality of computing devices is the computing device of claim 1.

Images