A power consumption optimization method for switching chips and related equipment

Abstract

This application discloses a power consumption optimization method and related equipment for switching chips, which can avoid the situation of excessively high power consumption in switching chips. The method includes: during the operation of the switching chip, the switching chip can sample the traffic it is forwarding to obtain the sampled traffic. Then, the switching chip can use the sampled traffic to detect whether multiple AI chips are switching between the computation phase and the communication phase, thereby determining whether to adjust the power consumption for subsequent traffic to be forwarded. If multiple AI chips switch from the communication phase to the computation phase, the switching chip can reduce the power consumption for the traffic to be forwarded, forwarding the traffic according to the reduced power consumption; if multiple AI chips switch from the computation phase to the communication phase, the switching chip can increase the power consumption for the traffic to be forwarded, forwarding the traffic according to the increased power consumption.

Description

[0001] This application is a divisional application. The original application has the application number 202410223654.4 and the original application date is February 28, 2024. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of chip technology, and in particular to a power consumption optimization method for switching chips and related equipment. Background Technology

[0003] With the popularization of artificial intelligence (AI) training clusters, in order to improve the training efficiency of AI training clusters for neural network models, an interconnection infrastructure device with ultra-high bandwidth can be introduced into the AI ​​training cluster. This device can be used as an interconnection node to connect multiple computing nodes in the cluster, thereby reducing the communication latency between computing nodes and thus improving the training efficiency of the AI ​​training cluster.

[0004] The AI ​​training cluster provided by the related technology can include multiple computing nodes and interconnect nodes. The computing nodes can be represented as multiple AI chips, and the interconnect nodes as switching chips. The receiving end of the switching chip is connected to some of the AI ​​chips, and the sending end is connected to another group of AI chips. Model training performed by multiple AI chips involves multiple iterations, each consisting of a computing phase and a communication phase. During the computing phase, the communication volume between AI chips is close to zero, while during the communication phase, the communication volume is very high. The switching chip ensures normal communication between the AI ​​chips in both the computing and communication phases.

[0005] However, in order to ensure the communication quality between AI chips, the switching chip often needs to be in a full-load working state, resulting in excessive power consumption of the switching chip. Summary of the Invention

[0006] This application provides a power consumption optimization method and related equipment for switching chips, which can make the power consumption of the switching chip match the actual workload, thereby avoiding the situation of the switching chip having artificially high power consumption and achieving power consumption optimization.

[0007] The first aspect of this application provides a power consumption optimization method for a switching chip. The switching chip can be used to forward traffic between multiple AI chips, that is, to forward traffic from one part of the AI ​​chips to another part of the AI ​​chips for processing. Multiple AI chips can be used to train a neural network. During the training process of the neural network, the multiple AI chips can be in a computation phase or a communication phase (i.e., the working state of the multiple AI chips can be either a computation state or a communication state). When in the computation phase, the traffic between the multiple AI chips is close to zero; when in the communication phase, the traffic between the multiple AI chips is large. The method includes:

[0008] During the operation of the switching chip, it can sample the traffic being forwarded to obtain the sampled traffic. Then, the switching chip can use the sampled traffic to detect whether multiple AI chips are switching between the computing and communication phases, in order to determine whether to adjust the chip power consumption for the traffic to be forwarded.

[0009] If, based on the sampled traffic, it is determined that multiple AI chips are not switching between the computation and communication phases, the switching chip will not adjust the power consumption for the traffic to be forwarded, and will forward subsequent traffic according to the current power consumption. If, based on the sampled traffic, it is determined that multiple AI chips are just switching from the communication phase to the computation phase, the switching chip will reduce the power consumption for the traffic to be forwarded, and will forward subsequent traffic according to the reduced power consumption. If, based on the sampled traffic, it is determined that multiple AI chips are just switching from the computation phase to the communication phase, the switching chip will increase the power consumption for the traffic to be forwarded, and will forward subsequent traffic according to the increased power consumption.

[0010] As can be seen from the above method, the switching chip possesses power consumption adjustment capabilities. It can identify changes in the stage of the AI ​​chip based on the forwarded traffic and adaptively adjust the power consumption of the transmitted traffic accordingly. Specifically, when the AI ​​chip is in the communication stage, the switching chip can operate in a high-power state to forward traffic between AI chips; when the AI ​​chip is in the computation stage, the switching chip can operate in a low-power state to forward traffic between AI chips. In this way, the power consumption of the switching chip matches the actual workload, thereby avoiding artificially high power consumption and achieving power optimization.

[0011] In one possible implementation, the training process includes multiple sampling periods, with the current sampling period included. The sampled traffic is the traffic sampled in the current sampling period. Based on the sampled traffic, the switching chip detects whether multiple AI chips are switching between the computation and communication phases. This includes the switching chip detecting whether multiple AI chips are switching between the computation and communication phases based on the traffic sampled in the previous sampling period and the traffic sampled in the current sampling period. In the aforementioned implementation, the entire training process can be divided into multiple statistical periods, each of which can contain multiple sampling periods. Assuming that the current sampling period of the current statistical period has been reached, the switching chip can sample the traffic being transmitted by the switching chip in the current sampling period to obtain the traffic sampled in the current sampling period. Since the traffic sampled in the previous sampling period is known, the switching chip can compare the traffic sampled in the previous sampling period with the traffic sampled in the current sampling period to accurately determine whether multiple AI chips are switching between the computation and communication phases.

[0012] In one possible implementation, if multiple AI chips switch from the communication phase to the computation phase, the switching chip reduces power consumption for the traffic to be forwarded by: if the traffic sampled in the previous sampling period is less than the traffic sampled in the current sampling period, the switching chip determines that the multiple AI chips have switched from the communication phase to the computation phase and reduces power consumption for the traffic to be forwarded. Alternatively, if the traffic sampled in the previous sampling period is to a certain extent less than the traffic sampled in the current sampling period, the switching chip determines that the multiple AI chips have just switched from the communication phase to the computation phase, and then reduces power consumption for the traffic to be forwarded, using the reduced power for subsequent forwarding of the traffic.

[0013] In one possible implementation, if multiple AI chips switch from the computation phase to the communication phase, the switching chip increases the power consumption for the traffic to be forwarded by: if the traffic sampled in the previous sampling period is greater than the traffic sampled in the current sampling period, the switching chip determines that multiple AI chips have switched from the computation phase to the communication phase and increases the power consumption for the traffic to be forwarded. Alternatively, if the traffic sampled in the previous sampling period is to a certain extent greater than the traffic sampled in the current sampling period, the switching chip determines that multiple AI chips have just switched from the computation phase to the communication phase, and then increases the power consumption for the traffic to be forwarded, so that the increased power consumption is used for forwarding subsequent traffic.

[0014] In one possible implementation, the switching chip reduces power consumption for traffic to be forwarded by performing at least one of the following: progressively reducing the operating frequency of the switching chip, progressively reducing the operating voltage of the switching chip, and disabling the serializer and deserializer (SERDes) of the switching chip. The priority of progressively reducing the operating frequency is higher than that of progressively reducing the operating voltage, and the priority of progressively reducing the operating voltage is higher than that of disabling the SERDes. In the aforementioned implementation, when the switching chip needs to reduce power consumption for subsequent traffic, it can achieve this by reducing the operating frequency, reducing the operating voltage, and disabling the SERDes. Because the switching chip uses a progressively reducing operating frequency, progressively reducing operating voltage, and disabling some SERDes in this process, it can achieve a smooth adjustment in power consumption when multiple AI chips switch from the communication phase to the computing phase, resulting in a continuous and smooth reduction in traffic between the AI ​​chips it forwards.

[0015] In one possible implementation, the switching chip increases power consumption for traffic to be forwarded by: the switching chip performing at least one of the following actions when the traffic to be forwarded arrives: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and activating the switch chip's SERDES; or, the switching chip performing at least one of the following actions at a target time: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and activating the switch chip's SERDES, where the target time is before the arrival time of the traffic to be forwarded. In the aforementioned implementation, when the switching chip needs to increase power consumption for subsequent traffic, it can achieve this by activating SERDES, increasing the operating voltage, and increasing the operating frequency. Since the switching chip can sample and increase power consumption in real time (increasing power consumption when subsequent traffic arrives) or increase power consumption in advance (increasing power consumption at the target time) according to the actual situation, the long reset (activation) time of the SERDES in the switching chip can be avoided from affecting the communication between AI chips.

[0016] In one possible implementation, the training process includes multiple statistical periods, each including a previous statistical period and the current statistical period. The current statistical period includes a previous sampling period and the current sampling period. When the traffic to be forwarded arrives, the switching chip performs at least one of the following: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and activating the switching chip's SERDS. This includes: if the average duration of communication between multiple AI chips in the previous statistical period is zero, the switching chip performs at least one of the following when the traffic to be forwarded arrives: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and activating the switching chip's SERDS; or, if the average duration of communication between multiple AI chips in the previous statistical period and the average duration of computation between multiple AI chips in the previous statistical period are zero... If the average duration of a phase is not zero, and the standard deviation of the duration of multiple AI chips in the communication phase in the previous statistical period is greater than or equal to a preset first threshold, the switching chip performs at least one of the following actions when the traffic to be forwarded arrives: increases the operating frequency of the switching chip, increases the operating voltage of the switching chip, and starts the switching chip's SERDES; or, if the average duration of multiple AI chips in the communication phase and the average duration of multiple AI chips in the computation phase in the previous statistical period are not zero, and the standard deviation of the duration of multiple AI chips in the computation phase in the previous statistical period is greater than or equal to a preset second threshold, the switching chip performs at least one of the following actions when the traffic to be forwarded arrives: increases the operating frequency of the switching chip, increases the operating voltage of the switching chip, and starts the switching chip's SERDES. In the aforementioned process, multiple statistical periods also include the previous statistical period. The switching chip can acquire traffic characteristic parameters from the previous statistical period to generate the power adjustment strategy for the current statistical period. For example, in the previous statistical period, this includes the average duration of multiple AI chips in the communication phase, the average duration of multiple AI chips in the computation phase, the standard deviation of the duration of multiple AI chips in the communication phase, and the standard deviation of the duration of multiple AI chips in the computation phase. Therefore, in the current statistical period, when the switching chip needs to increase power consumption, it can use these traffic characteristic parameters to determine whether the current traffic waveform conforms to a typical AI traffic model. If, based on these traffic characteristic parameters, the current traffic waveform does not conform to a typical AI traffic model, the switching chip will choose to increase power consumption immediately to achieve the power increase.

[0017] In one possible implementation, the switching chip performs at least one of the following at the target time: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and starting the switching chip's SERDS. This includes: if the average duration of multiple AI chips in the communication phase and the average duration of multiple AI chips in the computation phase in the previous statistical period are not zero, the standard deviation of the duration of multiple AI chips in the communication phase in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of multiple AI chips in the computation phase in the previous statistical period is less than a preset second threshold, the switching chip performs at least one of the following at the target time: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and starting the switching chip's SERDS. In the aforementioned implementation, if the current traffic waveform conforms to a typical AI traffic model based on these traffic characteristic parameters, the switching chip chooses to increase power consumption in advance to achieve the power consumption increase, thereby avoiding the impact of the long reset (startup) time of the SERDS in the switching chip on the communication between AI chips.

[0018] In one possible implementation, the average duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the average duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period. In the aforementioned implementation, the switching chip can sample the traffic being transmitted by the switching chip in multiple sampling periods of the previous statistical period, and determine the number of times the multiple AI chips are in the communication phase, the number of times the multiple AI chips are in the calculation phase, the duration of each communication phase and the duration of each calculation phase of the multiple AI chips in the previous statistical period based on the traffic sampled in multiple sampling periods. Based on these traffic characteristic parameters, the average duration of the multiple AI chips in the communication phase, the average duration of the multiple AI chips in the calculation phase, the standard deviation of the duration of the multiple AI chips in the communication phase, and the standard deviation of the duration of the multiple AI chips in the calculation phase in the previous statistical period can be calculated.

[0019] In one possible implementation, the difference between the target time and the arrival time of the traffic to be forwarded is calculated based on the average duration of the computation phase of multiple AI chips in the previous statistical period and the standard deviation of the duration of the computation phase of multiple AI chips in the previous statistical period.

[0020] A second aspect of this application provides a switching chip for forwarding traffic between multiple AI chips used to train a neural network. During the training process, the multiple AI chips are either in a computation phase or a communication phase. The switching chip includes: a sampling module for sampling the traffic forwarded by the switching chip to obtain the sampled traffic; a detection module for detecting whether the multiple AI chips are switching between the computation phase and the communication phase based on the sampled traffic; a first adjustment module for reducing the power consumption of the traffic to be forwarded if the multiple AI chips switch from the communication phase to the computation phase; or a second adjustment module for increasing the power consumption of the traffic to be forwarded if the multiple AI chips switch from the computation phase to the communication phase.

[0021] In one possible implementation, the training process includes multiple sampling periods, including the current sampling period. The sampled traffic is the traffic sampled in the current sampling period. The detection module is used to detect whether multiple AI chips are switching between the computation phase and the communication phase based on the traffic sampled in the previous sampling period and the traffic sampled in the current sampling period.

[0022] In one possible implementation, the first adjustment module is used to determine that multiple AI chips switch from the communication phase to the computing phase and reduce power consumption for the traffic to be forwarded if the traffic sampled in the previous sampling period is less than the traffic sampled in the current sampling period.

[0023] In one possible implementation, the second adjustment module is used to determine that multiple AI chips switch from the computation phase to the communication phase and increase power consumption for the traffic to be forwarded if the traffic sampled in the previous sampling period is greater than the traffic sampled in the current sampling period.

[0024] In one possible implementation, the first adjustment module is configured to perform at least one of the following: progressively reducing the operating frequency of the switching chip, progressively reducing the operating voltage of the switching chip, and shutting down the serializer and deserializer (SERDES) of the switching chip, wherein progressively reducing the operating frequency of the switching chip has a higher priority than progressively reducing the operating voltage of the switching chip, and progressively reducing the operating voltage of the switching chip has a higher priority than shutting down the SERDES of the switching chip.

[0025] In one possible implementation, the second adjustment module is configured to: at the time when the traffic to be forwarded arrives, perform at least one of the following: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDEs of the switching chip; or, at a target time, perform at least one of the following: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDEs of the switching chip, wherein the target time is prior to the time when the traffic to be forwarded arrives.

[0026] In one possible implementation, the training process includes multiple statistical periods, each including a previous statistical period and the current statistical period. The current statistical period includes a previous sampling period and the current sampling period. A second adjustment module is configured to: if the average duration of communication between the multiple AI chips in the previous statistical period is zero, perform at least one of the following when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the switch chip's SERDES; or, if the average duration of communication between the multiple AI chips in the previous statistical period and the average duration of computation between the multiple AI chips in the previous statistical period are not zero, and the average duration of computation between the multiple AI chips in the previous statistical period is zero... If the standard deviation of the duration of the communication phase is greater than or equal to a preset first threshold, at least one of the following shall be performed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDE of the switching chip; or, if the average duration of the multiple AI chips in the communication phase in the previous statistical period and the average duration of the multiple AI chips in the calculation phase in the previous statistical period are not zero, and the standard deviation of the duration of the multiple AI chips in the calculation phase in the previous statistical period is greater than or equal to a preset second threshold, at least one of the following shall be performed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDE of the switching chip.

[0027] In one possible implementation, the second adjustment module is configured to perform at least one of the following at a target time if the average duration of multiple AI chips in the communication phase and the average duration of multiple AI chips in the computation phase in the previous statistical period are not zero, the standard deviation of the duration of multiple AI chips in the communication phase in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of multiple AI chips in the computation phase in the previous statistical period is less than a preset second threshold: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and starting the SERDES of the switching chip.

[0028] In one possible implementation, the average duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the average duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period.

[0029] In one possible implementation, the difference between the target time and the arrival time of the traffic to be forwarded is calculated based on the average duration of the computation phase of multiple AI chips in the previous statistical period and the standard deviation of the duration of the computation phase of multiple AI chips in the previous statistical period.

[0030] A third aspect of this application provides an interconnection device including a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the device performs the method as described in the first aspect or any possible implementation of the first aspect.

[0031] A fourth aspect of this application provides a computer storage medium storing one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method described in the first aspect or any possible implementation of the first aspect.

[0032] A fifth aspect of this application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or any possible implementation of the first aspect.

[0033] In this embodiment, a switching chip can be used to transmit traffic between multiple AI chips. These AI chips can be used to train a neural network model, and during training, they can switch between computation and communication phases. During operation, the switching chip samples the traffic it is forwarding to obtain the sampled traffic. Then, the switching chip uses the sampled traffic to detect whether the multiple AI chips are switching between computation and communication phases, thereby determining whether to adjust the power consumption for subsequent traffic to be forwarded. If multiple AI chips switch from the communication phase to the computation phase, the switching chip can reduce the power consumption for the traffic to be forwarded, forwarding the traffic at the reduced power consumption. If multiple AI chips switch from the computation phase to the communication phase, the switching chip can increase the power consumption for the traffic to be forwarded, forwarding the traffic at the increased power consumption. In the aforementioned process, the switching chip has power adjustment capabilities, enabling it to identify changes in the phase of the AI ​​chips based on the forwarded traffic and adaptively adjust the power consumption of the transmitted traffic accordingly. It is evident that when the AI ​​chip is in the communication phase, the switching chip can operate in a high-power state to forward traffic between AI chips. When the AI ​​chip is in the computing phase, the switching chip can operate in a low-power state to forward traffic between AI chips. In this way, the power consumption of the switching chip can match the actual workload, thereby avoiding the situation of the switching chip having artificially high power consumption and achieving power optimization. Attached Figure Description

[0034] Figure 1 A schematic diagram of the structure of a training cluster provided in an embodiment of this application;

[0035] Figure 2 A schematic diagram of the AI ​​traffic model provided in the embodiments of this application;

[0036] Figure 3 A schematic diagram of the structure of a switching chip provided in an embodiment of this application;

[0037] Figure 4 A flowchart illustrating a power consumption optimization method for a switching chip provided in an embodiment of this application;

[0038] Figure 5 A schematic diagram of the statistical period and sampling period provided in the embodiments of this application;

[0039] Figure 6 Another flowchart illustrating the power consumption optimization method for the switching chip provided in this application embodiment;

[0040] Figure 7 Another schematic diagram illustrating the statistical period and sampling period provided in the embodiments of this application;

[0041] Figure 8 A schematic diagram of the structure of a switching chip provided in an embodiment of this application;

[0042] Figure 9 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0043] This application provides a power consumption optimization method and related equipment for switching chips, which can make the power consumption of the switching chip match the actual workload, thereby avoiding the situation of the switching chip having artificially high power consumption and achieving power consumption optimization.

[0044] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0045] With the increasing popularity of AI training clusters, in order to improve the training efficiency of AI training clusters for neural network models, an interconnection infrastructure device with ultra-high bandwidth can be introduced into the AI ​​training cluster. This device can be used as an interconnection node to connect multiple computing nodes in the cluster, thereby reducing the communication latency between computing nodes and thus improving the training efficiency of the AI ​​training cluster.

[0046] The AI ​​training cluster provided by the related technology can include multiple computing nodes and interconnect nodes. The computing nodes can be represented as multiple AI chips, and the interconnect nodes as switching chips. The receiving end of the switching chip is connected to some of the AI ​​chips, and the sending end is connected to another group of AI chips. Model training performed by multiple AI chips involves multiple iterations, each consisting of a computing phase and a communication phase. During the computing phase, the communication volume between AI chips is close to zero. In this case, the traffic that the switching chip needs to forward between AI chips is almost zero. During the communication phase, the communication volume between AI chips is quite large. In this case, the traffic that the switching chip needs to forward between AI chips is very large. Because the switching chip has ultra-high bandwidth, it can successfully forward large amounts of traffic from some AI chips to others, thus ensuring normal communication between the AI ​​chips.

[0047] However, in order to ensure the communication quality between AI chips, the switching chip often needs to be in a fully loaded state. During the communication phase, the switching chip needs to forward a lot of traffic, and the power consumption of the switching chip can match the actual traffic forwarding requirements. However, during the computing phase, the switching chip needs to forward very little traffic, and the power consumption of the switching chip is much greater than the actual traffic forwarding requirements, which will lead to the switching chip's power consumption being artificially high.

[0048] To address the aforementioned issues, this application provides a power consumption optimization method for switching chips, which can be implemented in conjunction with artificial intelligence (AI) technology. AI technology is a discipline that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence. AI technology achieves optimal results by perceiving the environment, acquiring knowledge, and using that knowledge. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. Using artificial intelligence for data processing is a common application of AI.

[0049] In practical applications, AI technology can use neural network models to complete user tasks and meet user needs. For example, an image from a user can be input into a neural network model to determine its category. Similarly, a user's voice can be input into a neural network model to obtain corresponding text. A user's question can be input into a neural network model to obtain a corresponding answer, and so on. The neural network models in these practical applications are all pre-trained neural network models, which can be trained using training clusters. Figure 1 A schematic diagram of the structure of the training cluster provided in the embodiments of this application is shown below. Figure 1 As shown, the training cluster includes a switching chip and multiple AI chips. The following sections will introduce the AI ​​chips and the switching chip respectively:

[0050] AI chips can be implemented in various ways. For example, an AI chip can be a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), or a tensor processing unit (TPU), and so on. In a training cluster, multiple AI chips can be used to collaboratively train a neural network model. The training process can involve multiple iterations, each including a computation phase and a communication phase. When multiple AI chips are in the computation phase, the communication volume between them is minimal, approaching zero. When multiple AI chips are in the communication phase, the communication volume is significant, far exceeding zero. Therefore, throughout the training process, multiple AI chips are either in the computation phase or the communication phase, constantly switching between these two stages. The bandwidth required for transmission between multiple AI chips is very similar across different computational stages. Similarly, the bandwidth required for transmission between multiple AI chips is also very similar across different communication stages. Therefore, the bandwidth required for transmission between multiple AI chips throughout the entire training process can be represented by a typical square wave shape. This square wave can also be called the AI ​​bandwidth model, such as... Figure 2 As shown ( Figure 2 (A schematic diagram of the AI ​​traffic model provided in the embodiments of this application).

[0051] A switching chip connects multiple AI chips. Generally, the receiving end of the switching chip is connected to a portion of the AI ​​chips, and the sending end is connected to another portion. Therefore, the switching chip can forward traffic between multiple AI chips (e.g., the number of received packets, received bytes, transmitted packets, and transmitted bytes, etc.). In other words, the switching chip can forward traffic from the receiving portion of the AI ​​chips to the sending portion. Because the amount of traffic that the receiving AI chips need to send to the sending AI chips is very similar and small in different computing stages, but very similar and large in different communication stages, if the switching chip is constantly operating under high load, its power consumption will be extremely high. To optimize the power consumption of the switching chip, it can be improved to allow it to adjust its power consumption according to the stage in which the multiple chips are operating.

[0052] like Figure 3 As shown ( Figure 3 (This is a schematic diagram of a switching chip provided in an embodiment of this application). The switching chip provided in this embodiment includes a sampling module, a detection module, and an adjustment module. The sampling module, also known as a port monitoring circuit, is connected to the ports of the receiving end and the transmitting end. It can sample the traffic forwarded by the switching chip in real time (e.g., the traffic forwarded by the switching chip during a certain time period) to obtain the sampled traffic. The detection module can detect whether multiple AI chips are switching between the computation and communication phases based on the sampled traffic and notify the adjustment module of the detection result. The adjustment module includes a configuration register that stores various traffic characteristic parameters (these traffic characteristic parameters are often calculated based on previously sampled traffic and stored in the register). These traffic characteristic parameters can serve as the current power consumption adjustment strategy. If the detection result indicates that multiple AI chips are switching from the communication phase to the computation phase, the adjustment module reduces the power consumption for the traffic to be transmitted based on the power consumption adjustment strategy. If the detection result indicates that multiple AI chips are switching from the computation phase to the communication phase, the adjustment module increases the power consumption for the traffic to be transmitted based on the power consumption adjustment strategy. This demonstrates that the switching chip can adjust its power consumption in forwarding traffic based on the stage changes of multiple AI chips, thereby optimizing its performance.

[0053] To further understand the power consumption adjustment process of the switching chip, the following section provides a detailed description of this process using specific embodiments. It should be noted that the switching chip can divide the entire training process into multiple statistical cycles (each statistical cycle has a fixed duration, and its size can be set according to actual needs; no limitation is made here). Each statistical cycle contains multiple sampling cycles (each sampling cycle has a fixed duration, and its size can be set according to actual needs; no limitation is made here). Since the operation of the switching chip in each statistical cycle is similar, the following section selects any one of the multiple statistical cycles for description and considers this statistical cycle as the current statistical cycle. Because the current power consumption adjustment strategy applied by the switching chip in the current statistical cycle comes from multiple parameters calculated by the switching chip in the previous statistical cycle, the operations performed by the switching chip in the previous statistical cycle will be described below. Figure 4 A flowchart illustrating the power consumption optimization method for a switching chip provided in this application embodiment is shown below. Figure 4 As shown, the method includes:

[0054] 401. In the previous statistical period, the switching chip determined the number of times multiple AI chips were in the computing phase, the number of times multiple AI chips were in the communication phase, the duration of each computing phase for multiple AI chips, and the duration of each communication phase for multiple AI chips.

[0055] In this embodiment, upon entering the previous statistical period, since the previous statistical period can be divided into multiple sampling periods, the switching chip can sample the traffic forwarded by the switching chip in each sampling period, thereby obtaining the traffic sampled by the switching chip in each sampling period. Based on the traffic sampled by the switching chip in the multiple sampling periods of the previous statistical period, the switching chip can perform the following operations:

[0056] If the traffic sampled by the switching chip in several consecutive sampling cycles is less than the preset traffic threshold (the size of this threshold can be set according to actual needs, and is not limited here), it can be regarded as the switching chip forwarding a small amount of traffic (usually close to zero) between multiple AI chips in these several consecutive sampling cycles. The switching chip can determine that multiple AI chips are in the calculation stage at a time, and use the number of these several consecutive sampling cycles and the duration of a single sampling cycle to perform calculations, thereby obtaining the duration of multiple AI chips in the calculation stage at a time.

[0057] If the traffic sampled by the switching chip in several consecutive sampling periods is greater than or equal to the preset traffic threshold, it can be considered that the switching chip is forwarding large traffic between multiple AI chips in these several consecutive sampling periods. The switching chip can determine that multiple AI chips are in the communication phase at a time, and use the number of these several consecutive sampling periods and the duration of a single sampling period to calculate the duration of multiple AI chips in the communication phase at a time.

[0058] By continuously repeating the above operations until the end of the previous statistical period, the switching chip can determine the number of times multiple AI chips were in the calculation phase, the number of times multiple AI chips were in the communication phase, the duration of each calculation phase, and the duration of each communication phase during the previous statistical period.

[0059] For example, such as Figure 5 As shown ( Figure 5 (This is a schematic diagram of the statistical period and sampling period provided in the embodiments of this application), where the statistical period is T. sample-period The sampling period is T sample-interval A T sample-period It can be divided into T sample-interval Let the previous statistical period be T. sample-period-0 In T sample-period-0 In the middle, the switching chip is in each T sample-interval All forwarded traffic is sampled to obtain the data for each T. sample-interval The flow rate obtained from sampling. The switching chip can make the following judgments:

[0060] If several consecutive T sample-interval The sampled traffic volumes are all high-volume. The switching chip can record them as multiple AI chips in a single communication phase, and then record these several T... sample-interval Quantity multiplied by T sample-interval The duration of the communication is T, which represents the duration of a single communication session between multiple AI chips. i .

[0061] If several consecutive T sample-interval The sampled traffic is close to 0, so the switching chip can treat it as multiple AI chips in the computation phase at a time, and record these several T... sample-interval Quantity multiplied by T sample-interval The duration of each AI chip's single computation phase is obtained by measuring the duration of the computation phase.

[0062] Therefore, the switching chip can statistically determine the value at T. sample-period-0In the diagram, the number of times n (n is usually an integer greater than or equal to 2) that multiple AI chips are in the communication phase, the number of times m (m is usually an integer greater than or equal to 2) that multiple AI chips are in the computation phase, and the duration T1, T2, ..., T of each communication phase for multiple AI chips are given. n And the duration t1, t2, ..., t of each AI chip in the computation phase. m .

[0063] 402. The switching chip calculates the number of times multiple AI chips were in the communication phase in the previous statistical period and the duration of each communication phase in the previous statistical period, to obtain the average duration of the communication phase of multiple AI chips in the previous statistical period.

[0064] 403. The switching chip calculates the number of times multiple AI chips were in the calculation phase in the previous statistical period and the duration of each calculation phase in the previous statistical period, to obtain the average duration of multiple AI chips in the calculation phase in the previous statistical period.

[0065] 404. The switching chip calculates the average duration of multiple AI chips in the communication phase during the previous statistical period and the duration of multiple AI chips in the communication phase each time during the previous statistical period, and obtains the standard deviation of the duration of multiple AI chips in the communication phase during the previous statistical period.

[0066] 405. The switching chip calculates the average duration of multiple AI chips in the calculation phase during the previous statistical period and the duration of multiple AI chips in the calculation phase each time during the previous statistical period, and obtains the standard deviation of the duration of multiple AI chips in the calculation phase during the previous statistical period.

[0067] After obtaining the number of times multiple AI chips were in the computation phase, the number of times they were in the communication phase, the duration of each computation phase, and the duration of each communication phase in the previous statistical period, the switching chip can detect whether the number of times the multiple AI chips were in the computation phase and the number of times they were in the communication phase are close. If they are close (e.g., equal or with a difference of one), the switching chip can perform the following calculations to obtain the corresponding traffic characteristic parameters:

[0068] In the previous statistical period, if the switching chip can average the number of times multiple AI chips were in the communication phase and the duration of each communication phase, thus obtaining the average duration of the multiple AI chips in the communication phase, then similarly, the switching chip can average the number of times multiple AI chips were in the computation phase and the duration of each computation phase, thus obtaining the average duration of the multiple AI chips in the computation phase.

[0069] Furthermore, the switching chip can calculate the average duration of multiple AI chips in the communication phase, the number of times each AI chip is in the communication phase, and the duration of each communication phase, thus obtaining the standard deviation of the communication phase duration. Similarly, the switching chip can calculate the average duration of multiple AI chips in the computation phase, the number of times each AI chip is in the computation phase, and the duration of each computation phase, thus obtaining the standard deviation of the computation phase duration. After obtaining these traffic characteristic parameters, the switching chip can store these parameters in a register to serve as a power consumption adjustment strategy for the switching chip in the current statistical cycle.

[0070] As in the example above, statistics were compiled for T. sample-period-0 In the process, after multiple AI chips have been in the communication phase (n) and the computation phase (m) multiple times, the switching chip can detect whether n is close to m. If they are very close, the switching chip can calculate the value of n in time T using the following formula. sample-period-0 In the context of AI chips, the average duration of communication between multiple chips is as follows:

[0071]

[0072] Similarly, the switching chip can also be calculated using the following formula at T sample-period-0 In the context of AI chips, the average time spent in the computation phase is as follows:

[0073]

[0074] In addition, the switching chip can also calculate the value at T using the following formula. sample-period-0 In the context of multiple AI chips, the standard deviation of the duration of communication phase is as follows:

[0075]

[0076] Similarly, the switching chip can also be calculated using the following formula at T sample-period-0 In the context of multiple AI chips, the standard deviation of the time spent in the computation phase is as follows:

[0077]

[0078] After obtaining these flow characteristic parameters, the switching chip can store them in a register to serve as the T. sample-period-1 The power consumption adjustment strategy in the process.

[0079] The following section describes the operations performed by the switching chip during the current statistical period. Figure 6 Another flowchart illustrating the power consumption optimization method for the switching chip provided in this application embodiment is shown below. Figure 6 As shown, the method includes:

[0080] 601. The switching chip samples the traffic forwarded by the switching chip to obtain the sampled traffic.

[0081] In this embodiment, after entering the current statistical period, the switching chip can sample the traffic forwarded by the switching chip to obtain the sampled traffic. It should be noted that the current statistical period may contain multiple sampling periods. Since the operation performed by the switching chip in each sampling period of the current statistical period is similar, the following description uses any one sampling period in the current statistical period as an illustration, and refers to this sampling period as the current sampling period. In the current sampling period, the switching chip can sample the traffic that the switching chip is forwarding, thereby obtaining the traffic sampled in the current sampling period.

[0082] 602. The switching chip detects whether multiple AI chips are switching between the computation and communication phases based on the sampled traffic.

[0083] 603. If multiple AI chips switch from the communication stage to the computing stage, the switching chip reduces the power consumption for the traffic to be forwarded.

[0084] 604. If multiple AI chips switch from the computing phase to the communication phase, the switching chip increases the power consumption for the traffic to be forwarded.

[0085] After acquiring the sampled traffic, the switching chip can use the sampled traffic to detect whether multiple AI chips are switching between the computing phase and the communication phase, in order to determine whether to adjust the power consumption for the traffic to be forwarded.

[0086] If the sampled traffic indicates that multiple AI chips are not switching between the computation and communication phases (meaning they are still in either phase), the switching chip will not adjust the power consumption for the traffic to be forwarded, and will forward subsequent traffic according to the current power consumption.

[0087] If, based on the sampled traffic, it is determined that multiple AI chips have just switched from the communication phase to the computing phase, that is, multiple AI chips have entered the computing phase from the communication phase, the switching chip reduces the power consumption of the traffic to be forwarded according to the power consumption adjustment strategy, so as to forward the subsequent traffic to be forwarded according to the reduced power consumption.

[0088] If, based on the sampled traffic, it is determined that multiple AI chips have just switched from the computing phase to the communication phase, that is, multiple AI chips have entered the communication phase from the computing phase, the switching chip will increase the power consumption for the traffic to be forwarded according to the power consumption adjustment strategy, so as to forward the subsequent traffic to be forwarded according to the increased power consumption.

[0089] Specifically, the switching chip can detect whether multiple AI chips are switching between the computation and communication phases in the following ways:

[0090] After acquiring the traffic sampled in the current sampling period, since the traffic sampled in the previous sampling period has also been acquired, the switching chip can compare the traffic sampled in the previous sampling period with the traffic sampled in the current sampling period, thereby detecting whether multiple AI chips are switching between the computing phase and the communication phase.

[0091] If the traffic sampled in the previous sampling period is equal to or approximately equal to the traffic sampled in the current sampling period, the switching chip determines that the multiple AI chips have not switched between the computation and communication phases. In other words, the multiple AI chips are still in either the computation or communication phase, and the switching chip does not adjust the power consumption for the traffic to be forwarded, but forwards the traffic to be forwarded in subsequent sessions with the current power consumption. For example, ... Figure 7 As shown ( Figure 7 (Another schematic diagram of the statistical period and sampling period provided in the embodiments of this application), in T sample-period-1 In the middle, let T sample-interval-2 For the current sampling period, T sample-interval-1 For the previous sampling period, since in T sample-interval-1 The flow rate collected in T is equal to the flow rate collected in T sample-interval-2 The traffic collected is all high-volume, so the switching chip can determine that the multiple AI chips have not switched between stages and are still in the communication stage. Therefore, the power consumption of subsequent forwarding traffic is not adjusted.

[0092] If the traffic sampled in the previous sampling period is less than the traffic sampled in the current sampling period, the switching chip determines that multiple AI chips have just switched from the communication phase to the computing phase. In other words, multiple AI chips have moved from the communication phase to the computing phase. The switching chip then reduces the power consumption of the traffic to be forwarded according to a power adjustment strategy, so that the reduced power consumption can be used to forward subsequent traffic. Continuing with the example above, in T... sample-period-1 In the middle, let T sample-interval-4 For the current sampling period, T sample-interval-3 For the previous sampling period, since in T sample-interval-3 The flow rate collected in T is greater than that in T sample-interval-4 Since the traffic collected is used to identify multiple AI chips and move them from the communication stage to the computing stage, the power consumption of subsequent forwarding traffic can be reduced.

[0093] If the traffic sampled in the previous sampling period is significantly greater than the traffic sampled in the current sampling period, the switching chip determines that multiple AI chips have just switched from the computation phase to the communication phase. In other words, multiple AI chips have moved from the computation phase to the communication phase. The switching chip then increases the power consumption for the traffic to be forwarded according to a power adjustment strategy, so that the increased power consumption can be used to forward subsequent traffic. Continuing with the example above, in T... sample-period-1 In the middle, let T sample-interval-6 For the current sampling period, T sample-interval-5 For the previous sampling period, since in T sample-interval-5 The flow rate collected in T is less than that in T sample-interval-6 Since the traffic collected is used to identify multiple AI chips and move from the computing stage to the communication stage, the switching chip can improve the power consumption of subsequent forwarding traffic.

[0094] More specifically, switching chips can reduce power consumption for traffic to be forwarded in the following ways:

[0095] After determining that multiple AI chips are switching from the communication phase to the computing phase, the switching chip performs at least one of the following operations: (1) Gradually reduce the operating frequency of the switching chip, for example, reducing its operating frequency by one level every 100µs. (2) Gradually reduce the operating voltage of the switching chip, for example, reducing its operating voltage by one level every 1000µs. (3) Turn off the serializer and deserializer (serdes) of the switching chip, for example, turning off the serdes for 1ms. In this way, the switching chip, after reducing its operating frequency / operating voltage / turning off the serdes, will reduce the transmission traffic.

[0096] In this process, gradually reducing the operating frequency of the switching chip has a higher priority than gradually reducing the operating voltage of the switching chip, and gradually reducing the operating voltage of the switching chip has a higher priority than shutting down the switch's SERDS. For example, the switching chip continuously reduces its own operating frequency, and when the operating frequency can no longer be reduced, the switching chip then reduces its own operating voltage. Similarly, the switching chip continuously reduces its own operating voltage, and when the operating voltage can no longer be reduced, the switching chip then shuts down its own SERDS.

[0097] More specifically, switching chips can reduce power consumption for traffic to be forwarded in several ways:

[0098] (a) After determining that multiple AI chips have switched from the computing phase to the communication phase, the switching chip performs at least one of the following actions when the traffic to be forwarded arrives: (1) Increase the operating frequency of the switching chip, for example, increase the operating frequency of the switching chip itself to the highest operating frequency, etc. (2) Increase the operating voltage of the switching chip, for example, increase the operating voltage of the switching chip itself to the highest operating voltage, etc. (3) Start the SERDEs of the switching chip, for example, start each SERDE directly by the switching chip, etc.

[0099] (b) Since the SERDE startup time is relatively long (usually 500us), it will affect the efficiency of the switching chip in forwarding traffic. Therefore, the switching chip can increase its power consumption in advance. That is, the switching chip can perform at least one of the following at a certain time before the arrival of the traffic to be forwarded, i.e., at the target time: (1) Increase the operating frequency of the switching chip, for example, increase the operating frequency of the switching chip itself to the highest operating frequency, etc. (2) Increase the operating voltage of the switching chip, for example, increase the operating voltage of the switching chip itself to the highest operating voltage, etc. (3) Start the SERDE of the switching chip, for example, start each SERDE directly by the switching chip, etc.

[0100] More specifically, the switching chip can determine whether to increase power consumption on the fly or in advance by:

[0101] Since the switching chip has stored the various flow characteristic parameters calculated in the previous statistical period, which is the power consumption adjustment strategy, when the switching chip needs to increase power consumption in the current statistical period, it can determine whether to increase power consumption immediately or in advance based on these flow characteristic parameters.

[0102] If the average duration of communication between multiple AI chips in the previous statistical period was zero, it indicates that multiple AI chips were in a dormant state during the previous statistical period. The switching chip can immediately increase power consumption, therefore, when the traffic to be forwarded arrives, the switching chip can perform at least one of the following: activate the switching chip's SERDES, increase the switching chip's operating voltage, and increase the switching chip's operating frequency (generally, activating the switching chip's SERDES has a higher priority than increasing the switching chip's operating voltage, and increasing the switching chip's operating voltage has a higher priority than increasing the switching chip's operating frequency). Continuing with the example above, if T... average If the value is 0, the switching chip can increase the power consumption for subsequent traffic as it arrives, so as to forward the traffic according to the increased power consumption.

[0103] If the average duration of communication phase for multiple AI chips in the previous statistical period and the average duration of computation phase for multiple AI chips in the previous statistical period are not zero, and the standard deviation of the duration of communication phase for multiple AI chips in the previous statistical period is greater than or equal to a preset first threshold (the size of this threshold can be set according to actual needs and is not limited here), it indicates that the traffic between multiple AI chips does not conform to a typical AI traffic model (i.e., the duration of different communication phases is not equal; in a typical AI traffic model, the duration of different communication phases is usually equal). The switching chip can immediately increase power consumption, and at least one of the following will be executed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDES of the switching chip. As in the example above, if T... average >0, t average >0 and T standard-deviation >T sd-threshold (The aforementioned first threshold) allows the switching chip to increase power consumption for subsequent traffic as it arrives, and then forward the traffic with the increased power consumption.

[0104] If the average duration of communication and computation phases of multiple AI chips in the previous statistical period are not zero, and the standard deviation of computation phase duration of multiple AI chips in the previous statistical period is greater than or equal to a preset second threshold (the size of this threshold can be set according to actual needs and is not limited here), it indicates that the traffic between multiple AI chips does not conform to a typical AI traffic model (i.e., the durations of different computation phases are not equal; in a typical AI traffic model, the durations of different communication phases are usually equal). The switching chip can immediately increase power consumption, and at least one of the following will be executed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the switch chip's SERDES. As in the example above, if T...average >0, t average >0 and t standard-deviation >t sd-threshold (The aforementioned second threshold) allows the switching chip to increase power consumption for subsequent traffic as it arrives, and then forward the traffic with the increased power consumption.

[0105] If the average duration of multiple AI chips in the communication phase and the average duration of multiple AI chips in the computation phase in the previous statistical period are not zero, the standard deviation of the duration of multiple AI chips in the communication phase in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of multiple AI chips in the computation phase in the previous statistical period is less than a preset second threshold, it indicates that the traffic between multiple AI chips conforms to a typical AI traffic model, and the switching chip can increase power consumption in advance. Then, at least one of the following will be executed at the target time: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDES of the switching chip.

[0106] The difference between the target time and the arrival time of the traffic to be forwarded is calculated based on the average duration of multiple AI chips in the calculation phase in the previous statistical period and the standard deviation of the duration of multiple AI chips in the calculation phase in the previous statistical period.

[0107] As in the example above, T average >0, t average >0, T standard-deviation <T sd-threshold And t standard-deviation <t sd-threshold The switching chip can increase its power consumption for subsequent traffic at a certain point in advance, using the increased power consumption to forward that traffic. This point in time is t0, where t0 = t1 + (t... average -t standard-deviation -T serdes-up Since the power consumption increase is performed when the Ai chip enters the communication phase from the calculation phase, t1 is the initial moment when the Ai chip was in the calculation phase most recently. It can be seen that t0 is located between t1 and t2. t2 is the moment when the Ai chip enters the current communication phase from the most recent calculation phase (that is, the initial moment when the Ai chip is in the current calculation phase). After entering the communication phase, at some time after t2, that is, the moment when the subsequent traffic arrives.

[0108] In this embodiment, a switching chip can be used to transmit traffic between multiple AI chips. These AI chips can be used to train a neural network model, and during training, they can switch between computation and communication phases. During operation, the switching chip samples the traffic it is forwarding to obtain the sampled traffic. Then, the switching chip uses the sampled traffic to detect whether the multiple AI chips are switching between computation and communication phases, thereby determining whether to adjust the power consumption for subsequent traffic to be forwarded. If multiple AI chips switch from the communication phase to the computation phase, the switching chip can reduce the power consumption for the traffic to be forwarded, forwarding the traffic at the reduced power consumption. If multiple AI chips switch from the computation phase to the communication phase, the switching chip can increase the power consumption for the traffic to be forwarded, forwarding the traffic at the increased power consumption. In the aforementioned process, the switching chip has power adjustment capabilities, enabling it to identify changes in the phase of the AI ​​chips based on the forwarded traffic and adaptively adjust the power consumption of the transmitted traffic accordingly. It is evident that when the AI ​​chip is in the communication phase, the switching chip can operate in a high-power state to forward traffic between AI chips. When the AI ​​chip is in the computing phase, the switching chip can operate in a low-power state to forward traffic between AI chips. In this way, the power consumption of the switching chip can match the actual workload, thereby avoiding the situation of the switching chip having artificially high power consumption and achieving power optimization.

[0109] Furthermore, in the embodiments of this application, when the switching chip needs to reduce power consumption, it adopts a scheme of gradually reducing the operating frequency, gradually reducing the operating voltage, and shutting down some SERDEs. This allows the switching chip to achieve a smooth adjustment in power consumption when switching from the communication stage to the computing stage, so that the forwarded traffic shows a continuous decrease and a smooth transition.

[0110] Furthermore, in this embodiment of the application, when the switching chip needs to increase power consumption, it determines whether to increase power consumption immediately or in advance by judging whether the waveform of the current traffic conforms to a typical AI traffic model. This can avoid the long reset (startup) time of the SERDES in the switching chip affecting the communication between AI chips.

[0111] The above is a detailed description of the power consumption optimization method for the switching chip provided in the embodiments of this application. The switching chip provided in the embodiments of this application will be described below. Figure 8 A schematic diagram of the structure of the switching chip provided in the embodiments of this application is shown below. Figure 8 As shown, the switching chip is used to forward traffic between multiple AI chips. These AI chips are used to train a neural network. During the training process, the multiple AI chips can be in either the computation or communication phase. The switching chip includes:

[0112] The sampling module 801 is used to sample the traffic forwarded by the switching chip to obtain the sampled traffic;

[0113] The detection module 802 is used to detect whether multiple AI chips are switching between the computing phase and the communication phase based on the sampled traffic.

[0114] The first adjustment module 803 is used to reduce power consumption for traffic to be forwarded when multiple AI chips switch from the communication phase to the computing phase; or,

[0115] The second adjustment module 804 is used to improve power consumption for traffic to be forwarded when multiple AI chips switch from the computing stage to the communication stage.

[0116] In one possible implementation, the training process includes multiple sampling periods, including the current sampling period. The sampled traffic is the traffic sampled in the current sampling period. The detection module is used to detect whether multiple AI chips are switching between the computation phase and the communication phase based on the traffic sampled in the previous sampling period and the traffic sampled in the current sampling period.

[0117] In one possible implementation, the first adjustment module is used to determine that multiple AI chips switch from the communication phase to the computing phase and reduce power consumption for the traffic to be forwarded if the traffic sampled in the previous sampling period is less than the traffic sampled in the current sampling period.

[0118] In one possible implementation, the second adjustment module is used to determine that multiple AI chips switch from the computation phase to the communication phase and increase power consumption for the traffic to be forwarded if the traffic sampled in the previous sampling period is greater than the traffic sampled in the current sampling period.

[0119] In one possible implementation, the first adjustment module is configured to perform at least one of the following: progressively reducing the operating frequency of the switching chip, progressively reducing the operating voltage of the switching chip, and shutting down the serializer and deserializer (SERDES) of the switching chip, wherein progressively reducing the operating frequency of the switching chip has a higher priority than progressively reducing the operating voltage of the switching chip, and progressively reducing the operating voltage of the switching chip has a higher priority than shutting down the SERDES of the switching chip.

[0120] In one possible implementation, the second adjustment module is configured to: at the time when the traffic to be forwarded arrives, perform at least one of the following: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDEs of the switching chip; or, at a target time, perform at least one of the following: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDEs of the switching chip, wherein the target time is prior to the time when the traffic to be forwarded arrives.

[0121] In one possible implementation, the training process includes multiple statistical periods, each including a previous statistical period and the current statistical period. The current statistical period includes a previous sampling period and the current sampling period. A second adjustment module is configured to: if the average duration of communication between the multiple AI chips in the previous statistical period is zero, perform at least one of the following when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the switch chip's SERDES; or, if the average duration of communication between the multiple AI chips in the previous statistical period and the average duration of computation between the multiple AI chips in the previous statistical period are not zero, and the average duration of computation between the multiple AI chips in the previous statistical period is zero... If the standard deviation of the duration of the communication phase is greater than or equal to a preset first threshold, at least one of the following shall be performed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDE of the switching chip; or, if the average duration of the multiple AI chips in the communication phase in the previous statistical period and the average duration of the multiple AI chips in the calculation phase in the previous statistical period are not zero, and the standard deviation of the duration of the multiple AI chips in the calculation phase in the previous statistical period is greater than or equal to a preset second threshold, at least one of the following shall be performed when the traffic to be forwarded arrives: increase the operating frequency of the switching chip, increase the operating voltage of the switching chip, and start the SERDE of the switching chip.

[0122] In one possible implementation, the second adjustment module is configured to perform at least one of the following at a target time if the average duration of multiple AI chips in the communication phase and the average duration of multiple AI chips in the computation phase in the previous statistical period are not zero, the standard deviation of the duration of multiple AI chips in the communication phase in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of multiple AI chips in the computation phase in the previous statistical period is less than a preset second threshold: increasing the operating frequency of the switching chip, increasing the operating voltage of the switching chip, and starting the SERDES of the switching chip.

[0123] In one possible implementation, the average duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the average duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the number of times the multiple AI chips were in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the communication phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the communication phase during the previous statistical period and the duration of each communication phase during the previous statistical period; the standard deviation of the duration of multiple AI chips in the computation phase during the previous statistical period is calculated based on the average duration of the multiple AI chips in the computation phase during the previous statistical period and the duration of each computation phase during the previous statistical period.

[0124] In one possible implementation, the difference between the target time and the arrival time of the traffic to be forwarded is calculated based on the average duration of the computation phase of multiple AI chips in the previous statistical period and the standard deviation of the duration of the computation phase of multiple AI chips in the previous statistical period.

[0125] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of this application, and the resulting technical effects are the same as those of the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in the embodiment of this application, and it will not be repeated here.

[0126] This application also relates to an interconnection device, which can be presented as a chip. The chip includes a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in a storage unit to cause the chip to perform the power optimization method described in the above embodiments. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0127] For details, please refer to Figure 9 , Figure 9This is a schematic diagram of the chip provided in an embodiment of this application. The chip can be represented as a neural network processor switching chip 900. The switching chip 900 is mounted as a coprocessor on the AI ​​chip, and the AI ​​chip assigns tasks (e.g., forwarding traffic from the AI ​​chip). The core part of the switching chip is the arithmetic circuit 903, which is controlled by the controller 904 to extract matrix data from the memory and perform multiplication operations.

[0128] In some implementations, the arithmetic circuit 903 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 903 is a two-dimensional pulsating array. The arithmetic circuit 903 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 903 is a general-purpose matrix processor.

[0129] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 902 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 901 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 908.

[0130] Unified memory 906 is used to store input and output data. Weight data is directly transferred to weight memory 902 via Direct Memory Access Controller (DMAC) 905. Input data is also transferred to unified memory 906 via DMAC.

[0131] BIU stands for Bus Interface Unit 913, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 909.

[0132] The Bus Interface Unit (BIU) 913 is used by the instruction fetch memory 909 to fetch instructions from external memory, and also by the memory access controller 905 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0133] The DMAC is mainly used to move input data from external memory DDR to unified memory 906, or to weight data to weight memory 902, or to input data to input memory 901.

[0134] The vector computation unit 907 includes multiple arithmetic processing units that, when necessary, further process the output of the arithmetic circuit 903, such as vector multiplication, vector addition, exponential operations, logarithmic operations, magnitude comparisons, etc. It is mainly used to cooperate with the arithmetic circuit 903 to complete parameter calculations for power consumption adjustment strategies, etc.

[0135] In some implementations, the vector computation unit 907 can store the processed output vector into the unified memory 906. For example, the vector computation unit 907 can apply a linear function; or a nonlinear function, to the output of the arithmetic circuit 903.

[0136] The instruction fetch buffer 909 connected to the controller 904 is used to store the instructions used by the controller 904;

[0137] Unified memory 906, input memory 901, weighted memory 902, and instruction fetch memory 909 are all on-chip memories. External memory is proprietary to this switching chip hardware architecture.

[0138] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the above program.

[0139] This application also relates to a computer storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform the steps performed by the aforementioned switching chip.

[0140] This application also relates to a computer program product that stores instructions that, when executed by a computer, cause the computer to perform the steps performed by the aforementioned switching chip.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A power consumption optimization method for switching chips, characterized in that, The method comprises: The exchange chip forwards traffic between multiple artificial intelligence (AI) chips and samples the forwarded traffic to obtain sampled traffic; The exchange chip detects whether the multiple AI chips switch between a computing phase and a communication phase based on the sampled traffic; If the multiple AI chips switch from the communication phase to the computing phase, the exchange chip reduces power consumption for the traffic to be forwarded; or If the multiple AI chips switch from the computing phase to the communication phase, the exchange chip increases power consumption for the traffic to be forwarded. The training process performed by the multiple AI chips includes multiple sampling periods, the multiple sampling periods include a current sampling period, the sampled traffic is traffic sampled in the current sampling period, and the exchange chip detects whether the multiple AI chips switch between the computing phase and the communication phase based on the sampled traffic comprises: The exchange chip detects whether the multiple AI chips switch between the computing phase and the communication phase based on traffic sampled in a previous sampling period and traffic sampled in the current sampling period.

2. The method of claim 1, wherein, The exchange chip reduces power consumption for the traffic to be forwarded if the multiple AI chips switch from the communication phase to the computing phase comprises: If the traffic sampled in the previous sampling period is less than the traffic sampled in the current sampling period, the exchange chip determines that the multiple AI chips switch from the communication phase to the computing phase and reduces power consumption for the traffic to be forwarded.

3. The method of claim 1, wherein, The exchange chip increases power consumption for the traffic to be forwarded if the multiple AI chips switch from the computing phase to the communication phase comprises: If the traffic sampled in the previous sampling period is greater than the traffic sampled in the current sampling period, the exchange chip determines that the multiple AI chips switch from the computing phase to the communication phase and increases power consumption for the traffic to be forwarded.

4. The method of claim 2, wherein, The exchange chip reduces power consumption for the traffic to be forwarded comprises: The exchange chip performs at least one of the following: gradually reducing a working frequency of the exchange chip, gradually reducing a working voltage of the exchange chip, and shutting down a serializer / deserializer (serdes) of the exchange chip, wherein the priority of gradually reducing the working frequency of the exchange chip is higher than the priority of gradually reducing the working voltage of the exchange chip, and the priority of gradually reducing the working voltage of the exchange chip is higher than the priority of shutting down the serdes of the exchange chip.

5. The method of claim 3, wherein, The exchange chip increases power consumption for the traffic to be forwarded comprises: The exchange chip performs at least one of the following at the time when the traffic to be forwarded arrives: increasing a working frequency of the exchange chip, increasing a working voltage of the exchange chip, and starting a serializer / deserializer (serdes) of the exchange chip; or The switch chip executes at least one of the following at a target time: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, the target time being before the time when the traffic to be forwarded arrives.

6. The method of claim 5, wherein, The training process includes a plurality of statistical periods, including a previous statistical period and a current statistical period, the current statistical period including the previous sampling period and the current sampling period, the switch chip executing at least one of the following at the time when the traffic to be forwarded arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, including: If the average duration of the plurality of AI chips in the communication stage in the previous statistical period is zero, the switch chip executes at least one of the following at the time when the traffic to be forwarded arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip; or, If the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, and the standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is greater than or equal to a preset first threshold, the switch chip executes at least one of the following at the time when the traffic to be forwarded arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip; or, If the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, and the standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is greater than or equal to a preset second threshold, the switch chip executes at least one of the following at the time when the traffic to be forwarded arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip.

7. The method of claim 5, wherein, The switch chip executes at least one of the following at a target time: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, the target time being before the time when the traffic to be forwarded arrives. The switch chip executes at least one of the following at a target time: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, the target time being before the time when the traffic to be forwarded arrives. If the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, the standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is less than a preset second threshold, the switching chip performs at least one of the following at a target time: increasing the working frequency of the switching chip, increasing the working voltage of the switching chip, and starting the serdes of the switching chip.

8. The method according to claim 6 or 7, characterized in that, The average duration of the plurality of AI chips in the communication stage in the previous statistical period is calculated based on the number of times that the plurality of AI chips are in the communication stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the communication stage in the previous statistical period; The average duration of the plurality of AI chips in the calculation stage in the previous statistical period is calculated based on the number of times that the plurality of AI chips are in the calculation stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the calculation stage in the previous statistical period; The standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is calculated based on the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the communication stage in the previous statistical period; The standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is calculated based on the average duration of the plurality of AI chips in the calculation stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the calculation stage in the previous statistical period.

9. The method of claim 7, wherein, The difference between the target time and the time when the traffic to be forwarded arrives is calculated based on the average duration of the plurality of AI chips in the calculation stage in the previous statistical period and the standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period.

10. A switch chip, comprising: The switching chip is configured to forward traffic between a plurality of AI chips, and the switching chip comprises: a sampling module configured to sample the forwarded traffic to obtain sampled traffic; a detection module configured to detect whether the plurality of AI chips switch between a calculation stage and a communication stage based on the sampled traffic; a first adjustment module configured to reduce power consumption for traffic to be forwarded if the plurality of AI chips switch from the communication stage to the calculation stage; or a second adjustment module configured to increase power consumption for the traffic to be forwarded if the plurality of AI chips switch from the calculation stage to the communication stage. The training process performed by the plurality of AI chips comprises a plurality of sampling periods, the plurality of sampling periods comprising a current sampling period, the sampled traffic being traffic sampled in the current sampling period, and the detection module is configured to detect whether the plurality of AI chips switches between the calculation stage and the communication stage based on the sampled traffic in a previous sampling period and the sampled traffic in the current sampling period.

11. The switch chip of claim 10, wherein, The first adjustment module is configured to determine that the plurality of AI chips switches from the communication stage to the calculation stage and reduce power consumption for the to-be-forwarded traffic if the sampled traffic in the previous sampling period is less than the sampled traffic in the current sampling period.

12. The switch chip of claim 10, wherein, The second adjustment module is configured to determine that the plurality of AI chips switches from the calculation stage to the communication stage and increase power consumption for the to-be-forwarded traffic if the sampled traffic in the previous sampling period is greater than the sampled traffic in the current sampling period.

13. The switch chip of claim 11, wherein, The first adjustment module is configured to perform at least one of the following: gradually reducing the working frequency of the switch chip, gradually reducing the working voltage of the switch chip, and shutting down the serdes of the switch chip, wherein the priority of gradually reducing the working frequency of the switch chip is higher than the priority of gradually reducing the working voltage of the switch chip, and the priority of gradually reducing the working voltage of the switch chip is higher than the priority of shutting down the serdes of the switch chip.

14. The switch chip of claim 12, wherein, The second adjustment module is configured to: perform at least one of the following at the time when the to-be-forwarded traffic arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip; or perform at least one of the following at a target time: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, the target time being before the time when the to-be-forwarded traffic arrives.

15. The switch chip of claim 14, wherein, The training process comprises a plurality of statistical periods, the plurality of statistical periods comprising a previous statistical period and a current statistical period, the current statistical period comprising the previous sampling period and the current sampling period, and the second adjustment module is configured to: perform at least one of the following at the time when the to-be-forwarded traffic arrives: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, if the average duration of the plurality of AI chips in the communication stage in the previous statistical period is zero; or perform at least one of the following at a target time: increasing the working frequency of the switch chip, increasing the working voltage of the switch chip, and starting the serdes of the switch chip, the target time being before the time when the to-be-forwarded traffic arrives. If the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, and the standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is greater than or equal to a preset first threshold, at least one of the following is performed at the moment when the traffic to be forwarded arrives: increasing the working frequency of the switching chip, increasing the working voltage of the switching chip, and starting the serdes of the switching chip; or, If the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, and the standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is greater than or equal to a preset second threshold, at least one of the following is performed at the moment when the traffic to be forwarded arrives: increasing the working frequency of the switching chip, increasing the working voltage of the switching chip, and starting the serdes of the switching chip.

16. The switch chip of claim 14, wherein, The second adjustment module is configured to, if the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the average duration of the plurality of AI chips in the calculation stage in the previous statistical period are not zero, the standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is less than a preset first threshold, and the standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is less than a preset second threshold, at least one of the following is performed at the target moment: increasing the working frequency of the switching chip, increasing the working voltage of the switching chip, and starting the serdes of the switching chip.

17. The switch chip according to claim 15 or 16, wherein, The average duration of the plurality of AI chips in the communication stage in the previous statistical period is calculated based on the number of times that the plurality of AI chips are in the communication stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the communication stage in the previous statistical period; The average duration of the plurality of AI chips in the calculation stage in the previous statistical period is calculated based on the number of times that the plurality of AI chips are in the calculation stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the calculation stage in the previous statistical period; The standard deviation of the duration of the plurality of AI chips in the communication stage in the previous statistical period is calculated based on the average duration of the plurality of AI chips in the communication stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the communication stage in the previous statistical period; The standard deviation of the duration of the plurality of AI chips in the calculation stage in the previous statistical period is calculated based on the average duration of the plurality of AI chips in the calculation stage in the previous statistical period and the duration of each time that the plurality of AI chips are in the calculation stage in the previous statistical period.

18. The switch chip of claim 16, wherein, The difference between the target moment and the moment when the traffic to be forwarded arrives is calculated based on an average length of time that the plurality of AI chips are in the calculation stage in the previous statistical period and a standard deviation of the length of time that the plurality of AI chips are in the calculation stage in the previous statistical period.

19. An interconnect device, characterized by The apparatus includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, and when the code is executed, the interconnection apparatus executes the method according to any one of claims 1 to 9.

20. A computer storage medium, comprising, The computer storage medium stores one or more instructions that, when executed by one or more computers, cause the one or more computers to implement the method according to any one of claims 1 to 9.

21. A computer program product, characterised in that, The computer program product stores instructions that, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 9.

Images