Method, apparatus, and computer equipment for dynamic clock frequency adjustment of large model accelerators

Abstract

This disclosure provides a method, apparatus, and computer device for dynamic clock frequency adjustment of a large model accelerator, relating to the field of integrated circuit design, particularly to the field of power consumption management of artificial intelligence chips. The implementation scheme is as follows: obtaining the operation type and working stage of the large model accelerator executing the current computing task; determining the actual running batch size and theoretical full-load batch size required to execute the current computing task; in response to the working stage being the decoding stage, adjusting the frequency of the computing clock based on the ratio of the actual running batch size to the theoretical full-load batch size; in response to the working stage being the pre-filling stage, adjusting the frequency of the on-chip network clock and the transfer rate of the memory system based on the ratio of the theoretical full-load batch size to the actual running batch size; during the execution of the current computing task, when a transient switch occurs due to a step jump in the computing load, adjusting the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering.

Description

Technical Field

[0001] This disclosure relates to the field of integrated circuit design, particularly to the field of power management for artificial intelligence chips, and especially to a method, apparatus, computer device, computer-readable storage medium, and computer program product for dynamic clock frequency adjustment of large-scale accelerators. Background Technology

[0002] The inference process of a large language model accelerator is divided into a pre-filling stage and a decoding stage, with drastically different data load characteristics in these two stages. The pre-filling stage has limited computing power, leading to high-frequency data reuse and resulting in excessive memory and on-chip network bandwidth. Conversely, the decoding stage is constrained by memory bandwidth, causing computational units to idle extensively while waiting for data. Currently, the computational units, on-chip networks, and memory systems in existing accelerators typically operate at a fixed maximum clock frequency, making it impossible to adjust according to the actual load requirements of different stages and computation types. Furthermore, the sudden changes in load at the start or end of parallel computing tasks can cause excessive drops or rises in the supply voltage, affecting the chip's operational stability and reliability. Therefore, dynamically adjusting the clock frequency of each component during the computation process of a large language model accelerator has become an important technical direction. Summary of the Invention

[0003] This disclosure provides a method, apparatus, computer device, computer-readable storage medium, and computer program product for dynamic clock frequency adjustment of large model accelerators.

[0004] According to one aspect of this disclosure, a method for dynamic clock frequency adjustment for a large model accelerator is provided, comprising: obtaining the operation type and working stage of the large model accelerator executing the current computing task, wherein the working stage includes a pre-filling stage and a decoding stage, and the operation type includes dense operation and sparse operation; determining the actual batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the large model accelerator's computing unit reaching full load under the operation type of the current computing task, wherein the theoretical full-load batch size includes a first theoretical full-load batch size under sparse operation and a second theoretical full-load batch size under dense operation; and, in response to the working stage being the decoding stage, adjusting the clock frequency based on the actual batch size... The frequency of the computing clock of the computing unit is adjusted according to the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size so that the computing power of the computing unit after adjustment matches the actual computing power required; in response to the pre-filling stage of the working stage, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted according to the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size so that the data transfer bandwidth from the memory system to the computing unit after adjustment matches the actual bandwidth required; and during the execution of the current computing task, when the computing load of the computing unit undergoes a step jump transient switch, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering.

[0005] In some embodiments, the reference computing clock frequency is the rated clock frequency of the computing unit under full-load operation. In response to the working stage being the decoding stage, the frequency of the computing clock of the computing unit is adjusted based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size, so that the adjusted computing power of the computing unit matches the actual required computing power. This includes: in response to the current computing task's operation type being sparse operation, calculating a first ratio of the actual running batch size to the first theoretical full-load batch size; adjusting the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the first ratio; in response to the current computing task's operation type being dense operation, calculating a second ratio of the actual running batch size to the second theoretical full-load batch size; and adjusting the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the second ratio.

[0006] In some embodiments, the reference memory system rate is the rated data transfer rate of the memory system under full bandwidth operation. In response to the operation phase being a pre-filling phase, based on the ratio of a first theoretical full-load batch size or a second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth. This includes: in response to the current computing task's operation type being sparse operation, calculating a third ratio of the first theoretical full-load batch size to the actual running batch size; adjusting the memory system transfer rate to the rate value corresponding to the product of the reference memory system rate and the third ratio; in response to the current computing task's operation type being dense operation, calculating a fourth ratio of the second theoretical full-load batch size to the actual running batch size; and adjusting the memory system transfer rate to the rate value corresponding to the product of the reference memory system rate and the fourth ratio.

[0007] In some embodiments, the reference on-chip network clock frequency is the rated clock frequency of the on-chip network of the large model accelerator in full-bandwidth operation. In response to the operation phase being a pre-filling phase, based on the ratio of a first theoretical full-load batch size or a second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth. The method further includes: in response to the current computing task's operation type being sparse operation, adjusting the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a third ratio; and in response to the current computing task's operation type being dense operation, adjusting the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a fourth ratio.

[0008] In some embodiments, during the execution of the current computing task, when a transient switch occurs where the computing load of the computing unit undergoes a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering, including: adjusting the frequency of the computing clock to the preset low-frequency clock threshold at a first moment before starting parallel computing in the current computing task; and restoring the frequency of the computing clock to the frequency of the computing clock before the first moment at a second moment after the parallel computing has been started.

[0009] In some embodiments, during the execution of the current computing task, when a transient switch occurs where the computing load of the computing unit undergoes a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering. This further includes: adjusting the frequency of the computing clock to the preset low-frequency clock threshold at a third moment before the end of parallel computing in the current computing task; and restoring the frequency of the computing clock to the frequency of the computing clock before the third moment at a fourth moment after the end of parallel computing.

[0010] According to one aspect of this disclosure, a dynamic clock frequency adjustment device for a large-scale accelerator is provided, comprising: a runtime data acquisition module configured to acquire the operation type and working stage of the large-scale accelerator executing the current computing task, wherein the working stage includes a pre-filling stage and a decoding stage, and the operation type includes dense computation and sparse computation; a batch size determination module configured to determine the actual runtime batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the large-scale accelerator's computing units reaching full load under the operation type of the current computing task, wherein the theoretical full-load batch size includes a first theoretical full-load batch size under sparse computation and a second theoretical full-load batch size under dense computation; and a computing frequency adjustment module configured to, in response to the working stage being the decoding stage, adjust the actual runtime data based on the current runtime data acquisition stage and the current operating stage, the large-scale accelerator's current operating stage, ... The algorithm includes several modules: a running batch size adjustment module, configured to adjust the frequency of the computing unit's clock based on the ratio of the running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size, so that the adjusted computing power of the computing unit matches the actual required computing power; a transmission bandwidth adjustment module, configured to adjust the frequency of the on-chip network clock of the large model accelerator and the transmission rate of the memory system based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, so that the adjusted data transmission bandwidth from the memory system to the computing unit matches the actual required bandwidth; and a frequency transition buffer module, configured to adjust the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering during the execution of the current computing task when the computing unit's computing load undergoes a step jump transient switch.

[0011] According to another aspect of this disclosure, a computer device is provided, comprising: at least one processor; and a memory having a computer program stored thereon, wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform the methods provided above in this disclosure.

[0012] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0013] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0014] According to one or more embodiments of this disclosure, chip power consumption optimization and underlying power supply network stability can be achieved without sacrificing the performance of large language models.

[0015] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0016] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0017] Figure 1 This is a flowchart illustrating a dynamic clock frequency adjustment method for a large-model accelerator according to an exemplary embodiment.

[0018] Figure 2 This is a schematic diagram illustrating a dynamic clock frequency adjustment process according to an exemplary embodiment.

[0019] Figure 3 This is a schematic diagram illustrating a dynamic balancing process according to an exemplary embodiment.

[0020] Figure 4 This is a schematic block diagram illustrating a dynamic clock frequency adjustment device for a large-model accelerator according to an exemplary embodiment.

[0021] Figure 5 An example computer device is shown in which any of the embodiments described herein may be implemented. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0024] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0025] In related technologies, large language model accelerators typically operate their computing units, on-chip networks, and memory systems at a fixed maximum clock frequency when performing computational tasks. Because the data load characteristics differ between the pre-filling and decoding phases of the computation process, and because the accelerator supports both dense and sparse computation modes, the fixed-frequency operation cannot meet the actual load requirements of each subsystem under different stages and computation types, resulting in unnecessary dynamic power consumption on resource-idle subsystems.

[0026] First, during the decoding phase, because the actual batch size is much smaller than the batch size required to fully load the computing unit, most computing resources are idle, while the computing clock still runs at full load, resulting in unnecessary power consumption by the computing unit. This problem is particularly pronounced in sparse computing mode, because the peak power of sparse computing is higher than that of dense computing, the batch size required for full load is larger, and the proportion of idle computing power under the same actual batch size is higher, leading to more severe power waste. Similarly, during the pre-filling phase, the computing unit is running at full load, while the memory system and on-chip network bandwidth have a large amount of spare bandwidth. If the clock frequency is maintained at full bandwidth, unnecessary dynamic power consumption will also occur in the subsystems with idle bandwidth.

[0027] Secondly, at the moment a parallel computing task starts or ends, the workload of the computing unit changes drastically between light and full load, causing a significant change in the current on the power supply network within a very short time. This huge rate of current change (di / dt effect) within a very short time can cause excessive drops or rises in the supply voltage, seriously affecting the operational stability and lifespan of the accelerator hardware.

[0028] To address this, embodiments of this disclosure propose a more effective dynamic clock frequency adjustment method for large model accelerators. By independently and dynamically adjusting the computing clock of the computing unit, the clock of the on-chip network, and the transmission rate of the memory system according to the working stage and operation type of the computing task, and switching the computing clock to a low-frequency threshold for transition buffering at the load jump moment of the parallel computing task, the actual load requirements of each subsystem can be matched with each other and the power supply voltage fluctuations caused by the di / dt effect can be effectively suppressed. Thus, the dynamic power consumption of the accelerator is reduced and the power supply stability is guaranteed without sacrificing the performance of the large model.

[0029] Exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart illustrating a dynamic clock frequency adjustment method for a large-model accelerator according to an exemplary embodiment.

[0031] like Figure 1 As shown, the dynamic clock frequency adjustment method for large model accelerators includes steps S101 to S105.

[0032] In step S101, the operation type and working stage of the current computing task executed by the large model accelerator are obtained. The working stage includes the pre-filling stage and the decoding stage, and the operation type includes dense operation and sparse operation.

[0033] In the example, during the computation of the large language model, the accelerator first needs to determine the current stage of the computation task and the type of operation being performed, so as to adjust the clock frequency of each subsystem accordingly. The working stages are divided into a pre-filling stage and a decoding stage. These two stages have fundamentally different data load characteristics. The pre-filling stage performs parallel computation on all data in the input sequence, which is a computationally constrained scenario; while the decoding stage generates each word-by-word, which is a bandwidth-constrained scenario. The operation types are divided into dense operations and sparse operations. The peak computing power of the corresponding computing units differs, resulting in different batch sizes required to fully load the computing resources. The identification of the working stages and operation types can be monitored and obtained on the software side. For example, the current stage can be determined based on the scheduling information of the current computation task, and the operation type can be determined based on whether sparse acceleration is enabled for the operator to be executed. The identification results are then provided to the load dynamic balancing control module on the hardware side for reading.

[0034] In step S102, the actual batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the large model accelerator's computing unit reaching full load under the operation type of the current computing task are determined. The theoretical full-load batch size includes the first theoretical full-load batch size under sparse operation and the second theoretical full-load batch size under dense operation.

[0035] In the example, after determining the current computing task's working stage and operation type, two key parameters need to be obtained: the actual running batch size and the theoretical full-load batch size. The actual running batch size refers to the real business load of the current computing task. This value can be obtained by the software by monitoring the actual business traffic of the current computing task, and its size changes dynamically with changes in computing requests. The theoretical full-load batch size is a fixed value determined by the accelerator's own hardware characteristics, representing the batch size required to fully utilize all the computing resources of the computing unit. Since the accelerator supports both dense and sparse computing modes, and the peak computing power of the computing unit is higher in sparse computing mode than in dense computing mode, the batch size required to fully utilize the sparse computing power is correspondingly larger. Therefore, the theoretical full-load batch size can include a first theoretical full-load batch size under sparse computing and a second theoretical full-load batch size under dense computing, where the first theoretical full-load batch size is larger than the second theoretical full-load batch size. These two values ​​can be determined based on hardware parameters such as the number of computing units and the peak computing power of a single computing unit under the corresponding operation type. The relationship between the actual operating batch size and the theoretical full-load batch size forms the core basis for subsequent adjustment of the clock frequency of each subsystem.

[0036] In step S103, in response to the working stage being the decoding stage, the frequency of the computing clock of the computing unit is adjusted based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size so that the computing power of the computing unit after adjustment matches the actual required computing power.

[0037] In the example, when the accelerator is in the decoding phase, due to the word-by-word generation characteristics, the actual batch size is usually much smaller than the theoretical full-load batch size required to fully utilize the computing unit's power. Most of the computing unit's resources are idle. If the clock frequency remains at full load during this period, the idle resources will generate a large amount of unnecessary dynamic power consumption. Therefore, during the decoding phase, the clock frequency needs to be reduced based on the ratio of the actual batch size to the theoretical full-load batch size to match the adjusted computing unit's power with the actual required power. If sparse computation is being performed, the ratio of the actual batch size to the first theoretical full-load batch size can be used as the adjustment basis; if dense computation is being performed, the ratio of the actual batch size to the second theoretical full-load batch size can be used. Since the first theoretical full-load batch size is larger than the second theoretical full-load batch size, the ratio calculated in the sparse computation mode is smaller for the same actual batch size, meaning a greater reduction in the clock frequency, which corresponds to the higher idle rate in the sparse computation mode.

[0038] In step S104, in response to the working phase being the pre-filling phase, the frequency of the on-chip network clock and the transfer rate of the memory system of the large model accelerator are adjusted based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, so that the data transfer bandwidth from the adjusted memory system to the computing unit matches the actual required bandwidth.

[0039] In the example, when the accelerator is in the pre-filling phase, due to the need for large-scale parallel computation of all data in the input sequence, the actual batch size is usually much larger than the theoretical full-load batch size required to fully load the computing unit. The computing unit operates at full capacity, and computing resources are fully utilized. However, in this computationally constrained scenario, the data transfer bandwidth between the memory system and the on-chip network is relatively excessive and not fully utilized. Maintaining the on-chip network clock frequency and transmission rate at full bandwidth would result in wasted power. Therefore, during the pre-filling phase, the frequency of the on-chip network clock and the transmission rate of the memory system need to be reduced based on the ratio of the theoretical full-load batch size to the actual running batch size, so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual bandwidth required. If sparse computation is being performed, the ratio of the first theoretical full-load batch size to the actual running batch size can be used as the adjustment basis; if dense computation is being performed, the ratio of the second theoretical full-load batch size to the actual running batch size can be used as the adjustment basis.

[0040] In step S105, during the execution of the current computing task, when the computing load of the computing unit undergoes a transient switch with a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering.

[0041] In the example, during the accelerator's execution of the current computational task, when a large number of parallel computing units start or stop simultaneously, the computational load undergoes a step transition between light and full load. This transition causes a significant change in current on the power supply network within a very short time. This rapid current change triggers the di / dt effect, causing excessive voltage drops or rises within the chip, potentially damaging the device. Therefore, during the transient transition of the computational load, the frequency of the computing clock can be temporarily adjusted to a preset low-frequency clock threshold for a transition buffer. Lowering the computing clock frequency reduces the current change caused by the transition from light to full load, thereby reducing excessive voltage drops or rises caused by the di / dt effect. After the load transition is complete, the computing clock frequency is restored to its previous operating frequency. This transient adjustment mechanism, in conjunction with frequency adjustment based on the operating stage and computation type, eliminates power waste during operation and ensures power supply stability during load transitions.

[0042] Therefore, by independently and dynamically adjusting the clock frequencies of the computing units, on-chip networks, and memory system transfer rates according to the working stage and operation type of the large model accelerator's computing tasks, the large model accelerator can provide computing and transmission resources that match the current actual load requirements, avoiding unnecessary power consumption when resources are idle. At the same time, by adjusting the computing clock frequency to a low-frequency threshold for transition buffering at the load transition time of parallel computing tasks, the excessive drop and rise of the supply voltage caused by current changes can be effectively suppressed, thereby achieving a smooth transition of the supply voltage under load transition scenarios and improving the operational stability of the large language model accelerator in complex computing scenarios.

[0043] In some embodiments, the reference computing clock frequency is the rated clock frequency of the computing unit under full-load operation. In response to the working stage being the decoding stage, the frequency of the computing clock of the computing unit is adjusted based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size, so that the adjusted computing power of the computing unit matches the actual required computing power. This includes: in response to the current computing task's operation type being sparse operation, calculating a first ratio of the actual running batch size to the first theoretical full-load batch size; adjusting the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the first ratio; in response to the current computing task's operation type being dense operation, calculating a second ratio of the actual running batch size to the second theoretical full-load batch size; and adjusting the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the second ratio.

[0044] In the example, when adjusting the frequency of the computing clock during the decoding phase, the frequency adjustment ratio needs to be calculated based on the theoretical full-load batch size selected according to the current operation type. If a sparse operation is currently being performed, the actual required batch size can be obtained and divided by the first theoretical full-load batch size corresponding to the sparse operation to calculate the first ratio. This first ratio reflects the proportion of the actual computing power demand of the computing unit to its peak computing power in the sparse operation mode, that is, the utilization rate of the computing power required for the current actual batch size relative to the full load of sparse computing power. Subsequently, the reference computing clock frequency of the computing unit under full-load working state can be used as the base, multiplied by the first ratio, and the actual computing clock frequency can be adjusted to the frequency value obtained by the multiplication operation through a phase-locked loop.

[0045] In the example, if a dense operation is currently being performed, the actual batch size required for the current operation can be obtained and divided by the second theoretical full-load batch size corresponding to the dense operation to calculate the second ratio. This second ratio reflects the proportion of the actual computing power demand of the computing unit to its peak computing power in the dense operation mode, that is, the utilization rate of the computing power required for the current actual batch size relative to the dense computing power at full load. Subsequently, the second ratio can be multiplied by the reference computing clock frequency of the computing unit under full-load operation, and the actual computing clock frequency can be adjusted to the frequency value obtained by the multiplication operation through a phase-locked loop. For example, the reference computing clock frequency is 1GHz, the current actual batch size is 1, the second theoretical full-load batch size under dense operation is 10, and the first theoretical full-load batch size under sparse operation is 20. If dense computation is currently being performed, the second ratio is 1 / 10, and the computation clock frequency can be adjusted to 1GHz multiplied by 1 / 10, i.e., 100MHz, through a phase-locked loop; if sparse computation is currently being performed, the first ratio is 1 / 20, and the computation clock frequency can be adjusted to 1GHz multiplied by 1 / 20, i.e., 50MHz, through a phase-locked loop.

[0046] Therefore, by selecting the corresponding theoretical full-load batch size calculation frequency adjustment ratio according to the operation type, and accurately adjusting the frequency of the computing clock accordingly, the computing unit can provide computing power that matches the actual load demand in both dense and sparse operation modes, thereby reducing dynamic power consumption waste caused by idle computing power.

[0047] In some embodiments, the reference memory system rate is the rated data transfer rate of the memory system under full bandwidth operation. In response to the operation phase being a pre-filling phase, based on the ratio of a first theoretical full-load batch size or a second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth. This includes: in response to the current computing task's operation type being sparse operation, calculating a third ratio of the first theoretical full-load batch size to the actual running batch size; adjusting the memory system transfer rate to the rate value corresponding to the product of the reference memory system rate and the third ratio; in response to the current computing task's operation type being dense operation, calculating a fourth ratio of the second theoretical full-load batch size to the actual running batch size; and adjusting the memory system transfer rate to the rate value corresponding to the product of the reference memory system rate and the fourth ratio.

[0048] In the example, when the computation task is in the pre-filling phase, the computing unit is already running at full capacity, while the actual bandwidth requirement of the memory system is only a portion of that at full bandwidth. If it continues to run at the baseline rate, the excess bandwidth will generate unnecessary dynamic power consumption. Therefore, when adjusting the memory system transfer rate during the pre-filling phase, it is necessary to select the corresponding theoretical full-load batch size based on the current operation type to calculate the rate adjustment ratio. The baseline memory system rate is the rated data transfer rate of the memory system under full-bandwidth operation. If sparse computation is currently being performed, the actual required batch size can be obtained, and the ratio of this first theoretical full-load batch size to the current actual batch size can be calculated to obtain a third ratio. This third ratio reflects the proportion of the memory system's actual bandwidth requirement to its full-bandwidth capacity under sparse computation mode during the pre-filling phase; that is, the utilization rate of the data transfer bandwidth required when the computing unit's computing power is just at full load relative to the memory system's peak bandwidth. Subsequently, this third ratio can be used as a rate reduction ratio, multiplied by the baseline memory system rate, and the actual transfer rate of the memory system adjusted to the rate value obtained from this product.

[0049] In the example, if dense computation is currently being performed, the actual batch size required for execution can be obtained, and the ratio of the second theoretical full-load batch size to the current actual batch size can be calculated to obtain a fourth ratio. This fourth ratio reflects the proportion of the actual bandwidth demand of the memory system to its full bandwidth capacity in the dense computation mode during the pre-filling phase; that is, the utilization rate of the data transfer bandwidth required when the computing unit's computing power is just fully loaded relative to the peak bandwidth of the memory system. Subsequently, this fourth ratio can be used as a rate-down ratio, multiplied by the baseline memory system speed, and the actual transfer rate of the memory system can be adjusted to the speed value obtained by this product. For example, the baseline memory system speed is 4800MT / s, the current actual batch size in the pre-filling phase is 100, the second theoretical full-load batch size under dense computation is 50, and the first theoretical full-load batch size under sparse computation is 80. If dense computation is currently being performed, the fourth ratio is 50 / 100, or 0.5, which can adjust the memory system's transfer rate to 4800MT / s multiplied by 0.5, or 2400MT / s. If sparse computation is currently being performed, the third ratio is 80 / 100, or 0.8, which can adjust the memory system's transfer rate to 4800MT / s multiplied by 0.8, or 3840MT / s.

[0050] Therefore, by adjusting the transmission rate to match the actual required bandwidth, the power consumption waste caused by bandwidth idleness in the memory system can be reduced without affecting the data supply of the computing unit.

[0051] In some embodiments, the reference on-chip network clock frequency is the rated clock frequency of the on-chip network of the large model accelerator in full-bandwidth operation. In response to the operation phase being a pre-filling phase, based on the ratio of a first theoretical full-load batch size or a second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth. The method further includes: in response to the current computing task's operation type being sparse operation, adjusting the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a third ratio; and in response to the current computing task's operation type being dense operation, adjusting the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a fourth ratio.

[0052] In the example, the on-chip network (CNN) is located on the data transmission path between the memory system and the computing unit, responsible for moving data from the memory system to the computing unit. During the pre-filling phase, while adjusting the transfer rate of the memory system, the clock frequency of the CNN also needs to be adjusted accordingly to ensure that the bandwidth of each link in the entire data transmission path remains consistent. The baseline CNN clock frequency is the rated clock frequency of the CNN when operating at full bandwidth. If a sparse operation is being performed, the third ratio can be used as the frequency reduction ratio and multiplied by the baseline CNN clock frequency to adjust the CNN clock frequency to the frequency value obtained by the product. If a dense operation is being performed, the fourth ratio can be used as the frequency reduction ratio and multiplied by the baseline CNN clock frequency to adjust the CNN clock frequency to the frequency value obtained by the product. The third and fourth ratios used here are the same as the ratios used when adjusting the memory system transfer rate, thus ensuring that the bandwidth of the CNN and the memory system is reduced proportionally.

[0053] Therefore, by synchronizing the frequency of the on-chip network clock with the transmission rate of the memory system, the bandwidth of the entire data transmission path can be uniformly reduced to a level that matches the actual required bandwidth, thereby reducing the dynamic power consumption waste caused by bandwidth idleness from multiple aspects.

[0054] In some embodiments, during the execution of the current computing task, when a transient switch occurs where the computing load of the computing unit undergoes a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering, including: adjusting the frequency of the computing clock to the preset low-frequency clock threshold at a first moment before starting parallel computing in the current computing task; and restoring the frequency of the computing clock to the frequency of the computing clock before the first moment at a second moment after the parallel computing has been started.

[0055] In the example, when the large-scale accelerator initiates parallel computing during a computational task, a large number of computing units simultaneously switch from idle to operational status, causing the computational load to jump from light to full load instantaneously. To mitigate the excessive voltage drop caused by this jump, the computing clock frequency can be adjusted to a preset low-frequency clock threshold at the first moment before parallel computing begins. This first moment can refer to the point in time when the accelerator receives the parallel computing start command but has not yet actually started the computing units. Lowering the computing clock frequency reduces the current change caused by the switch from light to full load, thus mitigating the excessive voltage drop caused by the di / dt effect. At the second moment after parallel computing has completed and entered a stable operational state, the computing clock frequency is restored to the frequency before the first moment, allowing the computing units to return to their previous operating frequency and continue executing the task.

[0056] By reducing the frequency before startup and restoring it after startup, a low-frequency transition interval is inserted during the transition from light load to full load. This transforms the original step change in current into a smoother transition, ensuring the stability of power supply during load transitions.

[0057] In some embodiments, during the execution of the current computing task, when a transient switch occurs where the computing load of the computing unit undergoes a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering. This further includes: adjusting the frequency of the computing clock to the preset low-frequency clock threshold at a third moment before the end of parallel computing in the current computing task; and restoring the frequency of the computing clock to the frequency of the computing clock before the third moment at a fourth moment after the end of parallel computing.

[0058] In the example, corresponding to the scenario at the start of parallel computing, as the parallel computing is about to end, a large number of computing units will simultaneously switch from working to idle states, and the computing load will jump from full load to light load instantaneously. At this time, the sharp drop in current will cause an excessive rise in supply voltage. Therefore, in the third moment before the parallel computing ends, the frequency of the computing clock can be adjusted to a preset low-frequency clock threshold, so that the computing units run at a lower frequency in the final stage before the end. The reduction in the computing clock frequency can reduce the current change caused by the switch from full load to light load, thereby mitigating the excessive voltage rise caused by the di / dt effect. Finally, in the fourth moment after the parallel computing ends and returns to a stable light load state, the frequency of the computing clock can be restored to the frequency of the computing clock before the third moment.

[0059] Therefore, by reducing the frequency before parallel computing ends and restoring it after it ends, it is equivalent to inserting a low-frequency transition interval during the transition from full load to light load. This forms a symmetrical protection mechanism with the transition buffer during the startup phase, thereby fully ensuring the power supply stability during load transitions.

[0060] Figure 2 This is a schematic diagram illustrating a dynamic clock frequency adjustment process according to an exemplary embodiment.

[0061] like Figure 2 As shown, the process first determines whether the current computation task being executed by the large model accelerator is in the pre-filling phase; otherwise, it proceeds to the decoding phase. In the pre-filling phase, it further determines whether the current computation is dense. If it is dense, the memory system's transfer rate is adjusted based on the ratio of the second theoretical full-load batch size to the actual running batch size, and the on-chip network clock frequency is adjusted based on the same ratio. If it is sparse, the memory system's transfer rate is adjusted based on the ratio of the first theoretical full-load batch size to the actual running batch size, and the on-chip network clock frequency is adjusted based on the same ratio. In the decoding phase, the process similarly determines whether the current computation is dense. If it is dense, the computation clock frequency is adjusted based on the ratio of the actual running batch size to the second theoretical full-load batch size; if it is sparse, the computation clock frequency is adjusted based on the ratio of the actual running batch size to the first theoretical full-load batch size. After the steady-state frequency adjustment is completed, the transient protection process begins. First, it is determined whether parallel computing has been started. If so, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold before startup, and the frequency of the computing clock is restored after startup is completed. Then, when the calculation is about to end, the frequency of the computing clock is adjusted to the preset low-frequency clock threshold again, and the frequency of the computing clock is restored after the calculation is completed.

[0062] Figure 3 This is a schematic diagram illustrating a dynamic balancing process according to an exemplary embodiment.

[0063] like Figure 3 As shown, the device comprises three controlled subsystems: DDR (Double Data Rate Synchronous Dynamic Random Access Memory), an on-chip network system, and a computing unit, along with a load balancing control module located at the top. Data flow is from DDR through the on-chip network system to the computing unit. The load balancing control module is connected to the three subsystems via three independent control signals: the DDR transfer rate, the on-chip network clock, and the computing clock. Based on the current computing task's stage and operation type, it independently and dynamically adjusts the DDR transfer rate, the on-chip network system clock frequency, and the computing unit's computing clock frequency via a phase-locked loop, ensuring that the operating speed of each subsystem matches the actual load requirements.

[0064] Embodiments of this disclosure also provide a dynamic clock frequency adjustment device for large-scale accelerators.

[0065] Figure 4 This is a schematic block diagram illustrating a dynamic clock frequency adjustment device 400 for a large-model accelerator according to an exemplary embodiment.

[0066] In some embodiments, the dynamic clock frequency adjustment device 400 for a large model accelerator may include a running data acquisition module 401, a batch size determination module 402, a calculation frequency adjustment module 403, a transmission bandwidth adjustment module 404, and a frequency transition buffer module 405.

[0067] The data acquisition module 401 is configured to acquire the operation type and working stage of the current computing task being performed by the large model accelerator. The working stage includes a pre-filling stage and a decoding stage, and the operation type includes dense operation and sparse operation.

[0068] The batch size determination module 402 is configured to determine the actual running batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the large model accelerator's computing unit reaching full load under the operation type of the current computing task. The theoretical full-load batch size includes the first theoretical full-load batch size under sparse operation and the second theoretical full-load batch size under dense operation.

[0069] The computing frequency adjustment module 403 is configured to adjust the computing clock frequency of the computing unit in response to the working phase being the decoding phase, based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size, so that the computing power of the computing unit after adjustment matches the actual required computing power.

[0070] The transmission bandwidth adjustment module 404 is configured to adjust the frequency of the on-chip network clock and the transmission rate of the memory system of the large model accelerator in response to the pre-filling phase of the working phase, based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, so that the adjusted data transmission bandwidth from the memory system to the computing unit matches the actual required bandwidth.

[0071] The frequency transition buffer module 405 is configured to adjust the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering when the computing load of the computing unit undergoes a transient switch during the execution of the current computing task.

[0072] It should be understood that Figure 4 The running data acquisition module 401, batch size determination module 402, frequency adjustment calculation module 403, transmission bandwidth adjustment module 404, and frequency transition buffer module 405 shown can respectively correspond to, for example, the following modules: Figure 1 The operations of steps S101, S102, S103, S104, and S105 are shown. Therefore, the details of each aspect will not be elaborated here.

[0073] In some embodiments, the reference computing clock frequency is the rated clock frequency of the computing unit under full-load operation. The computing frequency adjustment module 403 may include a first ratio calculation module 4031, a first frequency adjustment module 4032, a second ratio calculation module 4033, and a second frequency adjustment module 4034. The first ratio calculation module 4031 may be configured to calculate a first ratio between the actual running batch size and the first theoretical full-load batch size in response to the current computing task's operation type being sparse operation. The first frequency adjustment module 4032 may be configured to adjust the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the first ratio. The second ratio calculation module 4033 may be configured to calculate a second ratio between the actual running batch size and the second theoretical full-load batch size in response to the current computing task's operation type being dense operation. The second frequency adjustment module 4034 may be configured to adjust the frequency of the computing clock to the frequency value corresponding to the product of the reference computing clock frequency and the second ratio.

[0074] In some embodiments, the reference memory system rate is the rated data transfer rate of the memory system under full bandwidth operation. The transmission bandwidth adjustment module 404 may include a third ratio calculation module 4041, a first rate adjustment module 4042, a fourth ratio calculation module 4043, and a second rate adjustment module 4044. The third ratio calculation module 4041 may be configured to calculate a third ratio between the first theoretical full-load batch size and the actual running batch size in response to the current computing task's operation type being sparse operation. The first rate adjustment module 4042 may be configured to adjust the memory system's transmission rate to the rate value corresponding to the product of the reference memory system rate and the third ratio. The fourth ratio calculation module 4043 may be configured to calculate a fourth ratio between the second theoretical full-load batch size and the actual running batch size in response to the current computing task's operation type being dense operation. The second rate adjustment module 4044 may be configured to adjust the memory system's transmission rate to the rate value corresponding to the product of the reference memory system rate and the fourth ratio.

[0075] In some embodiments, the reference on-chip network clock frequency is the rated clock frequency of the on-chip network of the large model accelerator in full-bandwidth operation. The transmission bandwidth adjustment module 404 may further include a third frequency adjustment module 4045 and a fourth frequency adjustment module 4046. The third frequency adjustment module 4045 may be configured to adjust the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a third ratio, in response to the current computation task's operation type being sparse computation. The fourth frequency adjustment module 4046 may be configured to adjust the frequency of the on-chip network clock to the frequency value corresponding to the product of the reference on-chip network clock frequency and a fourth ratio, in response to the current computation task's operation type being dense computation.

[0076] In some embodiments, the frequency transition buffer module 405 may include a startup clock adjustment module 4051 and a startup clock recovery module 4052. The startup clock adjustment module 4051 may be configured to adjust the frequency of the computing clock to a preset low-frequency clock threshold at a first moment before parallel computing begins in the current computing task. The startup clock recovery module 4052 may be configured to restore the frequency of the computing clock to the frequency of the computing clock before the first moment at a second moment after the parallel computing has finished starting.

[0077] In some embodiments, the frequency transition buffer module 405 may further include an end clock adjustment module 4053 and an end clock recovery module 4054. The end clock adjustment module 4053 may be configured to adjust the frequency of the computing clock to a preset low-frequency clock threshold at a third time point before the end of parallel computation in the current computing task. The end clock recovery module 4054 may be configured to restore the frequency of the computing clock to the frequency of the computing clock before the third time point at a fourth time point after the end of parallel computation.

[0078] According to one aspect of this disclosure, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0079] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0080] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0081] Figure 5An example computer device 500 is shown in which any of the embodiments described herein may be implemented. The computer device 500 may be used to implement one or more components of the systems and methods described above. The computer device 500 may include a bus 502 or other communication mechanism for communicating information, and one or more processors 504 coupled to the bus 502 for processing information. The processor 504 may be, for example, one or more general-purpose microprocessors.

[0082] Computer device 500 may also include main memory 506, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 502, for storing information and instructions to be executed by processor 504. Main memory 506 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 504. Such instructions, when stored in a storage medium accessible to processor 504, can make computer device 500 a special-purpose machine customized to perform the operations specified in the instructions. Main memory 506 may include non-volatile media and / or volatile media. Non-volatile media may include, for example, optical discs or magnetic disks. Volatile media may include dynamic memory. Common media formats may include, for example, floppy disks, collapsible disks, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs (read-only optical disc drives), any other optical data storage media, any physical media with a perforated arrangement, RAM (random access memory), DRAM (dynamic random access memory), PROM (programmable read-only memory) and EPROM (erasable programmable read-only memory), FLASH-EPROM (fast erase programmable read-only memory), NVRAM (non-volatile random access memory), any other memory chips or tape cartridges, or network versions of the above.

[0083] Computer device 500 may implement the techniques described herein using custom hardwired logic, one or more ASICs (Application-Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays), firmware, and / or program logic, which, when combined with computer device 500, enable computer device 500 to become a special-purpose machine or to be programmed therein. According to one embodiment, the techniques described herein are executed by computer device 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 508. Executing the sequence of instructions contained in main memory 506 causes processor 504 to perform the processing steps described herein. For example, the processes / methods disclosed herein may be implemented by computer program instructions stored in main memory 506. When these instructions are executed by processor 504, they may perform the steps shown in the corresponding figures and as described above. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0084] Computer device 500 also includes a network interface 510 coupled to bus 502. Network interface 510 can provide bidirectional data communication coupled to one or more network links connected to one or more networks. As another example, network interface 510 can be a local area network (LAN) card to provide data communication connectivity with a compatible LAN (or a WAN component communicating with a WAN (wide area network)). Wireless links can also be implemented.

[0085] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or processor-implemented engine may be distributed across many geographic locations.

[0086] Each process, method, and algorithm described in the preceding sections can be embodied in a code module executed by one or more computer systems or computer processors including computer hardware, and can be fully or partially automated by them. These processes and algorithms can be implemented, in part or in whole, in a specific application circuit.

[0087] When the functions disclosed herein are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, can be embodied in the form of a software product. This software product can be stored in a storage medium and includes instructions to cause a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in the embodiments of this application. The storage medium may include a flash drive, a portable hard drive, ROM, RAM, a magnetic disk, an optical disk, another medium suitable for storing program code, or any combination thereof.

[0088] The embodiments disclosed herein can be implemented via a cloud platform, server, or group of servers that interact with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can install platform applications.

[0089] The various features and processes described above can be used independently or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in a non-specifically disclosed order, or multiple blocks or states may be combined in a single block or state. Exemplary blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the disclosed exemplary embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the disclosed exemplary embodiments.

[0090] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. An algorithm may consist of program code or instructions stored in memory (such as the non-transitory computer-readable storage medium described above). Such an algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not be explicitly programmed into the computer to perform the function, but may learn from training data to obtain a predictive model for performing that function.

[0091] The various operations of the exemplary methods described herein can be performed at least in part by one or more processors, which are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute the engine of a processor implementation whose operation is to perform one or more of the operations or functions described herein.

[0092] Similarly, the methods described herein can be implemented at least partially by a processor, where a specific processor or one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors can also run in a “cloud computing” environment or as “Software as a Service” (SaaS) to support the execution of the relevant operations. For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), which can be accessed via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0093] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or processor-implemented engine may be distributed across many geographic locations.

[0094] In this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are described and illustrated as independent operations, one or more individual operations may be performed concurrently, and these operations are not required to be performed in the order shown. Structures and functionalities presented as independent components in the example configuration may be implemented as combined structures or components. Similarly, structures and functionalities presented as individual components may be implemented as independent components. These and other variations, modifications, additions, and improvements are all within the scope of this document.

[0095] As used herein, “or” is inclusive rather than exclusive unless explicitly stated or indicated by context. Furthermore, “and” is both common and individual unless explicitly stated or indicated by context. Moreover, multiple instances may be provided for the resources, operations, or structures described herein as a single example. Furthermore, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are illustrated within the context of a particular illustrative configuration. The allocation of other functionalities is conceivable and may fall within the scope of various embodiments of this disclosure. Generally, structures and functionalities presented as independent resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functionalities presented as individual resources may be implemented as independent resources. These and other variations, modifications, additions, and improvements are all within the scope of embodiments of this disclosure. Therefore, this specification and accompanying drawings should be viewed in an illustrative rather than restrictive sense.

[0096] The terms “comprising” or “including” are used to indicate the presence of a subsequently stated feature, but do not preclude the addition of other features. Conditional language, in particular, such as “may,” “can,” or “may,” unless specifically stated or otherwise understood in the context of use, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, such conditional language generally does not imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments must include logic that, with or without user input or prompting, determines whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment.

Claims

1. A method for dynamic clock frequency adjustment for large-scale accelerators, characterized in that, The method includes: Obtain the operation type and working stage of the current computing task executed by the large model accelerator, wherein the working stage includes a pre-filling stage and a decoding stage, and the operation type includes dense operation and sparse operation; Determine the actual batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the large model accelerator's computing unit reaching full load under the operation type of the current computing task, wherein the theoretical full-load batch size includes the first theoretical full-load batch size under the sparse operation and the second theoretical full-load batch size under the dense operation; In response to the working stage being the decoding stage, the frequency of the computing clock of the computing unit is adjusted based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size so that the computing power of the computing unit after adjustment matches the actual required computing power. In response to the working phase being the pre-filling phase, based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock and the transfer rate of the memory system of the large-model accelerator are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth; and During the execution of the current computing task, when the computing load of the computing unit undergoes a transient switch with a step jump, the frequency of the computing clock is adjusted to a preset low-frequency clock threshold for transition buffering.

2. The method of claim 1, wherein, The reference computing clock frequency is the rated clock frequency of the computing unit under full load operation. The response to the operating stage is the decoding stage. Based on the ratio of the actual running batch size to the first theoretical full load batch size or the second theoretical full load batch size, the computing clock frequency of the computing unit is adjusted so that the adjusted computing power of the computing unit matches the actual required computing power, including: In response to the fact that the operation type of the current computing task is sparse operation, a first ratio of the actual running batch size to the first theoretical full-load batch size is calculated; The frequency of the computing clock is adjusted to the frequency value corresponding to the product of the reference computing clock frequency and the first ratio; In response to the current computing task being a dense computation, a second ratio is calculated between the actual running batch size and the second theoretical full-load batch size; and The frequency of the computing clock is adjusted to the frequency value corresponding to the product of the reference computing clock frequency and the second ratio.

3. The method according to claim 1 or 2, characterized in that, The baseline memory system rate is the rated data transfer rate of the memory system under full bandwidth operation. The response to the operating phase is the pre-filling phase. Based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, the frequency of the on-chip network clock of the large model accelerator and the transfer rate of the memory system are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth, including: In response to the fact that the operation type of the current computing task is sparse operation, a third ratio of the first theoretical full-load batch size to the actual running batch size is calculated; The transfer rate of the memory system is adjusted to the rate value corresponding to the product of the base memory system rate and the third ratio; In response to the current computing task being a dense computation, a fourth ratio is calculated between the second theoretical full-load batch size and the actual running batch size; and The transfer rate of the memory system is adjusted to the rate value corresponding to the product of the base memory system rate and the fourth ratio.

4. The method of claim 3, wherein, The reference on-chip network clock frequency is the rated clock frequency of the on-chip network of the large model accelerator in full-bandwidth operation. The response to the operating phase is the pre-filling phase. Based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual operating batch size, the frequency of the on-chip network clock and the transfer rate of the memory system of the large model accelerator are adjusted so that the adjusted data transfer bandwidth from the memory system to the computing unit matches the actual required bandwidth. The system also includes: In response to the fact that the operation type of the current computing task is sparse operation, the frequency of the on-chip network clock is adjusted to the frequency value corresponding to the product of the reference on-chip network clock frequency and the third ratio; and In response to the fact that the current computing task is a dense operation, the frequency of the on-chip network clock is adjusted to the frequency value corresponding to the product of the reference on-chip network clock frequency and the fourth ratio.

5. The method according to claim 1 or 2, characterized in that, During the execution of the current computing task, when the computing unit experiences a transient switch with a step change in computing load, adjusting the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering includes: At a first moment before initiating parallel computing in the current computing task, the frequency of the computing clock is adjusted to the preset low-frequency clock threshold; and At a second moment after the parallel computing has started, the frequency of the computing clock is restored to the frequency of the computing clock before the first moment.

6. The method of claim 5, wherein, The method of adjusting the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering during the execution of the current computing task, when the computing load of the computing unit undergoes a transient switch with a step jump, further includes: At the third moment before the parallel computation in the current computing task ends, the frequency of the computing clock is adjusted to the preset low-frequency clock threshold; and At the fourth moment after the parallel computation ends, the frequency of the computation clock is restored to the frequency of the computation clock before the third moment.

7. A dynamic clock frequency adjustment apparatus for a large model accelerator, comprising: The device includes: The data acquisition module is configured to acquire the operation type and working stage of the current computing task being performed by the large model accelerator, wherein the working stage includes a pre-filling stage and a decoding stage, and the operation type includes dense operation and sparse operation. The batch size determination module is configured to determine the actual running batch size required to execute the current computing task and the theoretical full-load batch size corresponding to the computing power of the computing unit of the large model accelerator reaching full load under the operation type of the current computing task, wherein the theoretical full-load batch size includes the first theoretical full-load batch size under the sparse operation and the second theoretical full-load batch size under the dense operation. The computing frequency adjustment module is configured to, in response to the working stage being the decoding stage, adjust the frequency of the computing clock of the computing unit based on the ratio of the actual running batch size to the first theoretical full-load batch size or the second theoretical full-load batch size, so that the computing power of the computing unit after adjustment matches the actual required computing power. A transmission bandwidth adjustment module is configured to, in response to the working phase being the pre-filling phase, adjust the frequency of the on-chip network clock and the transmission rate of the memory system of the large-model accelerator based on the ratio of the first theoretical full-load batch size or the second theoretical full-load batch size to the actual running batch size, so that the adjusted data transmission bandwidth from the memory system to the computing unit matches the actual required bandwidth; and The frequency transition buffer module is configured to adjust the frequency of the computing clock to a preset low-frequency clock threshold for transition buffering during the execution of the current computing task when the computing load of the computing unit undergoes a transient switch with a step jump.

8. A computer device, comprising: The computer device includes: At least one processor; A memory having a computer program stored thereon, wherein, when executed by the at least one processor, the computer program causes the at least one processor to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

10. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-6.

Images