Hardware acceleration system, control method, chip and electronic device

Abstract

The embodiments of the present disclosure provide a hardware acceleration system, a control method, a chip and an electronic device. The hardware acceleration system comprises at least one hardware acceleration unit. The hardware acceleration unit comprises a plurality of hardware circuit modules for pipeline operation, including a read data module, a data calculation module located in the back stage of the read data module in the pipeline, and a write data module located in the back stage of the data calculation module in the pipeline. Each hardware circuit module has a set of control interfaces. The set of control interfaces includes a configuration interface and a start interface, which are used to receive control commands to configure and start each hardware circuit module independently. Thus, the configuration and start of each hardware circuit module can be controlled independently to overlap the configuration time and the data calculation time after the start of the front and back stage hardware circuit modules, thereby improving the operation efficiency of the modules and the acceleration effect and data throughput rate of the hardware acceleration system.

Description

Technical Field

[0001] This disclosure relates to the field of electronic circuit technology, and in particular to hardware acceleration systems, control methods, chips, and electronic devices. Background Technology

[0002] Hardware acceleration is a crucial component of high-performance computing. The hardware acceleration unit, or execution unit, in a hardware acceleration system can be abstracted into three parts: a read-data-delivery (RDMA) module, multiple compute modules (COMPUTE[n]), and a write-back (WDMA) module. The operational flow of the hardware acceleration unit is as follows:

[0003] 1. Configure the registers of the controller's data read module, calculation module, and write-back module;

[0004] 2. Configure the controller with a start signal to enable the read data module, calculation module, and write-back module to start running simultaneously;

[0005] 3. Once the data reading module, calculation module, and write-back module have finished running, notify the controller to update the registers and begin executing new tasks.

[0006] This solution is simple to control and easy to implement.

[0007] However, this approach has certain drawbacks. Each acceleration task requires the registers of these three modules to be configured sequentially before the modules can be started. If a particular module has a large number of configuration commands, it will slow down the startup time of the entire hardware acceleration unit, thereby reducing the execution efficiency of hardware acceleration. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this disclosure is to provide a hardware acceleration system, control method, chip and electronic device to solve the problems in the related art.

[0009] This disclosure provides a hardware acceleration system, comprising: at least one hardware acceleration unit; the hardware acceleration unit comprising: a plurality of hardware circuit modules for pipeline operation; the plurality of hardware circuit modules comprising: a data reading module; a data calculation module located after the data reading module in the pipeline; and a data writing module located after the data calculation module in the pipeline; wherein each of the hardware circuit modules has a set of control interfaces; the set of control interfaces includes a configuration interface and a startup interface, for receiving control commands respectively to enable each of the hardware circuit modules to be configured and started independently.

[0010] In an embodiment of the first aspect, the hardware acceleration unit includes: at least one blocking circuit module for blocking preceding data input to the hardware circuit module when the hardware circuit module is configured.

[0011] In the first aspect of the embodiment, the number of blocking circuit modules is at least two, and the data calculation module and the data writing module are each configured with a blocking circuit module; or, the number of blocking circuit modules is one, and the blocking circuit module includes at least two blocking regions, which are respectively used by the data calculation module and the data writing module to block the preceding data.

[0012] In an embodiment of the first aspect, the set of control interfaces further includes: an idle indicator interface for outputting an idle indicator signal indicating that the hardware circuit module is in an idle state.

[0013] In the first aspect of the embodiment, each of the hardware circuit modules operates in a pipeline under the control of a preset logical timing control command, and the configuration time between the front-end and back-end hardware circuit modules in one or more tasks overlaps with the data calculation time after startup.

[0014] In an embodiment of the first aspect, the preset logical timing sequence includes: a first control timing sequence; the hardware acceleration unit is in a first operating mode for processing a single task under the control of the control command of the first control timing sequence; the first operating mode includes: each hardware circuit module is configured and started, and the configuration of the subsequent hardware circuit module is executed based on the completion of the configuration of the preceding hardware circuit module.

[0015] In an embodiment of the first aspect, the preset logic timing includes: a second control timing; the hardware acceleration unit, under the control of the control command of the second control timing, is in a second operating mode that processes multiple tasks; the second operating mode includes: each hardware circuit module continuously performing multiple combined actions, each combined action including configuration, startup and idle indication, each combined action corresponding to the processing of a task; and the configuration of the hardware circuit module located at the later stage is executed based on the completion of the configuration of the hardware circuit module at the previous stage.

[0016] In the first aspect of the embodiment, there are multiple hardware acceleration units; under the control of a preset logical timing control command, the multiple hardware acceleration units operate in a third operating mode to process multiple tasks in a pipeline manner: the third operating mode includes: allocating the multiple tasks in a manner that prioritizes allocating a single task to each hardware acceleration unit; the data to be processed by the write data module of the front-end hardware acceleration unit is dispersed into multiple data segments for processing in multiple write sub-tasks, and the read data module of the rear-end hardware acceleration unit executes multiple read sub-tasks corresponding to the multiple data segments, and the read sub-task corresponding to the same data segment is executed after the corresponding write sub-task is completed.

[0017] In an embodiment of the first aspect, the hardware acceleration system includes: at least one control module, which is connected to a set of control interfaces of each of the hardware circuit modules, for outputting control commands with preset timing logic.

[0018] The second aspect of this disclosure provides a control method for controlling a hardware acceleration system as described in any one of the first aspects. The method includes: generating control commands with preset logical timing; outputting the control commands to each of the hardware circuit modules so that each of the hardware circuit modules operates in a pipeline under the control of the preset logical timing control commands, and making the configuration time between the front-end and rear-end hardware circuit modules in one or more tasks overlap with the data calculation time after startup.

[0019] A third aspect of this disclosure provides a chip comprising: a hardware acceleration system as described in any one of the first aspects.

[0020] This disclosure provides a fourth aspect of an electronic device, comprising: a chip as described in the third aspect.

[0021] As described above, this disclosure provides a hardware acceleration system, control method, chip, and electronic device. The hardware acceleration system includes at least one hardware acceleration unit; the hardware acceleration unit includes multiple hardware circuit modules for pipeline operation; the multiple hardware circuit modules include: a data read module coupled to a data input interface; a data calculation module located after the data read module in the pipeline; and a data write module located after the data calculation module in the pipeline, coupled to a data output interface; wherein each hardware circuit module has a set of control interfaces; the set of control interfaces includes a configuration interface and a startup interface, used to receive control commands to enable each hardware circuit module to be configured and started independently. Therefore, by independently controlling the configuration and startup of each hardware circuit module, the configuration time and data calculation time after startup between the preceding and following hardware circuit modules overlap, improving the module's operating efficiency and enhancing the acceleration effect and data throughput of the hardware acceleration system. Attached Figure Description

[0022] Figure 1 A timing diagram illustrating the hardware acceleration unit performing a single task in an example demonstrating the relevant technology.

[0023] Figure 2 A timing diagram illustrating a single hardware acceleration unit processing multiple tasks in an example demonstrating the relevant technology.

[0024] Figure 3 A timing diagram illustrating multiple hardware acceleration units processing multiple tasks is shown in one example demonstrating the relevant technology.

[0025] Figure 4 A schematic diagram of the hardware acceleration unit in one embodiment of this disclosure is shown.

[0026] Figure 5 A schematic diagram of the hardware acceleration unit is shown in yet another embodiment of this disclosure.

[0027] Figure 6 This illustration shows a timing diagram of a single hardware acceleration unit processing a single task in one embodiment of the present disclosure.

[0028] Figure 7 This illustration shows a timing diagram of a single hardware acceleration unit processing multiple tasks in one embodiment of the present disclosure.

[0029] Figure 8 This illustration shows a timing diagram of multiple hardware acceleration units processing multiple tasks in one embodiment of the present disclosure.

[0030] Figure 9 A flowchart illustrating a control method according to an embodiment of this disclosure is shown. Detailed Implementation

[0031] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the information disclosed herein. This disclosure can also be implemented or its modules applied through other different specific embodiments. Various details in this disclosure can also be modified or changed according to different viewpoints and application modules without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this disclosure can be combined with each other.

[0032] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, so that those skilled in the art to which this disclosure pertains can readily implement it. This disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

[0033] In this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic represented in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics represented may be combined in any suitable manner in any one or a group of embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples represented in this disclosure, as well as the features of those different embodiments or examples.

[0034] Furthermore, the terms "first" and "second" are used for illustrative purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the representation of this disclosure, "a set" means two or more, unless otherwise explicitly specified.

[0035] For the purpose of clarity, devices unrelated to the description are omitted, and the same or similar components throughout the specification are given the same reference numerals.

[0036] Throughout this specification, when it is said that a device is "connected" to another device, this includes not only "direct connection" but also "indirect connection" by placing other components in between. Furthermore, when it is said that a device "comprises" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather implies that other constituent elements may be included.

[0037] While the terms first, second, etc., are used in some examples herein to refer to various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, first interface and second interface, etc., are used. Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, step, operation, element, module, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, modules, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0038] The technical terms used herein are for reference only to specific embodiments and are not intended to limit the scope of this disclosure. The singular form used herein includes the plural form unless the statement explicitly indicates otherwise. The word "comprising" as used in this specification means to specify a particular characteristic, region, integer, step, operation, element, and / or component, and does not exclude the presence or addition of other characteristics, regions, integers, steps, operations, elements, and / or components.

[0039] Although not explicitly defined, all terms, including technical and scientific terms used herein, shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in commonly used dictionaries shall be further interpreted as having a meaning consistent with the relevant technical literature and the message of the present disclosure, and shall not be over-interpreted as having an ideal or overly formulaic meaning unless otherwise defined.

[0040] Hardware acceleration systems are used in various integrated circuit processor chips, such as central processing units (CPUs), system-on-a-chip (SoCs), and even the currently popular artificial intelligence (AI) chips. These systems are increasingly embedded to help improve processor chip performance. Taking AI chips as an example, artificial intelligence is based on machine learning models, and current machine learning models are primarily based on neural networks. Neural networks, especially deep neural networks, require a large amount of computation in both the training and inference phases. For example, matrix convolution calculations using convolution kernels involve numerous multiplications, additions, and intermediate cache read / write operations. These operations are performed by hardware acceleration units, which is orders of magnitude faster than the processor chip calling memory to run software for computation, thus contributing to improved processor chip computing performance. Common examples of hardware acceleration systems include graphics processing units (GPUs) and tensor processing units (TPUs).

[0041] Furthermore, with the gradual upgrading of intelligent driving technology, the processor chips in in-vehicle intelligent systems need to process video data, image data, radar data, wireless communication data, or combinations thereof. Therefore, hardware acceleration systems are increasingly being embedded in processor chips. Besides processor chips, hardware acceleration systems can also be integrated into other types of chips, such as memory control chips, and are not limited to processor chips.

[0042] A hardware acceleration system can consist of multiple hardware acceleration units. As previously described, a hardware acceleration unit can be abstracted into three parts: a read-data module (RDMA), multiple computation modules (COMPUTE[n]), and a write-back module (WDMA). Here, "DMA" stands for Direct Memory Access. That is, the read-data module reads data directly from memory (which can be registers, main memory, etc.) without going through the processor; the computation module performs calculations based on the data to obtain the results; and the write-back module directly writes the results back to memory. The entire process is completed in a pipeline by these three modules, which are entirely hardware electronic circuits, resulting in high computation speed.

[0043] The read data module, calculation module, and write-back module all require configuration registers and must be controlled to start. Since the read data module, calculation module, and write-back module are all hardware circuit modules in the hardware acceleration unit and have certain commonalities, they are sometimes collectively referred to as "hardware circuit modules" in this disclosure.

[0044] It should be noted that the data read module, calculation module, and write-back module described in the instruction manual are only abstract representations of the circuits, and do not limit each hardware module to a single circuit implementation. For example, each hardware module can contain one hardware circuit, or each hardware module can contain a combination of multiple hardware circuits.

[0045] like Figure 1 The diagram shows a timing illustration of a hardware acceleration unit executing a single task in an example of the related technology.

[0046] When executing each acceleration task, the read data module, calculation module, and write-back module of the hardware acceleration unit need to be configured accordingly, such as configuring registers for each module. Figure 1 As can be seen, each module needs to be configured sequentially before starting the calculations together. Specifically... Figure 1 In the configuration process, the read data module is configured using `RDMAcfg`, the compute module using `COMPcfg`, and the write data module using `WDMAcfg`. Then, under the startup command (`Start cmd`), the read data module initiates the read data process (RDMA), the compute module initiates the compute process (`COMPUTE`), and the write data module initiates the write data process (RDMA). Understandably, if a particular hardware circuit module has many configuration commands, it will slow down the startup time of the entire acceleration unit and affect the execution efficiency of the hardware acceleration system.

[0047] To explain more clearly Figure 1 The impact of timing on the efficiency of hardware acceleration units can be further illustrated by specific examples of hardware acceleration units performing multitasking.

[0048] like Figure 2 The diagram illustrates the timing of a single hardware acceleration unit processing multiple tasks in an example of the related technology.

[0049] When multiple tasks run consecutively, as exemplified by two tasks, Task 0 and Task 1, the configuration and startup of the three hardware circuit modules of the hardware acceleration unit for the next task can only begin after the three hardware circuit modules of the previous task have been configured and have completed their respective read, compute, and write operations. Specifically, see... Figure 2 In the process of executing Task 0, the configuration process of the read data module, the calculation module, and the write-back module is RDMA cfg0, COMP cfg0, and WDMA cfg0, and the working process after startup is RDMA0, COMPUTE0, and WDMA0; when executing Task 1, the configuration process of the read data module, the calculation module, and the write-back module is RDMAcfg1, COMP cfg1, and WDMAcfg1, and the working process after startup is RDMA1, COMPUTE1, and WDMA1.

[0050] This leads to an increase in the idle state of each module. As can be seen in the timeline, there is a gap between the time taken by the two tasks to read data in each row, which will result in a decrease in data throughput.

[0051] For example Figure 3 The diagram shown illustrates the timing of pipelined processing between multiple hardware acceleration units in an example of related technologies.

[0052] In this embodiment, two hardware acceleration units, hardware acceleration unit 0 and hardware acceleration unit 1, are shown as an example.

[0053] like Figure 3 For example, hardware acceleration unit 0 and hardware acceleration unit 1 execute two tasks. During the execution of the first task, the configuration process of the three hardware circuit modules is represented by cfg00, and their respective post-startup working processes are represented by r00, c00, and w00. Similarly, the configuration process during the execution of the second task is represented by cfg01, and the respective post-startup working processes of the three hardware circuit modules are represented by r01, c01, and w01. Likewise, we can obtain cfg10, r10, c10, and w10, as well as cfg11, r11, c11, and w11, when hardware acceleration unit 1 executes the two tasks.

[0054] Understandably, the pipeline operation between hardware acceleration unit 0 and hardware acceleration unit 1 means that the configuration and module startup of the later-stage hardware acceleration unit 1 can only begin after the write operation of the write data module of the previous-stage hardware acceleration unit 0 is completed. Therefore, the latency of hardware acceleration unit 0 will slow down hardware acceleration unit 1, leading to an increase in the idle state of each hardware circuit module in each hardware acceleration unit, resulting in a decrease in data throughput.

[0055] It is evident that, due to the significant waiting time between hardware circuit modules or between hardware acceleration units in the hardware acceleration unit processing tasks, the idle time of the hardware circuit modules is relatively long, thus reducing execution efficiency.

[0056] Therefore, in this embodiment, the circuit structure of each hardware acceleration unit is designed to overlap the working time of the hardware circuit modules of the hardware acceleration unit in executing tasks, thereby improving the execution efficiency of the hardware acceleration system. The working time may include the configuration time and the computation time after startup for each hardware circuit module. Generally, the configuration times of the various hardware circuit modules in the hardware acceleration unit may not overlap to avoid register allocation conflicts.

[0057] like Figure 4 The diagram shown illustrates the structure of a hardware acceleration unit in one embodiment of this disclosure.

[0058] The hardware acceleration unit includes multiple hardware circuit modules for pipeline operations. These multiple hardware circuit modules include a data reading module, a data calculation module, and a data writing module. The data calculation module is located after the data reading module in the pipeline; the data writing module is located after the data calculation module in the pipeline.

[0059] To allow the hardware circuit modules to overlap in their operating time, each module needs to be controlled independently, such as configured or started independently. Specifically, each hardware circuit module—the data read module, data calculation module, and data write module—has a set of control interfaces. These control interfaces include a configuration interface and a start interface, used to receive control commands to enable each hardware circuit module to be configured and started independently. In some embodiments, the set of control interfaces may further include an idle indicator interface, used to output an idle indicator signal indicating that the hardware circuit module is in an idle state.

[0060] For example, in Figure 4The document also illustrates a control module, which is connected to a set of control interfaces for each hardware circuit module to independently send control commands or receive idle indication signals to each hardware circuit module. In some embodiments, each hardware acceleration unit may include one such control module for controlling the various hardware circuit modules within that hardware acceleration unit. Alternatively, in other possible embodiments, the control module may be implemented externally to the hardware acceleration unit; for example, a single control module may connect to and control the various hardware circuit modules in multiple hardware acceleration units.

[0061] When a hardware circuit module is configured with a register, it is necessary to restrict data entry. Therefore, in some embodiments, the hardware acceleration unit includes at least one blocking circuit module for blocking preceding data input to the hardware circuit module when it is being configured. Exemplarily, the blocking circuit module may be implemented based on a trigger, switch circuit, or other circuitry that can lock input port signals.

[0062] exist Figure 4 In this embodiment, the computing module is configured with a first blocking circuit module, and the data writing module is configured with a second blocking circuit module. However, Figure 4 This is just one example; there may be other implementations.

[0063] For example, such as Figure 5 The diagram illustrates the structure of a blocking circuit module in another embodiment of this disclosure. In this embodiment, there may be only one blocking circuit module, but it includes at least two blocking regions, such as a first blocking region and a second blocking region, which are respectively used by the data calculation module and the data writing module to block the preceding data. Using a single blocking circuit module containing multiple blocking regions to achieve data blocking for each hardware circuit module in a hardware acceleration unit improves the modularity of the circuit and facilitates unified replacement of the blocking circuit module.

[0064] As described above, each hardware circuit module in the hardware acceleration unit—namely, the data read module, the calculation module, and the data write module—can operate in a pipeline under the control of preset logical timing control commands (which can be output by the control module). This allows the configuration time of the preceding and following hardware circuit modules in one or more tasks to overlap with the data calculation time after startup, thus achieving time reuse and improving efficiency. Specifically, even if the configuration times of different hardware circuit modules do not overlap, there can be overlap in the configuration time of different hardware circuit modules and in their working time after startup, or there can be overlap in their working time.

[0065] In some embodiments, the preset logical timing sequence includes a first control timing sequence; the hardware acceleration unit, under the control of the control command of the first control timing sequence, is in a first operating mode that processes a single task. The first operating mode includes: each hardware circuit module is configured and then started, and the configuration of subsequent hardware circuit modules is executed based on the completion of the configuration of preceding hardware circuit modules.

[0066] For example, such as Figure 6 The diagram illustrates the timing of a single hardware acceleration unit processing a single task in one embodiment of this disclosure.

[0067] In this embodiment, as shown in the first line, the read data module first performs configuration, and the corresponding configuration process is displayed as RDMAcfg in the time dimension. After the configuration is completed, a start command (which can be issued by the control module) is received to the read data module to start the read data of the corresponding task, and the corresponding read process is displayed as RDMA in the time dimension.

[0068] See the second line. After the read data module is configured, the computation module can be configured, for example, after the read data module's startup command. The corresponding configuration process is displayed as COMP cfg in the time dimension. Similar to the read data module, after the computation module is configured, the computation of the corresponding task is started under the second startup command. The computation process is displayed as COMPUTE.

[0069] See the third line. After the computing module starts, the data writing module is configured, which is displayed as WDMAcfg. After the data writing module is configured, the corresponding task's data writing is started under the start command, and the process is displayed as WDMA.

[0070] As can be seen from the diagram, COMP cfg is completely covered by RDMA, while COMPUTE partially overlaps with RDMA; WDMA cfg is completely covered by COMPUTE, while WDMA partially overlaps with COMPUTE. (Compared to...) Figure 1 Timing diagram, Figure 6 The embodiment significantly reduces the time required for the entire process and improves acceleration efficiency.

[0071] In some embodiments, the preset logic timing includes a second control timing; the hardware acceleration unit is in a second operating mode for processing multiple tasks under the control of the control command of the second control timing; the second operating mode includes: each hardware circuit module continuously performing multiple combined actions, each combined action including configuration, startup and idle indication, each combined action corresponding to the processing of a task; and the configuration of the hardware circuit module located at the later stage is executed based on the completion of the configuration of the hardware circuit module at the previous stage.

[0072] For example, Figure 7 The diagram illustrates the timing of a single hardware acceleration unit processing multiple tasks in one embodiment of this disclosure.

[0073] In this embodiment, only two tasks are shown for example, distinguished by "0" and "1". Accordingly, 0 and 1 are added after the previously mentioned RDMAcfg, RDMA, COMP cfg, COMPUTE, WDMAcfg, WDMA, etc., to indicate actions in different tasks. The timing diagrams corresponding to a larger number of tasks can be inferred without question, therefore no further increase in number is made here.

[0074] For the two tasks, see reference. Figure 7 Based on this, we can obtain the RDMAcfg0, RDMAcfg1, RDMA0, RDMA1, COMP cfg0, COMP cfg1, COMPTE0, COMPTE1, WDMAcfg0, WDMAcfg1, WDMA0, and WDMA1 executed by the read data module, the calculation module, and the write data module, respectively.

[0075] Specifically, see the first line. The read data module executes the following process: After RDMAcfg0, the read data module is started by the start command to execute RDMA0. After execution, an idle indication command (wait idle cmd) is issued. Then, RDMAcfg1 is executed. After configuration is complete, RDMA1 is executed under the start command to wait for the next task.

[0076] See the second line. The calculation module performs the following process: After COMP cfg0, the data reading module is started by the start command to execute COMPUTE0. After execution, it enters the idle state. Then, COMP cfg1 is executed. After configuration is completed, COMPUTE1 is executed by the start command. After execution, it enters the idle state to await the next task.

[0077] See line 3. The write module executes the following process: After WDMAcfg0, the read data module is started by the start command to execute WDMA0. After execution, it enters the idle state. Then, WDMAcfg1 is executed. After configuration is complete, WDMA1 is executed by the start command. After execution, it enters the idle state to await the next task.

[0078] Comparison Figure 7 and Figure 2 It can be seen that, Figure 2 There can be no parallel processing time between the two tasks in the process, and Figure 7 This creates a temporal overlap, effectively improving the operating efficiency of the hardware acceleration unit.

[0079] In some embodiments, the hardware acceleration system includes multiple hardware acceleration units, which can work collaboratively and coordinate their working timing to create temporal overlap, further improving operating efficiency. Specifically, under the control of preset logical timing control commands, the multiple hardware acceleration units operate in a third operating mode, processing multiple tasks in a pipeline manner.

[0080] For example, such as Figure 8 The diagram illustrates the timing of multiple hardware acceleration units processing multiple tasks in one embodiment of this disclosure.

[0081] The third operating mode is to allocate the multiple tasks by prioritizing the allocation of a single task to each hardware acceleration unit. For example, taking two hardware acceleration units operating in a pipeline: hardware acceleration unit 0 and hardware acceleration unit 1, and combining the two exemplary tasks 0 and 1, hardware acceleration unit 0 is assigned to execute task 0, and hardware acceleration unit 1 is assigned to execute task 1. It can be seen that each hardware circuit module in hardware acceleration unit 0 and hardware acceleration unit 1 executes the following processes respectively.

[0082] Hardware acceleration unit 0: RDMAcfg0, RDMA0, COMP cfg0, COMPUTE0, WDMAcfg0, WDMA0;

[0083] Hardware acceleration unit 1: RDMAcfg1, RDMA1, COMP cfg1, COMPUTE1, WDMAcfg1, WDMA1.

[0084] Specifically, the read data module 0, calculation module 0, and write data module 0 in the hardware acceleration unit 0 operate in accordance with the first operating mode in the previous embodiment (e.g., ...). Figure 6 (Single task processing) or a second running mode (such as...) Figure 7 The assigned tasks are processed using a mechanism similar to that used in multitasking. Similarly, the read data module 1, the calculation module 1, and the write data module 1 in the hardware acceleration unit 1 process the assigned tasks using a mechanism similar to that used in the first or second operating mode in the previous embodiments.

[0085] Therefore, each hardware acceleration unit can be assigned a single task, so the read data module of the preceding hardware acceleration unit 0 only needs to be started once (see RDMA0), thus avoiding the initial memory access delay caused by the need to start multiple times due to being assigned a large number of tasks.

[0086] Furthermore, in the third operating mode, the data to be processed by the write data module of the front-end hardware acceleration unit is distributed into multiple data segments for processing in multiple write subtasks. The read data module of the rear-end hardware acceleration unit executes multiple read subtasks corresponding to the multiple data segments. The read subtask corresponding to the same data segment is executed after the corresponding write subtask is completed. For example... Figure 8 In this embodiment, WDMA0 corresponds to the two data segments 0 and 1 of the write data to be processed in the write data task, which are decomposed into subtasks WDMA0-1 and WDMA0-2, and each has a prior configuration process WDMAcfg0 and WDMAcfg1. Correspondingly, the read data task RDMA0 to be processed by the read data module of hardware acceleration unit 1 is decomposed into subtasks RDMA1-1 and RDMA1-2, and each has a prior configuration process RDMAcfg1 plus RDMAcfg2.

[0087] Thus, when hardware acceleration unit 0 and hardware acceleration unit 1 are running in the pipeline, the read data module 1 of hardware acceleration unit 1 can be started to execute a subtask after the write data module 0 of hardware acceleration unit 0 has completed a subtask. The advantage is that the RDMA1-1 of hardware acceleration unit 1 does not need to wait for the WDMA0-2 of hardware acceleration unit 0 to finish before it is started, thereby improving processing efficiency.

[0088] Comparison Figure 8 and Figure 3 It can be seen that, Figure 3 The read data module of hardware acceleration unit 0 was activated twice, which increases latency, and because... Figure 3 The modules in hardware acceleration unit 0 and subsequent hardware acceleration unit 1 have very little overlap during operation, resulting in a long total task execution time. This leads to a loose structure and a large width in the horizontal dimension of time. In contrast, Figure 8 The entire task execution process overlaps significantly, resulting in a short total task execution time. This presents a compact structure with a relatively small width in the horizontal dimension of time.

[0089] This disclosure also provides a hardware acceleration system, including at least one hardware acceleration unit as described in any of the previous embodiments.

[0090] This disclosure also provides a chip, including the hardware acceleration system. In some embodiments, the chip may be an AI processor chip. AI chips can be implemented in the following categories: general-purpose chips (such as GPUs, FPGAs), neural network processor chips (NPUs), FPGA-based semi-custom chips, fully custom ASIC chips, neuromorphic computing chips, etc. Of course, in other embodiments, the chip may also be other processor chips, such as CPUs, MCU chips, etc.

[0091] This disclosure also provides an electronic device, including the chip. In some embodiments, the electronic device includes, but is not limited to, a server, desktop computer, laptop computer, mobile terminal, smart wearable device, gateway device, vehicle body controller, security device, IoT device, etc.

[0092] More specifically, in some AI chip-based application examples, the AI ​​chip containing the hardware acceleration system can be used on the terminal device's server, which can be located in the cloud, i.e., a data center. The training phase of deep learning requires a huge amount of data and a large amount of computation, which a single processor cannot complete independently. Therefore, the training phase can only be implemented in the cloud.

[0093] In other AI chip application examples, AI chips containing the aforementioned hardware acceleration system can be applied to terminal devices such as mobile phones, security cameras, automobiles, smart home devices, and various IoT devices, where there is a demand for AI-based edge computing.

[0094] like Figure 9 The diagram illustrates a control method according to an embodiment of this disclosure. This control method can be applied to hardware acceleration systems in previous embodiments. Exemplarily, the control method can be applied to, for example... Figure 4 The control module in the embodiment.

[0095] The method includes:

[0096] Step S901: Generate the control commands for the preset logical timing.

[0097] Step S902: Output each of the control commands to each of the hardware circuit modules so that each of the hardware circuit modules operates in a pipeline under the control of the control commands of the preset timing logic, and so that the configuration time of the front-end and back-end hardware circuit modules in one or more tasks overlaps with the data calculation time after startup.

[0098] You can refer to this. Figure 4 In this embodiment, on the hardware acceleration unit side, each hardware circuit module has a set of control interfaces for sending and receiving its own control commands, such as configuration, startup, or idle indication. Therefore, the control module can generate control commands for each hardware circuit module separately, and issue them individually to independently control the configuration and startup of each hardware circuit module. The control commands are constructed according to the required tasks based on a preset logical timing sequence to achieve... Figure 6 , Figure 7 , Figure 8 The timing results of task execution create overlaps in configuration time and post-startup working time between modules and even hardware acceleration units, thereby improving operating efficiency.

[0099] As described above, this disclosure provides a hardware acceleration system, control method, chip, and electronic device. The hardware acceleration system includes at least one hardware acceleration unit; the hardware acceleration unit includes multiple hardware circuit modules for pipeline operation; the multiple hardware circuit modules include: a data read module coupled to a data input interface; a data calculation module located after the data read module in the pipeline; and a data write module located after the data calculation module in the pipeline, coupled to a data output interface; wherein each hardware circuit module has a set of control interfaces; the set of control interfaces includes a configuration interface and a startup interface, used to receive control commands to enable each hardware circuit module to be configured and started independently. Therefore, by independently controlling the configuration and startup of each hardware circuit module, the configuration time and data calculation time after startup between the preceding and following hardware circuit modules overlap, improving the module's operating efficiency and enhancing the acceleration effect and data throughput of the hardware acceleration system.

[0100] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A hardware acceleration system, characterized in that, include: At least one hardware acceleration unit; The hardware acceleration unit includes: multiple hardware circuit modules for assembly line operations; The plurality of hardware circuit modules include: Data reading module; The data calculation module is located after the data reading module in the pipeline; The data writing module is located after the data calculation module in the pipeline. Each of the hardware circuit modules has a set of control interfaces; the set of control interfaces includes a configuration interface and a startup interface, which are used to receive control commands to enable each of the hardware circuit modules to be configured and started independently. Each of the hardware circuit modules operates in a pipeline under the control of a preset logical timing control command, and the configuration time between the front-end and back-end hardware circuit modules in one or more tasks overlaps with the data calculation time after startup. There are multiple hardware acceleration units; under the control of preset logical timing control commands, the multiple hardware acceleration units operate in a third operating mode to process multiple tasks in a pipeline manner: the third operating mode includes: The multiple tasks were assigned in a manner that prioritized allocating a single task to each hardware acceleration unit. The data to be processed in the write data module of the front-end hardware acceleration unit is divided into multiple data segments for processing in multiple write subtasks. The read data module of the back-end hardware acceleration unit executes multiple read subtasks corresponding to the multiple data segments. The read subtask corresponding to the same data segment is executed after the corresponding write subtask is completed. Specifically, after the write data module of the front-end hardware acceleration unit finishes executing the write subtask corresponding to any data segment, the read data module of the back-end hardware acceleration unit is started to execute the read subtask corresponding to the data segment. The read subtask is executed before all the write subtasks corresponding to the remaining data segments are completed.

2. The hardware acceleration system according to claim 1, characterized in that, The hardware acceleration unit includes at least one blocking circuit module for blocking the storage space of the preceding data input configuration when one of the hardware circuit modules is configured.

3. The hardware acceleration system according to claim 2, characterized in that, The number of blocking circuit modules is at least two, and the data calculation module and the data writing module are each configured with a blocking circuit module; or, the number of blocking circuit modules is one, and the blocking circuit module contains at least two blocking regions, which are respectively used by the data calculation module and the data writing module to block the previous data.

4. The hardware acceleration system according to claim 1, characterized in that, The set of control interfaces also includes an idle indicator interface, used to output an idle indicator signal indicating that the hardware circuit module is in an idle state.

5. The hardware acceleration system according to claim 1, characterized in that, The preset logic timing includes: a first control timing; the hardware acceleration unit, under the control of the control command of the first control timing, is in a first operating mode that processes a single task; the first operating mode includes: Each hardware circuit module is configured and then started, and the configuration of subsequent hardware circuit modules is executed based on the completion of the configuration of the preceding hardware circuit modules.

6. The hardware acceleration system according to claim 1, characterized in that, The preset logic timing includes: a second control timing; the hardware acceleration unit, under the control of the control commands of the second control timing, is in a second operating mode that processes multiple tasks; the second operating mode includes: Each of the hardware circuit modules continuously performs multiple combined actions, each combined action including configuration, startup, and idle indication, and each combined action corresponds to the processing of a task; and the configuration of the hardware circuit module located at the next level is performed based on the completion of the configuration of the hardware circuit module at the previous level.

7. The hardware acceleration system according to claim 1, characterized in that, It includes at least one control module, which is connected to a set of control interfaces of each of the hardware circuit modules, for outputting control commands with preset timing logic.

8. A control method, characterized in that, The method for controlling a hardware acceleration system as described in any one of claims 1 to 7 includes: Generate control commands for each part of the preset logical timing sequence; Each of the aforementioned control commands is output to each of the aforementioned hardware circuit modules, so that each of the aforementioned hardware circuit modules operates in a pipeline under the control of the control commands of the preset timing logic, and the configuration time of the preceding and following hardware circuit modules in one or more tasks overlaps with the data calculation time after startup.

9. A chip, characterized in that, include: The hardware acceleration system as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: The chip as described in claim 9.

Images