Board card embedded with high-performance AI chip with multi-layer structure

Abstract

The invention discloses a board card with an embedded multi-layer structure high-performance AI chip. The board card comprises the AI chip, a boot starting module and a communication module, wherein the boot starting module and the communication module are connected with the AI chip; the boot starting module is used for initializing the board card; the communication module is used for realizing communication with external equipment; the AI chip adopts a master-slave heterogeneous structure and comprises a control core and a plurality of operation core groups, and the control core is connected with each operation core group through a ring network bus; each operation core group comprises a high-bandwidth storage controller and a plurality of operation cores connected with the high-bandwidth storage controller, a local cache and an acceleration core which are connected are arranged in each operation core, and the acceleration core has mixed precision operation capability. The AI chip is designed into a master-slave heterogeneous architecture, the control core focuses on task scheduling and management, and the operation core group focuses on calculation, so that the efficiency of multiple levels, including memory access efficiency, operation efficiency and parallel processing efficiency, is effectively improved, and the overall energy efficiency ratio of the system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer equipment, and specifically relates to a board with an embedded multi-level high-performance AI chip. Background Art

[0002] Artificial intelligence applications are becoming increasingly diverse, with varying requirements for computing and memory access. Currently, there are many approaches to achieving efficient execution of AI applications. One representative approach is to use software scheduling mechanisms to enable accelerator boards and hosts to collaboratively execute applications, improving task management and resource utilization. However, when complex data dependencies exist between tasks, scheduling overhead increases, and relying solely on software-level optimizations makes it difficult to fully realize the hardware potential of the board. Another representative approach is to increase the rate at which data is transferred to the computing unit by loading data while computing, further reducing memory access latency during task computations and thereby improving computational efficiency. However, for large-scale parameter models, the multi-level storage advantages of local caches and high-bandwidth storage have not yet been fully utilized.

[0003] For large-scale computing tasks, software scheduling modules are typically deployed on both the host and device sides to enable coordinated execution between the host and accelerator. The host-side scheduling module manages task scheduling and ensures that tasks are delivered to the device side; the device-side scheduling module allocates computing resources based on this scheduling information. When data dependencies exist between tasks, scheduling overhead increases and computing unit utilization decreases. In these situations, collaborative hardware and software operations can further unleash the computing potential of the board.

[0004] At present, in order to reduce the impact of memory access latency on computing performance, most rely on full utilization of computing units to achieve data calculation and loading synchronization operations. Due to the limited number of computing units and the complexity of computing tasks, computing performance is also affected by bandwidth bottlenecks. Therefore, further improving performance is inseparable from making full use of multi-level storage structures. Summary of the Invention

[0005] To address the above issues, the present invention proposes a board with an embedded multi-level high-performance AI chip. The AI ​​chip is designed as a master-slave heterogeneous architecture, with the control core focusing on task scheduling and management, and the computing core group focusing on calculation, effectively improving efficiency at multiple levels, including memory access efficiency, computing efficiency, and parallel processing efficiency, thereby improving the overall energy efficiency of the system.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions: A board with a built-in high-performance AI chip with a multi-layered structure, comprising: an AI chip, and a boot module and a communication module connected to the AI ​​chip; The boot module is used to initialize the board; The communication module is used to communicate with external devices; The AI ​​chip adopts a master-slave heterogeneous structure, including a control core and several computing core groups. The control core is connected to each computing core group through a ring network bus; each computing core group includes a high-bandwidth storage controller and several computing cores connected to the high-bandwidth storage controller. Each computing core is equipped with a connected local cache and acceleration core, and the acceleration core has mixed-precision computing capabilities.

[0007] Optionally, the control core loads data from external storage and performs pre-processing and fast access based on its own cache structure; The control core writes data into the high-bandwidth memory corresponding to each computing core group through the ring network bus; The computing core reads data from the high-bandwidth memory connected to it and loads it into the cache structure inside the computing core. The acceleration core reads data from the cache structure connected to it and performs parallel calculations. The calculation result data is then written back to the high-bandwidth memory, and then uniformly recovered and summarized by the control core for entering the next stage of processing or output.

[0008] Optionally, the cache structure of the control core consists of a primary data cache and a secondary data cache.

[0009] Optionally, the control core adopts a 64-bit RISC structure based on an autonomous instruction set, is used to run an operating system and user programs, and is responsible for global resource management and task scheduling.

[0010] Optionally, the acceleration core includes a mixed-precision multiplication-addition component, a mixed-precision addition component, and a mixed-precision conversion component arranged in parallel; the mixed-precision multiplication-addition component, the mixed-precision addition component, and the mixed-precision conversion component each include an independently working floating-point subcomponent and a fixed-point subcomponent; The floating-point subcomponent of the mixed-precision multiplication-addition component is used to perform a floating-point mixed multiplication-addition operation, and the calculation formula corresponding to the floating-point mixed multiplication-addition operation is: 16bit*16bit+32bit; The fixed-point subcomponent of the mixed-precision multiply-add component is used to perform one integer multiply-add operation, and the calculation formula corresponding to the one integer multiply-add operation is: 16bit*16bit+32bit, or to perform two integer multiply-add operations, and the calculation formula corresponding to the two integer multiply-add operations is 8bit*8bit+16bit; The floating-point subcomponent of the mixed-precision addition component is used to perform a single-precision floating-point addition operation, and the calculation formula corresponding to the single-precision floating-point addition operation is: 32bit+32bit; The fixed-point subcomponent of the mixed-precision addition component is used to perform an integer addition operation, where the calculation formula corresponding to the one integer addition operation is 32 bits + 32 bits, or to perform two integer addition operations, where the calculation formula corresponding to the two integer addition operations is 16 bits + 16 bits; The floating point subcomponent of the mixed precision conversion component is used to perform a single-precision floating point to half-precision floating point conversion operation; The fixed-point subcomponent of the mixed-precision conversion component is used to perform a 32-bit signed integer to 16-bit signed integer conversion operation or two 16-bit signed integer to 8-bit signed integer conversion operations.

[0011] Optionally, the cache structure of each computing core consists of a first-level data cache with a capacity of 256KB.

[0012] Optionally, each computing core is integrated with a communication engine for receiving memory access signals; The computing cores in each computing core group are arranged in an N*M array. Each computing core group integrates M memory access engines, one memory access engine corresponds to each column of N computing cores. The memory access engine realizes inter-core memory access requests and responses through the communication engine within the computing core.

[0013] Optionally, the cache structure of each computing core adopts continuous addressing to improve memory access operations between cores and between cores and high-bandwidth memories.

[0014] Optionally, the board further includes a maintenance module and a power supply module; The maintenance module is connected to the power supply module and the AI ​​chip respectively, and is used to control the power-on, reset, maintenance and power load monitoring of the board.

[0015] Optionally, a radiator is further provided on the board; the radiator is connected to the maintenance module, and the maintenance module controls the working state of the radiator according to the real-time temperature of the board.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The board provided in the present invention has an embedded multi-level high-performance AI chip. By designing the AI ​​chip as a master-slave heterogeneous architecture, the control core serves as the master core to focus on task scheduling and management, and the computing core in the computing core group serves as the slave core to focus on computing. This can effectively improve efficiency at multiple levels, including memory access efficiency, computing efficiency, and parallel processing efficiency, thereby improving the overall energy efficiency of the system.

[0017] Furthermore, N*M computing cores are integrated into the computing core group to form a slave core array structure. This design achieves efficient parallel processing through the sequential flow of data within the slave core array structure. The components supporting mixed-precision computation (i.e., mixed-precision multiplication and addition, mixed-precision addition, and mixed-precision conversion) utilize a classic low-latency architecture, enabling the use of matching computational precision to task requirements. This improves computational utilization of acceleration components while optimizing resource allocation.

[0018] Furthermore, the present invention proposes to use the cache structure of each computing core as a key technology to combine local data cache with high-bandwidth storage, which meets the needs of artificial intelligence applications for high-speed data transmission and improves the overall energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which: Figure 1 This is a schematic diagram of the structure of a board with an embedded multi-layered high-performance AI chip according to an embodiment of the present invention; Figure 2 A master-slave heterogeneous graph according to an embodiment of the present invention; Figure 3 This is a structural diagram of an acceleration component according to an embodiment of the present invention; Figure 4 This is a diagram of a multi-level heterogeneous storage structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] Unless otherwise specifically stated, the relative arrangement of the parts and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to actual proportional relationships. The techniques, methods and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods and equipment should be considered as part of the authorization specification. In all examples shown and discussed here, any specific values ​​should be interpreted as being merely exemplary, not as limitations. Therefore, other examples of the exemplary embodiments may also include different values. It should be noted that similar numbers and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0022] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0023] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0024] The application principle of the present invention is described in detail below with reference to the accompanying drawings.

[0025] like Figure 1 As shown, the present invention provides a board with a built-in multi-level high-performance AI chip, including: an AI chip, and a boot module and a communication module connected to the AI ​​chip; The boot module is used to initialize the board; The communication module is used to communicate with external devices; The AI ​​chip adopts a master-slave heterogeneous structure, including a control core and several computing core groups. The control core is connected to each computing core group through a ring network bus; each computing core group includes a high-bandwidth storage controller and several computing cores connected to the high-bandwidth storage controller (i.e. Figure 2 Each computing core has a connected local cache and an acceleration core, which has mixed precision computing capabilities. Figure 2 .

[0026] In this solution, the AI ​​chip is designed with a master-slave heterogeneous architecture, with the control core serving as the master core, focusing on task scheduling and management, and the computing cores within the computing core group serving as slave cores, focusing on computation. This effectively improves efficiency at multiple levels, including memory access, computational efficiency, and parallel processing efficiency, thereby enhancing the overall energy efficiency of the system. The ring bus not only transmits information and data between the control core and computing core group on the board but also enables efficient sharing of computing resources and data storage through distributed storage management. In specific implementations, the high-bandwidth memory controller supports HBM2E / HBM2 SDRAM access, with configurable storage capacities ranging from 32GB to 64GB. The measured memory access bandwidth exceeds 1024GB / s, providing sufficient data capacity and fast memory access speeds for model applications with large parameter counts. The communication module is an Ethernet module supporting 1000 / 100 / 10Mbps rates, ensuring stable and efficient communication between devices. The boot module has a capacity of 16MB, operates at 1.8V, and supports a maximum read and write speed of 100MHz. It is responsible for initializing the hardware and preparing for the loading of the operating system and programs. The high bandwidth storage controller is Figure 4 High-bandwidth storage and control in.

[0027] In a specific embodiment of the present invention, the control core loads data from external storage and performs pre-processing and fast access based on its own cache structure; The control core writes data into the high-bandwidth memory corresponding to each computing core group through the ring network bus; in a specific application process, the control core can write model parameters and input data into the high-bandwidth memory of each computing core group through the ring network bus; The computing core reads data from the high-bandwidth memory connected to it and loads it into the cache structure inside the computing core. The acceleration core reads data from the cache structure connected to it and performs parallel calculations. The calculation result data is then written back to the high-bandwidth memory, and then uniformly recovered and summarized by the control core for entering the next stage of processing or output.

[0028] In the above solution, the data interaction process between the control core and each computing core group is described in detail to improve the computing efficiency and task execution efficiency of the entire board.

[0029] In a specific embodiment of the present invention, the cache structure of the control core consists of a primary data cache and a secondary data cache.

[0030] In the specific implementation process, the capacity of the first-level data cache and the second-level data cache are 32KB and 512KB respectively. The two-level cache hierarchy reduces the memory access waiting time through the data pre-fetch mechanism, improves the data hit rate, ensures the efficient scheduling of tasks by the control core, optimizes resource allocation, and further improves the execution efficiency of the computing core. Figure 4 L1 and L2 within the central core.

[0031] In a specific embodiment of the present invention, the control core adopts a 64-bit RISC structure based on an autonomous instruction set, is used to run an operating system and user programs, and is responsible for global resource management and task scheduling.

[0032] The above solution specifically defines the structure of the control core so that the control core is used to manage computing and storage resources and provide services such as messaging, files, debugging, and low-power management.

[0033] In a specific embodiment of the present invention, the acceleration core includes a mixed-precision multiplication-addition component, a mixed-precision addition component, and a mixed-precision conversion component arranged in parallel; the mixed-precision multiplication-addition component, the mixed-precision addition component, and the mixed-precision conversion component each include an independently working floating-point subcomponent and a fixed-point subcomponent; The floating-point subcomponent of the mixed-precision multiplication-addition component is used to perform a floating-point mixed multiplication-addition operation, and the calculation formula corresponding to the floating-point mixed multiplication-addition operation is: 16bit*16bit+32bit (half precision*half precision+single precision, the result is single precision); The fixed-point subcomponent of the mixed-precision multiply-add component is used to perform one integer multiply-add operation, the corresponding calculation formula of which is 16 bits * 16 bits + 32 bits (the result is a 32-bit signed integer), or to perform two integer multiply-add operations, the corresponding calculation formula of which is 8 bits * 8 bits + 16 bits (the result is two 16-bit signed integers); The floating-point subcomponent of the mixed-precision addition component is used to perform a single-precision floating-point addition operation, and the calculation formula corresponding to the single-precision floating-point addition operation is: 32bit+32bit (the result is single precision); The fixed-point subcomponent of the mixed-precision addition component is used to perform an integer addition operation, the corresponding calculation formula of the one integer addition operation is 32 bits + 32 bits (the result is a 32-bit signed integer), or to perform two integer addition operations, the corresponding calculation formula of the two integer addition operations is 16 bits + 16 bits (the result is two 16-bit signed integers); The floating point subcomponent of the mixed precision conversion component is used to perform a single-precision floating point to half-precision floating point conversion operation; The fixed-point subcomponent of the mixed-precision conversion component is used to perform a 32-bit signed integer to 16-bit signed integer conversion operation or two 16-bit signed integer to 8-bit signed integer conversion operations.

[0034] In the above solution, the mixed-precision multiplication and addition components and mixed-precision addition components with classic low-latency structure, as well as the mixed-precision conversion components, can provide ultra-high-performance mixed-precision floating-point and fixed-point computing capabilities.

[0035] In a specific embodiment of the present invention, the cache structure of each computing core is composed of a first-level data cache with a capacity of 256KB. Figure 4 L1 in.

[0036] In the above solution, the 256KB first-level data cache space allows a certain amount of data to be retained inside the core and reused multiple times, reducing the number of memory accesses and alleviating storage bandwidth pressure; the high-bandwidth characteristics of the memory reduce data loading delays, thereby accelerating model training and inference processes, meeting the needs of artificial intelligence applications for high-speed data transmission and improving overall energy efficiency.

[0037] In a specific embodiment of the present invention, each computing core is integrated with a communication engine for receiving memory access signals; The computing cores in each computing core group are arranged in an N*M array. Each computing core group integrates M memory access engines, one memory access engine corresponds to each column of N computing cores. The memory access engine realizes inter-core memory access requests and responses through the communication engine within the computing core.

[0038] In a specific embodiment of the present invention, the cache structure of each computing core adopts continuous addressing to improve memory access operations between cores and between cores and high-bandwidth memories.

[0039] In a specific embodiment of the present invention, the board further includes a maintenance module and a power supply module; The maintenance module is connected to the power supply module and the AI ​​chip respectively, and is used to control the power-on, reset, maintenance and power load monitoring of the board.

[0040] In the above solution, during specific application, the power supply module can provide a maximum current of 1000A and can control the output voltage with a maximum resolution of 0.625Mv, ensuring that all components obtain stable and reliable power supply.

[0041] In a specific embodiment of the present invention, a heat sink is further provided on the board; the heat sink is connected to the maintenance module, and the maintenance module controls the working state of the heat sink according to the real-time temperature of the board.

[0042] In the above solution, efficient heat dissipation of the board can be achieved. The radiator can be a fan.

[0043] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the protection content of the present invention.

[0044] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A board with a built-in multi-layered high-performance AI chip, characterized in that: include: An AI chip, and a boot module and a communication module connected to the AI ​​chip; The boot module is used to initialize the board; The communication module is used to communicate with external devices; The AI ​​chip adopts a master-slave heterogeneous structure, including a control core and several computing core groups. The control core is connected to each computing core group through a ring network bus; each computing core group includes a high-bandwidth storage controller and several computing cores connected to the high-bandwidth storage controller. Each computing core is equipped with a connected local cache and acceleration core, and the acceleration core has mixed-precision computing capabilities.

2. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: The control core loads data from external storage and performs pre-processing and fast access based on its own cache structure; The control core writes data into the high-bandwidth memory corresponding to each computing core group through the ring network bus; The computing core reads data from the high-bandwidth memory connected to it and loads it into the cache structure inside the computing core. The acceleration core reads data from the cache structure connected to it and performs parallel calculations. The calculation result data is then written back to the high-bandwidth memory, and then uniformly recovered and summarized by the control core for entering the next stage of processing or output.

3. The board with a built-in multi-layered high-performance AI chip according to claim 2, characterized in that: The cache structure of the control core consists of a primary data cache and a secondary data cache.

4. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: The control core adopts a 64-bit RISC structure based on an independent instruction set, is used to run the operating system and user programs, and is responsible for global resource management and task scheduling.

5. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: The acceleration core includes a mixed-precision multiplication and addition component, a mixed-precision addition component, and a mixed-precision conversion component arranged in parallel; the mixed-precision multiplication and addition component, the mixed-precision addition component, and the mixed-precision conversion component each include an independently working floating-point subcomponent and a fixed-point subcomponent; The floating-point subcomponent of the mixed-precision multiplication-addition component is used to perform a floating-point mixed multiplication-addition operation, and the calculation formula corresponding to the floating-point mixed multiplication-addition operation is: 16bit*16bit+32bit; The fixed-point subcomponent of the mixed-precision multiply-add component is used to perform one integer multiply-add operation, and the calculation formula corresponding to the one integer multiply-add operation is: 16bit*16bit+32bit, or to perform two integer multiply-add operations, and the calculation formula corresponding to the two integer multiply-add operations is 8bit*8bit+16bit; The floating-point subcomponent of the mixed-precision addition component is used to perform a single-precision floating-point addition operation, and the calculation formula corresponding to the single-precision floating-point addition operation is: 32bit+32bit; The fixed-point subcomponent of the mixed-precision addition component is used to perform an integer addition operation, where the calculation formula corresponding to the one integer addition operation is 32 bits + 32 bits, or to perform two integer addition operations, where the calculation formula corresponding to the two integer addition operations is 16 bits + 16 bits; The floating point subcomponent of the mixed precision conversion component is used to perform a single-precision floating point to half-precision floating point conversion operation; The fixed-point subcomponent of the mixed-precision conversion component is used to perform a 32-bit signed integer to 16-bit signed integer conversion operation or two 16-bit signed integer to 8-bit signed integer conversion operations.

6. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: The cache structure of each computing core consists of a first-level data cache with a capacity of 256KB.

7. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: Each computing core is integrated with a communication engine for receiving memory access signals; The computing cores in each computing core group are arranged in an N*M array. Each computing core group integrates M memory access engines, one memory access engine corresponds to each column of N computing cores. The memory access engine realizes inter-core memory access requests and responses through the communication engine within the computing core.

8. The board with a built-in multi-layered high-performance AI chip according to claim 7, characterized in that: The cache structure of each computing core adopts continuous addressing to improve memory access operations between cores and between cores and high-bandwidth memory.

9. The board with a built-in multi-layered high-performance AI chip according to claim 1, characterized in that: The board also includes a maintenance module and a power supply module; The maintenance module is connected to the power supply module and the AI ​​chip respectively, and is used to control the power-on, reset, maintenance and power load monitoring of the board.

10. The board with a built-in multi-layered high-performance AI chip according to claim 9, characterized in that: The board is also provided with a radiator; the radiator is connected to the maintenance module, and the maintenance module controls the working state of the radiator according to the real-time temperature of the board.

Images