Hybrid precision data parallel transmission method for GPNPU array address mapping

Abstract

The invention discloses a hybrid precision data parallel transmission method for GPNPU array address mapping, which belongs to the technical field of microelectronics, realizes rapid data interaction between on-chip storage and a computing array by introducing row and column addresses of the array, and is characterized in that data is arranged in rows of the array, data format conversion is pre-processed before storage, and the data format conversion is pre-processed before storage. Data transmission supports at most 6-dimensional arrays and can be dynamically configured as 1-6-dimensional array transmission so as to adapt to different calculation requirements, when a calculation array requests data, an access address is formed through a row address and address offset, rapid filling of the data is achieved, and external data access is written into on-chip storage after address searching. According to the invention, the row and column addresses of the array are used as the interaction addresses of the on-chip memory and the array, the data are arranged in the on-chip memory according to the address of the array, and the conversion of the data format is pre-processed in the data memory, so that the problem of rapid data transmission is greatly solved, and the throughput rate of the calculation array is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of microelectronics technology, and particularly relates to a GPNPU array address mapping mixed-precision data parallel transmission method. BACKGROUND

[0002] GPNPU (general-purpose neural network processor) is a new processor architecture integrating general-purpose computing capability and neural network acceleration capability, and aims to solve the efficiency bottleneck faced by traditional schemes in processing AI workloads.

[0003] GPNPU usually integrates a large number of computing cores to meet application computing power, and it is a relatively difficult problem to maximize the computing power. In traditional NPU and CPU, the mode for transferring large blocks of data usually adopts a DMA (direct memory access) mode. A general DMA design includes three parts of control, data input and data output. The control generally includes the address of source data, the address of destination data, the length of transmission data and the generation of some states. The data input generally reads data from the address of source data in the form of a bus and counts the input data. When the data of the length of transmission data is read out, the reading of data is stopped. The data output generally also adopts the bus to send the input data to the storage space of the destination address until the transmission is completed.

[0004] In traditional NPU and CPU, the mode for transferring large blocks of data usually adopts a DMA (direct memory access) mode. This mode usually transfers the data in a storage space of one address to another address space, provides high-speed data transmission between peripherals and memories or between memories. This design has high efficiency for 1-dimensional block data transmission, but for multi-dimensional data block transmission, it needs to be converted into multiple 1-dimensional data transmission. The host computer needs to be configured once for each 1-dimensional data block transmission, which increases the burden of the host computer and affects the transmission efficiency. SUMMARY

[0005] In view of the above problems, the GPNPU array address mapping mixed-precision data parallel transmission method is provided to overcome the defects of the prior art and effectively solve the problems in the current market.

[0006] The technical scheme adopted by the application is as follows: the GPNPU array address mapping mixed-precision data parallel transmission method is provided, the row and column addresses of the array are introduced to realize fast data interaction between on-chip storage and computing arrays, the data is arranged in rows of the array, and the data format conversion is pre-processed before storage.

[0007] Further, the data transmission supports up to 6-dimensional arrays and can be dynamically configured for 1 to 6-dimensional array transmission to adapt to different computing requirements.

[0008] Further, when computing array requests data, the access address is formed by the row address and address offset, realizing the rapid filling of data.

[0009] Further, the external data access is written into the on-chip storage after address lookup, and the address translation is provided by the software compiler.

[0010] Further, the data in the on-chip storage unit is arranged according to the rows of the array, and each unit row has an independent address, which is mapped to the physical address of the storage.

[0011] Further, the data of the unit is arranged according to the position of each core processing unit, and the number of bits received by each core processing unit is related to the data precision, and the data conversion is completed in the core processing unit.

[0012] Further, the method further comprises buffering the output data of the computing unit to avoid the computing unit from stalling due to waiting for data output.

[0013] Further, the method is suitable for data types of various precisions, including single-precision floating point, half-precision floating point, BF16, 32-bit integer, 8-bit integer and 4-bit integer.

[0014] Further, the method effectively utilizes the bandwidth of the data path by optimizing the efficiency of the data path to solve the format conversion problem of different data bit widths.

[0015] Further, the method further comprises dynamically configuring the dimension of data transmission to adapt to the needs of different computing tasks.

[0016] The beneficial effects achieved by the above structure are as follows: the row and column addresses of the array are used as the addresses for interaction between the on-chip storage and the array, the data in the on-chip storage is arranged according to the addresses of the array, the data format conversion is pre-processed in the data storage, which greatly solves the problem of rapid data transmission and improves the throughput of the computing array. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The overall structure of the circuit of the present application, module 1 is an external data transmission module, which is used for delivery with external storage, such as data bus or DDR, module 2 is the on-chip storage unit mentioned in the present application, module 3 is a computing array; Figure 2 Each unit data in the on-chip storage is arranged according to the rows of the array. Figure 3 The storage structure of the on-chip storage, each unit row has an independent address, which is mapped to the physical address of the storage. Figure 4The data of the unit is arranged according to the position of each core, and 8-bit data corresponds to each core processing unit; Figure 5 The data of the unit is arranged according to the position of each core, and 4-bit data corresponds to each core processing unit, and each core processing unit receives 8-bit data, and the actual conversion is completed in the core processing unit.

[0018] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. EMBODIMENT

[0021] As Figures 1-5 shown, the present application proposes a mixed-precision data parallel transmission method for GPNPU array address mapping. The on-chip storage unit of the present application is designed to be tightly coupled with the computing array to minimize the distance and delay of data transmission. The on-chip storage unit is organized according to the row and column addresses of the computing array, and each storage unit corresponds to a specific computing core, thereby realizing fast access and transmission of data.

[0022] The computing array adopts a mesh structure and is composed of multiple processing units (PEs). Each PE is equipped with local storage and computing resources and can independently perform computing tasks.

[0023] Introduction of row and column addresses: in the data transmission process, the introduction of the row and column addresses of the array realizes fast data interaction between the on-chip storage and the computing array. The data is arranged in the row of the array in the on-chip storage, so that the data can be quickly filled according to the needs of the computing array.

[0024] Address translation mechanism: Address translation consists of two parts. First, the software compiler provides address mapping to convert the access address of the computation array to the physical address stored on-chip. Second, the external data access is written to the on-chip storage after address lookup, ensuring that data can be accurately transmitted to the designated location.

[0025] Multi-dimensional data support: The invention supports up to 6-dimensional array transmission and can be dynamically configured for 1 to 6-dimensional array transmission. This flexibility enables the method to adapt to the needs of different computing tasks, especially when dealing with complex neural network models.

[0026] During data transmission, the invention can handle multiple precision data types, including single-precision floating point, half-precision floating point, BF16, 32-bit integer, 8-bit integer, and 4-bit integer. Data format conversion is pre-processed in on-chip storage to ensure that data can be quickly transmitted to the computing unit in the correct format.

[0027] Data in on-chip storage is arranged row by row according to the array, and each unit row has an independent address that maps to the physical address of the storage. This design enables data to be quickly filled into the computation array, avoiding the stagnation of computing units due to waiting for data.

[0028] To further improve data transmission efficiency, the invention implements a data caching mechanism in the computation array. The data output by the computing unit is cached to avoid the stagnation of the computing unit due to waiting for data output. This caching mechanism can effectively improve the throughput of the computation array.

[0029] The device for mixed-precision data transmission between the on-chip storage and the computation array of the GPNPU includes an external data transmission module, an on-chip storage unit, and a computation array. The data in the on-chip storage unit is arranged row by row according to the array, and the data format conversion is pre-processed before storage to support fast data interaction and transmission.

[0030] On-chip storage unit: tightly coupled with the computation array, organized according to the row and column addresses of the computation array. Each storage unit corresponds to a specific computing core to minimize the distance and delay of data transmission.

[0031] Computation array: adopts a mesh structure composed of multiple processing elements (PEs). Each PE is equipped with local storage and computing resources, enabling independent execution of computing tasks.

[0032] External data transmission module: responsible for data exchange with external storage (such as DDR), transmitting external data to on-chip storage, and outputting computing results to external storage. Embodiment

[0033] Efficient computation of neural network models: In processing complex neural network models, the present application can quickly transmit and process multi-dimensional data, significantly improving computational efficiency. For example, in processing large-scale image recognition tasks, the present application can quickly transmit and process high-dimensional feature data.

[0034] Dynamic configuration and flexibility: The present application supports dynamic configuration of data transmission dimensions, which can adapt to the needs of different computing tasks. This flexibility makes the present application widely applicable in various application scenarios.

[0035] It should be noted that, in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device.

[0036] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

[0037] The above describes the present application and its embodiments, which are not limited, and the embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the present application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the present application.

Claims

1. A method for mixed-precision parallel data transmission based on GPNPU array address mapping, characterized in that: By introducing row and column addresses of the array, fast data interaction between on-chip storage and computing arrays is achieved, where data is arranged in rows of the array and data format conversion is preprocessed before storage.

2. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 1, characterized in that: Data transmission supports up to 6-dimensional arrays and can be dynamically configured to transmit 1- to 6-dimensional arrays to adapt to different computing needs.

3. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 2, characterized in that: When calculating array request data, the access address is formed by row address and address offset to achieve fast data filling.

4. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 3, characterized in that: External data access is written to on-chip storage after address lookup, and the address translation is provided by the software compiler.

5. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 4, characterized in that: The data in the on-chip storage cells are arranged in rows of the array, and each cell row has an independent address that is mapped to the physical address of the storage.

6. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 5, characterized in that: The data in each unit is arranged according to the position of each core processing unit. The number of bits received by each core processing unit is related to the data precision, and the data conversion is completed within the core processing unit.

7. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 6, characterized in that: The method also includes caching the output data of the computing unit to prevent the computing unit from stalling while waiting for data output.

8. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 7, characterized in that: The method is applicable to data types with various precisions, including single-precision floating-point, half-precision floating-point, BF16, 32-bit integers, 8-bit integers, and 4-bit integers.

9. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 8, characterized in that: The method optimizes the efficiency of the data path and effectively utilizes the bandwidth of the data path to address the format conversion problem of different data bit widths.

10. The mixed-precision data parallel transmission method for GPNPU array address mapping according to claim 9, characterized in that: The method also includes dynamically configuring the dimensions of data transfer to adapt to the needs of different computing tasks.

Images