Chip performance optimization method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of chip optimization, in particular to a chip performance optimization method and device, electronic equipment and a storage medium, and the method comprises the steps: planning input data and weight data through employing a three-dimensional MAC array additionally arranged in a target chip, and obtaining a distribution scheme; according to the distribution scheme and the trapezoidal shift register array, carrying out time-sharing loading on the input data, and shifting the input data subjected to time-sharing loading to obtain an input data stream; and according to the input data stream and the weight data, performing convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence to obtain an optimized pipeline calculation result of the target chip. According to the invention, the computing architecture based on the three-dimensional MAC array is optimized, the proportion of read-write operation is reduced, and the overall computing efficiency of the chip is improved; by optimizing the data processing flow, it is ensured that the calculation process can be continuously carried out, the proportion of calculation duration is increased, and time delay caused by read-write operation is shortened.

Description

Background Art

[0002] With the rapid development of artificial intelligence, the bottlenecks of existing computers with von Neumann architecture systems are gradually emerging. At present, the hardware running neural networks is roughly divided into two categories: deep learning accelerator chips and neuromorphic chips. In addition, Tsinghua University has developed hybrid brain-like chips that support heterogeneous fusion networks for general artificial intelligence. At present, most accelerators use a storage-computing separation architecture, that is, the traditional von Neumann architecture.

[0003] This type of chip uses increased transistor density to improve energy efficiency. For example, the TPU has gone from 45nm in the first generation to 7nm in the fourth generation. On the other hand, it uses near-memory design and domain-specific architecture design (DSA) to reduce power consumption. Due to the use of external storage, data requires a lot of transfer work between the computing unit and the storage unit, and read and write operations. The access power consumption caused by moving data will cause computing delays due to reading and writing. On the other hand, when the number of computing units gradually increases, the neural network needs to be finely split. Although the von Neumann architecture accelerator chip also has a multi-core architecture, most of them are serially processed from an algorithm perspective. The "von Neumann" TPU accelerator based on the separation of storage and calculation uses a systolic array based on the im2col calculation method. The design of high-bandwidth storage is designed for the high parallelism of neural networks, with an access bandwidth of 614GB / s. However, its structure is simple and easy to use for high-speed computing, and the mapping method is simple, and resources are scheduled according to the time complexity of the task.

[0004] Due to the high degree of parallelism of deep neural networks, some deep learning accelerator chips are also beginning to adopt decentralized architectures. Graphcore's IPU distributed storage and computing integrated multi-core architecture. Tsinghua University's Tianji series chips are brain-like chips with distributed multi-core architectures, such as Figure 2 As shown: 160 cores of Tianji X. The above distributed parallel multi-core architecture can increase the concurrent access bandwidth within a single chip to up to 6000GB / s. The concurrent transmission bandwidth between chip cores can reach 512GB / s.

[0005] However, most of the current many-core architectures use one-dimensional MAC arrays or two-dimensional MAC arrays, such as the first generation of Tianji (16 in one dimension), the second generation (16 in one dimension), Tianji X (4×32 in two dimensions), SpiNNaker (16×4 in two dimensions), and NVIDIA Simba (8×8 in two dimensions) in 2019. Therefore, the pipeline systems inside their chips are all pipeline systems based on two-dimensional MAC arrays.

[0006] In a two-dimensional MAC array, as the number of MACs increases, the efficiency of matching the size of the neural network gradually decreases, which further restricts the computing efficiency of the multi-core architecture chip. Figure 3 As shown in Figure 1, when the width of the graph is 7, the efficiency of the MAC array in terms of parallelism is 87.5%. When the MAC array is 256×256, such as the TPU, its efficiency is less than 22%. Figure 4 To change this state, when the number of MACs increases, the two-dimensional MAC array can be transformed into a three-dimensional MAC array. Figure 5 Take the three-dimensional MAC array structure as an example. The dimensions of the MAC array correspond to the dimensions of the neural network, the width direction of the image, the input depth direction of the image, and the output direction of the image. The number of MACs in the image height direction is not considered for the time being. The parallelism of the three-dimensional MACs is width W, input depth Cin, and output depth Cout. Figure 5 As shown: In the figure, there are 4 MACs in the width direction, 8 MACs in the output depth direction, and 4 MACs in the input degree direction.

[0007] Although the three-dimensional MAC array for the computing unit of the neural network has been developed, the corresponding pipeline system has not yet been designed. The present invention aims to solve the pipeline system equipped with the three-dimensional MAC array, reduce the proportion of read and write delays in the total time, and further enhance the computing efficiency of the chip.

[0008] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the invention

[0009] In order to solve the above technical problems, the present invention provides a chip performance optimization method, device, electronic device and storage medium.

[0010] In a first aspect, the present invention provides a chip performance optimization method, the technical solution of the method is as follows:

[0011] Using the three-dimensional MAC array added in the target chip, the input data and weight data are planned to obtain the allocation plan;

[0012] According to the allocation scheme and the trapezoidal shift register array, the input data is loaded in time-sharing mode, and the input data after the time-sharing loading is shifted to obtain an input data stream;

[0013] According to the input data stream and the weight data, convolution calculation is performed on each multiplier-adder unit in the three-dimensional MAC array in turn to obtain an optimized pipeline calculation result of the target chip.

[0014] The beneficial effects of a chip performance optimization method of the present invention are as follows:

[0015] The method of the present invention optimizes the computing architecture based on the three-dimensional MAC array, reduces the proportion of read and write operations, and improves the overall computing efficiency of the chip; by optimizing the data processing flow, it ensures that the computing process can be carried out continuously, increases the proportion of computing time, and reduces the time delay caused by read and write operations.

[0016] Based on the above solution, a chip performance optimization method of the present invention can also be improved as follows.

[0017] In an optional manner, the steps of planning the input data and weight data using the three-dimensional MAC array added in the target chip to obtain the allocation scheme include:

[0018] According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. The allocation scheme is obtained by determining a first mapping relationship between the input data and the three-dimensional MAC array, and a second mapping relationship between the weight data and the three-dimensional MAC array.

[0019] In the above optional manner, by dividing the input data according to the size of the three-dimensional MAC array and the size of the input data, and distributing the weight data to different computing units, it is possible to ensure that the data is reasonably distributed among the computing units. The allocation scheme can maximize the use of the computing power of the three-dimensional MAC array, reduce the transmission and waiting time of data during the calculation process, and thus improve the computing efficiency. At the same time, by determining the first mapping relationship between the input data and the three-dimensional MAC array, and the second mapping relationship between the weight data and the three-dimensional MAC array, the correctness of the data and the accuracy of the calculation can be ensured.

[0020] In an optional manner, the step of time-sharing loading the input data according to the allocation scheme and the ladder shift register array includes:

[0021] According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into a plurality of parts, and each part is sequentially loaded into a different trapezoidal shift register and then reorganized to obtain the input data after time-sharing loading.

[0022] In the above optional manner, by dividing the input data according to the allocation scheme and the length and shift speed of the trapezoidal shift register, and loading each part into different trapezoidal shift registers in turn for reorganization, time-sharing loading of the input data can be achieved. The above time-sharing loading method can ensure the continuity and stability of the data during the calculation process, avoid data loss and repeated calculation. At the same time, through the design of the trapezoidal shift register, the data transmission path and speed can be further optimized, and the efficiency of data processing can be improved.

[0023] In an optional manner, the step of performing convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and the weight data to obtain the optimized pipeline calculation result of the target chip includes:

[0024] According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain a product result;

[0025] Accumulating the product results in the input depth direction of the three-dimensional MAC array to obtain an output result of each multiplier-adder unit;

[0026] The output results of each multiplier-adder unit are integrated to obtain the optimized pipeline calculation result of the target chip.

[0027] In the above optional method, by multiplying each element in the input data stream with the corresponding weight data in sequence according to the mapping relationship between the input data stream and the weight data, and accumulating in the input depth direction of the three-dimensional MAC array, the output result of each multiplier unit can be obtained. The above calculation method can make full use of the parallel computing capability of the three-dimensional MAC array to realize the simultaneous calculation of multiple computing units, thereby improving the calculation speed. At the same time, by integrating the output results of each multiplier unit, the optimized pipeline calculation results of the target chip can be obtained to ensure the accuracy and completeness of the calculation results. The above optimized pipeline calculation method can significantly reduce the proportion of read and write operations and improve the overall computing efficiency of the chip.

[0028] In a second aspect, the present invention provides a chip performance optimization device, the technical solution of the device is as follows:

[0029] The chip performance optimization device comprises: an allocation module, a loading module and a calculation module;

[0030] The allocation module is used to: plan the input data and weight data using the three-dimensional MAC array added in the target chip to obtain an allocation plan;

[0031] The loading module is used to: load the input data in time-sharing mode according to the allocation scheme and the trapezoidal shift register array, and shift the input data after the time-sharing loading to obtain an input data stream;

[0032] The calculation module is used to perform convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and the weight data, so as to obtain the optimized target pipeline calculation result of the target chip.

[0033] The beneficial effects of a chip performance optimization device of the present invention are as follows:

[0034] The device of the present invention optimizes the computing architecture based on the three-dimensional MAC array, reduces the proportion of read and write operations, and improves the overall computing efficiency of the chip; by optimizing the data processing flow, it ensures that the computing process can be carried out continuously, increases the proportion of computing time, and reduces the time delay caused by read and write operations.

[0035] Based on the above solution, the chip performance optimization device of the present invention can also be improved as follows.

[0036] In an optional manner, the allocation module is specifically used to:

[0037] According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. The allocation scheme is obtained by determining a first mapping relationship between the input data and the three-dimensional MAC array, and a second mapping relationship between the weight data and the three-dimensional MAC array.

[0038] In the above optional manner, by dividing the input data according to the size of the three-dimensional MAC array and the size of the input data, and distributing the weight data to different computing units, it is possible to ensure that the data is reasonably distributed among the computing units. The allocation scheme can maximize the use of the computing power of the three-dimensional MAC array, reduce the transmission and waiting time of data during the calculation process, and thus improve the computing efficiency. At the same time, by determining the first mapping relationship between the input data and the three-dimensional MAC array, and the second mapping relationship between the weight data and the three-dimensional MAC array, the correctness of the data and the accuracy of the calculation can be ensured.

[0039] In an optional manner, the loading module is specifically used for:

[0040] According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into a plurality of parts, and each part is sequentially loaded into a different trapezoidal shift register and then reorganized to obtain the input data after time-sharing loading.

[0041] In the above optional manner, by dividing the input data according to the allocation scheme and the length and shift speed of the trapezoidal shift register, and loading each part into different trapezoidal shift registers in turn for reorganization, time-sharing loading of the input data can be achieved. The above time-sharing loading method can ensure the continuity and stability of the data during the calculation process, avoid data loss and repeated calculation. At the same time, through the design of the trapezoidal shift register, the data transmission path and speed can be further optimized, and the efficiency of data processing can be improved.

[0042] In an optional manner, the calculation module is specifically used to:

[0043] According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain a product result;

[0044] Accumulating the product results in the input depth direction of the three-dimensional MAC array to obtain an output result of each multiplier-adder unit;

[0045] The output results of each multiplier-adder unit are integrated to obtain the optimized target pipeline calculation result of the target chip.

[0046] In the above optional method, by multiplying each element in the input data stream with the corresponding weight data in sequence according to the mapping relationship between the input data stream and the weight data, and accumulating in the input depth direction of the three-dimensional MAC array, the output result of each multiplier unit can be obtained. The above calculation method can make full use of the parallel computing capability of the three-dimensional MAC array to realize the simultaneous calculation of multiple computing units, thereby improving the calculation speed. At the same time, by integrating the output results of each multiplier unit, the optimized pipeline calculation results of the target chip can be obtained to ensure the accuracy and completeness of the calculation results. The above optimized pipeline calculation method can significantly reduce the proportion of read and write operations and improve the overall computing efficiency of the chip.

[0047] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0048] The invention comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the chip performance optimization method of the present invention are implemented.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0050] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the chip performance optimization method of the present invention.

[0051] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0053] Figure 1 A schematic diagram of a process flow of an embodiment of a chip performance optimization method of the present invention;

[0054] Figure 2 The Tianjic X1 functional core (a) and the physical layout of the entire chip (b);

[0055] Figure 3 It is a schematic diagram of image parallelism;

[0056] Figure 4 It is a schematic diagram of single-dimension size matching efficiency;

[0057] Figure 5 Schematic diagram of the 4×4×8 structure design for the three-dimensional MAC array;

[0058] Figure 6 Design a schematic diagram for a chip performance optimization device;

[0059] Figure 7 A timing diagram of a chip performance optimization device;

[0060] Figure 8 A schematic diagram of the structure of an embodiment of a chip performance optimization device of the present invention;

[0061] Fig. 9 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0062] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0063] Figure 1 FIG. 1 is a flow chart showing an embodiment of a chip performance optimization method provided by the present invention, wherein the method is executed by an executor. Figure 1 As shown, the following steps are included:

[0064] S1. Using the three-dimensional MAC array added in the target chip, the input data and weight data are planned to obtain a distribution plan. In S1:

[0065] 1) The 3D MAC array is a collection of multiplication and addition units arranged in a 3D space, each unit can perform multiplication and addition operations independently. The above structure can significantly improve the parallelism of operations and speed up processing. Through the 3D layout, the 3D MAC array can more efficiently utilize the chip space, achieve higher computing density and faster computing speed, and is particularly suitable for applications such as deep learning and image processing that require a large number of multiplication and addition operations.

[0066] 2) Input data is the raw data to be processed, which can be any form of data such as images, audio, text, etc. After preprocessing, it is input into the chip for calculation. As the basis of calculation, the input data is transformed, analyzed and calculated by the processing unit in the chip to extract useful information or generate output results.

[0067] 3) Weight data is a parameter used to adjust the impact of input data in machine learning models such as neural networks. Weight data is determined through the training process and is used to weight input data during the inference phase. Weight data determines the importance of input data in the calculation process. By adjusting the weight, different calculation functions and output results can be achieved.

[0068] 4) The allocation scheme is a strategy to determine how to allocate input data and weight data to each computing unit in the three-dimensional MAC array. Through a reasonable allocation scheme, the utilization of computing resources can be optimized and the computing efficiency and accuracy can be improved.

[0069] Specifically, the controller uses the three-dimensional MAC array added in the target chip to plan the input data and weight data to obtain an allocation plan.

[0070] S2. According to the allocation scheme and the ladder shift register array, the input data is loaded in time-sharing mode, and the input data after time-sharing loading is shifted to obtain an input data stream. In S2:

[0071] 1) The trapezoidal shift register array is a special register structure used to transfer and reorganize input data in a three-dimensional MAC array. The trapezoidal shift register array uses a trapezoidal layout, which enables data to be moved and reorganized in multiple directions. The trapezoidal shift register array can flexibly distribute input data to different computing units while supporting parallel processing and pipelining of data.

[0072] 2) Time-sharing loading is a method of loading data into the computing unit in chronological order. Time-sharing loading can ensure that data is processed in the computing unit in order, avoiding data conflicts and computing delays.

[0073] 3) Shifting refers to the process of moving data between registers or computing units according to certain rules. Shifting operations can support data reorganization and parallel processing, improving computing efficiency and flexibility.

[0074] 4) Input data stream is a continuous data stream that is input into the three-dimensional MAC array in a certain order and format after time-sharing loading and shifting operations. The input data stream is the basis of the calculation process and determines how the calculation unit processes data and generates output results.

[0075] Specifically, the controller further loads the input data in time-sharing mode according to the allocation scheme and the trapezoidal shift register array, and shifts the input data after the time-sharing loading to obtain an input data stream.

[0076] S3. According to the input data stream and weight data, convolution calculation is performed on each multiplier-adder unit in the three-dimensional MAC array in turn to obtain the optimized pipeline calculation result of the target chip. In S3:

[0077] 1) The multiplier-adder unit is the basic computing unit in the three-dimensional MAC array, responsible for performing multiplication and addition operations. The multiplier-adder unit realizes data transformation and calculation by performing multiplication and addition operations, and is one of the most important computing resources in the chip.

[0078] 2) The pipeline calculation result is the final output result obtained after multiple multiplier-adder units in the three-dimensional MAC array calculate in sequence. The pipeline calculation result represents the chip's processing result of the input data and can be used for subsequent analysis, decision-making or generating other forms of output.

[0079] Specifically, the controller further performs convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and the weight data, and obtains the optimized pipeline calculation result of the target chip.

[0080] The technical solution of this embodiment optimizes the computing architecture based on the three-dimensional MAC array, reduces the proportion of read and write operations, and improves the overall computing efficiency of the chip; by optimizing the data processing flow, it ensures that the computing process can be carried out continuously, increases the proportion of computing time, and reduces the time delay caused by read and write operations.

[0081] In an optional manner, the steps of planning the input data and weight data using the three-dimensional MAC array added in the target chip to obtain the allocation scheme include:

[0082] According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. By determining the first mapping relationship between the input data and the three-dimensional MAC array, and the second mapping relationship between the weight data and the three-dimensional MAC array, an allocation scheme is obtained.

[0083] It should be noted that by dividing the input data into blocks that match the size of the MAC array, it can be ensured that each MAC can process the same amount of data at the same time, thereby improving the efficiency of parallel computing.

[0084] The allocation and mapping of weight data is a key step in realizing neural network calculations. Through correct allocation and mapping, it can ensure that each MAC can perform correct calculations based on the input data and weight data. At the same time, through the correct mapping relationship, it can ensure that each MAC processes data efficiently and outputs results. The generation of the allocation plan is to convert the mapping relationship into a specific calculation task allocation. Through a reasonable allocation plan, the calculation efficiency of the three-dimensional MAC array can be maximized.

[0085] In this embodiment, first, the specific size of the three-dimensional MAC array is determined, including the number of MACs in the width direction, the input depth direction, and the output depth direction. Then, according to the overall size of the input data, the input data is divided into multiple data blocks, and it is ensured that the multiple data blocks can be matched with a computing unit in the three-dimensional MAC array respectively. The above matching relationship not only helps to process data efficiently, but also maximizes the computing potential of the three-dimensional MAC array.

[0086] After the input data is divided into multiple blocks, the weight data is distributed to different computing units in the three-dimensional MAC array. The above distribution process needs to consider the size of the three-dimensional MAC array and the characteristics of the weight data. Through weight distribution, it can be ensured that each computing unit can obtain the required weight data, so as to perform accurate convolution calculations.

[0087] In order to ensure that the input data can be correctly and efficiently transmitted to each computing unit in the three-dimensional MAC array, it is necessary to determine the first mapping relationship between the input data and the three-dimensional MAC array. The first mapping relationship is formulated based on the multiple block divisions of the input data and the size and structure of the three-dimensional MAC array. Through the first mapping relationship, the transmission correspondence between the data block and the computing unit can be clearly known, thereby achieving accurate allocation and efficient processing of data.

[0088] Similar to the input data, the weight data also needs to establish a clear mapping relationship with the three-dimensional MAC array, that is, the second mapping relationship, which is formulated based on the allocation strategy of the weight data and the size and structure of the three-dimensional MAC array. Through the second mapping relationship, it can be ensured that each computing unit can correctly receive the required weight data, so as to perform accurate calculations.

[0089] After determining the first mapping relationship between the input data and the three-dimensional MAC array and the second mapping relationship between the weight data and the three-dimensional MAC array, a complete allocation scheme can be obtained. The allocation scheme describes in detail how to efficiently allocate the input data and the weight data to each computing unit in the three-dimensional MAC array to achieve efficient and accurate calculation.

[0090] Specifically, the controller further utilizes the three-dimensional MAC array added in the target chip to plan the input data and weight data to obtain an allocation plan.

[0091] In an optional manner, the step of time-sharing loading of input data according to the allocation scheme and the ladder shift register array includes:

[0092] According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into multiple parts, and each part is sequentially loaded into different trapezoidal shift registers and then reorganized to obtain the input data after time-sharing loading.

[0093] It should be noted that, during the implementation process, it is also necessary to consider how to design a pipeline system based on a three-dimensional MAC array to support the allocation scheme, including but not limited to: designing the shape of the shift register, determining the length and number of the shift register, and formulating the timing of data loading and calculation, so as to ensure that the data can be efficiently transmitted to each computing unit according to the predetermined mapping relationship, thereby achieving efficient computing performance.

[0094] In the process of time-sharing loading, first, according to the mapping relationship between the input data stream and the weight data, the pairing of each input data element and the corresponding weight data is determined. Figure 6As shown, the input data stream is obtained from the trapezoidal shift register, and each clock cycle provides multiple numbers (for example, 4×4=16 numbers simultaneously shifted out from Reg0, Reg1, Reg2, and Reg3), and the multiple numbers are propagated horizontally along the MAC to multiple MACs arranged horizontally. Then, for the multiplier-adder unit in each MAC, the received input data element is multiplied with the corresponding weight data. Since the weight data can read the weight number (for example, 32 weight numbers) per clock, the weight number is propagated upward along the height direction to multiple MACs (for example, 4 are propagated in the input depth direction of the three-dimensional MAC), so each multiplier-adder unit will perform a multiplication operation and reorganize in one clock cycle, and finally obtain the input data after time-sharing loading.

[0095] In addition, the pipeline mechanism of the three-dimensional MAC is consistent with the access bandwidth within a single core, and the weight data is read once and calculated at the same time, thus achieving seamless connection. Specifically, it ensures that the input data is ready every time the weight data is read. The use of the trapezoidal shift register array allows the input data to flow into the three-dimensional MAC array in an orderly manner and perform convolution calculations synchronously with the weight data. Therefore, each multiplier-adder unit can perform calculations while reading the weight data, thus achieving seamless connection of the pipeline mechanism.

[0096] In this embodiment, compared with the two-dimensional MAC shift register shifting out 1B of data at each moment, the three-dimensional MAC shift register can shift out multiple 1B of data at a time. Figure 6 As shown, the trapezoidal shift register uses 4 shift registers. The length of register Reg0 is 44B, the length of register Reg1 is 40B, the length of register Reg2 is 36B, and the length of register Reg3 is 32B. The characteristics of the above-mentioned trapezoidal shift register are: the number of data shifted out by the shift register at one time is the same as the number of three-dimensional MAC shift registers in the input depth direction. The number of heights of the trapezoid (the number of shift registers) is consistent with the number of three-dimensional MAC shift registers in terms of image parallelism. The length of the shortest side of the trapezoid controls the read-calculation ratio, and the reading time of the read-calculation ratio is less than the calculation time. The length of the longest side of the trapezoid is equal to the length of the shortest side plus the product of the number of shift registers and the number of data shifted out at one time.

[0097] based on Figure 6The three-dimensional MAC array is loaded with IA data in 4 clocks. After loading Reg0, Reg1 is loaded in the second clock, and Reg0 shifts 4 data to the left. Reg2 is loaded in the third clock, and Reg0 and Reg1 shift 4 data to the left at the same time. Reg3 is loaded in the fourth clock, and Reg0, Reg1, and Reg2 shift 4 data to the left at the same time. Starting from the fifth clock, the four shift registers simultaneously output 4 data to the MAC array. A total of 16 data.

[0098] The input data of the image obtains 16 numbers from the shift register. The above 16 numbers are propagated horizontally along the MAC to the 8 horizontal MACs (multiplier adder arrays). The weight reads 32B per clock, and the above 32 numbers are propagated upward along the height direction to the 4 MACs. The calculation of the input depth direction in the convolution calculation includes a multi-stage adder cascade. When the input depth direction and the height and width directions of the convolution window are added, the output is performed to obtain the input data after time-sharing loading. From the above structure, a 4×8 multiplier adder array is output, where each data is int32, and a total of 128B is output.

[0099] Specifically, according to the allocation scheme, the length of the trapezoidal shift register and the shift speed, the input data is divided into a plurality of parts, and each part is sequentially loaded into a different trapezoidal shift register and then reorganized to obtain the input data after time-sharing loading.

[0100] In an optional manner, the step of performing convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and the weight data to obtain the optimized pipeline calculation result of the target chip includes:

[0101] According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain the product result;

[0102] Accumulate the product results in the input depth direction of the three-dimensional MAC array to obtain the output result of each multiplier-adder unit;

[0103] The output results of each multiplier-adder unit are integrated to obtain the optimized pipeline calculation results of the target chip.

[0104] It should be noted that, according to the mapping relationship, the element currently to be processed is taken out from the input data stream, and the corresponding weight element is taken out from the weight data. In each multiplier-adder unit of the three-dimensional MAC array, a multiplication operation is performed. The above multiplication operation can be implemented by the multiplier in the hardware circuit. Each multiplier-adder unit processes multiple multiplication operations at the same time (depending on the parallelism of the array). The result of the multiplication operation (product result) is temporarily stored in the internal register of each multiplier-adder unit, ready for accumulation operation.

[0105] In the input depth direction of the three-dimensional MAC array, the product results in each multiplier-adder unit are accumulated. The above accumulation can be implemented by an accumulator in the hardware circuit, which can traverse all relevant product results and add the result results. The result of the accumulation operation represents the output result of each multiplier-adder unit, that is, a partial sum of the convolution operation.

[0106] When all the multiplier-adder units have completed their respective accumulation operations, the output results of each multiplier-adder unit need to be integrated. The integration operation can be achieved by setting up additional adders or accumulators inside the chip, or by transmitting partial sums to an external processor through a data bus for further processing. Finally, the integrated result is the optimized pipeline calculation result of the target chip. The above results can be stored in the memory inside the chip, or transmitted to an external device through a data interface for subsequent use.

[0107] In addition, the steps of performing convolution calculations on each multiplier-adder unit in the three-dimensional MAC array in turn to obtain the optimized pipeline calculation results of the target chip can be performed in the form of a pipeline. That is, while the first input element and weight element are multiplied, the subsequent input elements and weight elements are also being prepared and mapped. As the pipeline progresses, each multiplier-adder unit will process different input elements and weight elements in turn and generate corresponding output results. The above output results will be gradually integrated until the entire convolution operation is completed.

[0108] In this embodiment, during the pipeline calculation, each operation includes 8 clock cycles. In the above 8 clock cycles, 4 clock cycles are used to read data from the input image IA, thereby achieving a duty cycle of 1:1 between the time of reading data and the time of performing calculation. The above ratio is determined by the length of the first shift register, such as Figure 7 The workflow shown is as follows: First, read the data IA, the time required is marked as (rd); then read the bias, the time required is marked as (B); then enter the latching phase, the time required is marked as (M); then, read the weight W and overlap with the calculation operation being performed at the same time. Since the end of the calculation will start the next data IA reading at the same time, the calculation time will be greater than rd. When the calculation is completed, the time to write back the result is recorded as (Wr).

[0109] In the three-dimensional MAC array structure adopted, when the structure is 4×2×16, the corresponding shift register lengths are 32, 34, 36, and 38 units respectively. Two data are shifted in each clock cycle, and the calculation is continued for 16 clock cycles, while only 4 clock cycles are used to read the data. At this time, the duty cycle of reading data and calculation becomes 1:3. After several of the above calculation cycles, a 4×16 array result can be output, and 256 bytes (B) of data are output in 8 clock cycles, which is the optimized pipeline calculation result of the target chip. In the optimized pipeline calculation, each calculation contains 8 clock cycles, of which 4 clock cycles are used to read the input data, and the other 4 clock cycles are used to calculate and output the results, achieving a balance between reading and writing and calculation time.

[0110] During the pipeline calculation, 8 clocks are calculated each time. Among the above 8 clocks, 4 clocks are used to read the data of the input image IA. Therefore, the duty ratio of reading data and calculating is 1:1. The ratio of reading and calculating is determined by the length of the first shift register, such as Figure 7 As shown, first the data IA is read in, the duration is recorded as (rd), then the bias is read in, the duration is recorded as (B), then the latching phase is entered, the duration is recorded as (M), then the weight W is read in and the calculation is performed at the same time, the calculation time is greater than rd, because the data IA is read in at the end of the calculation. When the calculation is completed, the write-back time is recorded as (Wr).

[0111] For the above pipeline system structure, when the three-dimensional MAC structure is 4×2×16, the corresponding shift register lengths are 32, 34, 36, and 38, respectively. Each clock shifts 2 data, calculates 16 clocks, and reads 4 clocks. The reading and calculation duty ratio is 1:3. After several cycles of calculation and output, a 4×16 array is output, and 256B is output in 8 clocks, and the pipeline calculation result after the target chip is optimized is obtained.

[0112] Specifically, the controller further multiplies the elements in the input data stream with the corresponding weight data according to the mapping relationship between the input data stream and the weight data to obtain the product result, and then accumulates the product result in the input depth direction of the three-dimensional MAC array to obtain the output result of each multiplier unit, and integrates the above output results to obtain the optimized pipeline calculation result of the target chip.

[0113] Figure 8 FIG. 2 is a schematic diagram showing a structure of an embodiment of a chip performance optimization device 200 provided by the present invention. Figure 8 As shown, the device 200 includes: an allocation module 210 , a loading module 220 and a calculation module 230 .

[0114] The allocation module 210 is used to: plan the input data and weight data using the three-dimensional MAC array added in the target chip to obtain an allocation plan;

[0115] The loading module 220 is used to: load the input data in time-sharing mode according to the allocation scheme and the ladder shift register array, and shift the input data after the time-sharing mode loading to obtain an input data stream;

[0116] The calculation module 230 is used to perform convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and weight data, so as to obtain the optimized target pipeline calculation result of the target chip.

[0117] In an optional manner, the allocation module 210 is specifically configured to:

[0118] According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. By determining the first mapping relationship between the input data and the three-dimensional MAC array, and the second mapping relationship between the weight data and the three-dimensional MAC array, an allocation scheme is obtained.

[0119] In an optional manner, the loading module 220 is specifically used for:

[0120] According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into multiple parts, and each part is sequentially loaded into different trapezoidal shift registers and then reorganized to obtain the input data after time-sharing loading.

[0121] In an optional manner, the calculation module 230 is specifically used for:

[0122] According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain the product result;

[0123] Accumulate the product results in the input depth direction of the three-dimensional MAC array to obtain the output result of each multiplier-adder unit;

[0124] The output results of each multiplier-adder unit are integrated to obtain the optimized target pipeline calculation result of the target chip.

[0125] The technical solution of this embodiment optimizes the computing architecture based on the three-dimensional MAC array, reduces the proportion of read and write operations, and improves the overall computing efficiency of the chip; by optimizing the data processing flow, it ensures that the computing process can be carried out continuously, increases the proportion of computing time, and reduces the time delay caused by read and write operations.

[0126] The above-mentioned parameters and steps for each module to implement corresponding functions in the chip performance optimization device 200 of this embodiment can refer to the parameters and steps in the embodiment of the chip performance optimization method above, and will not be repeated here.

[0127] like Fig. 9 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above chip performance optimization methods, specifically:

[0128] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements any chip performance optimization method provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 300 may also include other components for implementing device functions, which will not be described in detail here.

[0129] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above chip performance optimization methods.

[0130] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0131] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above chip performance optimization methods.

[0132] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0133] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0134] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0135] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A chip performance optimization method, characterized in that: include: Using the three-dimensional MAC array added in the target chip, the input data and weight data are planned to obtain the allocation plan; According to the allocation scheme and the trapezoidal shift register array, the input data is loaded in time-sharing mode, and the input data after the time-sharing loading is shifted to obtain an input data stream; According to the input data stream and the weight data, convolution calculation is performed on each multiplier-adder unit in the three-dimensional MAC array in turn to obtain an optimized pipeline calculation result of the target chip.

2. The chip performance optimization method according to claim 1, characterized in that: The steps of planning the input data and weight data by using the three-dimensional MAC array added in the target chip to obtain the allocation scheme include: According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. The allocation scheme is obtained by determining a first mapping relationship between the input data and the three-dimensional MAC array, and a second mapping relationship between the weight data and the three-dimensional MAC array.

3. The chip performance optimization method according to claim 1, characterized in that: The step of time-sharing loading of the input data according to the allocation scheme and the ladder shift register array comprises: According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into a plurality of parts, and each part is sequentially loaded into a different trapezoidal shift register and then reorganized to obtain the input data after time-sharing loading.

4. The chip performance optimization method according to claim 1, characterized in that: The step of sequentially performing convolution calculation on each multiplier-adder unit in the three-dimensional MAC array according to the input data stream and the weight data to obtain the optimized pipeline calculation result of the target chip comprises: According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain a product result; Accumulating the product results in the input depth direction of the three-dimensional MAC array to obtain an output result of each multiplier-adder unit; The output results of each multiplier-adder unit are integrated to obtain the optimized pipeline calculation result of the target chip.

5. A chip performance optimization system, characterized in that: include: distribution module, loading module and calculation module; The allocation module is used to: plan the input data and weight data using the three-dimensional MAC array added in the target chip to obtain an allocation plan; The loading module is used to: load the input data in time-sharing mode according to the allocation scheme and the trapezoidal shift register array, and shift the input data after the time-sharing loading to obtain an input data stream; The calculation module is used to perform convolution calculation on each multiplier-adder unit in the three-dimensional MAC array in sequence according to the input data stream and the weight data, so as to obtain the optimized target pipeline calculation result of the target chip.

6. The chip performance optimization device according to claim 5, characterized in that: The allocation module is specifically used for: According to the size of the three-dimensional MAC array and the size of the input data, the input data is divided into multiple blocks, the size of each block matches a computing unit of the three-dimensional MAC array, and the weight data is allocated to different computing units. The allocation scheme is obtained by determining a first mapping relationship between the input data and the three-dimensional MAC array, and a second mapping relationship between the weight data and the three-dimensional MAC array.

7. The chip performance optimization device according to claim 5, characterized in that: The loading module is specifically used for: According to the allocation scheme, the length and shift speed of the trapezoidal shift register, the input data is divided into a plurality of parts, and each part is sequentially loaded into a different trapezoidal shift register and then reorganized to obtain the input data after time-sharing loading.

8. The chip performance optimization device according to claim 5, characterized in that: The calculation module is specifically used for: According to the mapping relationship between the input data stream and the weight data, each element in the input data stream is multiplied with the corresponding weight data in sequence to obtain a product result; Accumulating the product results in the input depth direction of the three-dimensional MAC array to obtain an output result of each multiplier-adder unit; The output results of each multiplier-adder unit are integrated to obtain the optimized target pipeline calculation result of the target chip.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the chip performance optimization method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements the chip performance optimization method according to any one of claims 1 to 4.

Images