Chip performance optimization method, system and device and storage medium

Abstract

The invention relates to the technical field of chip optimization, in particular to a chip performance optimization method, system and device and a storage medium, and the method comprises the steps: determining an MAC structure size according to the number of MAC units corresponding to a target demand of a to-be-optimized chip and the storage capacity of each MAC unit; according to the size of the MAC structure, width segmentation and depth segmentation are carried out on a benchmark test network, a plurality of data blocks which are matched with the size of the MAC structure and comprise width segmentation results and depth segmentation results are obtained, and according to the overall performance of the plurality of data blocks, a plurality of data blocks are obtained; and determining an optimal configuration point for reducing the number of data movement times and minimizing the number of memory read-write times, and performing performance optimization on the to-be-optimized chip according to the optimal configuration point. According to the invention, by accurately matching the size and the storage capacity of the MAC structure, the calculation efficiency of the storage and calculation integrated chip is improved, the calculation power per unit area is optimized, and the manufacturing cost is saved.

Description

Background Art

[0002] With the rapid development of artificial intelligence, the bottlenecks of existing computers with von Neumann architecture systems are gradually emerging. At present, the hardware running neural networks is roughly divided into two categories: deep learning accelerator chips and neuromorphic chips. In addition, Tsinghua University has developed hybrid brain-like chips that support heterogeneous fusion networks for general artificial intelligence. At present, most accelerators use a storage-computing separation architecture, that is, the traditional von Neumann architecture.

[0003] This type of chip uses increased transistor density to improve energy efficiency, such as the TPU from the first generation of 45nm to the fourth generation of 7nm, and on the other hand, uses near-memory design and domain-specific architecture design (DSA) to reduce power consumption. Due to the use of external storage, data requires a lot of movement between the computing unit and the storage unit, and read and write operations. The access power consumption caused by moving data will cause computing delays due to reading and writing. On the other hand, when the number of computing units gradually increases, the neural network needs to be finely split. Although the von Neumann architecture accelerator chip also has a multi-core architecture, most of them are serial processing from an algorithm perspective. The "von Neumann" based on the separation of storage and computing has a simple structure and is easy to use for high-speed computing, and the mapping method is simple, and resources are scheduled according to the time complexity of the task.

[0004] Due to the high degree of parallelism of deep neural networks, some deep learning accelerator chips are also beginning to adopt a decentralized architecture. Currently, some deep learning accelerator chips also adopt a decentralized architecture: Graphcore's IPU distributed storage and computing integrated multi-core architecture. Figure 2 As shown, the brain-like chip with a distributed parallel many-core architecture has 160 computing cores. This distributed parallel many-core architecture can increase the concurrent access bandwidth within a single chip to as high as 6000GB / s, and the concurrent transmission bandwidth between chip cores can reach 512GB / s.

[0005] Although this integrated storage and computing distributed multi-core architecture can improve the parallelism of the access bandwidth of a single chip, the storage and computing chip contains storage units and computing units in a single computing core, and the design of the two is closely related to the computing power and efficiency of the chip. However, its internal storage and computing mechanism is currently mostly unreasonable, resulting in low computing efficiency of the chip. At present, the number of MACs and storage units in a single computing core are designed according to the ratio of 1 multiplier to 1KB. For artificial neural networks such as ResNet50, the computing efficiency is extremely low, about 1%; for artificial neural networks such as ResNet50, the computing efficiency is extremely low, about 1%.

[0006] The structure of MAC exists in one dimension, two dimension, three dimension and four dimension. The same number of MACs, under different structures, have very different computing efficiency when corresponding to different storage capacities. The capacity of MEM determines the splitting method, and the splitting method determines the computing mode. According to the mapping computing mode, it can be divided into two types: with partial and computing and without partial and computing. In the splitting, since the weight and data capacity allocated to a single computing core are smaller than the storage capacity of the computing core itself, there are:

[0007]

[0008] Where τ is the constant ratio of the data capacity of the neural network after splitting to the memory capacity. This ratio is less than 1 and is an empirical value based on the hardware characteristics of the mapping. MAC structure (W MAC ,Cin MAC ,Cout MAC ) changes with MEM capacity S mem The two changes are independent variables. When the storage capacity is larger, although the computing efficiency will be improved, the manufacturing cost will be quite high due to the price of SRAM. For the distributed storage and computing multi-core architecture, the storage capacity of MEM is designed according to the number and structure of MAC, which is extremely important for the efficiency and cost of the chip.

[0009] Therefore, it is urgent to provide a technical solution to solve the above problems. Summary of the invention

[0010] In order to solve the above technical problems, the present invention provides a chip performance optimization method, system, device and storage medium.

[0011] In a first aspect, the present invention provides a chip performance optimization method, the technical solution of the method is as follows:

[0012] Determine the MAC structure size based on the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit;

[0013] According to the MAC structure size, the benchmark test network is split into width and depth respectively to obtain multiple data blocks containing width split results and depth split results that match the MAC structure size, and according to the overall performance of the multiple data blocks, the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes is determined, and the performance of the chip to be optimized is optimized according to the optimal configuration point.

[0014] The beneficial effects of a chip performance optimization method of the present invention are as follows:

[0015] The method of the present invention improves the computing efficiency of the integrated storage and computing chip, optimizes the computing power per unit area, and saves manufacturing costs by accurately matching the MAC structure size with the storage capacity.

[0016] Based on the above solution, a chip performance optimization method of the present invention can also be improved as follows.

[0017] In an optional manner, the chip performance optimization method further includes:

[0018] According to the design indicators, the performance of the benchmark test network is analyzed to determine the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit.

[0019] In the above optional method, through performance analysis, the performance of the benchmark network under different design indicators can be accurately understood, so as to determine the number of MAC units and storage capacity that match the target requirements of the chip to be optimized. The above process ensures that the optimization plan can be directly targeted at the specific requirements of the chip to be optimized, avoiding unnecessary waste of resources and performance redundancy. Since the optimization plan is formulated based on accurate demand analysis, the efficiency and success rate of the optimization process can be greatly improved.

[0020] In an optional manner, the step of performing performance analysis on the benchmark test network according to the design indicators and determining the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit includes:

[0021] According to the processing speed of the chip to be optimized, a performance analysis is performed on the computing efficiency or throughput of the benchmark test network to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark test network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

[0022] In the above optional methods, by considering the processing speed of the chip to be optimized and the computing efficiency or throughput of the benchmark network, it is possible to ensure that the optimization solution meets the performance requirements in different application scenarios. The number of MAC units and the storage capacity are determined using the analysis results and the operating frequency, making the optimization process more scientific and reasonable. Through accurate calculation, it is possible to ensure that the storage capacity of each MAC unit is fully utilized, avoiding waste of resources.

[0023] In an optional manner, the step of determining the optimal configuration point according to the overall performance of the plurality of data blocks includes:

[0024] Constructing a relationship model between the overall performance and MEM storage capacity, and predicting the overall performance under different MEM storage capacities to obtain a model prediction result;

[0025] Based on the prediction results of the model, the target point with the highest sensitivity between the increment of the overall performance and the increment of the MEM storage capacity is determined, and the target point is used as a reference value for the design of the MEM storage capacity corresponding to a single MAC unit. The reference value is used to characterize the optimal configuration point of the MEM storage capacity.

[0026] In the above optional methods, by building a relationship model and making predictions, the changing trend of the overall performance under different MEM storage capacities can be understood in advance, providing a basis for optimization decisions. By determining the target point with the highest sensitivity as a reference value, the accuracy of the optimization scheme in terms of MEM storage capacity can be ensured. By optimizing the MEM storage capacity configuration, unnecessary resource waste can be reduced and the overall cost performance of the chip can be improved.

[0027] In an optional manner, the overall performance includes: processing time, data transmission time and data sorting time.

[0028] In the above optional method, the overall performance is refined into processing time, data transmission time and data sorting time, which can comprehensively evaluate the impact of the optimization scheme on chip performance. Optimizing the above three key time indicators can more effectively improve the overall performance of the chip.

[0029] In a second aspect, the present invention provides a chip performance optimization system, the technical solution of which is as follows:

[0030] The chip performance optimization system includes: a size determination module and a performance optimization module;

[0031] The size determination module is used to determine the MAC structure size according to the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit;

[0032] The performance optimization module is used to: perform width segmentation and depth segmentation on the benchmark test network according to the MAC structure size, obtain multiple data blocks containing width segmentation results and depth segmentation results that match the MAC structure size, and determine the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes according to the overall performance of the multiple data blocks, and optimize the performance of the chip to be optimized according to the optimal configuration point.

[0033] The beneficial effects of a chip performance optimization system of the present invention are as follows:

[0034] The system of the present invention improves the computing efficiency of the integrated storage and computing chip, optimizes the computing power per unit area, and saves manufacturing costs by accurately matching the MAC structure size and storage capacity.

[0035] Based on the above solution, the chip performance optimization system of the present invention can also be improved as follows.

[0036] In an optional manner, the chip performance optimization system further includes: a performance analysis module, wherein the performance analysis module is used to:

[0037] According to the design indicators, the performance of the benchmark test network is analyzed to determine the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit.

[0038] In the above optional method, through performance analysis, the performance of the benchmark network under different design indicators can be accurately understood, so as to determine the number of MAC units and storage capacity that match the target requirements of the chip to be optimized. The above process ensures that the optimization plan can be directly targeted at the specific requirements of the chip to be optimized, avoiding unnecessary waste of resources and performance redundancy. Since the optimization plan is formulated based on accurate demand analysis, the efficiency and success rate of the optimization process can be greatly improved.

[0039] In an optional manner, the performance analysis module is specifically used to:

[0040] According to the processing speed of the chip to be optimized, a performance analysis is performed on the computing efficiency or throughput of the benchmark test network to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark test network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

[0041] In the above optional methods, by considering the processing speed of the chip to be optimized and the computing efficiency or throughput of the benchmark network, it is possible to ensure that the optimization solution meets the performance requirements in different application scenarios. The number of MAC units and the storage capacity are determined using the analysis results and the operating frequency, making the optimization process more scientific and reasonable. Through accurate calculation, it is possible to ensure that the storage capacity of each MAC unit is fully utilized, avoiding waste of resources.

[0042] In a third aspect, a technical solution of an electronic device of the present invention is as follows:

[0043] The invention comprises a memory, a processor and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the chip performance optimization method of the present invention are implemented.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having the following technical solution:

[0045] Instructions are stored in the computer-readable storage medium. When the computer-readable storage medium reads the instructions, the computer-readable storage medium executes the steps of the chip performance optimization method of the present invention.

[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0048] Figure 1 A schematic diagram of a process flow of an embodiment of a chip performance optimization method of the present invention;

[0049] Figure 2 A chip functional core (a) and the physical layout of the entire chip (b);

[0050] Figure 3 Schematic diagram of MAC design examples with three different architectures;

[0051] Figure 4 Schematic diagram of the relationship between core storage capacity and mapping efficiency for different MAC structures;

[0052] Figure 5 Schematic diagram for the collaborative design of MAC structure and MEM capacity;

[0053] Figure 6 A schematic diagram of the structure of an embodiment of a chip performance optimization system of the present invention;

[0054] Figure 7 The figure is a schematic structural diagram of an embodiment of an electronic device of the present invention. DETAILED DESCRIPTION

[0055] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0056] Figure 1 FIG. 1 is a flow chart showing an embodiment of a chip performance optimization method provided by the present invention, wherein the method is executed by a controller. Figure 1 As shown, the following steps are included:

[0057] S1. Determine the MAC structure size according to the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit. In S1:

[0058] 1) Target requirements refer to the performance indicators or functional requirements that the chip to be optimized needs to meet. Target requirements serve as the starting point and benchmark for chip performance optimization and are used to guide subsequent design and optimization processes.

[0059] 2) The number of MAC units refers to the number of units in the chip used to perform multiply-accumulate operations (MAC). The number of MAC units affects the computing power and parallel processing capabilities of the chip and is one of the key parameters for chip performance optimization.

[0060] 3) The storage capacity of the MAC unit refers to the amount of data that each MAC unit can store. The storage capacity of the MAC unit determines the size of the data that the MAC unit can process when performing operations, affecting the computing efficiency and power consumption of the chip.

[0061] 4) MAC structure size is the physical or logical structure size of the MAC unit determined according to the target requirements and the number and storage capacity of MAC units. MAC structure size is the basis for chip design and affects the chip layout, power consumption and performance.

[0062] Specifically, the controller determines the MAC structure size according to the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit.

[0063] S2. According to the MAC structure size, the benchmark network is split into width and depth respectively, and multiple data blocks including the width split results and the depth split results matching the MAC structure size are obtained. According to the overall performance of the multiple data blocks, the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes is determined, and the performance of the chip to be optimized is optimized according to the optimal configuration point. In S2:

[0064] 1) The benchmark network is a standard test network or data set used to evaluate chip performance. It provides a consistent test environment to accurately evaluate the performance of the chip under different configurations.

[0065] 2) Width slicing is to split the benchmark network in the width direction (such as the number of features) to form multiple smaller data blocks. Width slicing improves the chip's parallel processing capability and computing efficiency by reducing the amount of data that each MAC unit needs to process.

[0066] 3) Deep slicing is to split the benchmark network in the depth direction (such as the number of layers or calculation steps) to form multiple smaller data blocks. Deep slicing improves the chip's computing pipeline and parallel processing capabilities by reducing the data dependency of each calculation step.

[0067] 4) Data blocks are smaller data sets or computing units formed after width and depth splitting. As the basic unit of chip processing, data blocks affect the number of data movements and memory read and write times of the chip.

[0068] 5) The optimal configuration point is the configuration point that can minimize the number of data movements and memory read and write times among multiple possible configurations while meeting the target requirements. As the ultimate goal of chip performance optimization, the optimal configuration point guides the specific implementation and parameter adjustment of chip design.

[0069] Specifically, the controller further performs width segmentation and depth segmentation on the benchmark test network according to the MAC structure size, obtains multiple data blocks containing width segmentation results and depth segmentation results that match the MAC structure size, and determines the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes based on the overall performance of the multiple data blocks, and optimizes the performance of the chip to be optimized based on the optimal configuration point.

[0070] The technical solution of this embodiment improves the computing efficiency of the integrated storage and computing chip, optimizes the computing power per unit area, and saves manufacturing costs by accurately matching the MAC structure size and storage capacity.

[0071] In an optional manner, the chip performance optimization method further includes:

[0072] According to the design indicators, the performance of the benchmark network is analyzed to determine the number of MAC units and the storage capacity of each MAC unit corresponding to the target requirements. Specifically:

[0073] According to the processing speed of the chip to be optimized, the computing efficiency or throughput of the benchmark network is performance analyzed to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

[0074] It should be noted that the design indicators include but are not limited to: ① Processing speed: the number of operations that the chip can perform per second (such as FLOPs, floating-point operations). ② Power consumption: the energy consumption of the chip when it is working. ③ Latency: the time required from input data to output results. ④ Area: the physical size of the chip. ⑤ Cost: the manufacturing cost of the chip.

[0075] Target requirements include but are not limited to: ① Application scenarios: what type of computing tasks the chip will be used for (such as image processing, natural language processing, etc.). ② Performance requirements: specific requirements for processing speed, latency, power consumption, etc.

[0076] For the benchmark network, one or more benchmark networks are selected, which can characterize the workload of the chip to be optimized in actual applications. For example, for image processing tasks, deep learning models such as ResNet and VGG can be selected; for natural language processing tasks, models such as BERT and GPT can be selected. The benchmark network has a certain representativeness and complexity to fully reflect the performance of MAC in actual applications. The benchmark network can be a convolutional neural network (CNN), a recurrent neural network (RNN), or any other type of neural network.

[0077] Perform performance analysis on the benchmark network to evaluate the performance under different MAC unit configurations. The above steps include but are not limited to: ① Computational efficiency analysis: Evaluate the computational efficiency of the benchmark network under different MAC unit numbers and storage capacity configurations (such as the number of images processed per second, the number of texts generated per second, etc.). ② Power consumption analysis: Measure the power consumption under different configurations to ensure that the power consumption is within the design indicator range. ③ Delay analysis: Evaluate the delay under different configurations to find the best configuration that meets the delay requirements.

[0078] Based on the performance analysis results, determine the number of MAC units and the storage capacity of each MAC unit corresponding to the target requirements. The above steps include but are not limited to: ① Performance and resource trade-offs: Trade-offs between performance (such as computing efficiency, latency) and resources (such as power consumption, area, cost) to find the best configuration that meets the target requirements. ② Sensitivity analysis: Perform sensitivity analysis on performance indicators (such as computing efficiency, latency) and resource indicators (such as power consumption, area) to determine which parameters have the greatest impact on performance, thereby adjusting the number of MAC units and storage capacity. ③ Iterative optimization: Through the iterative optimization process, continuously adjust the number of MAC units and storage capacity until the best configuration that meets the design indicators and target requirements is found.

[0079] In addition, after determining the total number of MAC units, the storage capacity of each MAC unit needs to be evaluated. After considering the segmentation rules of the neural network, the matching degree between the MAC structure size and the storage capacity, the time overhead of data sorting and data transmission, etc., by constructing an evaluation model of storage capacity and computing efficiency under different MAC structures, it is possible to find out the storage capacity of a single MAC unit when the computing efficiency is optimal under different MEM (memory) capacities.

[0080] In this embodiment, ResNet50 is selected as the benchmark test network. First, the computing efficiency or throughput (the amount of data that can be processed per second) of the ResNet50 benchmark test network is evaluated based on the design indicator of a single chip processing speed of 4000 frames per second. The total number of MACs is determined based on the starting frequency of a single chip, that is, the operating frequency of the chip, and the computing performance that can be achieved at the operating frequency. Then, the number of MAC units inside a single computing core and the minimum storage capacity are evaluated.

[0081] Specifically, the controller further performs a performance analysis on the computing efficiency or throughput of the benchmark network according to the processing speed of the chip to be optimized, obtains analysis results, and determines the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit according to the analysis results and the operating frequency of the benchmark network.

[0082] It should be noted that after determining the total number of MAC units and the storage capacity of each MAC unit, the structural size of the MAC can be determined. The structural size of the MAC includes but is not limited to dimensions (such as width, depth, etc.) and shapes (such as linear, matrix, etc.). The above steps can be designed based on the segmentation rules of the neural network and the degree of matching between the MAC structural size and the storage capacity. By continuously adjusting the structural size of the MAC, a solution that can meet the requirements of computing efficiency and reasonably control manufacturing costs can be found. Specifically, MACs of different structural sizes can be designed according to the characteristics of the selected benchmark network, as well as the power consumption and cost constraints in the target requirements. For example, different width and depth combinations can be used to construct a MAC array, and the computing efficiency and power consumption performance under each combination can be evaluated. Through comparative analysis, an optimal MAC structural size solution can be selected.

[0083] The MAC structure size is clear, including but not limited to the data width (i.e., how many bits of data can be processed each time) and depth (i.e., how many multiplication results can be accumulated) that the MAC array can process in parallel. The above size will determine how to split the benchmark network.

[0084] Width slicing refers to splitting each layer (or specific layer) in the benchmark network according to the data width to match the parallel processing capability of the MAC. Width slicing includes but is not limited to: ① Determine the slicing granularity: Determine how many sub-blocks each layer should be split into based on the data width of the MAC. For example, if the MAC can process 16-bit data in parallel, the weights and input data of each layer can be split into groups of 16 bits. ② Reorganize the data: Reorganize the data of each layer into a form that matches the size of the MAC structure. The above process involves operations such as rearrangement, packing, or padding of data. ③ Adjust the network structure: In the framework of the benchmark network, modify the structure of each layer to reflect the results of width slicing. The above process involves updating the parameters of the layer, the connection relationship, or adding additional layers to process the sliced ​​data.

[0085] Deep slicing refers to splitting each layer (or specific layer) in the benchmark network according to the depth to match the accumulation capacity of MAC. Deep slicing includes but is not limited to: ① Determine the slicing depth: Determine how many sub-layers each layer should be split into based on the accumulation depth of MAC. For example, if MAC can accumulate 8 multiplication results, each layer can be divided into 8 sub-layers, and each sub-layer handles one multiplication operation. ② Split layer operations: Within each layer, the calculation that was originally completed once is split into multiple times. The above process involves splitting the weight matrix into multiple small blocks and splitting the input data into multiple batches for processing accordingly. ③ Update the network structure: In the framework of the benchmark network, modify the structure of each layer to reflect the results of deep slicing. The above process involves adding additional layers to store intermediate results, updating the connection relationship of the layer, or adjusting the output of the layer.

[0086] In addition, the performance of the benchmark network after width and depth splitting can be evaluated to ensure that the benchmark network can maintain its original accuracy and efficiency. Then, according to the performance evaluation results, the splitting strategy is adjusted and optimized. The above process involves adjusting the splitting granularity, depth or reorganizing data to improve performance.

[0087] After the above steps, the benchmark network can be divided into multiple data blocks that match the size of the MAC structure. The multiple data blocks contain both the results of width segmentation (i.e., sub-blocks segmented by data width) and the results of depth segmentation (i.e., sub-layers segmented by depth). The multiple data blocks will be used as input for subsequent calculations performed on the MAC hardware, thereby ensuring that the benchmark network can be efficiently mapped to the MAC hardware structure in order to fully utilize the parallel processing and accumulation capabilities of the MAC.

[0088] In this embodiment, under the condition that the total number of MAC units is constant, the structure of MAC and the pipeline system can be flexibly defined. Figure 3As shown in the figure, three MAC design examples with different architectures are presented, all of which are built on the premise of the same total number of MAC units, namely 4×32, 4×4×8 and 4×2×16 configurations. The above differentiated MAC structures determine the strategy of neural network segmentation, requiring that the segmentation rules must strictly follow the structural size of the MAC to ensure maximum utilization of computing resources.

[0089] Taking the 4×32 MAC array as an example, the width dimension is 4 and the output depth dimension is 32. Therefore, in the process of neural network segmentation, it is necessary to ensure that the segmentation result of the image data in the width direction is an integer multiple of 4, and the output depth of the weight data should be an integer multiple of 32. When the above two dimensions are segmented to the minimum granularity, that is, the width is 4 and the output depth is 32, if the amount of data after segmentation still exceeds the processing capacity of a single computing core, it is necessary to further segment other dimensions, such as input depth, to adapt to the design capacity limit. In this process, the segmentation of input depth and weight input depth will introduce "partial sum" data, which needs to be sorted and transmitted, thereby increasing the time cost in the calculation process. Therefore, the calculation efficiency needs to comprehensively consider the calculation time, data sorting time and data transmission time for comprehensive evaluation. Among them, partial sum refers to when the input depth of the image and the input depth of the weight are segmented to adapt to the storage capacity of the computing core (core) when processing the neural network, some intermediate results will be generated, and the data of multiple intermediate results is called "partial sum". The above data is the intermediate cumulative result generated during the neural network calculation process due to the data being segmented to different computing cores for parallel processing. When processing the above data, data collation and data transmission are required so that these partial sums can be correctly used and accumulated in the calculation process to obtain the final output result.

[0090] In an optional manner, the step of determining the optimal configuration point according to the overall performance of the plurality of data blocks includes:

[0091] Construct a relationship model between overall performance and MEM storage capacity, and predict the overall performance under different MEM storage capacities to obtain the model prediction results;

[0092] According to the model prediction results, the target point with the highest sensitivity between the increment of overall performance and the increment of MEM storage capacity is determined, and the target point is used as the reference value for the design of MEM storage capacity corresponding to a single MAC unit. The reference value is used to characterize the optimal configuration point of MEM storage capacity.

[0093] It should be noted that in the process of constructing the relationship model between overall performance and MEM storage capacity, first, the overall performance data of multiple data blocks under different MEM storage capacities are collected. The data includes but is not limited to key performance indicators such as processing time, data transmission time, data sorting time, processing speed, throughput, and delay. The collected data is preprocessed, including denoising, normalization, etc., to improve the data quality and the accuracy of model training. Then, the model input features, i.e., MEM storage capacity, and output features, i.e., overall performance, are determined. According to the data characteristics and problem requirements, effective features are extracted for model training. Then, according to the characteristics of the data and the complexity of the problem, a suitable modeling method, such as linear regression, polynomial regression, support vector machine, neural network, etc., is selected to obtain a preprocessed data training model. The preprocessed data training model is used to continuously adjust the model parameters so that the model can accurately predict the overall performance under different MEM storage capacities. Finally, the generalization ability of the model can be evaluated by methods such as cross-validation to ensure the prediction accuracy of the model on different data sets. The model is optimized according to the verification results, such as adjusting the model structure, adding regularization terms, etc., to improve the prediction performance of the model, and finally a relationship model between the overall performance and MEM storage capacity is obtained.

[0094] In the process of predicting the overall performance under different MEM storage capacities, first, according to actual needs, the prediction range of MEM storage capacity is set to ensure that the prediction results can cover all possible configuration points. Then, the trained relationship model is used to predict the overall performance of each possible MEM storage capacity configuration point, and the prediction results under each configuration point are recorded for subsequent analysis.

[0095] In the process of determining the optimal configuration point, for each prediction result, the ratio of the increment of overall performance to the increment of MEM storage capacity, i.e., sensitivity, is calculated. Sensitivity reflects the degree of impact of changes in MEM storage capacity on overall performance. Within the prediction range, find the point with the highest sensitivity, i.e., the point with the most significant improvement in overall performance and the least increase in MEM storage capacity. The above point is the target point, which represents the optimal configuration point of MEM storage capacity. In actual applications, the target point can also be verified to ensure that its performance in the real environment is consistent with the prediction. Fine-tune the target point based on the verification results to further optimize the overall performance.

[0096] In the process of applying the optimal configuration point, first, the obtained optimal configuration point or the verified optimal configuration point is used as a reference value for the design of the MEM storage capacity corresponding to a single MAC unit. The above reference value will be used to guide the subsequent hardware design and optimization process. Then, the design of the MAC unit is adjusted according to the reference value to ensure that each MAC unit has the optimal MEM storage capacity configuration. At the same time, the overall layout and power consumption management of the chip are optimized to improve the overall performance and energy efficiency of the chip. In actual applications, the chip design can also be iteratively optimized based on feedback and performance evaluation results, and the structure of the MAC unit and the MEM storage capacity configuration can be continuously optimized to meet the ever-changing application requirements.

[0097] In this embodiment, when the total number and structure of MAC units are fixed, by adjusting the storage capacity of MEM (memory), the computing efficiency under different MEM capacities can be further evaluated. Figure 4 As shown, a computing efficiency model for different MAC structures under different MEM capacities can be constructed, that is, a relationship model between the overall performance and the MEM storage capacity.

[0098] Finally, based on Figure 4 , can select the best MEM storage capacity design. Figure 4 It can be seen that the mapping efficiency under different MAC structures increases with the increase of MEM storage capacity, but the growth rate varies in each interval. When the MEM storage capacity increases to a certain extent, the mapping calculation efficiency tends to saturation. When the mapping efficiency exceeds 70%, the storage capacity continues to increase, and the sensitivity between the mapping efficiency and the MEM increment gradually decreases. Taking ResNet 50Block 16 as an example, under different MAC structures, the mapping efficiency shows different trends with the change of MEM storage capacity. Based on the above design mechanism, the MAC array of 4×2×16 structure exhibits the same mapping efficiency at MEM capacities of 192KB and 256KB.

[0099] Therefore, when considering the computing power per unit area, for the MAC array of 4×2×16 structure, a MEM storage capacity of about 192KB is selected; for the MAC array of 4×32 structure, a MEM storage capacity of about 256KB is selected based on the mapping efficiency and computing power per unit area; the MAC array of 4×4×8 structure is more suitable for designing a MEM storage capacity of about 192KB. Through the above design mechanism, the MEM storage capacity inside a single computing core can be reasonably set, which not only ensures the maximization of computing efficiency, but also realizes the saving of manufacturing costs.

[0100] Specifically, the controller further constructs a relationship model between the overall performance and the MEM storage capacity, and predicts the overall performance under different MEM storage capacities to obtain the model prediction results; based on the model prediction results, the target point with the highest sensitivity between the increment of the overall performance and the increment of the MEM storage capacity is determined, and the target point is used as a reference value for the design of the MEM storage capacity corresponding to a single MAC unit. The reference value is used to characterize the optimal configuration point of the MEM storage capacity.

[0101] It should be noted that Figure 5 The schematic diagram for the coordinated design of MAC structure and MEM capacity directly describes the storage capacity and structural characteristics of two three-dimensional multi-layer memory access caches (3D MACs). The left figure shows a MAC numbered 128, whose MEM capacity is 128KB and consists of four smaller cubes. The cubes represent the data storage area inside the MAC. The right figure shows another MAC with a MEM capacity of 192KB. By comparing the storage capacity and internal structure of different MACs, it can be seen that different MEM capacities can be achieved by adjusting the size of each storage area (i.e., the volume of the cube) while maintaining the same number of data storage areas. The above design takes into account both storage efficiency and the feasibility of hardware implementation. Therefore, Figure 5 The schematic diagram of the coordinated design of MAC structure and MEM capacity effectively conveys the design concept of how to achieve different storage capacities by optimizing the internal structure.

[0102] Figure 6 FIG. 2 is a schematic diagram showing a structure of an embodiment of a chip performance optimization system 200 provided by the present invention. Figure 6 As shown, the system 200 includes: a size determination module 210 and a performance optimization module 220 .

[0103] The size determination module 210 is used to: determine the MAC structure size according to the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit;

[0104] The performance optimization module 220 is used to: perform width segmentation and depth segmentation on the benchmark test network according to the MAC structure size, obtain multiple data blocks containing width segmentation results and depth segmentation results that match the MAC structure size, and determine the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes based on the overall performance of the multiple data blocks, and optimize the performance of the chip to be optimized based on the optimal configuration point.

[0105] In an optional manner, the chip performance optimization system further includes a performance analysis module, and the performance analysis module is used to:

[0106] According to the design indicators, the performance of the benchmark network is analyzed to determine the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit.

[0107] In an optional manner, the performance analysis module is specifically used to:

[0108] According to the processing speed of the chip to be optimized, the computing efficiency or throughput of the benchmark network is performance analyzed to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

[0109] The technical solution of this embodiment improves the computing efficiency of the integrated storage and computing chip, optimizes the computing power per unit area, and saves manufacturing costs by accurately matching the MAC structure size and storage capacity.

[0110] The above parameters and steps for each module to implement corresponding functions in the chip performance optimization system 200 of this embodiment can refer to the parameters and steps in the embodiment of the chip performance optimization method above, and will not be repeated here.

[0111] like Figure 7 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above chip performance optimization methods, specifically:

[0112] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements any chip performance optimization method provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the electronic device 300 may also include other components for implementing device functions, which will not be described in detail here.

[0113] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above chip performance optimization methods.

[0114] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0115] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs any of the above chip performance optimization methods.

[0116] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0117] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0118] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0119] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A chip performance optimization method, characterized in that: include: Determine the MAC structure size based on the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit; According to the MAC structure size, the benchmark test network is split into width and depth respectively to obtain multiple data blocks containing width split results and depth split results that match the MAC structure size, and according to the overall performance of the multiple data blocks, the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes is determined, and the performance of the chip to be optimized is optimized according to the optimal configuration point.

2. The chip performance optimization method according to claim 1, characterized in that: Also includes: According to the design indicators, the performance of the benchmark test network is analyzed to determine the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit.

3. The chip performance optimization method according to claim 2, characterized in that: The step of performing performance analysis on the benchmark test network according to the design index to determine the number of MAC units corresponding to the target demand and the storage capacity of each MAC unit includes: According to the processing speed of the chip to be optimized, a performance analysis is performed on the computing efficiency or throughput of the benchmark test network to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark test network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

4. The chip performance optimization method according to claim 1, characterized in that: The steps of determining the optimal configuration point according to the overall performance of the multiple data blocks include: Constructing a relationship model between the overall performance and MEM storage capacity, and predicting the overall performance under different MEM storage capacities to obtain a model prediction result; Based on the prediction results of the model, the target point with the highest sensitivity between the increment of the overall performance and the increment of the MEM storage capacity is determined, and the target point is used as a reference value for the design of the MEM storage capacity corresponding to a single MAC unit. The reference value is used to characterize the optimal configuration point of the MEM storage capacity.

5. The chip performance optimization method according to any one of claims 1 to 4, characterized in that: The overall performance includes: processing time, data transmission time and data sorting time.

6. A chip performance optimization system, characterized in that: include: Sizing module and performance optimization module; The size determination module is used to determine the MAC structure size according to the number of MAC units corresponding to the target requirements of the chip to be optimized and the storage capacity of each MAC unit; The performance optimization module is used to: perform width segmentation and depth segmentation on the benchmark test network according to the MAC structure size, obtain multiple data blocks containing width segmentation results and depth segmentation results that match the MAC structure size, and determine the optimal configuration point for reducing the number of data movements and minimizing the number of memory reads and writes according to the overall performance of the multiple data blocks, and optimize the performance of the chip to be optimized according to the optimal configuration point.

7. The chip performance optimization system according to claim 6, characterized in that: Also included is a performance analysis module, the performance analysis module is used to: According to the design indicators, the performance of the benchmark test network is analyzed to determine the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit.

8. The chip performance optimization system according to claim 7, characterized in that: The performance analysis module is specifically used for: According to the processing speed of the chip to be optimized, a performance analysis is performed on the computing efficiency or throughput of the benchmark test network to obtain analysis results, and based on the analysis results and the operating frequency of the benchmark test network, the number of MAC units corresponding to the target requirements and the storage capacity of each MAC unit are determined.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the chip performance optimization method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements the chip performance optimization method according to any one of claims 1 to 5.

Images