Performance analysis device and method for neural network processor

Abstract

The embodiment of the invention discloses a performance analysis device and method for a neural network processor. The device comprises an initialization module and an execution module. The initialization module is configured to generate a particle swarm for a particle swarm optimization algorithm and a first population for a genetic algorithm according to an input data range and a hyper-parameter range of the neural network model. The execution module is configured to iteratively optimize the particle swarm by using a particle swarm optimization algorithm to determine an optimal solution of the first performance index when the neural network processor operates the neural network model. The initialization module is further configured to generate an elite population according to an optimal solution of the first performance indicator, and merge the elite population and the first population as an initial population.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of neural network, in particular to a performance analysis device and method of neural network processor. BACKGROUND

[0002] Currently, the architecture design and ecology of neural network processor (NPU) are different and not open, and the competitiveness analysis is mainly performed in a black box manner. For example, artificial intelligence benchmark (AI Benchmark), artificial intelligence mark (AIMark) and other benchmark testing tools are used to run a variety of pre-trained networks on different NPUs to evaluate performance. These methods can provide a certain degree of performance comparison, but there are some limitations.

[0003] Commonly used techniques usually rely on specific network and parameter settings, making it difficult to conduct in-depth analysis of performance in specific scenarios. In addition, traditional optimization methods also have shortcomings in search efficiency and adaptability, such as high computational cost of grid search and easy premature convergence of particle swarm optimization algorithm. These problems limit the comprehensive understanding and optimization of NPU performance in specific scenarios. With the continuous expansion of neural networks in mobile applications, the demand for NPU performance analysis and optimization is increasing, so a more efficient and more general method is needed to address the shortcomings in the prior art. SUMMARY

[0004] Embodiments of the present application provide a performance analysis device and method of neural network processor.

[0005] The technical solution of the embodiments of the present application is as follows:

[0006] The embodiments of the present application provide a performance analysis device of neural network processor, the device comprises: an initialization module configured to generate a particle swarm for a particle swarm optimization algorithm and a first population for a genetic algorithm according to an input data range and a hyperparameter range of a neural network model; an execution module configured to iteratively optimize the particle swarm using the particle swarm optimization algorithm to determine an optimal solution of a first performance indicator of the neural network processor when running the neural network model; the initialization module is further configured to generate an elite population according to the optimal solution of the first performance indicator, and merge the elite population and the first population as an initial population; the execution module is further configured to iteratively optimize the initial population using the genetic algorithm to determine an optimal solution of a second performance indicator of the neural network processor when running the neural network model.

[0007] In some embodiments of the present application, the initialization module comprises: a searching unit configured to determine a plurality of first parameter combinations and a plurality of second parameter combinations according to an input data range and a hyperparameter range of the neural network model; an encoding unit configured to encode the plurality of first parameter combinations to generate the particle swarm and encode the plurality of second parameter combinations to generate the first population; wherein the first parameter combinations correspond one-to-one to the particles in the particle swarm; and the second parameter combinations correspond one-to-one to the individuals in the first population.

[0008] In some embodiments of the present application, the execution module comprises: a model running unit configured to input the input data and the hyperparameters corresponding to each particle in the particle swarm into the neural network processor and run the neural network model; a data acquisition unit configured to acquire first performance data of the neural network processor when the input data and the hyperparameters corresponding to each particle in the particle swarm are run on the neural network model; an evaluation unit configured to determine a first fitness value of each particle in the particle swarm and a first global fitness value of the particle swarm according to the first performance data and a first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm; a determination unit configured to determine whether the first global fitness value and the number of iterations of the particle swarm optimization algorithm satisfy a first iteration stop condition; an iteration unit configured to update the speed and position of each particle in response to a determination result of the determination unit that the first iteration stop condition is not satisfied; and an output unit configured to output the first global fitness value as an optimal solution of the first performance indicator in response to a determination result of the determination unit that the first iteration stop condition is satisfied.

[0009] In some embodiments of the present application, the encoding unit is further configured to encode input data and hyperparameters corresponding to the optimal solution of the first performance indicator to generate the elite population; the model running unit is further configured to input the input data and hyperparameters corresponding to each individual of the initial population into the neural network processor and run the neural network model; the data acquisition unit is further configured to acquire second performance data of the neural network processor when the input data and hyperparameters corresponding to each individual of the initial population are run on the neural network model; the evaluation unit is further configured to determine a second fitness value of each individual of the initial population and a second global fitness value of the initial population according to the second performance data and a second fitness function; wherein the second global fitness value is the second fitness value corresponding to the individual with the maximum fitness value in the initial population; the determination unit is further configured to determine whether the second global fitness value and the number of iterations of the genetic algorithm satisfy a second iteration stop condition; the iteration unit is further configured to, in response to a determination result of the determination unit that the second iteration stop condition is not satisfied, perform selection, crossover and mutation operations on the individuals in the initial population to generate a new generation population; and the output unit is further configured to, in response to a determination result of the determination unit that the second iteration stop condition is satisfied, output the second global fitness value as the optimal solution of the second performance indicator.

[0010] In some embodiments of the present application, the iteration unit is specifically configured to perform directional mutation operations on individuals other than excellent individuals in the initial population, and randomly perform random mutation operations on part of the individuals in the initial population; wherein the excellent individuals include individuals whose second fitness values increase after the last iteration and individuals in the elite population whose second fitness values are greater than a preset value.

[0011] The embodiments of the present application further provide a performance analysis method of a neural network processor, the method comprising: generating a particle swarm of a particle swarm optimization algorithm and a first population of a genetic algorithm according to an input data range and a hyperparameter range of a neural network model; iteratively optimizing the particle swarm by using the particle swarm optimization algorithm to determine an optimal solution of a first performance indicator of a neural network processor when running the neural network model; generating an elite population according to the optimal solution of the first performance indicator, and merging the elite population and the first population to serve as an initial population; and iteratively optimizing the initial population by using the genetic algorithm to determine an optimal solution of a second performance indicator of the neural network processor when running the neural network model.

[0012] In some embodiments of the present application, the generating a particle swarm of a particle swarm optimization algorithm and a first population of a genetic algorithm according to an input data range and a hyperparameter range of a neural network model comprises: determining a plurality of first parameter combinations and a plurality of second parameter combinations according to the input data range and the hyperparameter range of the neural network model; encoding the plurality of first parameter combinations to generate the particle swarm and encoding the plurality of second parameter combinations to generate the first population; wherein the first parameter combinations correspond one-to-one to particles in the particle swarm; and the second parameter combinations correspond one-to-one to individuals in the first population.

[0013] In some embodiments of the present application, the iterative optimization of the particle swarm by the particle swarm optimization algorithm to determine an optimal solution of a first performance indicator of a neural network processor when running the neural network model comprises: inputting input data and hyperparameters corresponding to each particle in the particle swarm into the neural network processor and running the neural network model; obtaining first performance data of the neural network processor when the input data and hyperparameters corresponding to each particle in the particle swarm are run on the neural network model; determining a first fitness value of each particle in the particle swarm and a first global fitness value of the particle swarm according to the first performance data and a first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm; determining whether the first global fitness value and the number of iterations of the particle swarm optimization algorithm satisfy a first iteration stop condition; in response to not satisfying the first iteration stop condition, updating the speed and position of each particle; and in response to satisfying the first iteration stop condition, outputting the first global fitness value as the optimal solution of the first performance indicator.

[0014] In some embodiments of the present application, the generating an elite population according to the optimal solution of the first performance indicator comprises: encoding input data and hyperparameters corresponding to the optimal solution of the first performance indicator to generate the elite population; and the iterative optimization of the initial population by the genetic algorithm to determine the optimal solution of the second performance indicator of the neural network processor when running the neural network model comprises: inputting the input data and hyperparameters corresponding to each individual of the initial population into the neural network processor and running the neural network model; obtaining the second performance data of the neural network processor when the input data and hyperparameters corresponding to each individual of the initial population are running on the neural network model; determining the second fitness value of each individual of the initial population and the second global fitness value of the initial population according to the second performance data and a second fitness function; wherein the second global fitness value is the second fitness value corresponding to the individual with the maximum fitness value in the initial population; determining whether the second global fitness value and the number of iterations of the genetic algorithm satisfy a second iteration stop condition; in response to not satisfying the second iteration stop condition, performing selection, crossover and mutation operations on the individuals in the initial population to generate a new generation population; and in response to satisfying the second iteration stop condition, outputting the second global fitness value as the optimal solution of the second performance indicator.

[0015] In some embodiments of the present application, the mutation operation on the individuals in the initial population comprises: performing a directional mutation operation on the individuals other than excellent individuals in the initial population, and randomly performing a random mutation operation on part of the individuals in the initial population; wherein the excellent individuals include individuals whose second fitness values increase after the last iteration and individuals in the elite population whose second fitness values are greater than a preset value.

[0016] In some embodiments of the present application, the optimization solving complexity of the first performance indicator is lower than the optimization solving complexity of the second performance indicator.

[0017] Therefore, in the embodiments of the present application, the PSO algorithm is used to quickly determine the optimal solution of the first performance indicator, and the elite population generated by the optimal solution is combined with the first population of the genetic algorithm. This staged optimization strategy not only speeds up the convergence speed, but also effectively avoids the problem of local optimal solution. Among them, the elite population guides the genetic algorithm to search in a better area. At the same time, the merging operation maintains the diversity of the initial population and enhances the ability of the algorithm to explore unknown areas. The combination of PSO algorithm and genetic algorithm supports parallel processing, which can fully utilize the parallel computing capability of NPU, speed up the optimization process and reduce power consumption, so as to find the combination of hyperparameters that can make the NPU utilization rate of the model highest, the energy efficiency ratio best, and the calculation and operation strength reach the best performance. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0019] Figure 1 A schematic diagram of the structure of a performance analysis device for a neural network processor provided in an embodiment of the present application Figure 1 ;

[0020] Figure 2 A schematic diagram of the structure of a performance analysis device for a neural network processor provided in an embodiment of the present application Figure 2 ;

[0021] Figure 3 A schematic diagram of the structure of a performance analysis device for a neural network processor provided in an embodiment of the present application Figure 3 ;

[0022] Figure 4 A schematic diagram of a search unit flow diagram provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of a performance analysis device for a neural network processor provided in an embodiment of the present application Figure 4 ;

[0024] Figure 6 A schematic diagram of the first fitness function provided in an embodiment of the present application;

[0025] Figure 7 Schematic diagram of the second fitness function of the initial population provided in the embodiment of the present application Figure 1 ;

[0026] Figure 8 Schematic diagram of the second fitness function of the initial population provided in the embodiment of the present application Figure 2 ;

[0027] Figure 9 Schematic diagram of the second fitness function of the initial population provided in the embodiment of the present application Figure 3 .

[0028] Figure 10 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 1 .

[0029] Figure 11 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present applicationFigure 2 .

[0030] Figure 12 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 3 .

[0031] Figure 13 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 4 .

[0032] Figure 14 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 5 . DETAILED DESCRIPTION

[0033] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the embodiments of the present application, but are not intended to limit the scope of the embodiments of the present application.

[0034] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0035] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of the present invention. The terms used herein are for the purpose of describing the present invention only and are not intended to limit the present invention.

[0037] The embodiment of the present application provides a performance analysis device for a neural network processor, as shown in the attached Figure 1 FIG. 1 is a schematic diagram of a performance analysis device for a neural network processor provided in an embodiment of the present application. Figure 1 ; The performance analysis device of the neural network processor includes an initialization module 10 and an execution module 20.

[0038] In the embodiments of this application, refer to the attached Figure 1As shown, the initialization module 10 is configured to generate a particle swarm 1011 for a particle swarm optimization (PSO) algorithm and a first population 1012 for a genetic algorithm according to the input data range and the hyperparameter range of the neural network model.

[0039] wherein the input data range of the neural network model is the scale or distribution range of the original data values before the original data is input into the neural network model; wherein the original data values can be image or text data, etc. After the original data is input into the neural network model, normalization processing is usually performed; for example, the original range of image pixels is [0, 255], and the range after normalization processing is [0, 1]. In this way, different sources and different scales of features can be converted to a relatively consistent and stable numerical scale, so that the neural network can learn efficiently and stably. The hyperparameter range is the possible value interval of the parameters manually set (or searched automatically) by researchers before training the neural network model, and the value range is set based on experience, initial population 2012 requirements, computing resources and preliminary experiments; the hyperparameters include learning rate, batch size and regularization parameter, etc.

[0040] The generation method of the initialization module 10 is independent of a specific NPU architecture or neural network model, and is suitable for various mobile end NPUs and different neural network architectures. For example, both convolutional neural networks and recurrent neural networks can use the initialization module 10 to generate corresponding populations, providing a basis for subsequent performance analysis and optimization.

[0041] Referring to the accompanying drawings Figure 1 As shown, the execution module 20 is configured to use the PSO algorithm to iteratively optimize the particle swarm 1011 to determine the optimal solution 1013 of the first performance indicator of the neural network processor when running the neural network model.

[0042] wherein the first performance indicator is the NPU utilization rate. The PSO algorithm is a stochastic optimization method inspired by the behavior of biological populations in nature, which simulates the foraging behavior of bird flocks to make particles move in the search space and find the optimal solution. Through the PSO algorithm, the device can quickly find a solution that makes the NPU utilization rate higher among a large number of parameter combinations, reducing the time cost of finding a suitable parameter combination.

[0043] In addition, the genetic algorithm is an optimization technique based on the principles of natural selection and genetics, which simulates the biological evolution process in nature and finds the optimal solution through a series of operations such as selection, crossover, mutation, etc. By combining the PSO algorithm and the genetic algorithm, the global search capability of the genetic algorithm and the fast convergence characteristics of the PSO algorithm can be fully utilized to form a more efficient optimization method. This method can quickly find the approximate position of the global optimal solution and approximate the optimal solution through detailed search.

[0044] Referring to FIG. 1, Figure 1 As shown in FIG. 1, the initialization module 10 is further configured to generate an elite population 2011 according to the optimal solution 1013 of the first performance indicator, and merge the elite population 2011 and the first population 1012 as an initial population 2012.

[0045] It should be noted that the merged initial population 2012 accelerates the convergence speed and improves the search efficiency in the genetic algorithm stage. Specifically, the PSO algorithm first finds parameter combinations that make the NPU utilization rate high, and these combinations are added to the initial population 2012 of the genetic algorithm as high-quality genes, so that the overall quality of the initial population 2012 is higher and closer to the global optimal solution, thereby reducing the number of iterations required to find the optimal solution. At the same time, the merging operation increases the diversity of the initial population 2012, which helps to avoid local optimal solutions and improve global search capability. Among them, the local optimal solution refers to a solution that makes the objective function optimal in a certain region, but this solution is not the global optimal solution in the entire search space. If the optimization process falls into a local optimal solution, it will not be able to find a better or even optimal solution in the global range, resulting in the inability to achieve the best performance of key indicators such as model performance.

[0046] It can be understood that, as shown in FIG. 1, Figure 2 As shown in FIG. 1, the device first generates a particle swarm 1011 and a first population 1012 according to the input data range and the hyperparameter range of the neural network model; then uses the PSO algorithm to quickly determine the optimal solution of the first performance indicator (such as NPU utilization rate); then merges the elite population 2011 generated by the optimal solution with the first population 1012 of the genetic algorithm to form an initial population 2012 for further global optimization. This phased optimization strategy not only accelerates the convergence speed, but also effectively avoids the problem of local optimal solution. Among them, the introduction of the elite population 2011 provides a high-quality initial population 2012 for the genetic algorithm, guiding the search to concentrate in a better area. At the same time, the merging operation maintains the diversity of the initial population 2012, enhancing the algorithm's ability to explore unknown regions. The combination of PSO algorithm and genetic algorithm supports parallel processing, which can fully utilize the parallel computing capability of NPU, speed up the optimization process and reduce power consumption, in order to find the hyperparameter combination that can make the NPU utilization rate of the model highest, the energy efficiency ratio best, and the calculation and operation strength reach the best performance.

[0047] In the embodiments of the present application, as shown in FIG. 1, Figure 3 and FIG. 2, Figure 4 As shown in FIG. 1, the initialization module 10 includes a search unit 101 and an encoding unit 102; the search unit 101 is configured to determine a plurality of first parameter combinations and a plurality of second parameter combinations according to the input data range and the hyperparameter range of the neural network model.

[0048] Here is an example, referring to FIG. 1Figure 3 S1011: input data range and hyperparameter range to neural network model. For example, the input data range is (N, H, W, C) and the hyperparameter range is (filters, kernel_size, strides, etc.). S1012: combine the data range and the hyperparameter range to form the total search space. S1013: generate particle swarm 1011 using real coding in PSO algorithm. The generated particle swarm 1011 has position, acceleration, individual best position and global best position numerical attributes. Through these attributes, the particle can constantly adjust its own position in the search space to find the optimal solution. S1014: generate the first population 1012 using binary coding in genetic algorithm. It should be noted that the initial population 2012 of the genetic algorithm also includes the optimal solution in the PSO algorithm. In actual use scenarios, some variables are often fixed, and other acceleration problems are focused on. This example is not limited to Conv2D operation, but is common to common convolutional neural networks.

[0049] where N represents the batch size of the input data, i.e. the number of samples input at a time; H represents the height of the input data; W represents the width of the input data; C represents the number of channels of the input data, and for an RGB image, the number of channels is 3; filters is the convolution kernel, which is a small two-dimensional matrix; kernel_size is the size of the convolution kernel; strides is the step size of the convolution kernel sliding on the input data.

[0050] It should be noted that the search unit 101 can use random search, grid search or sampling-based method to determine the parameter combination. Random search randomly samples parameter values in the search space, which is suitable for high-dimensional space; grid search comprehensively covers all possible combinations of the search space; and the sampling-based method generates parameter combinations according to a specific sampling strategy. According to the selected search strategy, the search unit 101 starts the search algorithm to generate multiple sets of initial parameter combinations; each set of parameter combinations includes the size of the input data and the specific value of the hyperparameter, ensuring to cover different areas of the search space to improve the possibility of finding the global optimal solution. The search unit 101 outputs the parameter combinations to the encoding unit 102 to provide a basis for the execution of the subsequent optimization algorithm.

[0051] In the embodiments of the present application, with reference to the accompanying Figure 1 As shown in the figure, the encoding unit 102 is configured to encode the multiple first parameter combinations to generate the particle swarm 1011, and encode the multiple second parameter combinations to generate the first population 1012; wherein the first parameter combination corresponds to the particle in the particle swarm 1011 one by one; and the second parameter combination corresponds to the individual in the first population 1012 one by one.

[0052] It should be noted that, with reference to the accompanyingFigure 3 and attached Figure 3 As shown, the encoding unit 102 converts the parameter combination into a specific representation form that needs to be able to be effectively processed by the optimization algorithm (such as the PSO algorithm or genetic algorithm). For example, a set of parameters (such as the learning rate, convolution kernel size, step size, etc.) is converted into a binary string or a real number vector. Different optimization algorithms have different requirements for parameter encoding. The PSO algorithm usually uses real number encoding because the position and velocity of the particles are updated in a continuous space; while the genetic algorithm can use binary encoding, real number encoding, or other forms of encoding, depending on the implementation of the algorithm and the requirements of the problem. Binary encoding is used here. By generating a variety of parameter combinations, the search unit 101 ensures the diversity of the initial population 2012 and prevents the optimization process from falling into a local optimal solution. The encoding unit 102 can also decode the parameters corresponding to the particles. Here, the PSO algorithm randomly initializes the particle swarm using the combination of the possible value ranges of the input data range and the hyperparameter range as the total search space.

[0053] In the embodiments of this application, refer to the attached Figure 5 and attached Figure 3 As shown, the execution module 20 includes:

[0054] The model running unit 201 is configured to input the input data and hyperparameters corresponding to each particle in the particle swarm 1011 into the neural network processor and run the neural network model.

[0055] Here, refer to the attached Figure 3 As shown, after the model and data are pushed to the NPU to be tested, the runtime environment is initialized to load the model and allocate resources; then, data is input into the model in batches for inference or training, giving full play to the parallel computing advantages of the NPU and improving processing efficiency.

[0056] The data acquisition unit 202 is configured to obtain input data corresponding to each particle in the particle swarm 1011 and first performance data of the neural network processor when the hyperparameters are run on the neural network model.

[0057] Among them, refer to the attached Figure 5 As shown, the first performance data is the model performance (such as accuracy, recall rate, etc.). When the input data and hyperparameters corresponding to each particle in the particle swarm 1011 are fed into the neural network model during or after the operation, the data acquisition unit 202 obtains the first performance data of the model through the interface with the neural network processor or the monitoring tool, so that the evaluation unit 203 can calculate the first fitness value.

[0058] The evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1):

[0059]

[0060] Wherein, U-rate is the NPU utilization rate, model performance is the model performance, model flops is the calculation amount of the model, that is, the floating point operation times of the model, which is an index for measuring the calculation complexity of the deep learning model, npu T flops is the floating point operation times per second of the NPU, which is an index for measuring the calculation capacity of the NPU, and npu Tflops is a constant; model flops is calculated according to the structure and hyperparameters of the model, and the floating point operation times of each layer are summed to obtain the total floating point operation times of the entire model.

[0061] As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1): Figure 6 As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1): Figure 5 As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1):

[0062] As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1):

[0063] As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1): Figure 5 As shown in FIGS. 1, 2 and 3, the evaluation unit 203 is configured to determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the maximum fitness value in the particle swarm 1011. The first fitness function is shown in the following formula (1):

[0064]

[0065] wherein, is the velocity of the i-th particle in the d-th dimension at the k+1-th iteration, omega is the inertia weight used to control the influence of the current velocity on the new velocity, omega can be set as a fixed value according to experience or dynamically adjusted to balance the global and local search ability, is the velocity of the i-th particle in the d-th dimension at the k-th iteration, c1 is the individual learning factor controlling the learning strength of the particle on its own historical optimal position, r1 is a random number generated in the interval [0, 1] for increasing the randomness and exploration ability of the algorithm, is the individual optimal position of the i-th particle in the d-th dimension at the k-th iteration, is the position of the i-th particle in the d-th dimension at the k-th iteration, c2 is the group learning factor controlling the learning strength of the particle on the group optimal position, r2 is a random number generated in the interval [0, 1], is the group optimal position at the k-th iteration in the d-th dimension. c1 and c2 are usually set to the same value according to experience to balance the learning influence of individuals and groups.

[0066] It should be noted that, with reference to the accompanying Figure 5 , the determination unit 204 determines whether the first iteration stop condition is satisfied. If not, the iteration unit 205 will perform an update operation, i.e. update the velocity and position of each particle; the velocity update is based on the best position of the particle itself (individual optimal) and the best position of the particle group 1011 (global optimal), while the position update is based on the newly calculated velocity, and returns to the search unit 101 for iteration. By continuously adjusting the velocity and position of the particles, the iteration unit 205 guides the particle group 1011 to move in the search space, gradually approaching a better solution. This process continues until the determination unit 204 detects that the stop condition is met. At this time, the individual optimal and group optimal are updated, and the optimization process continues.

[0067] With reference to the accompanying Figure 5 , the output unit 206 is configured to output the first global fitness value as the optimal solution 1013 of the first performance indicator in response to the determination result of the determination unit 204 that the first iteration stop condition is satisfied.

[0068] With reference to the accompanying Figure 3 , the specific process of the PSO algorithm to optimize the particle group 1011 is as follows:

[0069] S3011: randomly initialize the particle swarm 1011 as the total search space with input data and hyperparameter ranges; S3012: evaluate the first fitness value according to the first fitness function; S3013: update the individual optimal and global optimal positions; S3014: update the speed and position of each particle; S3015: if the first iteration stop condition is met, proceed in sequence, if the first iteration stop condition is not met, return to S3013 and iterate; S3016: output the global optimal value; S3017: decode the corresponding parameter combination and use it as the elite population 2011.

[0070] It can be understood that, with reference to the accompanying drawings Figure 5 and the accompanying drawings Figure 5 As shown in the accompanying drawings, the execution module 20 uses the PSO algorithm to iteratively optimize the particle swarm 1011 to determine the optimal solution 1013 of the first performance indicator. During the optimization process, the model running unit 201 inputs the input data and hyperparameters corresponding to the particles into the neural network processor and runs the model, the data acquisition unit 202 acquires the first performance data, the evaluation unit 203 determines the fitness value according to the first performance data and the first fitness function, and the determination unit 204 determines whether the first iteration stop condition is met. If not, the iteration unit 205 updates the speed and position of the particles; if so, the output unit 206 outputs the global optimal value and analyzes the corresponding input data and hyperparameter combination to generate the elite population 2011. The elite population 2011 is merged into the initial population 2012 of the genetic algorithm as a source of high-quality genes. In this way, the convergence process of the genetic algorithm can be accelerated, the global search capability can be improved, and a better solution can be found. This merging method combines the advantages of the PSO algorithm and the genetic algorithm, improves the optimization efficiency and effect, reduces power consumption, and improves the performance and efficiency of neural network applications.

[0071] In the embodiments of the present application, as shown in the accompanying drawings Figure 5 As shown in the accompanying drawings, a framework diagram of a performance analysis device of a neural network processor, according to the accompanying drawings Figure 3 The following device structure is described. The execution module 20 is also configured to use the genetic algorithm to iteratively optimize the initial population 2012 to determine the optimal solution of the second performance indicator of the neural network processor when running the neural network model.

[0072] Wherein the initial population 2012 is named as initial population B and initial population C respectively. The genetic algorithm optimizes the two initial populations B and C iteratively, and the difference between the initial population B and the initial population C lies in the second performance index and the second fitness function during the optimization of the genetic algorithm. The second performance index of the initial population B includes the energy efficiency ratio, and the second performance index of the initial population C includes the balance degree of performance and operation intensity. The initial population B evaluates the energy efficiency ratio through the second fitness function, and the initial population C evaluates the optimal value through the second fitness function, that is, the balance degree of performance and operation intensity. In addition, the genetic algorithm has strong global search capability and can effectively explore the entire search space to avoid falling into a local optimal solution. The optimal solution of the second performance index is output to provide a detailed and accurate reference for the competitiveness analysis of the NPU.

[0073] In the embodiments of the present application, referring to Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 5 and Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 3 As shown in Figs. 1 and 2, after the execution module 20 determines the optimal solution 1013 of the first performance index by iteratively optimizing the particle swarm 1011 using the PSO algorithm, the parameter solution set corresponding to the optimal solution 1013 of the first performance index is decoded to generate an elite population 2011 (referred to as A). The genetic algorithm multiplexes and fixes the encoding of the elite population 2011 to the subsequent initial population B and initial population C, and adds it as part of the initial population 2012 to avoid completely randomizing the initial population 2012 to greatly shorten the convergence time.

[0074] In the embodiments of the present application, referring to Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 5 and Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 5 As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps.

[0075] As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 5 As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps.

[0076] As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 3 As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps.

[0077] As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. Figure 3 As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps. As shown in Figs. 1 and 2, the model running unit 201 is configured to perform the following steps.

[0078] Here, referring to FIG. 2, the model and data are pushed to the NPU to be tested, and then the runtime environment is initialized to load the model and allocate resources. Subsequently, the data is batched and input to the model for inference or training, fully exerting the parallel computing advantage of the NPU and improving the processing efficiency. Figure 3

[0079] Here, referring to FIG. 2, the model and data are pushed to the NPU to be tested, and then the runtime environment is initialized to load the model and allocate resources. Subsequently, the data is batched and input to the model for inference or training, fully exerting the parallel computing advantage of the NPU and improving the processing efficiency. Figure 3

[0080] Here, referring to FIG. 2, the model and data are pushed to the NPU to be tested, and then the runtime environment is initialized to load the model and allocate resources. Subsequently, the data is batched and input to the model for inference or training, fully exerting the parallel computing advantage of the NPU and improving the processing efficiency. Figure 5

[0081] The evaluation unit 203 is further configured to determine the second fitness value of each individual of the initial population 2012 and the second global fitness value of the initial population 2012 according to the second performance data and the second fitness function; wherein the second global fitness value is the second fitness value corresponding to the individual with the maximum fitness value in the initial population 2012.

[0082] Here, referring to FIG. 2, the model and data are pushed to the NPU to be tested, and then the runtime environment is initialized to load the model and allocate resources. Subsequently, the data is batched and input to the model for inference or training, fully exerting the parallel computing advantage of the NPU and improving the processing efficiency. Figure 5

[0083] The determination unit 204 is further configured to determine whether the second global fitness value and the number of iterations of the genetic algorithm satisfy a first iteration stop condition; the first iteration stop condition includes that the second global fitness value satisfies a third threshold condition and / or the number of iterations of the genetic algorithm satisfies a fourth threshold condition. Figure 5 The determination unit 204 is further configured to determine whether the second global fitness value and the number of iterations of the genetic algorithm satisfy a first iteration stop condition; the first iteration stop condition includes that the second global fitness value satisfies a third threshold condition and / or the number of iterations of the genetic algorithm satisfies a fourth threshold condition.​​​​

[0084] Refer to the attached Figure 5 As shown, the iteration unit 205 is further configured to perform selection, crossover and mutation operations on individuals in the initial population 2012 to generate a new generation population in response to the judgment result of the determination unit 204 that the first iteration stop condition is not met.

[0085] Here, refer to the attached Figure 5 As shown, the determination unit 204 determines whether the first iterative stopping condition is met. If not, the iteration unit 205 performs selection, crossover, and mutation operations on the individuals in the initial population 2012 to generate a new generation of population. This process continues until the determination unit 204 detects that the stopping condition is met. If so, the optimization process continues.

[0086] Refer to the attached Figure 7 As shown, the output unit 206 is further configured to output a second global fitness value as an optimal solution of the second performance indicator in response to the determination result of the determination unit 204 that the first iteration stop condition is satisfied.

[0087] Refer to the attached Figure 8 As shown, in the optimization process of the genetic algorithm, the second fitness function of the initial population B is shown in the following formula (4):

[0088]

[0089] Among them, Power Efficiency is the energy efficiency ratio, and power per second is the time rate at which the device consumes energy during operation. It is an important indicator for measuring the energy efficiency of the equipment.

[0090] Refer to the attached Figure 9 As shown, the second fitness function of the initial population C is shown in the following formula (5):

[0091]

[0092] Optimal is the optimal value, which represents the balance between performance and operation intensity. Memory usage refers to the amount of memory occupied by a program or device during operation. It is one of the important indicators for measuring the performance and resource consumption of a program or device.

[0093] Refer to the attached Figure 9 As shown, attached Figure 8 It is attached Figure 5Further explanation is given in the figure. Among them, Step 1: maximize workload parallelism; Step 2: maximize data flow parallelism; Step 3: actual available PE number under limited PE array size; Step 4: available PE number under fixed PE array size; Step 5: PE number limited by storage resources under fixed storage capacity; Step 6: PE utilization rate is reduced due to insufficient average memory bandwidth; Step 7: PE utilization rate is reduced due to insufficient memory bandwidth. Steps 1-7 intuitively show the change of processor performance under different restrictions. Specifically, Steps 1 and 2 show the performance in the ideal case, Steps 3 to 5 introduce hardware and storage restrictions, and the performance begins to decline; Steps 6 and 7 further reduce the performance due to insufficient memory bandwidth. Steps 1-7 help to understand the influence of different factors on the performance of the processor.

[0094] Referring to the accompanying drawings Figure 3 The specific process of optimizing the initial population B by the genetic algorithm is as follows:

[0095] S4011: merge the elite population 2011 and the first population 2012 to form the initial population B; S4012: evaluate the energy efficiency ratio by the second fitness function; S4013: perform selection, crossover and mutation operations; S4014: whether the second iteration stop condition is met, if the first iteration stop condition is met, sequentially perform, if the first iteration stop condition is not met, return to step S4013 and iterate; S4015: output the global optimal value and the corresponding parameter combination. The global optimal value refers to the value of the objective function at the global optimal solution.

[0096] The specific process of optimizing the initial population C by the genetic algorithm is as follows:

[0097] S5011: merge the elite population 2011 and the first population 2012 to form the initial population C; S5012: evaluate the optimal value by the second fitness function; S5013: perform selection, crossover and mutation operations; S5014: whether the second iteration stop condition is met, if the first iteration stop condition is met, sequentially perform, if the first iteration stop condition is not met, return to step S5013 and iterate; S5015: output the global optimal value and the corresponding parameter combination.

[0098] It can be understood that, referring to the accompanying drawings Figure 5 and the accompanying drawings Figure 5As shown, the optimal solution 1013 of the first performance index optimized by the PSO algorithm is merged into the initial population 2012 as the elite population 2011. The model running unit inputs the individuals of the initial population 2012 into the neural network processor and runs the model; the data acquisition unit obtains the performance data corresponding to each individual, including the model performance and power consumption, etc.; the evaluation unit calculates the fitness value of each individual according to these data and the second fitness function, and determines the global optimal solution; the determination unit determines whether the iteration stopping condition is met, if not, the iteration unit performs selection, crossover and mutation operations on the population to generate a new generation of population, and continues iteration optimization. If the condition is met, the output unit outputs the global optimal value and the corresponding parameter combination. In this way, the initial population 2012 contains excellent individuals of the elite population 2011, which have been screened by the particle swarm 1011 optimization and have a high fitness value, so that the excellent individuals in the initial population 2012 can guide the population to evolve towards the optimal solution more quickly, reducing the blindness of the search and shortening the iteration number, and the global optimal solution can be converged more quickly. Through the PSO algorithm, the optimal solution is quickly located around the region in the early stage, and then the global search ability is further optimized through the genetic algorithm. This combination strategy can ensure the convergence speed while improving the global search ability and avoiding premature convergence to a local optimal solution.

[0099] In the embodiments of the present application, reference is made to the accompanying drawings Figure 5 As shown, the iteration unit 205 is specifically configured to perform directional mutation operation on the individuals other than the excellent individuals in the initial population 2012, and randomly perform random mutation operation on part of the individuals in the initial population 2012; wherein the excellent individuals include the individuals whose second fitness value increases after the last iteration and the individuals in the elite population 2011 whose second fitness value is greater than a preset value.

[0100] It should be noted that reference is made to the accompanying drawings Figure 10 As shown, the mutation includes directional mutation and random mutation. The directional mutation refers to retaining part of the excellent DNA in the original initial population 2012 and other DNA fragments that lead to better fitness function evaluation after random mutation, only mutating the remaining DNA fragments, retaining the excellent DNA to accelerate convergence, and ensuring globality to avoid premature convergence; the random mutation is to reduce the possibility of premature convergence to a local optimal solution.

[0101] In some embodiments of the present application, the initialization module 10 is further configured to merge the elite population 2011 and the first population 1012, and add multiple human prior individuals as the initial population 2012.

[0102] It should be noted that artificial priors are based on past experience and knowledge, providing some potentially high-performing parameter combinations. These combinations are encoded as individuals and added to the initial population 2012, thereby guiding the search direction and enriching the population diversity. Furthermore, the use of artificial priors can help narrow the search space, reduce search costs, and improve search efficiency.

[0103] The present application also provides a method for analyzing the performance of a neural network processor. Figure 11 As shown, the method includes the following steps S1-S4:

[0104] S1: Generate a particle swarm 1011 of a particle swarm optimization algorithm and a first population 1012 of a genetic algorithm according to an input data range and a hyperparameter range of a neural network model;

[0105] The generation method of the initialization module 10 is independent of a specific NPU architecture or neural network model and is applicable to various mobile NPUs and different neural network architectures. For example, whether it is a convolutional neural network or a recurrent neural network, the initialization module 10 can be used to generate the corresponding population, providing a basis for subsequent performance analysis and optimization.

[0106] S2: Iteratively optimizing the particle swarm 1011 using a particle swarm optimization algorithm to determine an optimal solution 1013 for a first performance indicator of the neural network processor when running the neural network model;

[0107] The first performance metric is NPU utilization. The PSO algorithm enables the device to quickly find a solution that maximizes NPU utilization among numerous parameter combinations, reducing the time required to find the right parameter combination. By combining the PSO algorithm with a genetic algorithm, the global search capabilities of the genetic algorithm and the rapid convergence of the PSO algorithm can be fully utilized, resulting in a more efficient optimization method. This method can both quickly locate an approximate global optimal solution and approach the optimal solution through a detailed search.

[0108] S3: Generate an elite population 2011 according to the optimal solution 1013 of the first performance indicator, and merge the elite population 2011 and the first population 1012 to form the initial population 2012.

[0109] It's important to note that the merged initial population 2012 accelerates convergence and improves search efficiency during the genetic algorithm phase. Specifically, the PSO algorithm first identifies parameter combinations that maximize NPU utilization. These combinations are then added to the genetic algorithm's initial population 2012 as high-quality genes. This improves the overall quality of initial population 2012 and brings it closer to the global optimal solution, thereby reducing the number of iterations required to find the optimal solution. Furthermore, the merge operation increases the diversity of initial population 2012, helping to avoid local optimal solutions and improving global search capabilities.

[0110] S4: Iteratively optimize the initial population 2012 using a genetic algorithm to determine an optimal solution for the second performance indicator of the neural network processor when running the neural network model.

[0111] The initial populations 2012 are named initial population B and initial population C. The genetic algorithm performs optimization iterations on each of these two populations. The difference between initial populations B and C lies in their second performance indicator and second fitness function. Initial population B's second performance indicator includes energy efficiency, while initial population C's second performance indicator includes the balance between performance and operational intensity. Initial population B uses the second fitness function to evaluate energy efficiency, while initial population C uses the second fitness function to evaluate the optimal value, i.e., the balance between performance and operational intensity.

[0112] It should be noted that this method first generates a particle swarm 1011 and a first population 1012 based on the input data range and hyperparameter range of the neural network model. The PSO algorithm then rapidly determines the optimal solution for the first performance metric (e.g., NPU utilization). The elite population 2011 generated from this optimal solution is then merged with the first population 1012 from the genetic algorithm to form the initial population 2012. Finally, the genetic algorithm iteratively optimizes the two initial populations B and C, respectively, to obtain the optimal solution for the second performance metric for both initial populations B and C. This phased optimization strategy not only accelerates convergence but also effectively avoids the problem of local optimal solutions. The introduction of the elite population 2011 provides the genetic algorithm with a high-quality initial population 2012, guiding the search towards more optimal areas. Furthermore, the merging operation maintains the diversity of the initial population 2012, enhancing the algorithm's ability to explore uncharted territory. The combination of the PSO algorithm and the genetic algorithm supports parallel processing and can fully utilize the parallel computing capabilities of the NPU to accelerate the optimization process and reduce power consumption, so as to find the hyperparameter combination that can maximize the NPU utilization of the model, achieve the best energy efficiency ratio, and achieve the best performance in computing and operation intensity.

[0113] Figure 2 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 11 , refer to the attached Figure 12 As shown, the specific steps of S1 include the following steps S101-S102:

[0114] S101: Determine multiple first parameter combinations and multiple second parameter combinations according to an input data range and a hyperparameter range of a neural network model.

[0115] S102: Encode multiple first parameter combinations to generate a particle swarm 1011, and encode multiple second parameter combinations to generate a first swarm 1012; wherein the first parameter combinations correspond one-to-one to the particles in the particle swarm 1011; and the second parameter combinations correspond one-to-one to the individuals in the first swarm 1012.

[0116] It should be noted that different optimization algorithms have different requirements for parameter encoding. Since the positions and velocities of particles are updated in a continuous space, the PSO algorithm usually uses real number encoding. By generating a variety of parameter combinations, the search unit 101 ensures the diversity of the particle swarm 1011 and prevents the optimization process from falling into a local optimal solution.

[0117] Figure 3 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 12 , refer to the attached Figure 13 As shown, the specific steps of S2 include the following steps S201-S206:

[0118] S201: Input the input data and hyperparameters corresponding to each particle in the particle swarm 1011 into the neural network processor and run the neural network model.

[0119] Here, after the model and data are pushed to the NPU to be tested, the runtime environment is initialized to load the model and allocate resources; then, data is input into the model in batches for inference or training, giving full play to the parallel computing advantages of the NPU and improving processing efficiency.

[0120] S202: Obtain first performance data of the neural network processor when the input data and hyperparameters corresponding to each particle in the particle swarm 1011 are run on the neural network model.

[0121] The first performance data is the model performance (such as accuracy, recall, etc.). When the input data and hyperparameters corresponding to each particle in the particle swarm 1011 are fed into the neural network model during or after the operation, the data acquisition unit 202 obtains the first performance data of the model through an interface with the neural network processor or a monitoring tool, so that the evaluation unit 203 can calculate the first fitness value.

[0122] S203: Determine a first fitness value of each particle in the particle swarm 1011 and a first global fitness value of the particle swarm 1011 according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the largest fitness value in the particle swarm 1011.

[0123] The NPU utilization rate is taken as the first fitness function, and a first fitness value is evaluated through the first fitness function. The evaluation unit 203 executes the first fitness function in the search main program on the server side, and is responsible for processing and analyzing the collected data to evaluate the performance of the model. Specifically, the evaluation unit 203 obtains the first performance data of the model during runtime from the data collection unit 202, and then calculates the fitness value of each particle in the particle swarm 1011 according to the first fitness function, to measure the performance of each particle under the current parameter combination. The evaluation unit 203 also finds the optimal solution in the population as the direction and target of subsequent optimization, and feeds back the evaluation result to the determination unit 204.

[0124] S204: Determine whether the first global fitness value and the iteration number of the particle swarm optimization algorithm satisfy a first iteration stop condition; the first iteration stop condition includes that the first global fitness value satisfies a first threshold condition and / or the iteration number of the particle swarm optimization algorithm satisfies a second threshold condition.

[0125] S205: In response to not satisfying the first iteration stop condition, update the speed and position of each particle.

[0126] It should be noted that if the first iteration stop condition is not satisfied, the iteration unit 205 will perform an update operation, i.e., update the speed and position of each particle; the update of the speed is based on the best position of the particle itself (individual optimal) and the best position of the particle swarm 1011 (global optimal), and the update of the position is based on the newly calculated speed, and returns to the search unit 101 for iteration. By continuously adjusting the speed and position of the particles, the iteration unit 205 guides the particle swarm 1011 to move in the search space, and gradually approaches a better solution. This process continues until the determination unit 204 detects that the stop condition is satisfied. At this time, the individual optimal and the population optimal are updated, and the optimization process continues.

[0127] S206: In response to satisfying the first iteration stop condition, output the first global fitness value as the optimal solution 1013 of the first performance indicator.

[0128] It is understood that the execution module 20 uses the PSO algorithm to iteratively optimize the particle swarm 1011 and determine the optimal solution 1013 for the first performance indicator. During the optimization process, the model execution unit 201 inputs the input data and hyperparameters corresponding to the particles into the neural network processor and runs the model. The data acquisition unit 202 obtains the first performance data. The evaluation unit 203 determines the fitness value based on the first performance data and the first fitness function. The determination unit 204 determines whether the first iterative stopping condition is met. If not, the iteration unit 205 updates the particle velocity and position. If so, the output unit 206 outputs the global optimal value and analyzes the corresponding input and hyperparameter combination parameters to generate an elite population 2011. This elite population 2011 is merged into the initial population 212 of the genetic algorithm as a source of high-quality genes. This accelerates the convergence process of the genetic algorithm, improves global search capabilities, and facilitates the search for more optimal solutions. This combined approach combines the advantages of the PSO algorithm and the genetic algorithm, improving optimization efficiency and effectiveness, reducing power consumption, and enhancing the performance and efficiency of neural network applications.

[0129] Figure 4 A schematic diagram of a performance analysis method for a neural network processor provided in an embodiment of the present application Figure 13 , refer to the attached Figure 14 As shown, the specific steps of S3 include the following steps S301-S302:

[0130] S301 : Encode the input data and hyperparameters corresponding to the optimal solution 1013 of the first performance indicator to generate an elite population 2011 .

[0131] S302: Merge the elite population 2011 and the first population 1012 to form the initial population 2012.

[0132] In the embodiment of the present application, after the elite population 2011 and the first population 1012 are merged, multiple artificial prior individuals may be added to serve as the initial population 2012 .

[0133] It should be noted that artificial priors provide some potentially high-performing parameter combinations. These combinations are encoded as individuals and added to the initial population 2012, thereby guiding the search direction and enriching population diversity. Furthermore, the use of artificial priors can help narrow the search space, reduce search costs, and improve search efficiency. The encoded parameter combinations facilitate selection, crossover, and mutation operations in particle swarm optimization and genetic algorithms, enabling the search for global optimal solutions in complex search spaces. Binary encoding is initialized using the genetic algorithm, and crossover and mutation can be easily implemented using bitwise operations.

[0134] Figure 5A flowchart of a performance analysis method of a neural network processor provided by an embodiment of the present application Figure 14 , refer to the drawings ​ As shown in FIG. 4, the specific steps of S4 include the following steps S401-S406:

[0135] S401: input the input data and hyperparameters corresponding to each individual of the initial population 2012 into the neural network processor and run the neural network model.

[0136] Here, after the model and data are pushed to the NPU to be tested, the runtime environment is initialized to load the model and allocate resources, and then the data is batched into the model for inference or training, fully exerting the parallel computing advantage of the NPU and improving the processing efficiency.

[0137] S402: obtain the second performance data of the neural network processor when the input data and hyperparameters corresponding to each individual of the initial population 2012 are run on the neural network model.

[0138] Wherein, when the input data and hyperparameters corresponding to each particle in the particle swarm 1011 are sent into the neural network model for running, the data acquisition unit 202 obtains the model performance data through the interface or monitoring tool of the neural network processor. At the same time, the running power consumption of the neural network processor is monitored and recorded in real time by means of a power consumption instrument. Then, the data acquisition unit 202 integrates the model performance and power consumption data to form a comprehensive performance evaluation data set for the evaluation unit 203 to calculate the second fitness value.

[0139] S403: determine the second fitness value of each individual of the initial population 2012 and the second global fitness value of the initial population 2012 according to the second performance data and the second fitness function; wherein the second global fitness value is the second fitness value corresponding to the individual with the maximum fitness value in the initial population 2012.

[0140] It should be noted that the evaluation unit 203 executes the second fitness function in the search main program on the server side, which is responsible for processing and analyzing the collected data to evaluate the performance of the model. Specifically, the evaluation unit 203 obtains the second performance data and power consumption data of the model running from the data acquisition unit 202, and then calculates the fitness value of each particle in the particle swarm 1011 according to the second fitness function, which measures its performance under the current parameter combination. The evaluation unit 203 will also find the optimal solution in the group as the direction and target of subsequent optimization, and feed back the evaluation result to the iteration unit 205.

[0141] S404: determining whether the second global fitness value and the iteration number of the genetic algorithm satisfy a first iteration stop condition; the first iteration stop condition comprises that the second global fitness value satisfies a third threshold condition and / or the iteration number of the genetic algorithm satisfies a fourth threshold condition.

[0142] S405: in response to the first iteration stop condition not being satisfied, performing selection, crossover and mutation operations on the individuals in the initial population 2012 to generate a new generation population.

[0143] In the embodiments of the present application, the mutation operation on the individuals in the initial population 2012 comprises: performing a directional mutation operation on the individuals in the initial population 2012 except for excellent individuals, and performing a random mutation operation on part of the individuals in the initial population 2012 randomly; wherein the excellent individuals comprise individuals whose second fitness values increase after the last iteration and individuals in the elite population 2011 whose second fitness values are greater than a preset value.

[0144] Here, the determination unit 204 determines whether the first iteration stop condition is satisfied. If not, the iteration unit 205 performs selection, crossover and mutation operations on the individuals in the initial population 2012 to generate a new generation population. This process continues until the determination unit 204 detects that the stop condition is satisfied, and if so, the optimization process continues.

[0145] S406: in response to the first iteration stop condition being satisfied, outputting the second global fitness value as an optimal solution of the second performance indicator.

[0146] It should be noted that the optimal solution 1013 of the first performance index obtained by optimizing the PSO algorithm is taken as the elite population 2011 and merged into the initial population 2012. The model running unit inputs the individuals of the initial population 2012 to the neural network processor and runs the model; the data acquisition unit acquires the performance data corresponding to each individual, including model performance and power consumption; the evaluation unit calculates the fitness value of each individual according to these data and the second fitness function, and determines the global optimal solution; the determination unit determines whether the iteration stop condition is met, if not, the iteration unit performs selection, crossover and mutation operations on the population to generate a new generation of population, and continues iteration optimization. If the condition is met, the output unit outputs the global optimal value and the corresponding parameter combination. In this way, the initial population 2012 contains excellent individuals of the elite population 2011, which have been screened by the particle swarm 1011 optimization and have a high fitness value, so that the excellent individuals in the initial population 2012 can guide the population to evolve towards the optimal solution more quickly, reducing the blindness of the search and shortening the iteration number, and enabling the global optimal solution to be converged more quickly. Through the PSO algorithm, the optimal solution is quickly located around the region in the early stage, and then the global search capability of the genetic algorithm is further optimized. This combination strategy can ensure the convergence speed while improving the global search capability and avoiding premature convergence to a local optimal solution.

[0147] It should be understood that, in various embodiments of the embodiments of the present application, the size of the serial number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial number of the above embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments. It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0148] The serial number of the above embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments. The methods disclosed in the several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments. The features disclosed in the several product embodiments provided by the present application can be combined arbitrarily without conflict to obtain new product embodiments. The features disclosed in the several method or device embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

[0149] The above merely provides the specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the embodiments of the present application, which should be covered in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A performance analysis device for a neural network processor, the device comprising: an initialization module, configured to generate a particle swarm for a particle swarm optimization algorithm and a first population for a genetic algorithm according to an input data range and a hyperparameter range of a neural network model; an execution module configured to iteratively optimize the particle swarm using a particle swarm optimization algorithm to determine an optimal solution for a first performance indicator of the neural network processor when running the neural network model; The initialization module is further configured to generate an elite population according to the optimal solution of the first performance indicator, and merge the elite population and the first population as an initial population.

2. The apparatus according to claim 1, wherein the execution module is further configured to iteratively optimize the initial population using a genetic algorithm to determine an optimal solution for a second performance indicator of the neural network processor when running the neural network model.

3. The device according to claim 2, wherein the initialization module comprises: A search unit configured to determine a plurality of first parameter combinations and a plurality of second parameter combinations according to an input data range and a hyperparameter range of the neural network model; An encoding unit is configured to encode the multiple first parameter combinations to generate the particle swarm, and to encode the multiple second parameter combinations to generate the first population; wherein the first parameter combinations correspond one-to-one to the particles in the particle swarm; and the second parameter combinations correspond one-to-one to the individuals in the first population.

4. The apparatus according to claim 3, wherein the execution module comprises: a model running unit, configured to input input data and hyperparameters corresponding to each particle in the particle swarm into the neural network processor and run the neural network model; A data acquisition unit is configured to obtain input data corresponding to each particle in the particle swarm and first performance data of the neural network processor when the hyperparameters are run on the neural network model; an evaluation unit configured to determine a first fitness value of each particle in the particle swarm and a first global fitness value of the particle swarm based on the first performance data and a first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the largest fitness value in the particle swarm; a determining unit configured to determine whether the first global fitness value and the number of iterations of the particle optimization algorithm meet a first iteration stopping condition; an iterative unit configured to update the speed and position of each particle in response to a determination result of the determining unit that the first iteration stop condition is not satisfied; The output unit is configured to output the first global fitness value as an optimal solution of the first performance indicator in response to the judgment result of the judgment unit that the first iteration stopping condition is satisfied.

5. The apparatus according to claim 4, wherein the encoding unit is further configured to encode input data and hyperparameters corresponding to the optimal solution of the first performance indicator to generate the elite population; The model running unit is further configured to input the input data and hyperparameters corresponding to each individual of the initial population into the neural network processor and run the neural network model; The data acquisition unit is further configured to obtain input data corresponding to each individual of the initial population and second performance data of the neural network processor when the hyperparameters are run on the neural network model; The evaluation unit is further configured to determine a second fitness value of each individual in the initial population and a second global fitness value of the initial population according to the second performance data and the second fitness function; wherein, The second global fitness value is the second fitness value corresponding to the individual with the largest fitness value in the initial population; The determining unit is further configured to determine whether the second global fitness value and the number of iterations of the genetic algorithm meet a second iteration stopping condition; The iteration unit is further configured to, in response to the determination result of the determination unit being that the second iteration stopping condition is not satisfied, perform selection, crossover and mutation operations on the individuals in the initial population to generate a new generation population; The output unit is further configured to output the second global fitness value as an optimal solution of the second performance indicator in response to a determination result of the determination unit that a second iteration stop condition is satisfied.

6. The apparatus according to claim 5, wherein the iteration unit is specifically configured to perform a directed mutation operation on individuals other than the excellent individuals in the initial population, and to randomly perform a random mutation operation on some individuals in the initial population; wherein, The excellent individuals include individuals whose second fitness values ​​increase after the last iteration and individuals in the elite population whose second fitness values ​​are greater than a preset value.

7. A performance analysis method for a neural network processor, the method comprising: Generate a particle swarm of the particle optimization algorithm and a first swarm of the genetic algorithm according to the input data range and the hyperparameter range of the neural network model; Iteratively optimizing the particle swarm using a particle optimization algorithm to determine an optimal solution for a first performance indicator of the neural network processor when running the neural network model; generating an elite population according to the optimal solution of the first performance indicator, and merging the elite population and the first population as an initial population; The initial population is iteratively optimized using a genetic algorithm to determine an optimal solution for a second performance indicator of the neural network processor when running the neural network model.

8. The method according to claim 7, wherein generating a particle swarm of a particle optimization algorithm and a first population of a genetic algorithm according to an input data range and a hyperparameter range of a neural network model comprises: Determining a plurality of first parameter combinations and a plurality of second parameter combinations according to an input data range and a hyperparameter range of the neural network model; The multiple first parameter combinations are encoded to generate the particle swarm, and the multiple second parameter combinations are encoded to generate the first population; wherein the first parameter combinations correspond one-to-one to the particles in the particle swarm; and the second parameter combinations correspond one-to-one to the individuals in the first population.

9. The method according to claim 8, wherein the iterative optimization of the particle swarm using a particle optimization algorithm to determine an optimal solution for a first performance indicator of a neural network processor when running the neural network model comprises: Inputting the input data and hyperparameters corresponding to each particle in the particle swarm into the neural network processor and running the neural network model; Obtaining first performance data of the neural network processor when input data and hyperparameters corresponding to each particle in the particle swarm are run on the neural network model; determining a first fitness value of each particle in the particle swarm and a first global fitness value of the particle swarm according to the first performance data and the first fitness function; wherein the first global fitness value is the first fitness value corresponding to the particle with the largest fitness value in the particle swarm; Determining whether the first global fitness value and the number of iterations of the particle optimization algorithm meet a first iteration stopping condition; In response to the first iteration stop condition not being met, updating the velocity and position of each particle; In response to satisfying a first iteration stopping condition, the first global fitness value is output as an optimal solution of the first performance indicator.

10. The method according to claim 9, wherein generating an elite population based on the optimal solution of the first performance indicator comprises: Encoding input data and hyperparameters corresponding to the optimal solution of the first performance indicator to generate the elite population; The iterative optimization of the initial population using a genetic algorithm to determine an optimal solution for a second performance indicator of the neural network processor when running the neural network model includes: Inputting the input data and hyperparameters corresponding to each individual of the initial population into the neural network processor and running the neural network model; Obtaining second performance data of the neural network processor when input data and hyperparameters corresponding to each individual of the initial population are run on the neural network model; Determining a second fitness value of each individual in the initial population and a second global fitness value of the initial population according to the second performance data and the second fitness function; wherein the second global fitness value is the second fitness value corresponding to the individual with the maximum fitness value in the initial population; Determining whether the second global fitness value and the number of iterations of the genetic algorithm meet a second iteration stopping condition; In response to a second iteration stopping condition not being met, performing selection, crossover, and mutation operations on individuals in the initial population to generate a new generation population; In response to satisfying a second iteration stop condition, outputting the second global fitness value as an optimal solution for the second performance indicator.

11. The method according to claim 10, wherein the performing of a mutation operation on individuals in the initial population comprises: Directed mutation operations are performed on individuals other than the excellent individuals in the initial population, and random mutation operations are randomly performed on some individuals in the initial population; wherein the excellent individuals include individuals whose second fitness values ​​increase after the last iteration and individuals in the elite population whose second fitness values ​​are greater than a preset value. 12 . The method according to claim 7 , wherein the optimization complexity of the first performance indicator is lower than the optimization complexity of the second performance indicator.

Images