Energy efficiency ratio tuning method for neural network processor and related device

Abstract

The embodiment of the application provides a kind of energy efficiency ratio optimization method of neural network processor and related equipment, belong to computer processing technical field, the method includes: obtaining the matrix parameter set of neural network processor, and based on the matrix parameter set, determine the working energy efficiency ratio function of neural network processor;Based on the working energy efficiency ratio function and the matrix parameter set, construct the bayesian distribution model, wherein the bayesian distribution model includes the acquisition function for evaluating the quality of matrix parameter set;Bayesian optimization processing is carried out to matrix parameter set using acquisition function, to obtain target matrix parameter set;Based on target matrix parameter set, control neural network processor enters running state.The energy efficiency ratio of the application can improve NPU, reduce energy consumption.

Description

Technical Field

[0001] This application relates to the field of computer processing technology, and in particular to a method for optimizing the energy efficiency ratio of a neural network processor and related equipment. Background Technology

[0002] A Neural Processing Unit (NPU) is a processor specifically designed for neural network computations. Generally, intelligent computing clusters leverage the matrix multiplication computations within the NPU to perform parallel deep learning tasks, thereby maximizing the performance advantages of the intelligent computing cluster. Therefore, how to divide the matrices on the NPU into blocks and then combine and reuse these blocks becomes crucial for optimizing NPU matrix multiplication. Energy efficiency ratio (EER) is an important indicator of NPU energy consumption; a higher EER indicates that the NPU can perform more computational work with the same energy consumption.

[0003] In related technologies, to maximize the energy efficiency of the NPU and achieve optimal operation, a traversal search method is used to determine the target matrix parameters from multiple matrix parameters of the NPU matrix. However, the vast search space poses a significant challenge to determining the target matrix parameters. Consequently, the tuning methods employed in related technologies cannot effectively improve the NPU's energy efficiency, leading to excessive energy consumption by the NPU. Summary of the Invention

[0004] The main objective of this application is to propose a method and related equipment for optimizing the energy efficiency ratio of a neural network processor, aiming to improve the energy efficiency ratio of the NPU and reduce energy consumption.

[0005] To achieve the above objectives, a first aspect of this application proposes a method for optimizing the energy efficiency ratio of a neural network processor, the method comprising:

[0006] Obtain the set of matrix parameters of the neural network processor, and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters;

[0007] A Bayesian distribution model is constructed based on the working energy efficiency ratio function and the matrix parameter set. The Bayesian distribution model includes a sampling function for evaluating the quality of the matrix parameter set.

[0008] The target matrix parameter set is obtained by performing Bayesian optimization on the matrix parameter set using the acquisition function.

[0009] The neural network processor is controlled to enter the running state based on the target matrix parameter set.

[0010] In some embodiments, the neural network processor includes a target matrix for performing computational processing, the target matrix including a plurality of matrix elements;

[0011] Obtain the set of matrix parameters for the neural network processor, including:

[0012] Based on a preset first value range, at least one block factor is determined;

[0013] Based on the preset second value range, a cyclic expansion factor matching the block factor is determined, wherein the block factor is used to divide the target matrix into multiple smaller sub-matrices, and the cyclic expansion factor represents the merging calculation of multiple matrix elements in the sub-matrices;

[0014] Based on a preset third value range, the current operating frequency of the neural network processor is determined;

[0015] The set of matrix parameters is obtained based on the block factor, the loop expansion factor, and the current running frequency.

[0016] In some embodiments, the neural network processor is provided with multiple cache spaces of different levels;

[0017] After determining the energy efficiency ratio function of the neural network processor based on the set of matrix parameters, the following steps are also included:

[0018] The frequency of use of the submatrix is ​​predicted to obtain frequency level information, and the submatrix is ​​stored in a cache space that matches the frequency level information;

[0019] The neural network processor is controlled to enter the working state based on the matrix parameter set, so that the neural network processor can process the data in all cache spaces and obtain the computational amount and average power information per unit time.

[0020] Based on the ratio of computational information to average power information, the energy efficiency ratio information corresponding to the matrix parameter set is obtained.

[0021] In some embodiments, a Bayesian distribution model is constructed based on the working energy efficiency ratio function and the set of matrix parameters, including:

[0022] Based on multiple sets of matrix parameters and the working energy efficiency ratio function, the corresponding multiple energy efficiency ratio information is determined;

[0023] Based on multiple energy efficiency ratio information and the corresponding set of matrix parameters, a mean function to characterize the correlation between multiple energy efficiency ratio information and a covariance function to characterize the correlation between the set of matrix parameters and energy efficiency ratio information are constructed.

[0024] A Bayesian distribution model is constructed based on the mean function and the covariance function.

[0025] In some embodiments, the acquisition function includes a first acquisition function;

[0026] The first acquisition function is determined through the following steps, including:

[0027] The mean function is updated based on the preset cumulative distribution function to obtain the updated mean function;

[0028] The covariance function is updated based on the preset probability density function to obtain the first updated covariance function;

[0029] By superimposing the update mean function and the first update covariance function, a first acquisition function for evaluating the quality of the matrix parameter set is constructed.

[0030] In some embodiments, the acquisition function further includes a second acquisition function;

[0031] The second acquisition function is determined through the following steps, including:

[0032] The second updated covariance function is obtained by updating the covariance function based on the preset hyperparameters;

[0033] By superimposing the mean function and the second update covariance function, a second acquisition function for evaluating the quality of the matrix parameter set is constructed.

[0034] In some embodiments, the matrix parameter set is subjected to Bayesian optimization using an acquisition function to obtain a target matrix parameter set, including:

[0035] The acquisition function is optimized to obtain the set of update matrix parameters. Based on the set of update matrix parameters and the working energy efficiency ratio function, the updated energy efficiency ratio information is generated.

[0036] Calculate the energy efficiency ratio difference between the updated energy efficiency ratio information and the initial optimal energy efficiency ratio information, update the acquisition function based on the updated energy efficiency ratio information to obtain the updated acquisition function, and update the initial optimal energy efficiency ratio based on the updated energy efficiency ratio information;

[0037] The updated acquisition function is used as the new acquisition function to iteratively update the matrix parameter set until the energy efficiency ratio difference is less than the preset threshold difference. Then, the updated matrix parameter set is used as the target matrix parameter set.

[0038] To achieve the above objectives, a second aspect of this application provides an energy efficiency ratio optimization device for a neural network processor, the device comprising:

[0039] The acquisition module is used to acquire the set of matrix parameters of the neural network processor and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters.

[0040] The module is used to construct a Bayesian distribution model based on the working energy efficiency ratio function and the matrix parameter set. The Bayesian distribution model includes a sampling function for evaluating the quality of the matrix parameter set.

[0041] The Bayesian optimization module is used to perform Bayesian optimization on the set of matrix parameters using the acquisition function to obtain the target set of matrix parameters.

[0042] The target execution module is used to control the neural network processor to enter the running state based on the target matrix parameter set.

[0043] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method of the first aspect described above.

[0044] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of the first aspect described above.

[0045] This application proposes a method and related equipment for optimizing the energy efficiency ratio of a neural network processor. The method includes obtaining a set of matrix parameters for the neural network processor and determining its operating energy efficiency ratio function based on this set. A Bayesian distribution model is constructed based on the operating energy efficiency ratio function and the set of matrix parameters. This Bayesian distribution model includes a sampling function used to evaluate the quality of the set of matrix parameters. The optimization device can clearly define the sampling direction from multiple sets of matrix parameters according to the sampling function, avoiding energy waste caused by blindly sampling the set of matrix parameters. Next, the set of matrix parameters is optimized using the sampling function to obtain a target set of matrix parameters, thereby selecting the set with the most sampling value from other sets of matrix parameters for which energy efficiency ratio information has not yet been calculated. Finally, the neural network processor is controlled to enter the running state based on the target set of matrix parameters. In this way, the neural network processor can efficiently process the current task while improving its energy efficiency ratio and reducing energy consumption. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of an optional application scenario for the energy efficiency ratio optimization device for the neural network processor provided in this application embodiment;

[0047] Figure 2 This is an optional flowchart of the energy efficiency ratio optimization method for neural network processors provided in the embodiments of this application;

[0048] Figure 3 yes Figure 2 An optional implementation flowchart for step 101;

[0049] Figure 4 yes Figure 2An optional implementation flowchart following step 101;

[0050] Figure 5 yes Figure 2 An optional implementation flowchart for step 102;

[0051] Figure 6 This is another optional implementation flowchart of the energy efficiency ratio optimization device for the neural network processor provided in the embodiments of this application;

[0052] Figure 7 This is another optional implementation flowchart of the energy efficiency ratio optimization device for the neural network processor provided in the embodiments of this application;

[0053] Figure 8 yes Figure 2 An optional implementation flowchart for step 103;

[0054] Figure 9 This is an optional flowchart of the energy efficiency ratio optimization device for the neural network processor provided in the embodiments of this application;

[0055] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0059] First, let's analyze some of the terms used in this application:

[0060] Artificial Neural Networks (ANNs) are abstractions of the human brain's neuronal network from an information processing perspective. They establish simple models and create different networks with varying connection methods. In engineering and academia, they are often simply referred to as neural networks or neural network-like systems. A neural network is a computational model composed of numerous interconnected nodes (or neurons). Each node represents a specific output function, called an activation function; the connection between any two nodes represents a weighted value for the signal passing through that connection, called a weight. This is analogous to the memory of an artificial neural network.

[0061] Neural network tasks mainly include pattern recognition, data prediction, system control, feature importance identification, regression tasks, and classification tasks. Because neural network tasks involve constraints and interactions related to large-scale parameters, nonlinear mappings, training data, and hardware resources, their complexity is higher than other tasks.

[0062] Next, the background technology related to the embodiments of this application will be described.

[0063] An NPU is a processor specifically designed for neural network computations. Generally, intelligent computing clusters leverage the matrix multiplication computations within the NPU to perform parallel deep learning tasks, thereby helping the intelligent computing clusters to realize their performance advantages. Consequently, how to divide the matrices on the NPU into blocks and combine and reuse these blocks becomes the key to optimizing NPU matrix multiplication.

[0064] Data centers and computing clusters, as core infrastructures for the current information technology industry, bear the enormous data processing demands of modern society and have become a crucial support for the global digital economy. Especially with the rapid advancements in artificial intelligence technology, intelligent computing clusters (also known as "intelligent computing clusters") are gradually becoming a key driver of AI research and industrial development. Intelligent computing clusters are built upon advanced intelligent computing processors such as Graphics Processing Units (GPUs) and NPUs, which are capable of efficiently handling complex AI tasks and large-scale data computation needs.

[0065] Furthermore, with the rapid development of artificial intelligence technology, the architecture of current intelligent computing clusters has undergone significant changes compared to traditional data centers and supercomputer clusters. In traditional clusters, computing tasks primarily rely on the general-purpose computing power provided by the Central Processing Unit (CPU), thus the CPU, as the core of the node, is responsible for processing various types of tasks. However, the CPU's versatility across a wide range of tasks means that the CPU in intelligent computing clusters has relatively limited ability to handle parallel tasks, especially when facing highly parallel computing tasks such as deep learning and large-scale image processing, where efficiency is low.

[0066] In comparison, the NPU, as a processor specifically designed for artificial intelligence tasks, can process a large number of matrix operations and data flow calculations in neural networks with extremely high efficiency. Therefore, it is widely used in intelligent computing clusters and can better leverage its performance advantages when performing deep learning tasks, significantly improving the intelligent computing power of intelligent computing clusters.

[0067] In other words, in current intelligent computing clusters, the NPU (Neural Processing Unit) has become the basic unit providing intelligent computing power, supporting the large-scale artificial intelligence computing needs. A large intelligent computing cluster may contain tens of thousands of NPUs. The existence of these NPUs gives the intelligent computing cluster super-powerful parallel computing capabilities, enabling it to efficiently handle artificial intelligence tasks, such as the training and inference of deep learning models. Among these, energy efficiency ratio is an important indicator for measuring the energy consumption of an NPU. A higher energy efficiency ratio means that, with the same energy consumption, the NPU can complete more computational work.

[0068] In related technologies, to maximize the energy efficiency of the NPU and achieve optimal operation, a traversal search method is used to determine the target matrix parameter set from multiple matrix parameters of the NPU matrix. However, the enormous search space poses a significant challenge to determining the target matrix parameter set. Consequently, the tuning methods employed in related technologies cannot effectively improve the NPU's energy efficiency, leading to excessive energy consumption by the NPU.

[0069] Specifically, taking some of the world's top intelligent computing clusters as examples, the peak power consumption of the US Frontier supercomputer system reaches 22.7 megawatts (MW); the peak power consumption of the Aurora supercomputer system reaches 24.7 MW; and China's Sunway Ocean Light cluster has a peak power consumption in the same order of magnitude. According to calculations, a cluster with a power consumption of 20 MW would have an annual electricity cost exceeding 100 million yuan, corresponding to carbon dioxide emissions exceeding 70,000 tons. These figures reflect that while intelligent computing clusters provide powerful computing capabilities, they also bring enormous energy and environmental challenges.

[0070] Based on this, embodiments of this application provide a method and related equipment for optimizing the energy efficiency ratio of a neural network processor, aiming to improve the energy efficiency ratio of the NPU and reduce energy consumption.

[0071] The energy efficiency ratio optimization method and related equipment for neural network processors provided in this application are specifically described through the following embodiments. First, it should be noted that the energy efficiency ratio optimization method for neural network processors in this application relates to the field of computer processing technology. It can be applied to terminals or servers, as long as the deployed system includes a neural network processor and requires optimization of its energy efficiency ratio. In some embodiments, the terminal can be a smartphone, tablet computer, laptop computer, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms; these are merely illustrative examples and are not limited to the above forms.

[0072] It should also be noted that in this application embodiment, when it involves information related to user characteristics such as basic user information or user identity, the user's permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when this application embodiment needs to obtain sensitive personal information of a user, the user's individual permission or consent will be obtained first. Only after obtaining the user's individual permission or consent will the necessary data for the normal operation of this application embodiment be obtained. For example, before using the neural network processor energy efficiency ratio tuning device (hereinafter referred to as "tuning device") proposed in this application embodiment to determine the target matrix parameter set of the NPU using the neural network processor energy efficiency ratio tuning method (hereinafter referred to as "tuning method") proposed in this application embodiment, the consent of the personnel managing the relevant servers will be obtained; otherwise, the matrix parameter set used to determine the target matrix parameters cannot be obtained. Furthermore, all relevant data obtained in this application embodiment is authorized data after obtaining consent, and will not be elaborated upon here.

[0073] like Figure 1 As shown, Figure 1 This is a schematic diagram of an optional application scenario for the energy efficiency ratio optimization device for the neural network processor provided in this application embodiment. Figure 1 In the example, the optimization method proposed in this application embodiment can be applied to a server cluster. Figure 1The server cluster includes multiple servers, such as server a, server b, and server c (this is just an example and does not represent a limitation on the number of servers). Assuming a specific intelligent computing task is being processed, each server deploys an NPU, and the relevant parameters (matrix parameter set) affecting NPU operation need to be adjusted. To minimize energy consumption while the NPU enters a working state, the matrix parameter set is input into the tuning device proposed in this embodiment. The tuning device, using the tuning method proposed in this embodiment, can determine and output the target matrix parameter set that optimizes the NPU's energy efficiency without trying all possible matrix parameter sets. This target matrix parameter set is then applied to the NPU. It is understood that the energy consumption of the NPU entering a working state based on the target matrix parameter set is the lowest among all feasible decision solutions for the matrix parameter set.

[0074] After understanding the example application scenarios of the tuning device proposed in this application, the tuning method proposed in the embodiments of this application will be described next.

[0075] In this application embodiment, the description will be from the perspective of the tuning device, such as Figure 2 As shown, Figure 2 This is an optional flowchart of the energy efficiency ratio optimization method for neural network processors provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, the following steps 101 to 104. When the tuning device executes the tuning method, the specific process is as follows. It should be noted that this embodiment... Figure 2 The order of steps 101 to 104 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0076] Step 101: Obtain the set of matrix parameters of the neural network processor, and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters.

[0077] Step 101 will be described in detail below.

[0078] The Neural Processing Unit (NPU) is a hardware processor specifically designed to accelerate neural network computing tasks. Compared to traditional Central Processing Units (CPUs) and Graphics Processing Units (GPUs), NPUs provide servers with highly efficient neural network computing support. In practical applications, to solve neural network processing tasks, intelligent computing clusters built on multiple servers are needed to handle more complex neural network processing tasks. That is, the NPU can be deployed in a single server; or, the NPU can be deployed across multiple servers in the intelligent computing cluster; or, when the intelligent computing cluster also includes smart devices such as smart cameras and smart home devices, the NPU can also be deployed in these smart devices. This application does not limit the actual deployment scenario of the NPU. Regardless of the scenario, the optimization method proposed in this application can be used to optimize the NPU's energy efficiency ratio, thereby improving the NPU's energy efficiency ratio and reducing its energy consumption while completing neural network tasks.

[0079] Furthermore, one of the core functions of the NPU is to efficiently perform matrix multiplication operations. Matrix multiplication is the most commonly used basic operator in deep learning computation, widely applied in operations such as forward propagation, backward propagation, and weight updates in neural networks. For example, performing matrix multiplication in the NPU is represented as C = AB, where matrix A is an M×K matrix, matrix B is a K×N matrix, and matrix C is an M×N matrix. By performing matrix multiplication operations, the NPU can complete a series of complex neural network tasks.

[0080] Furthermore, compared to traditional NPU triple cyclic matrix multiplication, when the matrix dimension is quite large, partitioning the matrix can significantly improve computational efficiency. In other words, when processing a specific target neural network task C, determining how to divide task C into multiple smaller submatrices c = ab and how to define the computational mode for each submatrix is ​​crucial for improving the energy efficiency of the NPU.

[0081] The following will explain in detail how to optimize the energy efficiency ratio of the NPU, and the beneficial effects of the optimization method proposed in the embodiments of this application will gradually become apparent during the description.

[0082] In some embodiments, such as Figure 3 As shown, Figure 3 yes Figure 2 An optional implementation flowchart in step 101, obtaining the matrix parameter set of the neural network processor, includes the following steps 201 to 204:

[0083] Step 201: Determine at least one block factor based on a preset first value range.

[0084] Step 202: Based on the preset second value range, determine the cyclic expansion factor that matches the block factor, wherein the block factor is used to divide the target matrix into multiple smaller sub-matrices, and the cyclic expansion factor represents the merging calculation of multiple matrix elements in the sub-matrices.

[0085] Step 203: Determine the current operating frequency of the neural network processor based on the preset third value range.

[0086] Step 204: Obtain the matrix parameter set based on the block factor, the loop expansion factor, and the current running frequency.

[0087] Steps 201 to 204 are described in detail below.

[0088] In some embodiments, the neural network processor includes a target matrix for performing computational processing, the target matrix comprising multiple matrix elements. The target matrix, used to perform computational processing in the neural network, is the core part of the NPU's computation. The target matrix consists of multiple matrix elements, which can be numbers, vectors, or other data types, depending on the specific application and requirements of the neural network. The arrangement and combination of matrix elements in the target matrix determine the type and precision of computations that the NPU can perform. Furthermore, the NPU performs computational processing on the matrix elements of the target matrix using specific operational rules (such as matrix multiplication, addition, convolution, etc.) to process and transform data in the neural network task.

[0089] The matrix parameter set represents the sum of NPU-related parameters that need to be adjusted. This set includes the block factor, loop unrolling factor, and current running frequency. The block factor is a parameter used to divide the target matrix into multiple smaller sub-matrices. For example, for the target matrix C, the block factors include m, n, and k. The target matrix can be divided into multiple sub-matrices c = ab based on the block factors, where a = m × k and b = k × n. The size of the block factor directly affects the size and number of the resulting sub-matrices.

[0090] Furthermore, the cyclic expansion factor characterizes the magnitude of the combined computation of multiple matrix elements in a submatrix, thereby improving computational efficiency through cyclic expansion. For example, assuming a cyclic expansion factor of 1, this means that in each iteration, the NPU will only compute one matrix element from a submatrix, resulting in a serialized computation process and low efficiency. Conversely, assuming a cyclic expansion factor of 4, the NPU will compute all four matrix elements from a submatrix simultaneously in each iteration, thus reducing the number of iterations and improving the NPU's computational efficiency.

[0091] Furthermore, the current operating frequency of a neural network processor refers to the clock rate at which the processor executes tasks. It determines the number of instruction cycles the processor can execute per second, and is typically expressed in Hertz (Hz). A higher current operating frequency allows the NPU to perform more computational operations per unit of time, meaning that a higher-frequency NPU can process more data in the same amount of time. However, this also leads to increased power consumption; that is, an increase in NPU frequency is directly proportional to an increase in power consumption. Therefore, setting an appropriate frequency for the NPU to ensure that it can efficiently process neural network tasks while reducing energy consumption is crucial.

[0092] Furthermore, the block factor, cycle unrolling factor, and current operating frequency interact, thus jointly affecting the computational efficiency and energy consumption of NPU matrix multiplication. Based on this, the block factor, cycle unrolling factor, and current operating frequency are considered as factors influencing NPU operating efficiency and energy efficiency ratio, resulting in a set of matrix parameters. For example, when the block factor includes m, n, and k, and the cycle unrolling factor includes U... m U n U k When the NPU's current operating frequency is f, the matrix parameter set is (m, n, k, U). m U n U k f).

[0093] It should be noted that the number of cyclic expansion factors is usually matched with the number of block factors. The specific number of block factors can be set according to the actual situation. The embodiments of this application are only for illustrative purposes and do not imply any limitation on this.

[0094] Furthermore, to determine the set of matrix parameters subsequently input to the Bayesian distribution model, the first range of predefined values ​​for the block factor, the second range of values ​​for the cyclic expansion factor, and the third range of values ​​for the current running frequency are first obtained. Generally, due to the limitation of the NPU cache size, the values ​​of m, n, and k are usually in the integer range of [1, 128], U m U n U k Typically, within the range of [1, 16], the current operating frequency f of the NPU has multiple adjustable options, usually more than 10 selectable frequencies, with the specific value range determined by the specific NPU model. The first value range, the second value range, and the third value range can all be set according to actual conditions, and this application embodiment does not impose any restrictions on this.

[0095] In some embodiments, such as Figure 4 As shown, Figure 4 yes Figure 2An optional implementation flowchart following step 101, after determining the operating energy efficiency ratio function of the neural network processor based on the matrix parameter set, further includes steps 301 to 303:

[0096] Step 301: Predict the frequency of use of the submatrix to obtain frequency level information, and store the submatrix in a cache space that matches the frequency level information.

[0097] Step 302: Based on the matrix parameter set, control the neural network processor to enter the working state so that the neural network processor can process the data in all cache spaces to obtain the computational load information and average power information per unit time.

[0098] Step 303: Based on the ratio of computational information to average power information, obtain the energy efficiency ratio information corresponding to the matrix parameter set.

[0099] Steps 301 to 303 are described in detail below.

[0100] In some embodiments, the operating energy efficiency ratio function of the neural network processor is determined based on the set of matrix parameters, as shown in Equation <1> below:

[0101]

[0102] Here, Performance represents the computational performance of the NPU, defined as the number of matrix multiplications performed per unit time, and is therefore equal to the reciprocal of the time taken to complete one matrix multiplication. Power represents the average power consumed by the NPU during matrix multiplication. Both Performance and Power can be calculated in real time by the server hosting the NPU or other related devices.

[0103] In some embodiments, the neural network processor has multiple cache spaces of different levels, and the processing efficiency and storage capacity of the different cache spaces are usually different. Specifically, the NPU usually has a multi-level cache structure, including a first-level cache (L1 cache), a second-level cache (L2 cache), and global shared memory. Each cache layer has its specific capacity and access speed differences. In order to improve the energy efficiency of matrix multiplication operations, when performing block matrix multiplication, the size of each block of matrix data must be reasonably planned to ensure that it can be loaded into the cache exactly and avoid the performance loss caused by cache overflow.

[0104] Furthermore, the L1 cache is typically the cache closest to the computation unit, offering extremely low latency but with a small capacity. Therefore, the L1 cache usually stores the data most frequently processed by the NPU, such as the matrix block currently being processed. The L2 cache has a relatively large capacity but is slower to access than the L1 cache. The L2 cache is typically used to store large blocks of data that have already been loaded, avoiding frequent access to global memory. Therefore, optimization of NPU block matrix multiplication often involves storing larger data blocks in the L2 cache. Global shared memory is the slowest part of the NPU storage system and is typically used to store complete matrix data. Due to its high access cost, when optimizing matrix multiplication, frequent access to global memory should be minimized, relying more on data in the L1 and L2 caches.

[0105] Furthermore, after dividing the target matrix into multiple submatrices using different partitioning factors, the usage frequency of each submatrix is ​​first predicted to obtain frequency level information. This determines the required cache space for the submatrixes to facilitate subsequent data processing by the NPU. The usage frequency of a submatrix can be determined by analyzing historical data, matrix usage patterns, or through specific prediction algorithms.

[0106] Furthermore, the frequency level information is matched with different cache spaces. For example, the frequency level information includes a first-level frequency, a second-level frequency, and a third-level frequency. The first-level frequency matches the L1 cache, the second-level frequency matches the L2 cache, and the third-level frequency matches the global shared memory. Of course, the specific number of frequency level information and its matching relationship with different cache spaces can be adaptively adjusted according to the actual situation, and this application embodiment does not impose any limitations on this.

[0107] Furthermore, the neural network processor is controlled to enter the working state based on the matrix parameter set, so that the neural network processor processes the data in all cache spaces to obtain computational information representing the specific value of Performance and average power information representing the specific value of Power, thereby determining the energy efficiency ratio information under the current matrix parameter set.

[0108] Furthermore, to reduce the calculation error of the energy efficiency ratio information, the average calculated quantity information and average power information are often obtained by repeated measurements. The number of repeated measurements can be set according to the actual situation, usually more than 5 times, but the embodiments of this application do not limit this.

[0109] For example, when the set of matrix parameters is (m1, n1, k1, U) m1 U n1 U k1 When f1), the NPU enters the running state and obtains the energy efficiency ratio information EER1; when the matrix parameter set is (m2, n2, k2, U ....m2 U n2 U k2 When f1 and f2 are in operation, the NPU enters the running state and obtains the energy efficiency ratio information EER2. EER1 and EER2 are usually different depending on the set of matrix parameters.

[0110] Step 102: Construct a Bayesian distribution model based on the working energy efficiency ratio function and the set of matrix parameters, wherein the Bayesian distribution model includes a sampling function for evaluating the quality of the set of matrix parameters.

[0111] Step 102 is described in detail below.

[0112] In some embodiments, since the relationship between the NPU's energy efficiency ratio information and different sets of matrix parameters cannot be described by a simple analytical formula, this application embodiment constructs a Bayesian distribution model based on Bayesian optimization to characterize the distribution relationship between the set of matrix parameters and the NPU's energy efficiency ratio. This Bayesian distribution model includes a sampling function for evaluating the quality of the set of matrix parameters. The tuning device can determine the sampling direction from multiple sets of matrix parameters according to the sampling function, avoiding energy waste caused by blindly sampling the set of matrix parameters.

[0113] In some embodiments, such as Figure 5 As shown, Figure 5 yes Figure 2 An optional implementation flowchart in step 102, which constructs a Bayesian distribution model based on the working energy efficiency ratio function and the set of matrix parameters, includes the following steps 401 to 403:

[0114] Step 401: Based on multiple sets of matrix parameters and the working energy efficiency ratio function, determine the corresponding multiple energy efficiency ratio information.

[0115] Step 402: Based on multiple energy efficiency ratio information and the corresponding set of matrix parameters, construct a mean function to characterize the correlation between multiple energy efficiency ratio information and a covariance function to characterize the correlation between the set of matrix parameters and the energy efficiency ratio information.

[0116] Step 403: Construct a Bayesian distribution model based on the mean function and covariance function.

[0117] Steps 401 to 403 are described in detail below.

[0118] In some embodiments, a Bayesian distribution model is constructed to approximate the true objective function. The Gaussian process (GP) is a commonly used surrogate model in Bayesian optimization, which describes the probability distribution of the objective function through the mean function μ(x) and k(x, x′), as shown in Equation <2> below:

[0119] EER(x)~GP(μ(x),k(x,x))<2>

[0120] Where x = (m, n, k, U) m U n U k f) is the set of input matrix parameters; GP represents a Gaussian process; μ(x) = E[EER(x)] is the mean function characterizing the correlation between multiple energy efficiency ratio information; k(x, x′) = E[(EER(x)-μ(x))(EER(x′)-μ(x′))] is the covariance function characterizing the correlation between the set of matrix parameters and energy efficiency ratio information; E represents the expected value.

[0121] It should be noted that random forests, neural networks, etc., can also be selected as surrogate models in the Bayesian optimization process. The specific choice can be made according to the actual situation, and this application does not limit this. The Bayesian distribution model constructed based on the existing energy efficiency ratio information and the corresponding matrix parameter set can capture the complex interactions between multiple different types of processors, and thus take into account the impact of these interactions on NPU energy consumption. This helps to reduce the exploration of unknown matrix parameter sets in subsequent sampling processes, thereby reducing computational costs and unnecessary energy waste.

[0122] In some embodiments, such as Figure 6 As shown, Figure 6 This is another optional implementation flowchart of the energy efficiency ratio tuning device for the neural network processor provided in this application embodiment. The acquisition function includes a first acquisition function, which is determined through the following steps 501 to 503:

[0123] Step 501: Update the mean function based on the preset cumulative distribution function to obtain the updated mean function.

[0124] Step 502: Update the covariance function based on the preset probability density function to obtain the first updated covariance function.

[0125] Step 503: Overlay the update mean function and the first update covariance function to construct the first acquisition function for evaluating the quality of the matrix parameter set.

[0126] Steps 501 to 503 are described in detail below.

[0127] In some embodiments, the NPU can obtain different energy efficiency ratio (EER) information by entering the operating state based on different sets of matrix parameters. It determines the current optimal EER information from multiple existing EER information sets, typically selecting the one with the largest EER information as the current optimal EER information. Next, the first acquisition function, as shown in equation <3>, is constructed, representing the expected improvement (EI):

[0128] EI(x)=E[max(0,EER(x)-EER best )]

[0129] = (μ(x)-EER) best )Φ(Z)+σ(x)φ(Z) <3>

[0130] Among them, EER best This represents the current optimal energy efficiency ratio; E represents the expected value; Z = (μ(x) - EER) best μ(x) and σ(x) are the predicted mean and standard deviation of the Gaussian process at x, respectively. The predicted mean is calculated by the mean function, and the standard deviation is calculated by taking the square root of the covariance function. Φ(Z) is the preset cumulative distribution function (CDF), (μ(x)-EER) / σ(x); best Φ(Z) is the update mean function; φ(Z) is the preset probability density function (PDF); and σ(x)φ(Z) is the first update covariance function.

[0131] In some embodiments, such as Figure 7 As shown, Figure 7 This is another optional implementation flowchart of the energy efficiency ratio optimization device for the neural network processor provided in this application embodiment. The acquisition function further includes a second acquisition function, which is determined through the following steps 601 to 602:

[0132] Step 601: Update the covariance function based on the preset hyperparameters to obtain the second updated covariance function.

[0133] Step 602: Superimpose the mean function and the second update covariance function to construct a second acquisition function for evaluating the quality of the matrix parameter set.

[0134] Steps 601 to 602 are described in detail below.

[0135] In some embodiments, the following formula is constructed. <4> The second acquisition function shown is the upper confidence bound (UCB):

[0136] UCB(X) = μ(x) + k·σ(x) <4>

[0137] Where μ(x) and σ(x) are the predicted mean and standard deviation of the Gaussian process at x, respectively; k·σ(x) is the second update covariance function; k is a hyperparameter used to control the balance between exploration and exploitation. The common value range of k is between 1 and 10. In practical applications, a smaller value of k can be used for a smoother objective function, while a larger value of k is usually chosen to enhance exploration for a highly uncertain or complex objective function. That is, the specific value of k can be set according to the actual situation, and the embodiments of this application do not limit this.

[0138] Step 103: Use the acquisition function to perform Bayesian optimization on the matrix parameter set to obtain the target matrix parameter set.

[0139] Step 103 will be described in detail below.

[0140] Furthermore, compared to the traditional method of debugging multiple matrix parameter sets one by one, the embodiments of this application can use the acquisition function to perform Bayesian optimization processing on the matrix parameter set, and determine the sampling direction from multiple matrix parameter sets based on the Bayesian distribution model, and then sample other matrix parameter sets to achieve the purpose of selecting the most valuable sample from other matrix parameter sets that have not yet calculated energy efficiency ratio information, thereby reducing the repeated debugging of parameters and energy consumption.

[0141] When the sampling function is the first sampling function, the set of matrix parameters for the next sampling is selected by maximizing EI, i.e., the set of matrix parameters for the next sampling is x. * =argmaxEI(x). According to the formula for calculating EI, when μ(x) > EER... best When σ(x) is large, the EI value is relatively large, indicating that there is a high expectation of improvement at that point; while when σ(x) is large, even if μ(x) is not high, the EI may still be large, which can encourage the distribution model to explore regions with high uncertainty. In this way, the exploration (understanding unknown regions) and utilization (fine search in known good regions) are balanced, and the sampling accuracy is improved.

[0142] When the sampling function is the second sampling function, the next sampling point is selected by maximizing UCB(x), that is, x*=argmaxUCB(x).

[0143] Further, input the matrix parameter set into the Bayesian distribution model to obtain the corresponding energy efficiency ratio information. Select the optimal one as the target energy efficiency ratio information from the currently existing energy efficiency ratio information and the energy efficiency ratio information corresponding to the newly sampled matrix parameter set. Then, use the matrix parameter set corresponding to the target energy efficiency ratio information as the target matrix parameter set, and adjust the matrix parameters of each processor according to the target matrix parameter set, thereby completing the optimization.

[0144] Step 104, based on the target matrix parameter set, control the neural network processor to enter the operating state.

[0145] The following provides a detailed description of step 104.

[0146] In some embodiments, the existing matrix parameter set and the corresponding energy efficiency ratio information are (m1, n1, k1, U m1 , U n1 , U k1 , f1, EER1), (m2, n2, k2, U m2 , U n2 , U k2 , f2, EER2), the newly sampled matrix parameter set and the corresponding energy efficiency ratio information are (m3, n3, k3, U m3 , U n3 , U k3 , f3, EER3), and EER1 < EER2 < EER3. Then, determine EER3 as the optimal energy efficiency ratio information, and adjust the relevant parameters of the NPU according to (m3, n3, k3, U m3 , U n3 , U k3 , f3). It can be understood that in this state, the NPU has the best energy efficiency ratio and the least energy consumption.

[0147] It can be understood that the optimization method proposed in the embodiments of this application can not only effectively avoid the performance bottleneck caused by improper single computing unit matrix, but also more precisely balance the relationship between the computing speed and energy consumption, thereby achieving more efficient NPU energy efficiency ratio optimization.

[0148] In some embodiments, as Figure 8 shown, Figure 8 is Figure 2 an optional implementation flowchart of step 103 in

[0149] Step 701, perform optimization calculation on the acquisition function to obtain an updated matrix parameter set, and generate updated energy efficiency ratio information based on the updated matrix parameter set and the working energy efficiency ratio function.

[0150] Step 702: Calculate the energy efficiency ratio difference between the updated energy efficiency ratio information and the initial optimal energy efficiency ratio information, update the acquisition function based on the updated energy efficiency ratio information to obtain the updated acquisition function, and update the initial optimal energy efficiency ratio based on the updated energy efficiency ratio information.

[0151] Step 703: The updated acquisition function is used as the new acquisition function to iteratively update the matrix parameter set until the energy efficiency ratio difference is less than the preset threshold difference, and then the updated matrix parameter set is used as the target matrix parameter set.

[0152] Steps 701 to 703 are described in detail below.

[0153] In some embodiments, after sampling the matrix parameter set once, the Bayesian distribution model can update the model based on the original matrix parameter set and the corresponding energy efficiency ratio information, as well as the newly sampled matrix parameter set and the corresponding energy efficiency ratio information, and resample other matrix parameter sets whose energy efficiency ratio information is unknown. By repeatedly iterating the model update and resampling steps, the optimal solution is gradually approximated, and the accuracy of the final target matrix parameter set is ensured.

[0154] The initial optimal energy efficiency ratio (EER) refers to the best one among multiple EER information obtained by the NPU when it enters the working state based on different matrix parameter sets. This initial optimal EER information changes with the number of iterations. The acquisition function is optimized based on the initial optimal EER to resample other matrix parameter sets, resulting in an updated matrix parameter set. Next, the updated EER information corresponding to the updated matrix parameter set is determined according to the working EER function. The initial optimal EER information is updated based on the current updated EER information and the original EER information, thus obtaining a new acquisition function. Then, the matrix parameter set is sampled again using the new acquisition function to obtain another updated matrix parameter set. This iterative process is repeated until the EER difference is less than a preset threshold difference or a preset iteration termination condition is met, at which point the updated matrix parameter set is used as the target matrix parameter set.

[0155] In this process, after multiple iterations, when the new energy efficiency ratio (EER) is significantly lower than the existing EER and the improvement in the target EER is no longer significant, the optimization process converges, returning the target matrix parameter set. Alternatively, the termination condition for repeated iterations can be that the change between the newly sampled EER and the previously sampled EER is less than a threshold, the maximum number of iterations is reached, or the maximum iteration time is reached. The termination condition for repeated iterations can be adaptively adjusted according to the actual situation, and this application embodiment does not impose any limitations on it. Compared with related technologies, the Bayesian distribution model used in this application embodiment typically only requires dozens or even fewer iterations to determine the optimal matrix parameter set of the NPU. By effectively exploring the search space, it significantly reduces the computational resources and time costs required to find the global optimum, providing strong support for the green and efficient operation of the NPU.

[0156] It is understood that the optimization method proposed in this application intelligently samples the matrix parameter space by establishing a Bayesian distribution model. After each sampling, the model is updated and the search range is narrowed, gradually approaching the optimal solution. Compared with traditional grid search or random search, Bayesian optimization can find the global optimum in fewer evaluations, and is therefore suitable for optimization problems with high-dimensional and large-scale search spaces.

[0157] Furthermore, since energy efficiency ratio information needs to be obtained through experiments on real servers, which is costly and makes it difficult to quickly acquire a large number of data points, traditional fitting methods that rely on a large number of data points, such as neural network methods, cannot be used. In contrast, the optimization method proposed in this application can reduce the number of adjustments to NPU-related parameters while determining the optimal target matrix parameter set, thereby reducing energy consumption.

[0158] Furthermore, the iteration process of the Bayesian distribution model is adaptive, which can dynamically adjust the next search direction based on existing sample data. This makes the optimization process more intelligent and can gradually approach the global optimum in complex multivariate environments. At the same time, the Bayesian distribution model supports online optimization, which enables it to maintain high efficiency and energy efficiency under constantly changing computational loads, making it suitable for real-time application scenarios.

[0159] To enable readers to further understand the beneficial effects of the optimization method proposed in the embodiments of this application, a summary explanation is provided below.

[0160] The energy efficiency ratio optimization method for neural network processors proposed in this application seeks the globally optimal solution for energy efficiency ratio among matrix partitioning factor, loop unrolling factor, and current NPU operating frequency through Bayesian optimization, thereby achieving a balance between performance and energy efficiency, and has the following expected effects:

[0161] (1) Improve energy efficiency ratio:

[0162] The optimization method proposed in this application significantly reduces unnecessary energy consumption by optimizing matrix partitioning, loop unrolling, and frequency adjustment. Unlike traditional methods that simply pursue maximum performance, this optimization method can reduce total energy consumption while ensuring performance, thereby achieving optimal energy efficiency. Especially in large-scale deep learning tasks, the optimized NPU node can complete the same or even more computational work with lower power consumption. Experiments on artificial intelligence processors show that the optimal matrix partitioning improves energy efficiency by 5% compared to the default matrix partitioning; the optimal loop unrolling improves energy efficiency by 13% compared to the default loop unrolling; and the optimal frequency setting improves energy efficiency by 3% compared to the default frequency setting. Overall, this can achieve an NPU energy efficiency improvement of over 20%.

[0163] (2) Reduce memory access overhead:

[0164] The optimization method proposed in this application uses block matrix multiplication to decompose a large matrix into smaller blocks for computation. These smaller blocks can better match the multi-level cache of the NPU (such as L1 and L2 cache), thereby reducing the number of global memory accesses, reducing memory bandwidth pressure and latency. Reducing global memory access not only improves the computation speed, but also reduces energy consumption during data transmission, further optimizing overall energy efficiency.

[0165] (3) Improve computational efficiency:

[0166] The optimization method proposed in this application further improves the utilization of computing resources through loop unrolling factors. Loop unrolling not only reduces the overhead of loop control but also improves register reuse, making the data flow more efficient in the processor pipeline. This process reduces waiting time and improves computational efficiency, enabling more operations to be completed in the same amount of time, thereby enhancing the overall computing power of the NPU node.

[0167] (4) Flexible adaptation to different computing tasks:

[0168] The optimization method proposed in this application can automatically adjust parameters such as the partitioning factor, cycle unrolling factor, and current running frequency according to the characteristics of different computing tasks, enabling the optimization method to flexibly adapt to tasks of different scales and types. This means that NPU nodes can dynamically adjust their computing modes when facing different loads, achieving dual optimization of computing performance and energy efficiency without manual adjustment or complex pre-configuration.

[0169] (5) Reduce system operating costs:

[0170] The optimization method proposed in this application improves the energy efficiency ratio, enabling NPU nodes in the intelligent computing cluster to consume less power, significantly reducing energy consumption when performing large-scale computing tasks. For large-scale data centers and computing clusters, this not only reduces direct power consumption but also lowers indirect energy-related costs such as cooling and maintenance. Therefore, the application of this method will help data centers achieve energy conservation and consumption reduction while maintaining high-performance computing capabilities, thereby reducing long-term operating costs.

[0171] (6) Supporting the Sustainable Development Goals:

[0172] The optimization method proposed in this application can significantly reduce carbon emissions per unit computing task by improving the energy efficiency of NPU nodes, providing technical support for the green transformation of data centers and computing clusters, and helping to promote the development of related industries in a more environmentally friendly direction.

[0173] like Figure 9 As shown, Figure 9 This is an optional flowchart of the energy efficiency ratio optimization device for the neural network processor provided in this application embodiment. The optimization device includes the following modules 801 to 804:

[0174] The acquisition module 801 is used to acquire the set of matrix parameters of the neural network processor and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters.

[0175] Module 802 is used to construct a Bayesian distribution model based on the working energy efficiency ratio function and the matrix parameter set, wherein the Bayesian distribution model includes a sampling function for evaluating the quality of the matrix parameter set;

[0176] The Bayesian optimization module 803 is used to perform Bayesian optimization on the matrix parameter set using the acquisition function to obtain the target matrix parameter set.

[0177] The target execution module 804 is used to control the neural network processor to enter the running state based on the target matrix parameter set.

[0178] This application proposes a method and related equipment for optimizing the energy efficiency ratio of a neural network processor. The method includes obtaining a set of matrix parameters for the neural network processor and determining its operating energy efficiency ratio function based on this set. A Bayesian distribution model is constructed based on the operating energy efficiency ratio function and the set of matrix parameters. This Bayesian distribution model includes a sampling function used to evaluate the quality of the set of matrix parameters. The optimization device can clearly define the sampling direction from multiple sets of matrix parameters according to the sampling function, avoiding energy waste caused by blindly sampling the set of matrix parameters. Next, the set of matrix parameters is optimized using the sampling function to obtain a target set of matrix parameters, thereby selecting the set with the most sampling value from other sets of matrix parameters for which energy efficiency ratio information has not yet been calculated. Finally, the neural network processor is controlled to enter the running state based on the target set of matrix parameters. In this way, the neural network processor can efficiently process the current task while improving its energy efficiency ratio and reducing energy consumption.

[0179] The specific implementation of this optimization device is basically the same as the specific embodiment of the optimization method described above, and will not be repeated here.

[0180] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described tuning method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0181] like Figure 10 As shown, Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes:

[0182] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0183] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the tuning method of the embodiments of this application.

[0184] The input / output interface 903 is used to implement information input and output;

[0185] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0186] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0187] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0188] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described tuning method.

[0189] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0190] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0191] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0192] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0194] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0195] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0197] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0199] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for optimizing the energy efficiency ratio of a neural network processor, characterized in that, The method includes: Obtain the set of matrix parameters of the neural network processor, and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters; A Bayesian distribution model is constructed based on the working energy efficiency ratio function and the set of matrix parameters, wherein the Bayesian distribution model includes a sampling function for evaluating the quality of the set of matrix parameters; The set of matrix parameters is subjected to Bayesian optimization using the acquisition function to obtain the target set of matrix parameters; The neural network processor is controlled to enter the running state based on the target matrix parameter set; The construction of the Bayesian distribution model based on the working energy efficiency ratio function and the matrix parameter set includes: Based on the multiple sets of matrix parameters and the working energy efficiency ratio function, determine the corresponding multiple energy efficiency ratio information; Based on multiple energy efficiency ratio information and the corresponding set of matrix parameters, a mean function characterizing the correlation between multiple energy efficiency ratio information and a covariance function characterizing the correlation between the set of matrix parameters and the energy efficiency ratio information are constructed. The Bayesian distribution model is constructed based on the mean function and the covariance function. The step of performing Bayesian optimization on the matrix parameter set using the acquisition function to obtain the target matrix parameter set includes: The acquisition function is optimized to obtain an update matrix parameter set, and the updated energy efficiency ratio information is generated based on the update matrix parameter set and the working energy efficiency ratio function. Calculate the energy efficiency ratio difference between the updated energy efficiency ratio information and the initial optimal energy efficiency ratio information, update the acquisition function based on the updated energy efficiency ratio information to obtain the updated acquisition function, and update the initial optimal energy efficiency ratio based on the updated energy efficiency ratio information; The updated acquisition function is used as the new acquisition function to iteratively update the matrix parameter set until the energy efficiency ratio difference is less than a preset threshold difference. Then, the updated matrix parameter set is used as the target matrix parameter set.

2. The method according to claim 1, characterized in that, The neural network processor includes a target matrix for performing computational processing, the target matrix including multiple matrix elements; The acquisition of the matrix parameter set of the neural network processor includes: Based on a preset first value range, at least one block factor is determined; Based on a preset second value range, a cyclic expansion factor matching the block factor is determined, wherein the block factor is used to divide the target matrix into multiple smaller sub-matrices, and the cyclic expansion factor represents the merging calculation of multiple matrix elements in the sub-matrices; Based on a preset third value range, the current operating frequency of the neural network processor is determined; The set of matrix parameters is obtained based on the block factor, the cycle expansion factor, and the current running frequency.

3. The method according to claim 2, characterized in that, The neural network processor is equipped with multiple cache spaces of different levels; After determining the operating energy efficiency ratio function of the neural network processor based on the set of matrix parameters, the method further includes: The frequency of use of the submatrix is ​​predicted to obtain frequency level information, and the submatrix is ​​stored in the cache space that matches the frequency level information; Based on the set of matrix parameters, the neural network processor is controlled to enter the working state, so that the neural network processor processes the data in all the cache spaces to obtain the computational load information and average power information per unit time. Based on the ratio of the computational information to the average power information, the energy efficiency ratio information corresponding to the matrix parameter set is obtained.

4. The method according to claim 3, characterized in that, The acquisition function includes a first acquisition function; The first acquisition function is determined through the following steps, which include: The mean function is updated based on a preset cumulative distribution function to obtain an updated mean function; The covariance function is updated based on a preset probability density function to obtain a first updated covariance function; By superimposing the updated mean function and the first updated covariance function, a first acquisition function for evaluating the quality of the matrix parameter set is constructed.

5. The method according to claim 3, characterized in that, The acquisition function also includes a second acquisition function; The second acquisition function is determined through the following steps, which include: The covariance function is updated based on preset hyperparameters to obtain a second updated covariance function; By superimposing the mean function and the second update covariance function, a second acquisition function for evaluating the quality of the matrix parameter set is constructed.

6. A device for optimizing the energy efficiency ratio of a neural network processor, characterized in that, The device includes: An acquisition module is used to acquire the set of matrix parameters of the neural network processor and determine the working energy efficiency ratio function of the neural network processor based on the set of matrix parameters. A construction module is used to construct a Bayesian distribution model based on the working energy efficiency ratio function and the matrix parameter set, wherein the Bayesian distribution model includes a sampling function for evaluating the quality of the matrix parameter set; The Bayesian optimization module is used to perform Bayesian optimization processing on the matrix parameter set using the acquisition function to obtain the target matrix parameter set; The target execution module is used to control the neural network processor to enter the running state based on the target matrix parameter set; The construction of the Bayesian distribution model based on the working energy efficiency ratio function and the matrix parameter set includes: Based on the multiple sets of matrix parameters and the working energy efficiency ratio function, determine the corresponding multiple energy efficiency ratio information; Based on multiple energy efficiency ratio information and the corresponding set of matrix parameters, a mean function characterizing the correlation between multiple energy efficiency ratio information and a covariance function characterizing the correlation between the set of matrix parameters and the energy efficiency ratio information are constructed. The Bayesian distribution model is constructed based on the mean function and the covariance function. The step of performing Bayesian optimization on the matrix parameter set using the acquisition function to obtain the target matrix parameter set includes: The acquisition function is optimized to obtain an update matrix parameter set, and the updated energy efficiency ratio information is generated based on the update matrix parameter set and the working energy efficiency ratio function. Calculate the energy efficiency ratio difference between the updated energy efficiency ratio information and the initial optimal energy efficiency ratio information, update the acquisition function based on the updated energy efficiency ratio information to obtain the updated acquisition function, and update the initial optimal energy efficiency ratio based on the updated energy efficiency ratio information; The updated acquisition function is used as the new acquisition function to iteratively update the matrix parameter set until the energy efficiency ratio difference is less than a preset threshold difference. Then, the updated matrix parameter set is used as the target matrix parameter set.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the energy efficiency ratio optimization method of the neural network processor according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the energy efficiency ratio optimization method of the neural network processor according to any one of claims 1 to 5.

Images