Neural network model construction method, device and equipment based on operation speed analysis

Abstract

The application discloses a neural network model construction method and device based on operation speed analysis and equipment, the method comprises the following steps: measuring the operator operation speed of different algorithm models on different heterogeneous chips, and obtaining a model operator operation speed table; constructing a reinforcement learning data set according to the model operator in the model operator operation speed table; based on the reinforcement learning strategy, searching for the optimal operation operator in the reinforcement learning data set according to the preset strategy network and the preset reward function, and constructing an optimal neural network model. The neural network model constructed according to the optimal operation operator can improve the model operation speed on the basis of operator design, and the model accuracy can also be ensured by optimizing and combining the operators. Therefore, the application can solve the technical problem that the existing model compression and light weight processing technology cannot fundamentally improve the model operation speed while ensuring the model accuracy.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to a method, apparatus and device for constructing neural network models based on computational speed analysis. Background Technology

[0002] The energy consumption of intelligent AI chips is limited by their rated power, especially for lightweight intelligent chips deployed at the edge, where the energy resources available for model computation are very limited. In this case, algorithms typically need to minimize computational energy consumption while meeting accuracy requirements. Two common approaches are: 1) compressing the model to reduce the computational cost of a single inference; and 2) using mature, small, lightweight models, which, due to their lower energy consumption, are suitable for deployment on chips with lower energy requirements.

[0003] However, model compression technology cannot fully reduce energy consumption. Excessive pruning to meet energy consumption requirements, under the premise of limited energy resources, leads to more network structure damage and a decrease in model accuracy. Even though lightweight models have a smaller size, when running on heterogeneous chips, the underlying computing method may cause an excessive number of floating-point operations per second, resulting in excessive instantaneous computation, network congestion, and thus reducing the model's running speed. Summary of the Invention

[0004] This application provides a method, apparatus, and device for constructing neural network models based on computational speed analysis, which addresses the technical problem that existing model compression and lightweighting techniques cannot fundamentally improve model computational speed while ensuring model accuracy.

[0005] In view of this, the first aspect of this application provides a method for constructing a neural network model based on computational speed analysis, including:

[0006] The computation speed of model operators of different algorithm models on different heterogeneous chips is measured to obtain a table of model operator computation speeds. The table of model operator computation speeds includes the operator type, chip type and computation speed of the model operator.

[0007] Construct a reinforcement learning dataset based on the model operators in the model operator operation speed table;

[0008] Based on the reinforcement learning strategy, the optimal operation operator is searched in the reinforcement learning dataset according to the preset policy network and preset reward function, and the optimal neural network model is constructed.

[0009] Preferably, the step of measuring the operator operation speed of different algorithm models on different heterogeneous chips to obtain a model operator operation speed table includes:

[0010] The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips.

[0011] The average value of all the initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a model operator operation speed table is constructed.

[0012] Preferably, the step of measuring the operator operation speed of different algorithm models on different heterogeneous chips to obtain a model operator operation speed table further includes:

[0013] Extract structural hyperparameters from different algorithm models to generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

[0014] Preferably, the step of measuring the operator operation speed of model operators of different algorithm models on different heterogeneous chips to obtain a model operator operation speed table further includes:

[0015] The computation speed of operators corresponding to different model operators of different algorithm models is normalized.

[0016] Preferably, the step of searching for the optimal operator in the reinforcement learning dataset based on a preset policy network and a preset reward function, and constructing the optimal neural network model, includes:

[0017] Based on the reinforcement learning strategy, the model operators are defined as states, forming a state space;

[0018] A set of executable actions is defined for each state, forming an action space;

[0019] Using a preset policy network with optimal computing resources and optimal model performance as constraints, the optimal operator is iteratively searched in the reinforcement learning dataset based on the state space, the action space, and the preset reward function until the stopping condition is met, thus obtaining the optimal operation operator.

[0020] Construct an optimal neural network model based on the optimal operation operator.

[0021] The second aspect of this application provides a neural network model construction apparatus based on computational speed analysis, comprising:

[0022] The speed measurement unit is used to measure the operator operation speed of model operators of different algorithm models on different heterogeneous chips, and obtain a model operator operation speed table, which includes the operator type, chip type and operation speed of the model operator;

[0023] A dataset construction unit is used to construct a reinforcement learning dataset based on the model operators in the model operator operation speed table.

[0024] The optimal search unit is used to search for the optimal operation operator in the reinforcement learning dataset based on the reinforcement learning policy, according to the preset policy network and the preset reward function, and to construct the optimal neural network model.

[0025] Preferably, the speed measuring unit is specifically used for:

[0026] The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips.

[0027] The average value of all the initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a model operator operation speed table is constructed.

[0028] Preferably, it further includes:

[0029] The operator extraction unit is used to extract the structural hyperparameters of different algorithm models and generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

[0030] Preferably, it further includes:

[0031] The normalization unit is used to normalize the operator operation speed corresponding to different model operators of different algorithm models.

[0032] A third aspect of this application provides a neural network model construction device based on computing speed analysis, the device including a processor and a memory;

[0033] The memory is used to store program code and transmit the program code to the processor;

[0034] The processor is used to execute the neural network model construction method based on computational speed analysis as described in the first aspect, according to the instructions in the program code.

[0035] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0036] This application provides a method for constructing a neural network model based on computational speed analysis, including: measuring the computational speed of model operators of different algorithm models on different heterogeneous chips to obtain a table of model operator computational speeds, which includes the operator type, chip type, and operator computational speed of the model operators; constructing a reinforcement learning dataset based on the model operators in the table of model operator computational speeds; and searching for the optimal computational operator in the reinforcement learning dataset based on a reinforcement learning policy, a preset policy network, and a preset reward function, and constructing the optimal neural network model.

[0037] The neural network model construction method based on computational speed analysis provided in this application decomposes the model into multiple model operators. A computational speed table of model operators is constructed by measuring the computational speed of different operators on different heterogeneous chips. A reinforcement learning search is then performed based on an energy consumption optimization strategy to find the optimal operator. The neural network model constructed based on the optimal operator can improve the model's computational speed by optimizing the operator design, and the model's accuracy can also be ensured through optimized combination of operators. Therefore, this application can solve the technical problem that existing model compression and lightweight processing techniques cannot fundamentally improve model computational speed while maintaining model accuracy. Attached Figure Description

[0038] Figure 1 A flowchart illustrating the neural network model construction method based on computational speed analysis provided in this application embodiment;

[0039] Figure 2 This is a schematic diagram of the neural network model construction device based on computing speed analysis provided in an embodiment of this application. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0041] For easier understanding, please refer to Figure 1 The embodiments of the neural network model construction method based on computational speed analysis provided in this application include:

[0042] Step 101: Measure the operator operation speed of different algorithm models on different heterogeneous chips to obtain a table of model operator operation speeds. The table of model operator operation speeds includes the operator type, chip type, and operator operation speed of the model operator.

[0043] Further, step 101 includes:

[0044] The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips.

[0045] The average of all initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a table of model operator operation speeds is constructed.

[0046] Furthermore, step 101, preceding the following, also includes:

[0047] Extract the structural hyperparameters of different algorithm models to generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

[0048] Furthermore, step 101 also includes:

[0049] The computation speed of operators corresponding to different model operators of different algorithm models is normalized.

[0050] It should be noted that for each model operator of each algorithm model, a heterogeneous chip can be selected, and the model operator can be implemented on that chip. The operation time of the operator on the chip can be measured using a timing tool; the timing tool can be a CPU performance counter or other timing tools, which are not limited here. To ensure the accuracy of the measurement, this embodiment can also measure the operation speed of the model operator multiple times, i.e., the initial operation speed; then, the average value is calculated based on the initial operation speed to obtain the operator operation speed.

[0051] This measurement method can be used for all model operators to obtain their corresponding operation speeds, thereby constructing a model operator operation speed table. It is understood that each model operator in the table corresponds to a specific chip type and its corresponding operation speed; therefore, model operators can be specifically selected and combined based on this table to meet the needs of specific tasks. Furthermore, to facilitate comparative analysis between different chips, this embodiment can also use normalization processing to standardize the expression of the operation time of model operators on different chips, dividing the actual operation time by the minimum operation time to obtain the relative operation speed.

[0052] It should be noted that structural hyperparameters, in addition to including the number of network layers, the number of neurons, and the kernel size, can also be configured with other model-related parameters to generate model operators, such as fully connected layers and pooling; specific examples are not limited here. Furthermore, heterogeneous chip types include, but are not limited to, GPUs, CPUs, and FPGAs.

[0053] Step 102: Construct a reinforcement learning dataset based on the model operators in the model operator operation speed table.

[0054] Understandably, reinforcement learning datasets are used in subsequent reinforcement learning optimization processes to find model operators with optimal computing resources and model performance, and then build models based on these operators. The chip model corresponding to a type of model operator in the model operator computation speed table constitutes a set of data; multiple sets of datasets form a reinforcement learning dataset.

[0055] Step 103: Based on the reinforcement learning strategy, search for the optimal operation operator in the reinforcement learning dataset according to the preset policy network and preset reward function, and construct the optimal neural network model.

[0056] Further, step 103 includes:

[0057] Based on reinforcement learning strategies, model operators are defined as states, forming a state space;

[0058] Define a set of executable actions in each state to form an action space;

[0059] Using a pre-defined policy network with optimal computing resources and optimal model performance as constraints, the optimal operator is iteratively searched in the reinforcement learning dataset based on the state space, action space and pre-defined reward function until the stopping condition is met, and the optimal operation operator is obtained.

[0060] Construct the optimal neural network model based on the optimal operators.

[0061] It should be noted that the state space defined by the model operators is the parameter setting of each operator in the neural network model, such as the selection of the number of layers, the number of neurons, the size of the convolution kernel, etc. The executable actions in the action space refer to the actions of selecting appropriate operator parameters and network structure parameters under given computing resources. While considering optimal computing resources as a constraint, the adaptability of operator performance to the target task can also be considered; this is a point of focus in specific application scenarios. The preset policy network takes the current state as input and outputs a probability distribution to express the probability of selecting each action in the current state. Furthermore, the preset policy network can be built based on a multilayer perceptron or other types of network structures, which are not limited here. The preset reward function is negatively correlated with the energy consumption of the neural network structure, so it is generally defined as a negative index value. Moreover, the preset reward function is used to measure the performance of the neural network structure in the current state. This embodiment considers both the model's accuracy and computational speed, and can adopt a multi-objective optimization approach, defining the reward as a weighted sum of speed and accuracy, balancing the importance of both by adjusting the weight parameters.

[0062] Furthermore, the iterative optimization learning task based on the reinforcement learning strategy can be pre-trained based on the loss function. First, the parameters of the pre-defined policy network are initialized; this can be done using random initialization or pre-training. Second, interaction with the environment occurs: actions are selected based on the current policy network, and these actions are executed in the environment, observing the state and reward; the reward can be directly provided by the environment, such as a weighted sum of accuracy and computational speed. Next, the loss function is calculated: based on the observed state, actions, and reward, the loss function is calculated; the loss function is constructed based on the pre-defined reward function and is defined as:

[0063] J(θ) = -E πθ [R(s,a)logπ θ (a|s)]

[0064] Where θ represents the parameters of the preset policy network, and π θ This represents the policy network with parameter θ, where a and s represent the current action and current state, respectively, R(s,a) is the preset reward function, and E πθ In terms of strategy π θ The expected value is calculated as follows. The goal of training optimization is to minimize the loss function J(θ).

[0065] Then, the policy network parameters are updated using the policy gradient algorithm to update the parameters, thereby reducing the loss function. The formula for updating the policy network parameters is: Here, α is the learning rate. The interaction with the environment is then repeated, the loss function is calculated, and the policy network parameters are updated until the stopping condition is met. This yields the optimal operator, upon which the optimal neural network model is constructed. It can be observed that after multiple iterations, the resulting optimized policy network can be used to search for the optimal neural network structure that balances speed and performance under limited computing resources.

[0066] It's important to note that reinforcement learning training can be time-consuming, requiring a large number of samples and training epochs to optimize the policy network. Furthermore, the learning rate is a crucial hyperparameter that needs to be adjusted based on specific circumstances. For complex tasks and environments, more advanced reinforcement learning algorithms and techniques may be necessary to achieve better training results.

[0067] The neural network model construction method based on computational speed analysis provided in this application decomposes the model into multiple model operators. A computational speed table of model operators is constructed by measuring their computational speeds on different heterogeneous chips. A reinforcement learning search is performed based on an energy consumption optimization strategy to find the optimal operator. The neural network model constructed based on the optimal operator can improve the model's computational speed by optimizing the operator design, and the model's accuracy can also be ensured through optimized combination of operators. Therefore, this application can solve the technical problem that existing model compression and lightweighting techniques cannot fundamentally improve model computational speed while maintaining model accuracy.

[0068] For easier understanding, please refer to Figure 2 This application provides an embodiment of a neural network model construction device based on computational speed analysis, including:

[0069] The speed measurement unit 201 is used to measure the operator operation speed of model operators of different algorithm models on different heterogeneous chips, and obtain a model operator operation speed table. The model operator operation speed table includes the operator type, chip type and operator operation speed of the model operator.

[0070] Dataset construction unit 202 is used to construct a reinforcement learning dataset based on the model operators in the model operator operation speed table;

[0071] The optimal search unit 203 is used to search for the optimal operation operator in the reinforcement learning dataset based on the reinforcement learning policy, according to the preset policy network and the preset reward function, and to construct the optimal neural network model.

[0072] Furthermore, the speed measurement unit 201 is specifically used for:

[0073] The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips.

[0074] The average of all initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a table of model operator operation speeds is constructed.

[0075] Furthermore, it also includes:

[0076] The operator extraction unit 204 is used to extract the structural hyperparameters of different algorithm models and generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

[0077] Furthermore, it also includes:

[0078] Normalization unit 205 is used to normalize the operator operation speed corresponding to different model operators of different algorithm models.

[0079] This application also provides a neural network model building device based on computing speed analysis, the device including a processor and memory;

[0080] The memory is used to store program code and transfer the program code to the processor;

[0081] The processor is used to execute the neural network model construction method based on computing speed analysis in the above method embodiment according to the instructions in the program code.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of this application through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0086] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing a neural network model based on computational speed analysis, characterized in that, include: The computation speed of model operators for different algorithm models on different heterogeneous chips is measured to obtain a table of model operator computation speeds. This table includes the operator type, chip type, and computation speed of the model operator. The specific process is as follows: The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips. The average value of all the initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a model operator operation speed table is constructed. Construct a reinforcement learning dataset based on the model operators in the model operator operation speed table; Based on the reinforcement learning strategy, the optimal operation operator is searched in the reinforcement learning dataset according to the preset policy network and preset reward function, and the optimal neural network model is constructed.

2. The neural network model construction method based on computational speed analysis according to claim 1, characterized in that, The process of measuring the operator operation speed of different algorithm models on different heterogeneous chips to obtain a table of model operator operation speeds also includes: Extract structural hyperparameters from different algorithm models to generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

3. The neural network model construction method based on computational speed analysis according to claim 1, characterized in that, The method of measuring the operator operation speed of different algorithm models on different heterogeneous chips to obtain a model operator operation speed table also includes: The computation speed of operators corresponding to different model operators of different algorithm models is normalized.

4. The neural network model construction method based on computational speed analysis according to claim 1, characterized in that, The step of searching for the optimal computation operator in the reinforcement learning dataset based on the reinforcement learning strategy, according to the preset policy network and preset reward function, and constructing the optimal neural network model includes: Based on the reinforcement learning strategy, the model operators are defined as states, forming a state space; A set of executable actions is defined for each state, forming an action space; Using a preset policy network with optimal computing resources and optimal model performance as constraints, the optimal operator is iteratively searched in the reinforcement learning dataset based on the state space, the action space, and the preset reward function until the stopping condition is met, thus obtaining the optimal operation operator. Construct an optimal neural network model based on the optimal operation operator.

5. A neural network model construction device based on computational speed analysis, characterized in that, include: A speed measurement unit is used to measure the operator operation speed of model operators of different algorithm models on different heterogeneous chips, and to obtain a model operator operation speed table. The model operator operation speed table includes the operator type, chip type, and operator operation speed of the model operator. The speed measurement unit is specifically used for: The initial computation speed of each operator in different algorithm models was measured multiple times on different heterogeneous chips. The average value of all the initial operation speeds is calculated to obtain the operator operation speed of each model operator, and a model operator operation speed table is constructed. A dataset construction unit is used to construct a reinforcement learning dataset based on the model operators in the model operator operation speed table. The optimal search unit is used to search for the optimal operation operator in the reinforcement learning dataset based on the reinforcement learning policy, according to the preset policy network and the preset reward function, and to construct the optimal neural network model.

6. The neural network model construction device based on computational speed analysis according to claim 5, characterized in that, Also includes: The operator extraction unit is used to extract the structural hyperparameters of different algorithm models and generate model operators. The structural hyperparameters include the number of network layers, the number of neurons, and the size of the convolution kernel.

7. The neural network model construction device based on computational speed analysis according to claim 5, characterized in that, Also includes: The normalization unit is used to normalize the operator operation speed corresponding to different model operators of different algorithm models.

8. A neural network model construction device based on computational speed analysis, characterized in that, The device includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the neural network model construction method based on computing speed analysis as described in any one of the claims 1-4 according to the instructions in the program code.

Images