Neural network searching method, device and computer equipment

Abstract

The application relates to a neural network searching method, device and computer equipment. The method comprises the following steps: determining a target network layer number; determining a plurality of initialized neural network models according to the target network layer number, performing expression capability judgment on each initialized neural network model to obtain a corresponding judgment result; for any initialized neural network model, if the judgment result represents that the expression capability of the initialized neural network model meets a preset condition, determining in-chip storage information and off-chip storage information of the initialized neural network model, and if the in-chip storage information and the off-chip storage information of the initialized neural network model meet a storage condition, determining performance data of the initialized neural network model according to a target bandwidth utilization rate; and determining a target neural network model according to the performance data of each initialized neural network model. The method can quickly obtain the target neural network model which has good performance and high precision when deployed on a chip.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a neural network search method, apparatus, and computer device. Background Technology

[0002] The design of network structure is key to improving the classification performance of network. Neural Architecture Search (NAS), as a network structure design method, essentially transforms the process of manually adjusting neural networks into an automated task to discover more complex neural network architectures.

[0003] NAS (Neural Architecture Search) generally falls into two categories: the first is neural network architecture search based on reinforcement learning and evolutionary algorithms, and the second is neural network architecture search based on gradient descent. Reinforcement learning-based neural network architecture search, taking NAS-net (Neural Network Architecture Search) as an example, suffers from a particularly large search space due to the variable number of operators, specifications, connections, and network layers, resulting in an exceptionally long search time. Gradient descent-based neural network architecture search, using Darts as an example, requires less search time, but hardware information cannot be searched using gradients, making it impossible to establish connections between the search model and the hardware data model. This leads to poor performance of the searched neural network model on deployed hardware.

[0004] It is evident that current neural network architecture search methods suffer from a trade-off between search speed and the performance and accuracy of the found neural network model on a deployed chip. Summary of the Invention

[0005] Therefore, it is necessary to provide a neural network search method, apparatus, computer device, computer-readable storage medium, and computer program product to address the aforementioned technical problems, so that the searched neural network model can simultaneously achieve both accuracy and on-chip performance.

[0006] In a first aspect, this application provides a neural network search method, the method comprising:

[0007] Determine the target number of network layers;

[0008] During the i-th iteration search process, multiple initial neural network models are determined according to the number of target network layers. The expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0.

[0009] For any of the initial neural network models, if the judgment result indicates that the expressive power of the initial neural network model meets the preset conditions, the on-chip storage information and off-chip storage information of the initial neural network model are determined.

[0010] For any of the initialization neural network models, if the on-chip storage information and the off-chip storage information of the initialization neural network model meet the storage conditions, the performance data of the initialization neural network model is determined based on the target bandwidth utilization.

[0011] The target neural network model is determined based on the performance data of each of the initial neural network models.

[0012] In one embodiment, the method further includes:

[0013] If the judgment results corresponding to each of the initial neural network models indicate that the expressive ability of the initial neural network model does not meet the preset condition, the next iteration search process is entered. The steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment results corresponding to each initial neural network model are repeated until an initial neural network model corresponding to the judgment result indicating that the expressive ability of the initial neural network model meets the preset condition is obtained, or until the number of iterations reaches the target number.

[0014] After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

[0015] In one embodiment, determining the on-chip storage information and off-chip storage information of the initialized neural network model includes:

[0016] The chip computation diagram is determined based on the original network layers of the initialized neural network model;

[0017] Establish the layer mapping relationship between the original network layer and the chip computation graph;

[0018] Obtain the sub-chip storage information of each layer of the computation graph in the chip computation graph, and obtain the off-chip storage information of the initialized neural network model;

[0019] Based on the layer mapping relationship and the sub-chip storage information of the computation graph of each layer, the on-chip storage information of the initialized neural network model is determined.

[0020] In one embodiment, determining the performance data of the initialized neural network model based on the target bandwidth utilization, when the on-chip storage information and the off-chip storage information of the initialized neural network model meet the storage conditions, includes:

[0021] Obtain target storage parameters, which include target off-chip storage parameters and target on-chip storage parameters;

[0022] The on-chip storage information and off-chip storage information of the initialized neural network model are compared with the target off-chip storage parameters and the target on-chip storage parameters to obtain the comparison result of the initialized neural network model. The comparison result includes a first comparison result or a second comparison result. The first comparison result indicates that the on-chip storage information and off-chip storage information meet the storage conditions, and the second comparison result indicates that the on-chip storage information and off-chip storage information of the initialized neural network model do not meet the storage conditions.

[0023] Based on the first comparison result, the performance data of the initialized neural network model is determined according to the target bandwidth utilization.

[0024] In one embodiment, comparing the on-chip storage information and off-chip storage information of the initialized neural network model with the target off-chip storage parameters and the target on-chip storage parameters to obtain a comparison result includes:

[0025] The first comparison result is obtained when the on-chip storage information is less than or equal to the target on-chip storage parameter, and the off-chip storage information is less than or equal to the target off-chip storage parameter; or,

[0026] The second comparison result is obtained when the on-chip storage information is greater than the target on-chip storage parameter, and / or the off-chip storage information is greater than the target on-chip storage parameter.

[0027] In one embodiment, the method further includes:

[0028] If the comparison results corresponding to each of the initial neural network models are the second comparison results, proceed to the next iteration search process. Use the comparison results corresponding to each of the initial neural network models as constraints, repeatedly determine multiple initial neural network models according to the target network layer number, judge the expressive power of each initial neural network model, and obtain the judgment results corresponding to each initial neural network model, until the initial neural network model corresponding to the first comparison result is obtained, or until the number of iterations reaches the target number.

[0029] After adjusting the target network layer number and / or the target storage parameters, the process of repeatedly determining multiple initial neural network models based on the adjusted target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

[0030] In one embodiment, the performance data includes inference time, and determining the performance data of the initialized neural network model based on the target bandwidth utilization includes:

[0031] Determine the target bandwidth utilization rate;

[0032] Based on the target bandwidth utilization and the target storage parameters, the inference time of the initial neural network model is determined, and the inference time represents the time it takes for the initial neural network model to complete the target scenario task.

[0033] In one embodiment, determining the target neural network model based on the performance data of each of the initialized neural network models includes:

[0034] If the performance data of the initialized neural network model meets the target performance conditions, the initialized neural network model is used as the target neural network model; or,

[0035] If the performance data of each of the initial neural network models does not meet the target performance condition, the next iteration search process is entered. The steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model are repeated until the target neural network model is obtained, or until the number of iterations reaches the target number.

[0036] After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

[0037] Secondly, this application also provides a neural network search device, the device comprising:

[0038] The network layer determination module is used to determine the target network layer number;

[0039] The expressive ability judgment module is used to determine multiple initial neural network models based on the number of target network layers during the i-th iteration search process, judge the expressive ability of each initial neural network model, and obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0;

[0040] The storage information determination module is used to determine the on-chip storage information and off-chip storage information of any of the initial neural network models, provided that the judgment result indicates that the expressive ability of the initial neural network model meets the preset conditions.

[0041] The performance data determination module is used to determine the performance data of any of the initialized neural network models based on the target bandwidth utilization, provided that the on-chip storage information and the off-chip storage information of the initialized neural network model meet the storage conditions.

[0042] The target network determination module is used to determine the target neural network model based on the performance data of each of the initial neural network models.

[0043] In one embodiment, the neural network search device further includes an iteration device. The iteration device is used to enter the next iteration search process when the judgment results corresponding to each of the initial neural network models all indicate that the expressive ability of the initial neural network model does not meet the preset condition. This process repeats the steps of determining multiple initial neural network models based on the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model, until an initial neural network model corresponding to the judgment result indicating that the expressive ability of the initial neural network model meets the preset condition is obtained, or until the number of iterations reaches the target number. After adjusting the target network layer number, the process repeats the steps of determining multiple initial neural network models based on the adjusted target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model.

[0044] In one embodiment, the storage information determination module is further configured to: determine the chip computation graph based on the original network layer of the initialized neural network model; establish a layer mapping relationship between the original network layer and the chip computation graph; obtain the sub-on-chip storage information of each layer of the computation graph in the chip computation graph, and obtain the off-chip storage information of the initialized neural network model; and determine the on-chip storage information of the initialized neural network model based on the layer mapping relationship and the sub-on-chip storage information of each layer of the computation graph.

[0045] In one embodiment, the performance data determination module is further configured to acquire target storage parameters, the target storage parameters including target off-chip storage parameters and target on-chip storage parameters; compare the on-chip storage information and off-chip storage information of the initialized neural network model with the target off-chip storage parameters and the target on-chip storage parameters to obtain a comparison result of the initialized neural network model, the comparison result including a first comparison result or a second comparison result, the first comparison result indicating that the on-chip storage information and off-chip storage information meet the storage conditions, and the second comparison result indicating that the on-chip storage information and off-chip storage information of the initialized neural network model do not meet the storage conditions; if the first comparison result is obtained, determine the performance data of the initialized neural network model based on the target bandwidth utilization.

[0046] In one embodiment, the performance data determination module is further configured to obtain the first comparison result when the on-chip storage information is less than or equal to the target on-chip storage parameter and the off-chip storage information is less than or equal to the target off-chip storage parameter; or to obtain the second comparison result when the on-chip storage information is greater than the target on-chip storage parameter and / or the off-chip storage information is greater than the target on-chip storage parameter.

[0047] In one embodiment, the iterative device is further configured to, when the comparison results corresponding to each of the initial neural network models are all the second comparison results, enter the next iterative search process, using the comparison results corresponding to each of the initial neural network models as constraints, repeatedly determine multiple initial neural network models based on the target network layer number, judge the expressive power of each of the initial neural network models, and obtain the judgment results corresponding to each of the initial neural network models, until the initial neural network model corresponding to the first comparison result is obtained, or until the number of iterations reaches the target number; after adjusting the target network layer number and / or the target storage parameters, repeatedly determine multiple initial neural network models based on the adjusted target network layer number, judge the expressive power of each of the initial neural network models, and obtain the judgment results corresponding to each of the initial neural network models.

[0048] In one embodiment, the performance data includes inference time, and the performance data determination module is further configured to determine the target bandwidth utilization rate; and to determine the inference time of the initial neural network model based on the target bandwidth utilization rate and the target storage parameters, wherein the inference time characterizes the time it takes for the initial neural network model to complete the target scenario task.

[0049] In one embodiment, the target network determination module is further configured to: 1) use the initial neural network model as the target neural network model if the performance data of the initial neural network model meets the target performance conditions; or 2) enter the next iteration search process if the performance data of each initial neural network model does not meet the target performance conditions, repeating the steps of determining multiple initial neural network models based on the target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model, until the target neural network model is obtained, or until the number of iterations reaches the target number; or 3) after adjusting the target network layer number, repeating the steps of determining multiple initial neural network models based on the adjusted target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model.

[0050] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0051] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0052] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0053] The aforementioned neural network search method, apparatus, computer device, computer-readable storage medium, and computer program product determine the target network layer number; during the i-th iteration search process, multiple initial neural network models are determined based on the target network layer number, and the expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model; for any initial neural network model, if the judgment result indicates that the expressive power of the initial neural network model meets preset conditions, the on-chip storage information and off-chip storage information of the initial neural network model are determined; for any initial neural network model, if the on-chip storage information and off-chip storage information of the initial neural network model meet storage conditions, the performance data of the initial neural network model is determined based on the target bandwidth utilization rate; based on the performance data of each initial neural network model, the target neural network model is determined. The neural network search method, apparatus, computer device, computer-readable storage medium, and computer program product provided in this application first perform a search based on a manually set target network layer number, which improves the search speed, and then combines expressive power, on-chip storage information, off-chip storage information, and target bandwidth utilization rate to search for the target neural network model, thereby enabling the rapid acquisition of a target neural network model with good performance and high accuracy on the deployed chip. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a neural network search method in one embodiment;

[0055] Figure 2 This is a flowchart illustrating step 106 in one embodiment;

[0056] Figure 3 This is a flowchart illustrating step 108 in one embodiment;

[0057] Figure 4 This is a flowchart illustrating step 108 in one embodiment;

[0058] Figure 5 This is a flowchart illustrating a neural network search method in one embodiment;

[0059] Figure 6 This is a block diagram of the neural network search device in one embodiment;

[0060] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] Neural network model search has become a method for model design, and it mainly falls into two categories: the first is neural network model search based on reinforcement learning and evolutionary algorithms, and the second is neural network model search based on gradient descent. Taking differential neural network models as an example, gradient descent-based neural network model search cannot feed back low-level chip information to the model design level.

[0063] Neural network architecture search suffers from an exceptionally large search space due to the variable number of operators, specifications, connections, and network layers per layer. This results in extremely long search times. For example, Google's initial reinforcement learning-based NAS-net required over 1000 GPU (graphics processing unit) days to run on a single GPU. While gradient descent-based neural networks, such as Darts (Differentiable Architecture Search), require shorter search times, hardware information cannot be searched using gradients, making it impossible to establish connections between the search model and the hardware data model. In other words, although the neural network model found by Darts is small, it takes a long time to run on hardware.

[0064] Currently, research on model expressiveness has become an important direction, as it represents another method for searching neural network models. Model expressiveness assesses the performance of a model by using Gaussian-distributed input data and evaluating the output. It enables rapid searching for optimal network structure characteristics within a discrete space; however, it lacks constraints regarding the network being searched and the target task, lacks targeted datasets for searching, results in poor network specificity, and lacks constraints related to hardware information.

[0065] Since the storage size and bandwidth utilization of neural network models of different sizes can be accurately assessed during the compilation and deployment of hardware, and the width of the neural network model has a significant impact on the model's accuracy, it is of great practical value to find a search method that can balance the width of the neural network model and the resulting changes in parameters and accuracy, in order to maximize accuracy, reduce the number of parameters, and balance hardware storage and bandwidth utilization, so as to achieve low storage consumption and high bandwidth utilization.

[0066] Based on this, embodiments of this application provide a neural network search method to solve the above problems, achieve fast search of neural network structures, and obtain a neural network model with good performance on the deployed chip and high accuracy.

[0067] In one embodiment, such as Figure 1 As shown, a neural network search method is provided. This embodiment illustrates the method applied to a server, but it is understood that the method can also be applied to a terminal, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. The neural network search process can be completed in a large GPU server. In this embodiment, the method includes the following steps:

[0068] Step 102: Determine the target number of network layers.

[0069] The target network layer number refers to the number of network layers in the target neural network model to be searched. The initial target network layer number can be set manually according to different application scenarios, and this application embodiment does not specifically limit the target network layer number. For example, the application scenario can be face recognition, object classification, object detection, image recognition, etc.

[0070] Step 104: During the i-th iteration search, multiple initial neural network models are determined based on the number of target network layers. The expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0.

[0071] In this embodiment, i represents the iteration number, a positive integer starting from 1. First, a search space can be selected; however, this embodiment does not specifically limit the search space. The initialized neural network model is a neural network model with a number of layers that matches the target number of layers, found based on the search space. For example, expressive power can include the accuracy of the neural network model. Taking a classification task as an example, on a publicly available dataset CIFAR-10, the classification accuracy of the initialized neural network model is obtained using data from CIFAR-10; this accuracy is the precision of the initialized neural network model. The result can be used to characterize whether the precision of the initialized neural network model meets the preset target precision.

[0072] Step 106: For any initialized neural network model, if the judgment result indicates that the expressive power of the initialized neural network model meets the preset conditions, determine the on-chip storage information and off-chip storage information of the initialized neural network model.

[0073] In this embodiment, taking the expressive power, including the accuracy of the neural network model, as an example, the preset condition can be a preset target accuracy. If the accuracy of the initialized neural network model is equal to or greater than the preset target accuracy, a judgment result indicating that the expressive power of the initialized neural network model meets the preset condition is obtained. If the expressive power meets the preset condition, it can be guaranteed that the neural network model can reach the accuracy requirement after training. On-chip storage information can include the amount of storage occupied by the initialized neural network model after it is deployed on the chip. Off-chip storage information can include the amount of storage occupied by the initialized neural network model after it is deployed on the chip. The initialized neural network model can be quantized and compiled to calculate the on-chip storage information and off-chip storage information of the initialized neural network model.

[0074] Step 108: For any initialized neural network model, if the on-chip storage information and off-chip storage information of the initialized neural network model meet the storage conditions, determine the performance data of the initialized neural network model based on the target bandwidth utilization.

[0075] In this embodiment, the storage conditions can be manually set target storage parameters. These target storage parameters can include target off-chip storage parameters and target on-chip storage parameters. The target off-chip storage parameters are the spatial parameters of off-chip storage, i.e., the maximum amount of storage the target neural network model can occupy in the off-chip storage. The target on-chip storage parameters are the spatial parameters of on-chip storage, i.e., the maximum amount of storage the target neural network model can occupy in the on-chip storage. The target bandwidth utilization rate is the maximum bandwidth that the target neural network model can occupy during deployment on hardware. The target bandwidth utilization rate can be manually set according to different application scenarios.

[0076] If the on-chip storage information used to initialize the neural network model does not exceed the target on-chip storage parameters, and the off-chip storage information does not exceed the target off-chip storage parameters, performance simulation can be performed by setting parameters such as the target bandwidth utilization rate on the simulator to obtain performance data.

[0077] Step 110: Determine the target neural network model based on the performance data of each initialized neural network model.

[0078] In this embodiment, if the performance data of at least one of the initialized neural network models meets the preset performance conditions, then that initialized neural network model is selected as the target neural network model. The performance data can be the chip inference time, i.e., the time required for the neural network model to run on the chip to complete the scenario task. The preset performance conditions can be a manually set target chip inference time. If the performance data of none of the initialized neural network models meets the preset performance conditions, the performance data is fed back to the model search system for a re-search. The model search system is the algorithm that carries the neural network search method of this application.

[0079] The neural network search method provided in this application determines the target network layer number; during the i-th iteration search process, multiple initial neural network models are determined based on the target network layer number, and the expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model; for any initial neural network model, if the judgment result indicates that the expressive power of the initial neural network model meets preset conditions, the on-chip storage information and off-chip storage information of the initial neural network model are determined; for any initial neural network model, if the on-chip storage information and off-chip storage information of the initial neural network model meet storage conditions, the performance data of the initial neural network model is determined based on the target bandwidth utilization rate; based on the performance data of each initial neural network model, the target neural network model is determined. The neural network search method provided in this application first searches based on a manually set target network layer number, which improves the search speed, and then combines expressive power, on-chip storage information, off-chip storage information, and target bandwidth utilization rate to search for the target neural network model, thereby enabling the rapid acquisition of a target neural network model with good performance and high accuracy on the deployed chip.

[0080] In one embodiment, the neural network search method further includes:

[0081] If the judgment results corresponding to each initial neural network model indicate that the expressive ability of the initial neural network model does not meet the preset conditions, the next iteration search process is entered. The steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment results corresponding to each initial neural network model are repeated until the initial neural network model corresponding to the judgment result indicating that the expressive ability of the initial neural network model meets the preset conditions is obtained, or until the number of iterations reaches the target number.

[0082] After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network is repeated, and the expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model is obtained.

[0083] The target number is the number of search iterations for the target, which can be set according to actual needs.

[0084] For example, the target network has 4 layers, and the target number of iterations is 10. In the first iteration, if the results of the search for multiple initial neural network models determined based on the target network's layer count all indicate that their expressive power does not meet the preset conditions, the process proceeds to the second iteration. Multiple initial neural network models are again determined based on the target network's layer count, and their expressive power is assessed. If the results of the second iteration still indicate that their expressive power does not meet the preset conditions, the process proceeds to the third iteration, repeating the above steps until an initial neural network model whose expressive power meets the preset conditions is obtained. Alternatively, if an initial neural network model whose expressive power meets the preset conditions is still not obtained after the 10th iteration, the target network's layer count can be increased to 5 layers, and the process of determining multiple initial neural network models based on the adjusted target network layer count, assessing the expressive power of each initial neural network model, and obtaining the corresponding assessment results for each initial neural network model can be repeated.

[0085] This embodiment of the present disclosure can search for an initial neural network model whose expressive power meets preset conditions within a limited number of searches based on the number of target network layers, thereby accelerating the search speed. Furthermore, by using both the number of target network layers and expressive power as constraints, the accuracy and performance of the final target neural network model are improved.

[0086] In one embodiment, such as Figure 2 As shown, in step 106, determining the on-chip storage information and off-chip storage information for initializing the neural network model may include:

[0087] Step 202: Determine the chip computation diagram based on the original network layers of the initialized neural network model.

[0088] The original network layer can be obtained by quantizing the initialized neural network model. The original network layer can include each layer of the network structure, and the number of layers in the network structure is the same as the number of layers in the target network. Each layer of the network structure in the original network layer is labeled and numbered. In the compiler, operator fusion and computational graph optimization are performed on the original network layer of the initialized neural network model to obtain the chip computational graph. The operator fusion and computational graph optimization techniques described here mainly target the efficient fusion between different computational networks to improve algorithm execution efficiency; computational graph optimization is the process of fine-tuning computations for different images, and it is also performed for computations of different networks. The chip computational graph is an abstract computational graph obtained by compiling the original network layer of the initialized neural network model into an operation process that conforms to chip language instructions. The chip computational graph can include multiple computational graphs, and each computational graph can include one operation process. There is a one-to-one correspondence between the operation process of each computational graph and each network structure in the original network layer.

[0089] For example, the weight parameters of an initial neural network model that meets preset expressive power conditions can be initialized using Gaussian distributed random data. Then, quantization is performed using random inputs to obtain the quantized original network layer. Quantization can be performed using 8-bit, 16-bit, or mixed quantization. Operator fusion can be achieved through convolution and the ReLU function (Rectified Linear Unit, a commonly used activation function in artificial neural networks), which can fuse operators into a single computational graph.

[0090] Step 204: Establish the layer mapping relationship between the original network layer and the chip computation graph.

[0091] In this chip computation graph, the computation process of each layer corresponds one-to-one with the network structure of each layer in the original network layer. During operator fusion and computation graph optimization of the original network layer that initializes the neural network model, the computation graphs in the chip computation graph can be numbered according to the numbers of each network structure in the original network layer. The correspondence between the numbers of each computation graph in the chip computation graph and the numbers of each network structure in the original network layer is the layer mapping relationship. For example, the optimized chip computation graph and the original quantized network layer are labeled and compared to establish the layer mapping relationship between the original network layer and the chip computation graph.

[0092] Step 206: Obtain the on-chip storage information of each layer of the chip computation graph, and obtain the off-chip storage information for initializing the neural network model.

[0093] The compiler compiles each layer of the computation graph in the chip computation graph to obtain the instruction set corresponding to each layer. Memory is allocated for each instruction set so that the instruction set corresponding to each layer can be simulated and operated within the allocated storage space. The memory allocated to the instruction set of the computation graph during operation is the sub-chip storage information. The off-chip storage information for initializing the neural network model can be directly obtained by determining the amount of off-chip storage occupied by the initialized neural network model deployed on the chip. For example, during the process of allocating storage space to compile the chip computation graph, the sub-chip storage information for each layer of the computation graph can be verified and recorded. This sub-chip storage information includes the storage size occupied by the on-chip SRAM (Static Random-Access Memory).

[0094] Step 208: Determine the on-chip storage information for initializing the neural network model based on the layer mapping relationship and the sub-chip storage information of each layer's computation graph.

[0095] After determining the sub-chip storage information of each layer's computation graph, the sub-chip storage information of each layer's network structure in the corresponding original network layer can be obtained based on the layer mapping relationship. The in-chip storage information for initializing the neural network model can include the sub-chip storage information of each layer's network structure in the original network layer, or it can include the sum of the sub-chip storage information of all layers' network structures.

[0096] In this embodiment of the disclosure, determining the off-chip storage information for initializing the neural network model and determining the on-chip storage information for initializing the neural network model through quantization compilation can incorporate the width requirement of the neural network model into the neural network search process, so that the target neural network model can make full use of the chip's on-chip and off-chip storage.

[0097] In one embodiment, such as Figure 3 As shown, in step 108, if the on-chip and off-chip storage information for initializing the neural network model meets the storage conditions, the performance data for initializing the neural network model is determined based on the target bandwidth utilization, which may include:

[0098] Step 302: Obtain the target storage parameters, which include the target off-chip storage parameters and the target on-chip storage parameters.

[0099] The target storage parameters can include target off-chip storage parameters and target on-chip storage parameters. Target off-chip storage parameters refer to the spatial parameters of off-chip storage, that is, the maximum amount of storage the target neural network model can occupy off-chip. Target on-chip storage parameters refer to the spatial parameters of on-chip storage, that is, the maximum amount of storage the target neural network model can occupy on-chip. Target storage parameters can be set according to actual needs.

[0100] Step 304: The on-chip storage information and off-chip storage information of the initialized neural network model are compared with the target off-chip storage parameters and the target on-chip storage parameters to obtain the comparison result of the initialized neural network model. The comparison result includes a first comparison result or a second comparison result. The first comparison result indicates that the on-chip storage information and off-chip storage information meet the storage conditions, and the second comparison result indicates that the on-chip storage information and off-chip storage information of the initialized neural network model do not meet the storage conditions.

[0101] Specifically, the off-chip storage information of the initialized neural network model can be compared with the target off-chip storage parameters. The comparison result may include information on redundancy or overflow in the off-chip storage. Alternatively, the sub-on-chip storage information of each layer of the network structure in the initialized neural network model can be compared with the target on-chip storage parameters; or, the sum of the sub-on-chip storage information of each layer of the network structure in the initialized neural network model can be compared with the target on-chip storage parameters. The comparison result may include information on redundancy or overflow in the on-chip storage. The storage condition is that the off-chip storage information of the initialized neural network model is less than or equal to the target off-chip storage parameters, and the on-chip storage information is less than or equal to the target on-chip storage parameters.

[0102] Step 306: After obtaining the first comparison result, determine the performance data of the initialized neural network model based on the target bandwidth utilization.

[0103] The target bandwidth utilization rate can be set according to different application scenarios. After obtaining the first comparison result, i.e., the initial neural network model meets the storage conditions, the target bandwidth utilization rate and target storage parameters can be set in the simulator to perform performance simulation on the initial neural network model and obtain performance data. Performance data can include chip inference time, i.e., the time it takes for the initial neural network model to complete the application scenario on the chip, such as a classification task.

[0104] In this embodiment, the width (storage) requirement of the neural network model is incorporated into the neural network search process. This ensures that performance data is calculated only when the requirements for expressiveness, on-chip storage, and off-chip storage are met, thereby accelerating the search speed of the neural network model and enabling the target neural network model to be searched to simultaneously achieve both performance and accuracy.

[0105] In one embodiment, in step 304, the on-chip storage information and off-chip storage information of the initialized neural network model are compared with the target off-chip storage parameters and the target on-chip storage parameters to obtain the comparison result, which may include:

[0106] A first comparison result is obtained when the on-chip storage information is less than or equal to the target on-chip storage parameters, and the off-chip storage information is less than or equal to the target off-chip storage parameters; or a second comparison result is obtained when the on-chip storage information is greater than the target on-chip storage parameters, or the off-chip storage information is greater than the target on-chip storage parameters.

[0107] Specifically, the off-chip storage information of the initialized neural network model can be compared with the target off-chip storage parameters. Alternatively, the sub-on-chip storage information of each layer of the network structure in the initialized neural network model can be compared with the target on-chip storage parameters; or, the sum of the sub-on-chip storage information of each layer of the network structure in the initialized neural network model can be compared with the target on-chip storage parameters. If either the on-chip storage information or the off-chip storage information exceeds the corresponding target storage information, a second comparison result is obtained, the performance inference step is terminated, the storage information is fed back to the model search system algorithm, and the search is restarted.

[0108] In this embodiment, the width (storage) requirement of the neural network model is incorporated into the neural network search process. This ensures that performance data is calculated only when the requirements for expressiveness, on-chip storage, and off-chip storage are met, thereby accelerating the search speed of the neural network model and enabling the target neural network model to be searched to simultaneously achieve both performance and accuracy.

[0109] In one embodiment, the neural network search method may further include:

[0110] If the comparison results for each initial neural network model are all the second comparison results, proceed to the next iteration search process. Use the comparison results for each initial neural network model as constraints, repeat the steps of determining multiple initial neural network models based on the target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment results for each initial neural network model, until the initial neural network model corresponding to the first comparison result is obtained, or until the number of iterations reaches the target number.

[0111] After adjusting the target network layer number and / or target storage parameters, the process involves repeatedly determining multiple initial neural network models based on the adjusted target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model.

[0112] For example, the target network has 4 layers, and the target number of iterations is 10. In the first iteration search, if the comparison results of multiple initial neural network models whose expressive power meets the preset conditions are all the second comparison results, then the second iteration search process begins. Each second comparison result is used as a constraint, and multiple initial neural network models are determined again based on the target network layer count. For example, if the second comparison result indicates that the on-chip storage information of layer 4 exceeds the target storage parameter, the storage parameter amount of layer 5 can be limited in the second iteration search. If the total off-chip storage information exceeds the target storage parameter, the parameter amount can be gradually limited and re-searched in the second iteration search, sorted by the storage size of each layer. The expressive power of the multiple initial neural network models obtained from the re-search is judged, and the comparison results of multiple initial neural network models whose expressive power meets the preset conditions are determined. If the comparison results of multiple initial neural network models whose expressive power meets the preset conditions obtained in the second search are still all the second comparison results, then the third iteration search process begins, repeating the above steps until the initial neural network model corresponding to the first comparison result is obtained. Alternatively, if the initial neural network model corresponding to the first comparison result is still not obtained after the 10th iteration search, the number of target network layers can be increased to 5 layers or the target storage parameters can be increased. Then, the steps of determining multiple initial neural network models based on the adjusted number of target network layers can be repeated, and the expressive power of each initial neural network model can be judged to obtain the judgment result corresponding to each initial neural network model can be obtained.

[0113] This embodiment of the present disclosure can search for an initial neural network model that meets the conditions of expressive power and on-chip and off-chip storage within a limited number of searches based on the number of target network layers, thereby accelerating the search speed and improving the accuracy and performance of the final target neural network model.

[0114] In one embodiment, performance data includes inference time. For example... Figure 4 As shown, in step 108, determining the performance data for initializing the neural network model based on the target bandwidth utilization may include:

[0115] Step 402: Determine the target bandwidth utilization.

[0116] The target bandwidth utilization rate is the maximum bandwidth that the target neural network model to be searched can occupy during the deployment of hardware. The target bandwidth utilization rate can be set according to different application scenarios.

[0117] Step 404: Determine the inference time of the initialized neural network model based on the target bandwidth utilization and target storage parameters. The inference time represents the time required for the initialized neural network model to complete the target scenario task.

[0118] The target scenario task refers to the computational objective that the neural network model needs to perform in the application scenario, such as face recognition and classification.

[0119] For example, the bandwidth utilization rate can be set as a percentage according to the application scenario in the simulator, such as a target bandwidth utilization rate of 40%. The inference performance of the initialized neural network model that meets the storage requirements is simulated to obtain the inference time. In a classification application scenario, if the time for the initialized neural network model to complete classification is 50 seconds, then the corresponding inference time for this initialized neural network model is 50 seconds.

[0120] In this embodiment, the performance requirements of the neural network model are incorporated into the neural network search process. This ensures that the target neural network model is obtained only when the requirements for expressive power, on-chip storage, off-chip storage, and performance are met. This accelerates the search speed of the neural network model and ensures that the searched target neural network model balances both performance and accuracy.

[0121] In one embodiment, determining the target neural network model in step 110 based on the performance data of each initialized neural network model may include:

[0122] If the performance data of the initial neural network model meets the target performance conditions, the initial neural network model is used as the target neural network model; or, if the performance data of each initial neural network model does not meet the target performance conditions, the next iteration search process is entered, and the steps of determining multiple initial neural network models according to the target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model are repeated until the target neural network model is obtained, or until the number of iterations reaches the target number.

[0123] After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network is repeated, and the expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model is obtained.

[0124] For example, the target performance condition can be a preset target inference time, such as 10 seconds. If the inference time of the initialized neural network model is less than or equal to the target inference time, the target performance condition is met. If the inference time of the initialized neural network model obtained in the simulator is 8 seconds, which is less than the target inference time of 10 seconds, then the initialized neural network model simultaneously meets the requirements of expressive power, storage, and performance, and is considered the target neural network model. If the performance data of the initialized neural network model does not meet the target performance condition, the iterative search can be repeated. When the target number of searches (when the reinforcement learning or evolutionary algorithm has converged to the best result) still fails to converge to the ideal neural network model, the number of network layers can be increased, and the above steps can be repeated.

[0125] In this embodiment, the performance requirements of the neural network model are incorporated into the neural network search process. This ensures that the target neural network model is obtained only when the requirements for expressiveness, on-chip storage, off-chip storage, and performance are met. This accelerates the search speed of the neural network model and ensures that the searched target neural network model balances both performance and accuracy.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] For a better understanding of the embodiments of this application, see [link to relevant documentation]. Figure 5This application provides a most complete embodiment. The neural network search method provided in this application fully utilizes the storage and bandwidth of the deployed chip to guide the neural network search, accelerating the convergence speed of the neural network model search (the speed of searching the neural network), while ensuring the chip inference time (the inference time of the neural network on the chip) and model accuracy. The specific method is as follows: First, according to different application scenarios, different bandwidth utilization rates, off-chip spatial parameters, and on-chip spatial parameters are manually set. The search space of the network and the target dataset are selected, and the number of initial network search layers is manually set, for example, 20 layers. Image and speech data streams will occupy bandwidth. In some scenarios, the bandwidth utilization rate of the neural network model calculation is 70-80%, and in some scenarios, the bandwidth utilization rate is 20-30%. Then, the expressive power of the initialized search network structure is judged to see if it meets the standard. If it meets the standard of a certain expressive power, it means that the neural network model can reach the accuracy requirements after training. Then, network layer labeling and compilation optimization are performed; if it does not meet the standard, it is fed back to the search system to increase the network structure parameters (width, number of layers, etc.) and search again.

[0128] The second step involves initializing the network weights using Gaussian-distributed random data based on the network structure obtained in the first step. Then, quantization is performed using random inputs; quantization can be 8-bit, 16-bit, or a hybrid approach, resulting in the original quantized model layers. The compiler then optimizes the compilation process, primarily through operator fusion (for example, convolution + ReLU can be fused into a single layer) and computational graph optimization. During optimization, the layers are compared with the original quantized model layers; the optimized computational graph still maintains the correspondence between its labels and the original model layers. Finally, the computational graph, after operator fusion and computational graph optimization by the compiler, establishes a layer mapping relationship with the original network graph. Then, memory allocation instructions are executed, allocating memory for each layer of the computation graph, generating instruction sets, and completing compilation. Simultaneously, based on pre-set on-chip space parameters, the on-chip storage redundancy of each layer of the current computation graph is calculated. If on-chip storage is insufficient, the storage deficit for the current layer is determined. Then, all layer on-chip storage information (redundancy and deficit) is used to calculate the parameter storage information for each layer of the original network graph through layer mapping relationships. After compilation, the total off-chip storage is calculated based on pre-set off-chip space parameters to determine if a reasonable storage size has been reached. If either on-chip or off-chip storage is insufficient, the next performance inference step is terminated, the storage information is fed back to the model search system, and the search is restarted until the expression is satisfied. The process involves defining the requirements for on-chip storage and off-chip storage, and then proceeding to the next step. For example, the space is allocated to compile the computational graph, and the storage information for each layer is verified. This storage information includes the remaining SRAM redundancy and the amount of storage overflow. Finally, the size of the compiled file is used to check whether it exceeds the reasonable range of off-chip storage. If either on-chip or off-chip storage exceeds the storage range, the network search system is notified to re-search, and adjustments are made based on the storage information of each layer. For example, if the parameter storage of layer 5 in the current search result exceeds the limit, the storage parameter quantity of layer 5 can be limited in the next search. If the overall off-chip storage exceeds the limit, the parameter quantity can be gradually limited according to the size of each layer's storage, and the search can be repeated.

[0129] The third step involves model performance inference. The simulator reads and sets the bandwidth utilization based on the upper-layer application scenario, then sets relevant parameters such as off-chip and on-chip spatial parameters, performs performance simulation, and finally feeds the performance data (inference time) back to the model search system for a re-search. The fourth step involves manually initializing the network layers and setting a certain number of search iterations based on steps one through three. Therefore, if a certain number of search iterations (when reinforcement learning or evolutionary algorithms have converged to the best result) still fails to converge to the ideal network structure model, the number of network layers can be increased, and the search process can be repeated until a neural network model that fully utilizes both on-chip and off-chip storage, meets accuracy requirements, and has optimal inference performance is found.

[0130] The neural network search method provided in this application combines the impact of the storage size and bandwidth of the deployed hardware on model search, providing a better direction for model search design. It solves the problems of how to use hardware information and how to achieve fast network structure search. It proposes a neural network search method guided by the utilization of hardware storage and bandwidth. This method first searches for networks with sufficient accuracy based on the number of layers and search space of the target network. Then, it performs quantization, compilation, simulation, and inference. Finally, it uses hardware storage and bandwidth information to provide feedback on the model inference time and whether each layer of the original model can be widened or replaced with operations with larger parameters, thereby guiding the model search. To accelerate the search speed, the neural network search method provided in this application first searches based on a manually set number of network layers, then judges the expressive power, performs quantization and compilation, calculates the on-chip storage data size of each layer and the overall off-chip storage information, and finally uses a simulator combined with bandwidth to feed back the performance of the searched network model to the model search system. This enables the rapid acquisition of networks with good performance on the deployed chip while maintaining high accuracy. It has significant value and meaning for designing different efficient network structures for different hardware architectures to fully utilize the computing, storage resources, and bandwidth of the underlying chip.

[0131] Based on the same inventive concept, this application also provides a device for implementing the neural network search described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more neural network search device embodiments provided below can be found in the limitations of the neural network search method described above, and will not be repeated here.

[0132] In one embodiment, see Figure 6 A neural network search device 600 is provided. The neural network search device 600 includes:

[0133] The network layer number determination module 602 is used to determine the target network layer number;

[0134] The expressive ability judgment module 604 is used to determine multiple initial neural network models based on the number of target network layers during the i-th iteration search process, judge the expressive ability of each initial neural network model, and obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0;

[0135] The storage information determination module 606 is used to determine the on-chip storage information and off-chip storage information of any initialized neural network model, provided that the judgment result indicates that the expressive ability of the initialized neural network model meets the preset conditions.

[0136] The performance data determination module 608 is used to determine the performance data of the initial neural network model based on the target bandwidth utilization, provided that the on-chip storage information and off-chip storage information of the initial neural network model meet the storage conditions for any initial neural network model.

[0137] The target network determination module 610 is used to determine the target neural network model based on the performance data of each initialized neural network model.

[0138] The neural network search device provided in this application determines the number of target network layers; during the i-th iteration search, multiple initial neural network models are determined based on the number of target network layers, and the expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model; for any initial neural network model, if the judgment result indicates that the expressive power of the initial neural network model meets preset conditions, the on-chip storage information and off-chip storage information of the initial neural network model are determined; for any initial neural network model, if the on-chip storage information and off-chip storage information of the initial neural network model meet storage conditions, the performance data of the initial neural network model is determined based on the target bandwidth utilization rate; based on the performance data of each initial neural network model, the target neural network model is determined. The neural network search device provided in this application first searches based on a manually set target network layer number, improving the search speed, and then combines expressive power, on-chip storage information, off-chip storage information, and target bandwidth utilization rate to search for the target neural network model, thereby enabling the rapid acquisition of target neural network models with good performance and high accuracy on the deployed chip.

[0139] In one embodiment, the neural network search device 600 further includes an iteration device. The iteration device is used to, when the judgment results corresponding to each initialized neural network model all indicate that the expressive power of the initialized neural network model does not meet a preset condition, enter the next iteration search process, repeatedly determining multiple initialized neural network models based on the target network layer number, judging the expressive power of each initialized neural network model, and obtaining the judgment results corresponding to each initialized neural network model, until an initialized neural network model corresponding to a judgment result indicating that the expressive power of the initialized neural network model meets the preset condition is obtained, or until the number of iterations reaches a target number; after adjusting the target network layer number, repeatedly determining multiple initialized neural network models based on the adjusted target network layer number, judging the expressive power of each initialized neural network model, and obtaining the judgment results corresponding to each initialized neural network model.

[0140] In one embodiment, the storage information determination module 606 is further configured to determine the chip computation graph based on the original network layer of the initialized neural network model; establish a layer mapping relationship between the original network layer and the chip computation graph; obtain the sub-on-chip storage information of each layer of the chip computation graph, and obtain the off-chip storage information of the initialized neural network model; and determine the on-chip storage information of the initialized neural network model based on the layer mapping relationship and the sub-on-chip storage information of each layer of the computation graph.

[0141] In one embodiment, the performance data determination module 608 is further configured to acquire target storage parameters, including target off-chip storage parameters and target on-chip storage parameters; compare the on-chip storage information and off-chip storage information of the initialized neural network model with the target off-chip storage parameters and target on-chip storage parameters to obtain a comparison result of the initialized neural network model. The comparison result includes a first comparison result or a second comparison result. The first comparison result indicates that the on-chip storage information and off-chip storage information meet the storage conditions, and the second comparison result indicates that the on-chip storage information and off-chip storage information of the initialized neural network model do not meet the storage conditions; if the first comparison result is obtained, the performance data of the initialized neural network model is determined based on the target bandwidth utilization.

[0142] In one embodiment, the performance data determination module 608 is further configured to obtain a first comparison result when the on-chip storage information is less than or equal to the target on-chip storage parameter and the off-chip storage information is less than or equal to the target off-chip storage parameter; or to obtain a second comparison result when the on-chip storage information is greater than the target on-chip storage parameter and / or the off-chip storage information is greater than the target on-chip storage parameter.

[0143] In one embodiment, the iterative device is further configured to, when the comparison results corresponding to each initial neural network model are all the second comparison results, enter the next iterative search process, using the comparison results corresponding to each initial neural network model as constraints, repeatedly determine multiple initial neural network models based on the target network layer number, judge the expressive power of each initial neural network model, and obtain the judgment results corresponding to each initial neural network model, until the initial neural network model corresponding to the first comparison result is obtained, or until the number of iterations reaches the target number; after adjusting the target network layer number and / or target storage parameters, repeatedly determine multiple initial neural network models based on the adjusted target network layer number, judge the expressive power of each initial neural network model, and obtain the judgment results corresponding to each initial neural network model.

[0144] In one embodiment, the performance data includes inference time. The performance data determination module 608 is further configured to determine the target bandwidth utilization; and based on the target bandwidth utilization and target storage parameters, determine the inference time for initializing the neural network model, wherein the inference time characterizes the time required for the initialized neural network model to complete the target scenario task.

[0145] In one embodiment, the target network determination module 610 is further configured to: use the initial neural network model as the target neural network model if the performance data of the initial neural network model meets the target performance conditions; or, if the performance data of each initial neural network model does not meet the target performance conditions, enter the next iteration search process, repeat the steps of determining multiple initial neural network models according to the number of target network layers, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model, until the target neural network model is obtained, or until the number of iterations reaches the target number; after adjusting the number of target network layers, repeat the steps of determining multiple initial neural network models according to the adjusted number of target network layers, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model.

[0146] Each module in the aforementioned neural network search device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0147] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a neural network search method.

[0148] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0150] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments. In another embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A neural network search method, characterized in that, The method includes: Determine the target number of network layers; During the i-th iteration search process, multiple initial neural network models are determined according to the number of target network layers. The expressive power of each initial neural network model is judged to obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0. For any of the initial neural network models, if the judgment result indicates that the expressive power of the initial neural network model meets the preset conditions, the chip computing graph is determined based on the original network layer of the initial neural network model; a layer mapping relationship is established between the original network layer and the chip computing graph; the sub-on-chip storage information of each layer of the computing graph in the chip computing graph is obtained, and the off-chip storage information of the initial neural network model is obtained; the on-chip storage information of the initial neural network model is determined based on the layer mapping relationship and the sub-on-chip storage information of each layer of the computing graph. For any of the initialization neural network models, if the on-chip storage information and the off-chip storage information of the initialization neural network model meet the storage conditions, the performance data of the initialization neural network model is determined based on the target bandwidth utilization. The target neural network model is determined based on the performance data of each of the initial neural network models.

2. The method according to claim 1, characterized in that, The method further includes: If the judgment results corresponding to each of the initial neural network models indicate that the expressive ability of the initial neural network model does not meet the preset condition, the next iteration search process is entered. The steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment results corresponding to each initial neural network model are repeated until an initial neural network model corresponding to the judgment result indicating that the expressive ability of the initial neural network model meets the preset condition is obtained, or until the number of iterations reaches the target number. After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

3. The method according to claim 1, characterized in that, When the on-chip storage information and off-chip storage information of the initialized neural network model meet the storage conditions, the performance data of the initialized neural network model is determined based on the target bandwidth utilization, including: Obtain target storage parameters, which include target off-chip storage parameters and target on-chip storage parameters; The on-chip storage information and off-chip storage information of the initialized neural network model are compared with the target off-chip storage parameters and the target on-chip storage parameters to obtain the comparison result of the initialized neural network model. The comparison result includes a first comparison result or a second comparison result. The first comparison result indicates that the on-chip storage information and off-chip storage information meet the storage conditions, and the second comparison result indicates that the on-chip storage information and off-chip storage information of the initialized neural network model do not meet the storage conditions. Based on the first comparison result, the performance data of the initialized neural network model is determined according to the target bandwidth utilization.

4. The method according to claim 3, characterized in that, The method further includes: If the comparison results corresponding to each of the initial neural network models are the second comparison results, proceed to the next iteration search process. Use the comparison results corresponding to each of the initial neural network models as constraints, repeatedly determine multiple initial neural network models according to the target network layer number, judge the expressive power of each initial neural network model, and obtain the judgment results corresponding to each initial neural network model, until the initial neural network model corresponding to the first comparison result is obtained, or until the number of iterations reaches the target number. After adjusting the target network layer number and / or the target storage parameters, the process of repeatedly determining multiple initial neural network models based on the adjusted target network layer number, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

5. The method according to claim 3, characterized in that, The performance data includes inference time, and determining the performance data of the initialized neural network model based on the target bandwidth utilization includes: Determine the target bandwidth utilization rate; Based on the target bandwidth utilization and the target storage parameters, the inference time of the initial neural network model is determined, and the inference time represents the time it takes for the initial neural network model to complete the target scenario task.

6. The method according to claim 1, characterized in that, The step of determining the target neural network model based on the performance data of each of the initialized neural network models includes: If the performance data of the initialized neural network model meets the target performance conditions, the initialized neural network model is used as the target neural network model; or, If the performance data of each of the initial neural network models does not meet the target performance condition, the next iteration search process is entered. The steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model are repeated until the target neural network model is obtained, or until the number of iterations reaches the target number. After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

7. A neural network search device, characterized in that, The device includes: The network layer determination module is used to determine the target network layer number; The expressive ability judgment module is used to determine multiple initial neural network models based on the number of target network layers during the i-th iteration search process, judge the expressive ability of each initial neural network model, and obtain the judgment result corresponding to each initial neural network model, where i is an integer greater than 0; The storage information determination module is used, for any of the initialized neural network models, when the judgment result indicates that the expressive power of the initialized neural network model meets preset conditions, to determine the chip computing graph based on the original network layer of the initialized neural network model; establish a layer mapping relationship between the original network layer and the chip computing graph; obtain the sub-on-chip storage information of each layer of the computing graph in the chip computing graph, and obtain the off-chip storage information of the initialized neural network model; and determine the on-chip storage information of the initialized neural network model based on the layer mapping relationship and the sub-on-chip storage information of each layer of the computing graph. The performance data determination module is used to determine the performance data of any of the initialized neural network models based on the target bandwidth utilization, provided that the on-chip storage information and the off-chip storage information of the initialized neural network model meet the storage conditions. The target network determination module is used to determine the target neural network model based on the performance data of each of the initial neural network models.

8. The apparatus according to claim 7, characterized in that, The neural network search device further includes an iterative device, which is used to enter the next iterative search process when the judgment results corresponding to each of the initial neural network models all indicate that the expressive ability of the initial neural network model does not meet the preset condition. The iterative device repeats the steps of determining multiple initial neural network models according to the target network layer number, judging the expressive ability of each initial neural network model, and obtaining the judgment results corresponding to each initial neural network model, until an initial neural network model corresponding to the judgment result indicating that the expressive ability of the initial neural network model meets the preset condition is obtained, or until the number of iterations reaches the target number. After adjusting the number of layers in the target network, the process of determining multiple initial neural network models based on the adjusted number of layers in the target network, judging the expressive power of each initial neural network model, and obtaining the judgment result corresponding to each initial neural network model is repeated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images