A neural network acquisition method and device

Abstract

A neural network acquisition method and device, the method and device provided in this application can be applied to the field of artificial intelligence, such as smart home, autonomous driving and other scenarios. In this application, the neural network acquisition device can obtain a target neural network with a relatively accurate estimated delay for a target hardware platform. The estimated delay is close to the actual delay of the target neural network on the target hardware platform, and the difference is within the standard deviation interval. The standard deviation interval is determined by the distribution of actual delays measured multiple times by the target neural network on the target hardware platform. Using the method of this application, the estimated delay can more realistically reflect the delay of the target neural network on the target hardware platform, so that users who need the target neural network can truly understand the performance of the target neural network to a certain extent, and better meet the needs of users.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method and device for acquiring a neural network. Background Art

[0002] Artificial neural networks (ANNs), also known as neural networks (NNs), are a mathematical model of algorithms that imitate the behavioral characteristics of animal neural networks to process information. Neural networks can be used to perform complex computational reasoning. Usually, a neural network includes multiple layers of networks, and each layer of the network can use different sub-modules. By splicing different sub-modules, neural networks with different structures can be formed. Neural networks with different structures have large differences in accuracy and overall network latency.

[0003] Different hardware platforms can run different neural networks. For example, a neural network with more layers can be run on the cloud or large servers. Given the powerful computing power (also called computing power) of the cloud or large servers, a neural network with more layers can achieve higher accuracy. However, for edge computing devices, due to their limited computing power, the latency of such a neural network with more layers running on edge computing devices will increase significantly, affecting the reasoning efficiency of the neural network.

[0004] Neural network search refers to searching for neural networks that are more suitable for hardware platforms based on user needs, such as the number of layers, accuracy, and overall network latency. At present, the latency of neural networks needs to be estimated during the neural network search process. There are two common methods: Method 1: Approximately measure the latency of the neural network on the hardware platform based on the computational complexity of the neural network itself (such as the number of floating-point operations included in the neural network). Method 2: Estimate the latency of the neural network based on the performance of the hardware platform (such as the number of floating-point operations that the hardware platform can process per second).

[0005] There is a large difference between the delay of the neural network estimated by these two methods and the actual delay of the neural network, which will eventually lead to a decrease in the accuracy of the neural network search. Summary of the invention

[0006] The present application provides a neural network acquisition method and device to ensure the accuracy of neural network delay estimation.

[0007] In a first aspect, an embodiment of the present application provides a neural network acquisition method, which can be executed by a neural network acquisition device. In the method, the neural network acquisition device can first determine a target hardware platform, and then acquire a target neural network for the target hardware platform. The difference between the estimated delay of the target neural network on the target hardware platform and the actual delay of the target neural network on the target hardware platform is within a standard deviation interval, and the standard deviation interval is determined based on the distribution of actual delays of the target neural network measured multiple times on the target hardware platform.

[0008] Through the above method, the estimated delay of the target neural network obtained by the neural network acquisition device on the target hardware platform is closer to the actual delay of the target neural network on the target hardware platform, and the difference between the estimated delay and the actual delay is within a smaller standard deviation range, so that the target neural network obtained by the neural network acquisition device is more suitable for the target hardware platform, thereby improving the accuracy of neural network search.

[0009] In one possible design, when the neural network acquisition device acquires the target neural network for the target hardware platform, it can first acquire a delay set, which includes the delays of multiple sub-modules on different hardware platforms; thereafter, the neural network acquisition device can acquire the target neural network for the target hardware platform based on the delay set, the target neural network is composed of part or all of the multiple sub-modules, and the estimated delay is determined based on the delay set.

[0010] Through the above method, since the delay set includes the delays of various sub-modules that may be used to constitute the target neural network on different hardware platforms, the neural network acquisition device can accurately estimate the delay of the target neural network on the target hardware platform when acquiring the target neural network for the target hardware platform. The estimated delay can be closer to the actual delay, thereby improving the accuracy of the estimated delay of the target neural network. In addition, due to the existence of the delay set, the neural network acquisition device can use the delay set to calculate the estimated delay more quickly when acquiring the target neural network, thereby reducing the time to acquire the target neural network and improving the speed of neural network search.

[0011] In one possible design, the number of layers of the target neural network conforms to a range of layer numbers, and the estimated delay is the neural network with the highest accuracy within a target delay range, and the target delay range is confirmed based on user input.

[0012] Through the above method, the target neural network acquired by the neural network acquisition device better meets the needs of users and can improve the user experience.

[0013] In one possible design, when determining the target hardware platform, the neural network acquisition device may first prompt the user to input or select the target hardware platform; after detecting the user's input or selection, the target hardware platform may be determined based on the user's input or selection.

[0014] Through the above method, the neural network acquisition device can determine the target hardware platform according to the user's selection or input, so that the target neural network can be acquired for the target hardware platform selected or input by the user later.

[0015] In one possible design, the delay set can be constructed by other devices and pre-configured to the neural network acquisition device, or it can be constructed by the neural network device itself. The method of constructing the delay set is described below. Specifically: sub-modules can be randomly selected from multiple sub-modules to form a test neural network. The number and number of layers of the test neural network are not limited here; then, the test neural network is run on different hardware platforms to measure the delays of each sub-module in the test neural network on different hardware platforms; and the delay set is constructed using the measured delays of each sub-module in the test neural network on different hardware platforms.

[0016] Optionally, the delays of each submodule in the test neural network on different hardware platforms may be measured and calibrated first. After calibration, a delay set may be constructed using the calibrated delays of each submodule in the test neural network on different hardware platforms.

[0017] Through the above method, the delay of each sub-module in the delay set on different hardware platforms is actually measured, which can ensure the accuracy of the delay of each sub-module in the delay set, and thus make the estimated delay of the target neural network more accurate.

[0018] In a possible design, for any of the different hardware platforms, the delay set records some or all of the following:

[0019] The correspondence between the delay of each submodule on the hardware platform and the identification information of the hardware platform;

[0020] The correspondence between the delay of each submodule on the hardware platform and the identification information of the processing unit in the hardware platform.

[0021] Through the above method, the delay set establishes the corresponding relationship between the delay, the hardware platform and the processing unit, so that the delay can be estimated based on the delay set calculation in the future.

[0022] In a possible design, for any hardware platform among different hardware platforms, if the hardware platform includes multiple processing units of the same model, the delay of the submodule on the hardware platform is equal to the time consumed by one processing unit in the hardware platform to execute the operation involved in the submodule;

[0023] If the hardware platform includes multiple processing units of different models, a delay of a sub-module on the hardware platform is the delay of a processing unit of one model executing the sub-module, and the delay of a processing unit of this model executing the sub-module is equal to the time consumed by the calculations involved in the execution of the sub-module by a processing unit of this model in the hardware platform.

[0024] Through the above method, the delay of the sub-module on the hardware platform is related to the processing unit of the hardware platform. That is to say, for hardware devices including processing units of the same model, the delay of the sub-module on the hardware device is the same, and the hardware devices including processing units of the same model can be regarded as the same hardware platform.

[0025] In one possible design, if the target hardware platform includes multiple processing units of the same model, the estimated delay is equal to the sum of the delays of the various sub-modules in the delay set that constitute the target neural network on the target hardware platform.

[0026] If the target hardware platform includes multiple processing units of different models, an estimated delay is equal to the sum of the delays of one model of processing units in the delay set executing various sub-modules that constitute the target neural network.

[0027] Through the above method, the estimated delay can be calculated by simple summation. The calculation method of the estimated delay is simple and fast, which ensures the efficiency of the neural network search.

[0028] In one possible design, if the target hardware platform includes multiple processing units of different models, the neural network acquisition device can also determine the target processing unit in the target hardware platform based on user input; in this case, the estimated delay is equal to the sum of the delays of the target processing unit in the delay set executing each sub-module that constitutes the target neural network.

[0029] Through the above method, the estimated delay can be quickly obtained through simple summation, which can further improve the efficiency of neural network search.

[0030] In one possible design, when the hardware platform involved in the delay set is different from the target hardware platform, if the hardware platform involved in the delay set includes a candidate hardware platform that has the same processing unit (such as the same model) as the target hardware platform, the delay of each sub-module on the candidate hardware platform can be used to obtain an estimated delay, and the estimated delay is equal to the sum of the delays of each sub-module of the target neural network on the candidate hardware platform.

[0031] Through the above method, the candidate hardware platform is equated with the target hardware platform, and the estimated delay can be calculated by simple summation. The calculation method of the estimated delay is simple and fast, which ensures the efficiency of the neural network search.

[0032] In one possible design, the standard deviation interval falls within {-0.6, 0.6}.

[0033] Through the above method, the range of the standard deviation interval is relatively small, indicating that the difference between the estimated delay and the actual delay of the target neural network on the target hardware platform is within a smaller range, and the estimated delay and the actual delay are relatively close.

[0034] In a second aspect, the present application provides a neural network acquisition device, which has the function of implementing the first aspect and any possible design of the first aspect. The device function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the device includes a determination module and an acquisition module, which can perform the corresponding functions in the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0035] In a third aspect, the present application also provides a neural network acquisition device, and the beneficial effects can be found in the description of the first aspect and any possible design of the first aspect, and will not be repeated here. The structure of the neural network acquisition device includes a processor and a memory, and the processor is configured to perform the corresponding functions in the above-mentioned first aspect and any possible design of the first aspect. The memory is coupled to the processor, which stores the necessary program instructions and data for the neural network acquisition device. The structure of the neural network acquisition device also includes a communication interface for communicating with other devices.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned methods.

[0037] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned methods.

[0038] In a sixth aspect, the present application also provides a computer chip, which is connected to a memory, and the chip is used to read and execute a software program stored in the memory to perform the methods of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1AA schematic diagram of the architecture of a system provided for this application;

[0040] Figure 1B A schematic diagram of the architecture of a system provided for this application;

[0041] Figure 1C A schematic diagram of the architecture of a system provided for this application;

[0042] Figure 2 A schematic diagram of a neural network search method provided in this application;

[0043] Figure 3A-3B A schematic diagram of a configuration interface of a target hardware platform provided for this application;

[0044] Figure 4 A schematic diagram of a configuration interface for a range of layers provided in this application;

[0045] Figures 5A-5B A schematic diagram of a configuration interface for a target delay range provided in this application;

[0046] Figure 6 A schematic diagram of the relationship between the middle value of the target delay range provided in this application and β;

[0047] Figure 7 Schematic diagram of the delay of different neural networks on edge devices provided for this application;

[0048] Figure 8 Schematic diagram of the accuracy of different neural networks provided for this application on edge devices;

[0049] Fig. 9 A schematic diagram of the delay of different neural networks provided in this application on terminal devices;

[0050] Fig.10 A schematic diagram of the accuracy of different neural networks provided for this application on terminal devices;

[0051] Fig.11 A schematic diagram showing the multiples of the speed of acquiring a target neural network using the neural network acquisition method provided by the present application and the speed of acquiring a target neural network in the prior art;

[0052] Fig.12 A schematic diagram of the structure of a neural network acquisition device provided in this application. DETAILED DESCRIPTION

[0053] like Figure 1A As shown, it is a schematic diagram of a system architecture applicable to an embodiment of the present application, wherein the system includes a neural network acquisition device 100 and a client 200, and the client 200 is deployed on the user side.

[0054] The embodiment of the present application does not limit the deployment location of the neural network acquisition device 100. For example, the neural network acquisition device 100 can be deployed in an edge data center, a cloud data center, or a terminal computing device. The neural network acquisition device 100 can also be deployed in a distributed manner in part or all of the environments of the edge data center, the cloud data center, and the terminal computing device.

[0055] The neural network acquisition device 100 may be a hardware device, such as a server, a server cluster, or a terminal computing device, and includes a bus 101, a processor 102, a communication interface 103, and a memory 104. The processor 102, the memory 104, and the communication interface 103 communicate with each other via the bus 101.

[0056] Among them, the processor 102 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (AI) chip, a system on chip (SoC) or a complex programmable logic device (CPLD), a graphics processing unit (GPU), etc.

[0057] The memory 104 may include a volatile memory, such as a random access memory (RAM). The memory 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 104 stores executable code, and the processor 102 executes the executable code to execute Figure 2 The memory 104 may also include other software modules required for running processes such as an operating system (such as multiple modules in the neural network acquisition device 100, such as the determination module 1201 and the acquisition module 1202). The operating system may be LINUX TM ,UNIX TM ,WINDOWSTM wait.

[0058] Among them, the determination module 1201 in the neural network acquisition device 100 can be used to determine the target hardware platform, and the acquisition module 1202 can be used to acquire the target neural network for the target hardware platform.

[0059] The processor 101 can call the computer execution instructions stored in the memory 102 so that the neural network acquisition device 100 can execute the following steps: Figure 2 The neural network acquisition device 100 in the illustrated embodiment executes steps 201 to 205 .

[0060] The communication interface 103 is used to send and receive data. When the processor 101 communicates with other devices, it can transmit data through the communication interface 103, such as obtaining a target hardware platform, or a delay set, sending a test neural network, etc.

[0061] In the embodiment of the present application, the neural network acquisition device 100 is used to execute the neural network acquisition method provided in the embodiment of the present application, and can acquire a target neural network for the target hardware platform after determining the target hardware platform, and the target neural network is a neural network with better performance on the target hardware platform. It should be noted that when the neural network acquisition device 100 is a server cluster, multiple servers in the server cluster can cooperate to execute the neural network acquisition method provided in the embodiment of the present application.

[0062] The user can operate on the client 200 and send information to the neural network acquisition device 100 through the client 200. For example, the user can input or select the target hardware platform on the client 200. After the client 200 detects the user's operation on the client 200, the client 200 can send the target hardware platform (identification information of the target hardware platform) input or selected by the user to the neural network acquisition device 100. The user can also input or select the range of layers that the target neural network needs to comply with on the client 200. After the client 200 detects the user's operation on the client 200, the client 200 can send the range of layers that the user input or selected to the neural network acquisition device 100. The user can also input or select the target delay range that the target neural network needs to comply with on the client 200. After the client 200 detects the user's operation on the client 200, the client 200 can send the target delay range that the user input or selected to the neural network acquisition device 100.

[0063] The neural network acquisition device 100 can also send a message to the client 200. For example, after searching for the target neural network, the neural network acquisition device 100 can push the target neural network to the client 200, and can also send relevant information of the target neural network, such as the number of layers of the target neural network, the accuracy of the target neural network, or the estimated delay to the client 200. The client 200 can display the target neural network and relevant information of the target neural network to the user for user viewing.

[0064] In an embodiment of the present application, the neural network acquisition device 100 can acquire a target neural network for the target hardware platform after determining the target hardware platform. When acquiring the target neural network for the target hardware platform, the delay of the target neural network on the target hardware platform (also referred to as the estimated delay of the target neural network) is estimated to be closer to the actual delay of the target neural network. Specifically, the difference between the estimated delay of the target neural network on the target hardware platform and the actual delay of the target neural network on the target hardware platform can be within a standard deviation interval, and the standard deviation interval is determined based on the distribution of actual delays of the target neural network measured multiple times on the target hardware platform.

[0065] In order to determine the deviation between the estimated delay and the actual delay of the target neural network on the target hardware platform, the distribution of the actual delay of the target neural network on the target hardware platform can be determined first, and the target neural network is run on the target hardware platform, so that the target neural network is run multiple times on the target hardware platform, and each time the target neural network is run, the time from obtaining input data to outputting data (that is, the actual delay) is measured to obtain multiple actual delays. According to the distribution of the multiple actual delays, a distribution curve of the actual delay of the target neural network on the target hardware platform is fitted, and the horizontal axis of the distribution curve is the actual delay of the target neural network on the target hardware platform, and the vertical axis is the distribution probability or frequency of the actual delay. For any actual delay among the multiple actual delays, the actual delay is located on the distribution curve, or the distance from the distribution curve is within a set range.

[0066] like Figure 1B As shown in FIG. 1 , a distribution curve of the actual delay of a target neural network provided by the present application on the target hardware platform is shown in FIG. Figure 1B It can be seen that the distribution curve is close to a normal distribution, that is, the actual delays measured multiple times have more actual delays distributed in the middle area and less actual delays distributed in the edge area.

[0067] The standard deviation is used to characterize the discreteness of the multiple actual delays, which is equal to Among them, x i is the actual delay of the ith test, It is the average value of the actual delay measured multiple times. After multiple calculations, the standard deviation interval is a subset of {-0.6, 0.6}, which means that the difference between the estimated delay and the actual delay of the target neural network on the target hardware platform is no more than 0.6 milliseconds (ms). In order to more accurately characterize the size of the standard deviation interval, the maximum and minimum values ​​of the standard deviation interval are 1% and -1% respectively to the average value of the actual delay measured multiple times.

[0068] like Figure 1C As shown, it is a distribution curve between the actual delay of different target neural networks (different target neural networks have different actual delays), the estimated delay of the target neural network on the target hardware platform and the actual delay of the target neural network on the target hardware platform, wherein the dark gray area is the standard deviation interval determined based on the actual delay distribution of different target neural networks on the target hardware platform. The light gray area is twice the standard deviation interval determined based on the actual delay distribution of different target neural networks on the target hardware platform.

[0069] from Figure 1C It can be seen that the difference between the estimated delay of the target neural network on the target hardware platform and the actual delay of the target neural network on the target hardware platform is close to zero, and for any target neural network, the difference between the estimated delay of the target neural network on the target hardware platform and the actual delay of the target neural network on the target hardware platform is within the standard deviation range.

[0070] In order to ensure that the estimated delay of the target neural network on the target hardware platform is close to the actual delay of the target neural network on the target hardware platform, the neural network acquisition device 100 can acquire the target neural network for the target hardware platform based on the delay set, wherein the delay set includes the pre-measured delays of each submodule on different hardware platforms, and each submodule includes the submodule required to construct the target neural network, and the different hardware platforms include the target hardware platform. Due to the existence of the delay set, it can be ensured that the delay of the target neural network on the target hardware platform estimated by the delay set (that is, the estimated delay of the target neural network on the target hardware platform) can be closer to the actual delay of the target neural network on the target hardware platform.

[0071] The following is combined with Figure 2 The present application provides a neural network acquisition method, which is performed by a neural network acquisition device 100. Figure 2 As shown, the method includes:

[0072] Step 201: The neural network acquisition device 100 acquires a delay set, which includes the delays of each submodule on different hardware platforms. The delay of a submodule on a hardware platform is the time length for the submodule to process input data, which is equal to the time interval between the submodule acquiring input data and the submodule outputting data. The different hardware platforms include the target hardware platform. The submodules include the submodules required to construct the target neural network.

[0073] In order to accurately estimate the delay of the entire neural network on different hardware platforms, so as to subsequently search for the best neural network that can run on a certain hardware platform, the neural network acquisition device 100 can perform some preliminary preparation operations, one of which is to determine the delay of each possible sub-module required to build the neural network on different hardware platforms.

[0074] A method for constructing a delay set provided in an embodiment of the present application is described below. The embodiment of the present application does not limit the device for executing the method. The method can be directly executed by the neural network acquisition device 100. After constructing the delay set, the neural network acquisition device 100 can save the delay set. The method can also be executed by other devices. After constructing the delay set, the other devices can send the delay set to the neural network acquisition device 100. In the embodiment of the present application, only the neural network acquisition device 100 executing the method is used as an example for description. The method includes the following steps:

[0075] Step 1: The neural network acquisition device 100 determines multiple submodules required to construct the test neural network. The submodules determined by the neural network acquisition device 100 are submodules that may be required to construct the test neural network. When actually constructing the submodules of the test neural network, only some of the multiple submodules may be required.

[0076] It should be noted here that the multiple sub-modules determined by the neural network acquisition device 100 include all sub-modules that may be used in the neural network searched during the subsequent neural network search. From the functional perspective of the neural network, the functions that can be achieved by the test neural network constructed by each sub-module need to cover the functions of the neural network searched during the subsequent neural network search.

[0077] For example, if the subsequent neural network search needs to search for both a neural network for face recognition and a neural network for target recognition, the multiple submodules determined by the neural network acquisition device 100 are all possible submodules required to construct a neural network for face recognition or target recognition, and the test neural network constructed by the multiple submodules can be used for face recognition or target recognition.

[0078] If the neural network search is subsequently performed and it is necessary to search for both the neural network for speech recognition and the neural network for specific speech detection (i.e., it is necessary to detect speech information including specific speech), the multiple submodules determined by the neural network acquisition device 100 are all possible submodules required to construct a neural network for speech recognition and specific speech detection, and the test neural network constructed by the multiple submodules can be used for speech recognition and specific speech detection.

[0079] Here, the function of the neural network to be searched later is face recognition as an example. The basic module of the test neural network is the face recognition submodule, which includes three parts, namely, the inverted residual block conv (MBConv) module, the channel attention module (CAM), and the spatial attention module (SAM). Among them, when the MBConv module uses convolution kernels of different sizes and expansion rates, multiple different face recognition submodules can be formed, and the different face recognition submodules are different submodules.

[0080] For example, the value set of the convolution kernel size of the MBConv module is {3, 5, 7}, that is, the convolution kernel size of 3, 5 or 7 can be selected. The value set of the expansion rate size of the MBConv module is {3, 4, 5, 6, 7}, that is, the expansion rates that can be configured for the MBConv module are 3, 4, 5, 6 or 7. There are 15 combinations of selecting convolution kernels of different sizes and configuring different expansion rates, which can form 15 different face recognition submodules. The 15 face recognition submodules are multiple submodules that may be used to build a test neural network.

[0081] Step 2: The neural network acquisition device 100 randomly selects a submodule from the multiple submodules to form a test neural network.

[0082] A neural network is composed of a specific number of network layers, and the submodules used in each network layer may be different. In constructing a test neural network, the number of layers of the test neural network can be determined first, and the number of layers of the test neural network is not limited here. The neural network acquisition device 100 can pre-configure the value range of the number of layers of the test neural network, and the neural network acquisition device 100 can determine a value from the value range of the number of layers, and then construct a test neural network with a number of layers equal to the value. The submodules used in each network layer in the test neural network can be randomly selected.

[0083] Still taking the function of the test neural network as face recognition, the range of the number of layers predefined by the neural network acquisition device 100 is [4, 20], that is, the minimum number of layers and the maximum number of layers are 4 and 20 respectively. A value is determined from [4, 20] as the number of layers, and then face recognition submodules are randomly selected from 15 face recognition submodules to form each network layer. The face recognition submodules in each network layer can be the same or different, thereby forming a test neural network.

[0084] Step 3: The neural network acquisition device 100 measures the delay of each submodule in the test neural network on different hardware platforms.

[0085] The neural network acquisition device 100 can be connected to different hardware platforms, which can be computing devices in edge data centers, terminal computing devices, or computing devices in cloud data centers. The neural network acquisition device 100 can be connected to the multiple different hardware platforms to transmit data. The embodiment of the present application does not limit the connection method between the neural network acquisition device 100 and different hardware platforms, which can be connected by wire or by wireless.

[0086] It should be noted that in the embodiments of the present application, a hardware platform is a hardware device composed of a specific number or specific type of processing units. The processing unit includes but is not limited to: a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), and any unit that can process data can be used as a processing unit.

[0087] The neural network acquisition device 100 can pre-acquire the number and model of the processing units in the multiple different hardware platforms, and the neural network acquisition device 100 can also save the identification information of the multiple different hardware platforms and / or the identification information of each processing unit in the multiple different hardware platforms. After the neural network acquisition device 100 acquires the delays of different submodules on the multiple different hardware platforms, it can use the acquired delays, the identification information of the multiple different hardware platforms, and the identification information of each processing unit in the multiple different hardware platforms to construct a delay set; it can also use the acquired delays and the identification information of the multiple different hardware platforms to construct a delay set; it can also use the acquired delays and the identification information of each processing unit to construct a delay set. The information recorded in the delay set can be specifically referred to the subsequent description.

[0088] The specific type of the identification information of the hardware platform is not limited here. The identification information of the hardware platform can be the device number or serial number of the hardware platform. Any information that can uniquely identify the hardware platform is applicable to the embodiments of the present application. Similarly, the specific type of the identification information of the processing unit is not limited here. The identification information of the processing unit can be the model or serial number of the processing unit. Any information that can uniquely identify the processing unit is applicable to the embodiments of the present application.

[0089] The hardware platform is composed of processing units and has a certain data processing capability. Different hardware platforms have different data processing capabilities. For example, the data processing capability of computing devices located in edge data centers is usually higher than that of terminal computing devices, and the data processing capability of computing devices located in cloud data centers is usually higher than that of computing devices located in edge data centers. The data processing capabilities of different types of terminal computing devices are also different.

[0090] It should be noted that the processing unit in the hardware platform is the main factor that determines the data processing capability. In the embodiments of the present application, the hardware devices including the same type of processing unit can be considered to belong to the same hardware platform. For example, there are two hardware devices, device 1 and device 2. Both devices include a GPU of the same model. The difference between device 1 and device 2 is that the resolution of the display is different. Device 1 and device 2 belong to the same hardware platform. In other words, the delay of testing the neural network on different hardware platforms and the delay of each sub-module on different hardware platforms are essentially the delay of testing the neural network on processing units of different models (that is, the delay of the processing unit executing the test neural network) and the delay of each sub-module on processing units of different models (that is, the delay of the processing unit executing each sub-module).

[0091] The data processing capability of the hardware platform has a certain influence on the delay of the test neural network and the delay of each submodule therein when the hardware platform runs the test neural network. For example, for a hardware platform including a processing unit with better performance, the data processing capability is larger, and more operations can be realized in a unit time. When running the test neural network, various operations involved in the test neural network (such as convolution, pooling, etc.) can be completed faster, and the output data of the test neural network can be obtained faster. In this way, the delay of the test neural network on the hardware platform and the delay of each submodule on the hardware platform are also smaller.

[0092] In the embodiment of the present application, the neural network acquisition device 100 can send the test neural network to different hardware platforms to request the different hardware platforms to run the test neural network. Further, for any hardware platform, the neural network acquisition device 100 can also obtain the delay of each submodule in the test neural network on the hardware platform from the hardware platform, and optionally, can also obtain the delay of the test neural network on the hardware platform.

[0093] For example, the neural network acquisition device 100 can send a neural network test request to the hardware platform, where the neural network test request includes a test neural network. The neural network test request is used to request the hardware platform to run the test neural network and measure the delay of each sub-module in the test neural network running on the hardware platform.

[0094] The neural network acquisition device 100 can instruct the processing unit in the hardware platform to execute various operations involved in the test neural network through the neural network test request, and obtain the output data of the test neural network. For example, the neural network acquisition device 100 can specify the GPU in the hardware platform to execute various operations involved in the test neural network. For the entire test neural network, the time interval from the GPU obtaining the input data of the test neural network to obtaining the output data of the test neural network can be used as the delay of the test neural network. For any sub-module in the test neural network, the time interval from the GPU obtaining the input data of the sub-module to obtaining the output data of the sub-module can be used as the delay of the sub-module on the hardware platform (in the scenario of using the GPU).

[0095] After receiving the neural network test request, the hardware platform can run the test neural network on the hardware platform and measure the delay of each submodule in the test neural network on the hardware platform. The delay of any submodule on the hardware platform is equal to the time interval between the submodule obtaining input data and obtaining output data.

[0096] Optionally, the neural network acquisition device 100 can also measure the delay of the test neural network on different hardware platforms. The delay of the test neural network on a hardware platform is the time length for the test neural network to process input data, which is equal to the time interval between the test neural network acquiring input data and the test neural network outputting data.

[0097] For any hardware platform, the neural network acquisition device 100 can send the test neural network to the hardware platform and request the hardware platform to measure the delay of each submodule in the test neural network. During the measurement process, the hardware platform can run the neural network multiple times to measure the delay of each submodule in the test neural network and the delay of the test neural network multiple times.

[0098] For example, the hardware platform can run the test neural network ten times, that is, perform ten inferences, and record the delays of each submodule in the test neural network and the delay of the test neural network. The hardware platform can obtain the delays of each submodule in the test neural network and the delay of the test neural network by taking the average value, or delete part of the recorded delays of each submodule in the test neural network and the delay of the test neural network, and take the average value of the remaining delays of each submodule in the test neural network and the delay of the test neural network to obtain the delays of each submodule in the test neural network and the delay of the test neural network. For example, the hardware platform can discard the delays of each submodule recorded in the first five inferences, and for any submodule, take the average value of the delays recorded in the last five times as the delay of the submodule on the hardware platform. Discard the delays of the test neural network recorded in the first five times, and take the average value of the delays recorded in the last five times as the delay of the test neural network on the hardware platform.

[0099] Here, the hardware platform's processing of the recorded delays of each sub-module in the test neural network and the delay of the test neural network is taken as an example to illustrate. In actual applications, the hardware platform can also directly send the recorded delays of each sub-module in the test neural network and the delay of the test neural network to the neural network acquisition device 100, and the neural network acquisition device 100 processes the recorded delays of each sub-module in the test neural network and the delay of the test neural network by the hardware platform.

[0100] It should be noted that when there are multiple processing units of the same model in the hardware platform, the neural network test request can instruct a processing unit in the hardware platform to execute various operations involved in the test neural network and obtain the output results of the test neural network, so that the processing unit executes various operations involved in the test neural network. For the entire test neural network, the time interval from the processing unit obtaining the input data of the test neural network to obtaining the output data of the test neural network can be used as the delay of the test neural network. The time interval from the processing unit obtaining the input data of the test neural network to obtaining the output data of the test neural network can also be used as the delay of the processing unit to execute the test neural network. For any submodule in the test neural network, the time interval from the processing unit obtaining the input data of the submodule to obtaining the output data of the submodule can be used as the delay of the submodule on the hardware platform (in the scenario using the processing unit). The time interval from the processing unit obtaining the input data of the submodule to obtaining the output data of the submodule can also be used as the delay of the processing unit to execute the submodule.

[0101] When there are multiple processing units of different models in the hardware platform, the neural network acquisition device 100 can send multiple neural network test requests, each neural network test request instructs a processing unit of a model in the hardware platform to execute various operations involved in the test neural network and obtain the output data of the test neural network. So that the processing units of different models execute various operations involved in the test neural network. For the entire test neural network, the time interval from the acquisition of the input data of the test neural network to the acquisition of the output data of the test neural network by the processing units of different models can be used as the delay of the test neural network on the hardware platform (in the scenario of using processing units of different models); the time interval from the acquisition of the input data of the test neural network to the acquisition of the output data of the test neural network by the processing units of different models can also be used as the delay of the execution of the test neural network by the processing units of different models. For any sub-module in the test neural network, the time interval from when processing units of different models obtain the input data of the sub-module to when they obtain the output data of the sub-module can be used as the delay of the sub-module on the hardware platform (in the scenario using processing units of different models); the time interval from when processing units of different models obtain the input data of the sub-module to when they obtain the output data of the sub-module can also be used as the delay of processing units of different models executing the sub-module.

[0102] For the delay of the test neural network on the hardware platform (using a processing unit of one model) and the delay of constructing each submodule in the test neural network on the hardware platform (using a processing unit of one model), after acquiring these two delays, the neural network acquisition device 100 can construct any one of the following three corresponding relationships:

[0103] The first type is the correspondence between delay, hardware platform, and processing unit in the hardware platform. The delay may include the delay of constructing each submodule in the test neural network on the hardware platform (using a scenario of a processing unit of one model), and may also include the delay of the test neural network on the hardware platform (using a scenario of a processing unit of one model).

[0104] The neural network acquisition device 100 can establish a correspondence between the delay and the identification information of the hardware platform and the identification information of the processing unit of this model.

[0105] It should be noted that, when the processing units included in multiple hardware platforms are the same (the model and quantity are the same), the neural network acquisition device 100 can measure the delay of the test neural network on each hardware platform and the delay of each submodule on each hardware platform respectively. The neural network acquisition device 100 can also only measure the delay of the test neural network on a hardware platform and the delay of each submodule on the hardware platform. After that, the neural network acquisition device 100 uses the measured delay as the delay corresponding to other hardware platforms, and establishes a corresponding relationship between the delay, the hardware platform, and the processing unit in the hardware platform for each hardware platform.

[0106] The second type is the correspondence between delay and hardware platform. The delay includes the delay of constructing each submodule in the test neural network on the hardware platform, and can also include the delay of the test neural network on the hardware platform.

[0107] The neural network acquisition device 100 can establish a corresponding relationship between the delay and the identification information of the hardware platform.

[0108] When there are multiple types of processing units in a hardware platform, for the hardware platform, the neural network acquisition device 100 can obtain the delay of different types of processing units executing the test neural network and the delay of different types of processing units executing each sub-module in constructing the test neural network.

[0109] Taking the delay in executing a test neural network as an example, the neural network acquisition device 100 can establish a correspondence between the delay of the multiple models of processing units executing the test neural network (the number of delays is consistent with the number of models of processing units, that is, multiple delays) and the hardware platform, that is, establish a correspondence between the multiple delays and the identification information of the hardware platform, indicating that one hardware platform can correspond to multiple delays.

[0110] For example, for hardware platform A, the multiple delays acquired by the neural network acquisition device 100 are delay 1 when processing unit 1 executes the test neural network, delay 2 when processing unit 2 executes the test neural network, and delay 3 when processing unit 3 executes the test neural network; the neural network acquisition device 100 can establish a correspondence between delay 1, delay 2, delay 3 and the identification information of hardware platform A, indicating that there are three possible delays for the test neural network on the hardware platform A.

[0111] The neural network acquisition device 100 may also first solve the average value of the delay of the multiple models of processing units executing the test neural network, and then establish a correspondence between the average values ​​of the multiple delays and the hardware platform, that is, establish a correspondence between the average values ​​of the multiple delays and the identification information of the hardware platform.

[0112] For example, for hardware platform A, the multiple delays acquired by the neural network acquisition device 100 are delay 1 when processing unit 1 executes the test neural network, delay 2 when processing unit 2 executes the test neural network, and delay 3 when processing unit 3 executes the test neural network; the neural network acquisition device 100 can solve the average value P1 of delay 1, delay 2, and delay 3, and establish a corresponding relationship between P1 and the identification information of hardware platform A.

[0113] For the delay in executing any submodule in the test neural network, similar to the delay in executing the test neural network, the neural network acquisition device 100 can establish a correspondence between the delay of each type of processing unit executing the submodule and the identification information of the hardware platform, indicating that one hardware platform can correspond to multiple delays.

[0114] For example, for hardware platform A, the multiple delays acquired by the neural network acquisition device 100 are delay 4 of processing unit 1 executing sub-module M in the test neural network, delay 5 of processing unit 2 executing sub-module M in the test neural network, and delay 6 of processing unit 3 executing sub-module M in the test neural network; the neural network acquisition device 100 can establish a correspondence between delay 4, delay 5, delay 6 and the identification information of hardware platform A, indicating that there are three possible delays for the sub-module on the hardware platform A.

[0115] The neural network acquisition device 100 may also calculate the average value of the delays of the multiple types of processing units executing the submodule, and then establish a correspondence between the average values ​​of the multiple delays and the identification information of the hardware platform.

[0116] For example, for hardware platform A, the multiple delays acquired by the neural network acquisition device 100 are delay 4 of processing unit 1 executing sub-module M in the test neural network, delay 5 of processing unit 2 executing sub-module M in the test neural network, and delay 6 of processing unit 3 executing sub-module M in the test neural network; the neural network acquisition device 100 can solve the average value P2 of delay 4, delay 5, and delay 6, and establish a corresponding relationship between P2 and the identification information of hardware platform A.

[0117] It should be noted that, for multiple hardware platforms including the same type and number of processing units, the neural network acquisition device 100 can measure the delay of the test neural network on each hardware platform and the delay of each submodule on each hardware platform respectively. The neural network acquisition device 100 can also only measure the delay of the test neural network on one hardware platform and the delay of each submodule on the hardware platform. After that, the neural network acquisition device 100 uses the measured delay as the delay corresponding to other hardware platforms and establishes the corresponding relationship between the delay and each hardware platform.

[0118] The third type is the correspondence between delay and processing unit. The delay includes the delay of the processing unit of this type to execute each submodule, and may also include the delay of the processing unit of this type to execute the test neural network. In this case, the correspondence between the delay and the processing unit is directly established.

[0119] The neural network acquisition device 100 can establish a corresponding relationship between the delay and the identification information of the processing unit.

[0120] It should be noted that, for the delay of the test neural network on the hardware platform, the identification information of the test neural network can be used to identify that the delay is a delay related to the test neural network. Taking the first correspondence as an example, a correspondence is essentially established between the identification information of the test neural network, the specific value of the delay, the identification information of the hardware platform, and the identification information of the processing unit of the hardware platform. For the delay of any submodule on the hardware platform, the identification information of the submodule can be used to identify that the delay is a delay related to the submodule. Taking the first correspondence as an example, a correspondence is essentially established between the identification information of the submodule, the specific value of the delay, the identification information of the hardware platform, and the identification information of the processing unit of the hardware platform.

[0121] Step 4: The neural network acquisition device 100 repeatedly executes steps 2 and 3 to ensure that the number of times each of the multiple submodules is selected to form a test neural network is greater than a threshold.

[0122] The threshold can be set according to actual needs. For example, to ensure the accuracy of the delay recorded by each submodule on different hardware platforms, the threshold can be set to a larger value; for another example, in order to achieve faster delay aggregation, the threshold can be set to a smaller value.

[0123] Step 5: The neural network acquisition device 100 constructs a delay set according to the delays of each submodule tested in steps 2 to 4.

[0124] By repeating steps 2 to 4 multiple times, different test neural networks can be constructed, and then the different test neural networks are sent to different hardware platforms, so as to obtain the delay of each sub-module on different hardware platforms when it is a component of a different test neural network.

[0125] Theoretically, the delay of testing the neural network on the hardware platform is equal to the sum of the delays of each submodule on the hardware platform. However, in actual tests, there are cases where the delay of testing the neural network on the hardware platform is less than the sum of the delays of each submodule on the hardware platform. This phenomenon is related to the existence of intersections between network layers in the test neural network.

[0126] For two non-intersecting network layers, the latter network layer needs to use the final output data of the previous network layer as input data, that is, the latter network layer starts to operate only after the previous network layer completes the operation. For the test neural network composed of non-intersecting network layers, the delay of the test neural network on the hardware platform is equal to the sum of the delays of each submodule on the hardware platform.

[0127] For two overlapping network layers, when the previous network layer has not completely calculated the final output data, the next network layer can use the data output by the previous network layer during the calculation process to perform calculations, which will lead to a reduction in the delay of testing the neural network.

[0128] In order to make the delay of testing the neural network on the hardware platform equal to the sum of the delays of each sub-module on the hardware platform, it is necessary to calibrate the delay of each sub-module on the hardware platform and update the delay of each sub-module on the hardware platform to achieve that the delay of testing the neural network on the hardware platform is equal to the sum of the delays of each calibrated sub-module on the hardware platform.

[0129] It should be noted that when calibrating the delay of each submodule on the hardware platform, only the delay of the submodule in the intersecting network layer on the hardware platform may be calibrated, or the delay of all submodules on the hardware platform may be calibrated.

[0130] The method for calibrating the delay of each sub-module on the hardware platform is not limited here. The following describes a method for calibrating the delay of each sub-module on the hardware platform provided in an embodiment of the present application.

[0131] First, the neural network acquisition device 100 can construct two curves, Curve 1 is a curve constructed by the delays actually measured by multiple test neural networks on different hardware platforms. Curve 2 is a curve constructed by the theoretical delays of multiple test neural networks on different hardware platforms, where the theoretical delay of a test neural network on a hardware platform is equal to the sum of the delays of each submodule in constructing the test neural network on the hardware platform.

[0132] From the above description, since there is a situation where the delay of the test neural network on the hardware platform is less than the sum of the delays of each sub-module on the hardware platform, curve 1 and curve 2 do not completely overlap.

[0133] The neural network acquisition device 100 can configure a fitting parameter α for curve 2, and use the fitting parameter to fit curve 1 and curve 2, so that the fitted curve 2 can overlap with curve 1, that is:

[0134] Curve 1 = α Curve 2

[0135] The neural network acquisition device 100 uses the fitting parameter α to calibrate the delay of each submodule on the hardware platform. For example, the product of the fitting parameter and the delay of each submodule on the hardware platform can be used as the calibrated delay of each submodule on the hardware platform.

[0136] At this point, the neural network acquisition device 100 can construct a delay set. According to the three possible correspondences constructed by the neural network acquisition device 100, the correspondence recorded in the delay set can be any of the following:

[0137] (1) The correspondence between delay, hardware platform, and processing unit of the hardware platform.

[0138] In this case, when the neural network acquisition device 100 searches for a delay related to a sub-module from the delay set, it can obtain the delay of the sub-module on the hardware platform by matching the identification information of the hardware platform and the identification information of the sub-module; it can also obtain the delay of the processing unit executing the sub-module by matching the identification information of the processing unit and the identification information of the sub-module.

[0139] In the case where the hardware platform includes multiple types of processing units, the neural network acquisition device 100 can directly obtain the delay of the processing unit of this type to execute the sub-module by matching the identification information of the processing unit and the identification information of the sub-module, or it can first obtain the delay of the sub-module on the hardware platform by matching the identification information of the hardware platform and the identification information of the sub-module, and then obtain the delay of the processing unit to execute the sub-module during the delay of the sub-module on the hardware platform by matching the identification information of the processing unit.

[0140] (2) The correspondence between delay and hardware platform.

[0141] In this case, when the neural network acquisition device 100 searches for a delay related to a submodule from the delay set, it can obtain the delay of the submodule on the hardware platform by matching the identification information of the hardware platform and the identification information of the submodule.

[0142] When the submodule has multiple delays on the hardware platform, a delay may be randomly selected as the search result, or the multiple delays may be averaged to obtain the average value of the delays, which is used as the search result.

[0143] (3) Delay, and the corresponding relationship between processing units.

[0144] In this case, when the neural network acquisition device 100 searches for a delay related to a submodule from the delay set, it can obtain the delay of the processing unit executing the submodule by matching the identification information of the processing unit and the identification information of the submodule.

[0145] After acquiring the delay set, the neural network acquisition device 100 can save the delay set. The present application does not limit the file format of the delay set saved by the neural network acquisition device 100. For example, the delay set can be saved in the memory in the form of a pickle file (file suffix .pkl) or in the memory in the form of a json file (file suffix .json).

[0146] In addition to acquiring the delay set, the neural network acquisition device 100 can also update the delay set. For example, when a new hardware platform is generated (the hardware platform includes a processing unit that is not involved in constructing the delay set), the neural network acquisition device 100 can construct a test neural network, and use a method similar to the above method to obtain the delay of each sub-module on the new hardware platform, and add the delay of each module on the new hardware platform to the delay set.

[0147] In addition, according to the above description, the delay set includes the delay of each sub-module on different hardware platforms, which can be essentially understood as the delay of each sub-module on processing units of different models. When the subsequent target hardware platform is not a tested hardware platform, but the processing unit included in the target hardware platform is the same model as the processing unit of a tested hardware platform, the delay of each sub-module in the delay set on the hardware platform can be used as the delay of each sub-module on the target hardware platform.

[0148] In addition to obtaining the delay set in advance, the neural network acquisition device 100 also needs to determine which hardware platform (for the convenience of explanation, the hardware platform is called the target neural network) is needed to search for the neural network (such as step 202) in order to search for a neural network suitable for a certain hardware platform later (for the convenience of explanation, the neural network finally searched out is called the target neural network). In addition to determining the target hardware platform, the neural network acquisition device 100 can also determine the range of the number of layers that the target neural network must meet and the target delay range of the target neural network (such as steps 203 to 204). The following describes the methods for determining the target hardware platform, the range of the number of layers, and the target delay range.

[0149] Step 202: The neural network acquisition device 100 determines a target hardware platform.

[0150] The target hardware platform can be specified by the user, such as Figure 3AAs shown, the neural network acquisition device 100 displays the input interface of the target hardware platform to the user, prompting the user to input the target hardware platform. The user can input the name of the target hardware platform or information that can indicate the target hardware platform (such as the number or device number of the target hardware platform, etc.), etc., which can uniquely identify the target hardware platform. After detecting the user's input, the neural network acquisition device 100 can determine the target hardware platform.

[0151] Optionally, the user can also input the identification information of the processing unit in the target hardware platform that needs to run the target test neural network, and this processing unit is the target processing unit.

[0152] The neural network acquisition device 100 may first determine whether the target hardware platform input by the user belongs to the multiple different hardware platforms on which the neural network is tested in step 201. If so, the subsequent steps are executed. Otherwise, the user is prompted that the target hardware platform input by the user is invalid and needs to be re-entered. The neural network acquisition device 100 may also first determine whether the identification information of the processing unit input by the user belongs to the identification information of the processing unit that executes the neural network test in step 201. If so, the subsequent steps are executed. Otherwise, the user is prompted that the processing unit input by the user is invalid and needs to re-enter the target hardware platform or the identification information of the processing unit.

[0153] like Figure 3B As described above, the neural network acquisition device 100 may also display a target hardware platform selection interface to the user, prompting the user to select a target hardware platform from a plurality of hardware platforms, and the plurality of hardware platforms may be a plurality of different hardware platforms on which the test neural network is run in step 201. The user may select a target hardware platform from the plurality of hardware platforms, and the neural network acquisition device 100 determines the target hardware platform according to the user's selection after detecting the user's selection operation.

[0154] Step 203: The neural network acquisition device 100 determines a range of layers of the target neural network.

[0155] Generally, a neural network running on a hardware platform will have a negative impact on its performance if it has too many or too few layers. Too many layers will improve the accuracy of the neural network, but will also increase the latency of the neural network. Too few layers can effectively reduce the latency of the neural network, but will reduce the accuracy of the neural network.

[0156] To this end, the neural network acquisition device 100 may first determine the range of the number of layers of the target neural network, so that a target neural network that meets the range of the number of layers can be searched for later. The range of the number of layers of the target neural network may specify the maximum value of the number of layers of the target neural network, and optionally, the range of the number of layers of the target neural network may also specify the minimum value of the number of layers of the target neural network.

[0157] The layer number range can also be specified by the user, such as Figure 4 As shown, the neural network acquisition device 100 displays an input interface of the target neural network layer range to the user, prompting the user to input the target neural network layer range. The user can input the target neural network layer range according to their own needs or experience, and the neural network acquisition device 100 can determine the target neural network layer range after detecting the user's input.

[0158] The range of the number of layers can also be pre-configured. The embodiment of the present application does not limit the way of pre-configuring the range of the number of layers. For example, for each hardware platform, a corresponding range of the number of layers is pre-configured in the neural network acquisition device 100. For edge hardware devices as the hardware platform, due to the limited computing power of edge hardware devices, the range of the number of layers corresponding to the edge hardware devices is a smaller range. For hardware devices in data centers as the hardware platform, due to the better performance of the hardware devices in data centers, neural networks with a larger number of layers can be run. The range of the number of layers corresponding to the hardware devices in the data center is a larger range. After the neural network acquisition device 100 determines the target hardware platform, it can determine the pre-configured range of the number of layers corresponding to the target hardware platform. For another example, the range of the number of layers is a fixed range, which is the default and has nothing to do with the hardware platform.

[0159] Step 204: The neural network acquisition device 100 determines a target delay range.

[0160] In principle, when acquiring a neural network for a certain hardware platform, the smaller the delay, the better, but a smaller delay will also lead to a lower accuracy of the neural network. In other words, the target neural network for the target hardware platform does not need to have the smallest delay, but the delay of the target neural network needs to fall within a target delay range.

[0161] There are many ways to characterize the target delay range. For example, the target delay range can be indicated by the maximum and minimum values ​​of the delay, such as indicating that the target delay range is 40 milliseconds to 60 milliseconds. For another example, the target delay range can be indicated by the target delay and the maximum difference with the target delay (adapting to the small or large time of the estimated delay and the target delay, the allowed target delay and the maximum difference of the estimated delay are the same scene), such as indicating that the target delay range is 10 milliseconds with the difference of 50 milliseconds of the target delay, that is, the target delay range is 40 milliseconds to 60 milliseconds. For another example, the target delay range can be indicated by allowing a preset percentage error with the target delay, such as indicating that a 20% error is allowed with the target delay of 50 milliseconds, that is, the estimated delay can have an error of 10 milliseconds (50*20%) with the target delay of 50 milliseconds, that is, the target delay range is 40 milliseconds to 60 milliseconds.

[0162] For another example, the maximum difference between the target delay and the estimated delay when the estimated delay is smaller than the target delay, and the maximum difference between the estimated delay and the target delay when the delay is larger than the target delay (applicable to scenarios where the maximum difference between the target delay and the delay allowed is different when the delay is smaller or larger than the target delay). For example, the target delay range may be indicated as a delay range where the maximum difference between the estimated delay and the target delay is 20 milliseconds when the estimated delay is 50 milliseconds larger than the target delay, and the maximum difference between the estimated delay and the target delay is 20 milliseconds when the estimated delay is 50 milliseconds smaller than the target delay, that is, the target delay range is 40 milliseconds to 70 milliseconds.

[0163] The embodiment of the present application does not limit the manner in which the neural network acquisition device 100 determines the target delay range. For example, the target delay range may be the delay range of the target neural network allowed by the user. The target delay range may be specified by the user, such as Figure 5A As shown, the neural network acquisition device 100 displays the input interface of the target delay range to the user, prompting the user to input the target delay range. The user can input the target delay range according to his own needs or experience, and the neural network acquisition device 100 can determine the target delay range after detecting the user's input.

[0164] For example, Figure 5B As described above, the neural network acquisition device 100 can also display the target delay and the configuration interface of the maximum difference with the target delay to the user, prompting the user to configure the target delay and the maximum difference allowed. The user can input the target delay and the maximum difference according to their own needs or experience, and the neural network acquisition device 100 can determine the target delay range after detecting the user's input.

[0165] The target delay range may also be pre-configured. The embodiment of the present application does not limit the method of pre-configuring the target delay range. For example, for each hardware platform, a target delay range is pre-configured in the neural network acquisition device 100. For edge hardware devices whose hardware platform is an edge hardware device, due to the limited computing power of the edge hardware device, in order to ensure the accuracy of the neural network, the neural network is allowed to have a larger delay, and a larger target delay range can be configured for the edge hardware device. For hardware devices whose hardware platform is a data center, due to the better performance of the hardware devices in the data center, the neural network with a larger number of layers can be run. The delay of the neural network will also be smaller than that on other hardware devices. The target delay range of the hardware devices in the data center can be set to a relatively small range. For another example, the target delay range is a fixed range, which is the default and has nothing to do with the hardware platform. For another example, the target delay range can be determined based on the delay of different test neural networks on the target hardware platform in step 201.

[0166] It should be noted that the embodiment of the present application does not limit the order of steps 202 to 204. When the target delay range is pre-configured, step 202 can be executed before step 204; when the layer number range is pre-configured, step 202 can be executed before step 203. In the embodiment of the present application, steps 203 and 204 may not be executed, and step 205 may be executed directly. The neural network acquisition device 100 selects the neural network with the smallest delay as the target neural network based on the delay set, and may also select the neural network with the highest accuracy as the target neural network, or may select multiple candidate neural networks from them, and provide the multiple candidate neural networks to the user, who confirms the target neural network.

[0167] Step 205: The neural network acquisition device 100 searches for a target neural network for a target hardware platform based on the delay set. The number of layers of the target neural network meets the layer number range, and when the target neural network runs on the target hardware platform, the delay falls within the target delay range and the target neural network has the highest running accuracy.

[0168] When executing step 205, the neural network acquisition device 100 may first construct a candidate neural network, and then select a candidate neural network from the candidate neural networks as a target neural network, wherein the number of layers of the candidate neural network conforms to the range of the number of layers.

[0169] The method of constructing the candidate neural network can be similar to the method of constructing the test neural network in step 202, that is, randomly selecting submodules from each submodule to construct a candidate neural network with a number of layers that meets the range of layer numbers. Other methods can also be used to construct the candidate neural network.

[0170] Taking the construction of a candidate neural network as an example, a method for constructing a candidate neural network provided in an embodiment of the present application is described below:

[0171] First, the neural network acquisition device 100 determines the maximum number of layers in the layer number range, and the maximum number is the maximum number of layers of the candidate neural network. Afterwards, for any layer of the candidate neural network, the neural network acquisition device 100 can determine a candidate submodule set of the layer, and the candidate submodule set includes one or more submodules and may also include an indication information for indicating not to construct the network layer of the layer.

[0172] The submodules included in the candidate submodule set of each layer may be the same or different. The candidate submodule set of each layer may be pre-configured. The method of pre-configuring the candidate submodule set of each layer is not limited here. For example, the candidate submodule set of each layer may be determined according to the construction method of multiple common neural networks and the type of submodules of each layer. For another example, it may be specified by the user, that is, the user may configure the candidate submodule set of the layer.

[0173] It is worth noting that the candidate submodule set of each layer is allowed to include an indication information indicating that the network layer is not to be constructed. When determining the submodules in a network layer of the candidate neural network, if the indication information is selected from the candidate submodule set of the layer, it indicates that the network layer does not need to be constructed. The existence of the indication information enables the number of layers of the constructed candidate neural network to be less than the maximum number of layers of the candidate neural network.

[0174] After determining the set of candidate submodules for each layer, the neural network acquisition device 100 can randomly select a submodule from the set of candidate submodules for each layer when constructing a candidate neural network (it can also randomly select indication information, in which case it means that the network layer will not be constructed) to construct the candidate neural network.

[0175] After the neural network acquisition device 100 constructs the candidate neural networks, a target neural network can be selected from the multiple candidate neural networks. Since the target neural network needs to meet the two conditions that the delay belongs to the target delay range and / or the accuracy is the highest, the neural network acquisition device 100 can determine the accuracy of each candidate neural network and the delay of each candidate neural network after constructing the candidate neural networks.

[0176] There are many ways to determine the accuracy of the candidate neural network. The following is an example of how to determine the accuracy of the candidate neural network using the candidate neural network for face recognition:

[0177] The neural network acquisition device 100 can set a training set and a test set. For example, the neural network acquisition device 100 can use the CASIA standard data set as a training set and the collection of the three data sets of LFW, Age-DB, and CFP-FP as a test set. The neural network acquisition device 100 uses the training set to train the candidate neural network and adjusts the weights of each network layer in the candidate neural network until the loss function of the candidate neural network converges. Afterwards, the trained candidate neural network is tested using the test set to test the performance of the candidate neural network and determine the accuracy of the candidate neural network. The accuracy can be equal to the ratio of the number of times the candidate neural network accurately performs face recognition based on the test set to the total number of times face recognition is performed, or it can be determined based on the value of the loss function of the candidate neural network.

[0178] When calculating the delay of a candidate neural network, the neural network acquisition device 100 can determine the various sub-modules that constitute the candidate neural network, and then obtain the delay of each sub-module on the target hardware platform from the delay set, and sum the delays of each sub-module on the target hardware platform. The sum is the delay of the candidate neural network.

[0179] The neural network acquisition device 100 acquires the delay of each submodule on the target hardware platform from the delay set. The acquisition method is different for different corresponding relationships included in the delay set, which are described below respectively:

[0180] (1) The delay set records the correspondence between the delay, the hardware platform, and the processing unit of the hardware platform.

[0181] The neural network acquisition device 100 can first search for each delay corresponding to the target hardware platform in the identification information of each hardware platform recorded in the delay set, and then obtain the delay of each sub-module on the target hardware platform from the delays corresponding to the target hardware platform according to the identification information of each sub-module.

[0182] If there are multiple processing units of different models in the target hardware platform, when the user inputs the identification information of the target processing unit, the neural network device 100 can first search for each delay corresponding to the target processing unit in the target hardware platform from the delay set according to the identification information of the target hardware platform and the identification information of the target processing unit, and then obtain the delay of each sub-module on the target hardware platform (in the scenario using the target processing unit) from the delays corresponding to the target processing unit in the target hardware platform according to the identification information of each sub-module.

[0183] The neural network device 100 can also directly search for the delay of the target processing unit executing each submodule based on the identification information of the target processing unit and the identification information of each submodule, and the sum of these delays is equal to the delay of the candidate neural network.

[0184] If there are multiple processing units of different models in the target hardware platform, and the user does not input the identification information of the target processing unit, the neural network acquisition device 100 can calculate the delay of the candidate neural network in the following two ways.

[0185] Method 1: The neural network acquisition device 100 calculates a delay of the candidate neural network.

[0186] The neural network acquisition device 100 searches for multiple delays of each submodule on the target hardware platform from the delay set according to the identification information of the target hardware platform and the identification information of each submodule, wherein one delay is the delay of a type of processing unit executing the submodule. For multiple delays of any submodule on the target hardware platform, the neural network acquisition device 100 can select a delay as the delay of the submodule on the target hardware platform (the method of selecting the delay is not limited here, for example, it can be randomly selected, or it can be selected from the default processing unit pre-configured for the target hardware platform, and the identification information of the default processing unit is stored in the neural network acquisition device 100), and then the delays of each submodule on the target hardware platform are summed to obtain the delay of the candidate neural network. For multiple delays of any submodule on the target hardware platform, the neural network acquisition device 100 can also average the multiple delays, and use the average delay as the delay of the submodule on the target hardware platform, and then the delays of each submodule on the target hardware platform are summed to obtain the delay of the candidate neural network.

[0187] Method 2: The neural network acquisition device 100 calculates multiple delays of the candidate neural network, each delay is a delay corresponding to a type of processing unit, and a delay can be understood as the delay of the processing unit of this type in executing the candidate neural network calculated based on the delay set.

[0188] For any type of processing unit, the neural network acquisition device 100 can determine the delay of each sub-module on the target hardware platform (in the scenario using the processing unit of this type) from the delay set based on the identification information of each sub-module, the identification information of the processing unit of this type, and the identification information of the target hardware platform, and obtain a delay of the candidate neural network by summing up.

[0189] For any type of processing unit, the neural network acquisition device 100 can determine a target neural network based on the delay corresponding to the type of processing unit in the multiple delays of each candidate neural network and the accuracy of each candidate neural network.

[0190] Since the target hardware platform includes multiple types of processing units, the neural network acquisition device 100 can determine multiple target neural networks, each target neural network corresponds to a type of processing unit, and the neural network acquisition device 100 can display the multiple target neural networks to the user (such as displaying the various sub-modules, number of layers and other related information of the target neural network), and provide identification information of the processing unit corresponding to each target neural network identification, and the identification information of the processing unit can be obtained from the delay set.

[0191] (2) The delay set records the correspondence between delay and hardware platform.

[0192] The neural network acquisition device 100 can first search for the various delays corresponding to the target hardware platform in the identification information of each hardware platform recorded in the delay set, and then obtain the delay of each sub-module on the target hardware platform from the various delays corresponding to the target hardware platform according to the identification information of each sub-module, and obtain the delay of the candidate neural network by summing up.

[0193] It should be noted that, when there are multiple processing units of different models in the target hardware platform, for the delay of executing any submodule, if the target hardware platform in the delay set corresponds to multiple delays for executing the submodule (for this situation, please refer to the above description of the corresponding relationship between delay and hardware platform). The neural network acquisition device 100 can select one delay from them as the delay of the submodule on the target hardware platform. The neural network acquisition device 100 can calculate the average value of the multiple delays for executing the submodule and use the average value as the delay of the submodule on the target hardware platform.

[0194] (3) The delay set records the correspondence between delay and processing units.

[0195] The neural network acquisition device 100 can first search for the various delays corresponding to the target processing unit in the identification information of each processing unit recorded in the delay set, and then obtain the delay of the target processing unit to execute each sub-module from the various delays corresponding to the target processing unit according to the identification information of each sub-module, and obtain the delay of the candidate neural network by summing up.

[0196] It should be noted that in the embodiment of the present application, the delay of the candidate neural network on the target hardware platform calculated by the neural network acquisition device 100 is calculated using the delay set, rather than the actual delay of the candidate neural network on the target hardware platform, but the estimated delay of the candidate neural network on the target hardware platform. Here, the delay of the candidate neural network on the target hardware platform calculated by the neural network acquisition device 100 is the estimated delay of the candidate neural network on the target hardware platform.

[0197] After determining the accuracy of each candidate neural network and the delay of each candidate neural network, the neural network acquisition device 100 can select a candidate neural network with the highest accuracy and a delay within the target delay range from each candidate neural network. This candidate neural network is the target neural network.

[0198] The neural network acquisition device 100 can also set a loss function L and select a target neural network from the multiple candidate neural networks according to the loss function L.

[0199] The loss function L set by the neural network acquisition device 100 is as follows:

[0200]

[0201] Among them, α and ref.val are constants, Loss can be the value of the loss function of the candidate neural network, or a value determined according to the inverse of the accuracy of the candidate neural network, Latency is the delay of the candidate neural network, β is a hyperparameter, β is determined according to the target delay range, and there is a linear relationship between β and the middle value of the target delay range (that is, the value of the delay located in the middle of the target delay range). When the target delay range is determined, β can be determined based on the linear relationship between β and the middle value of the target delay range.

[0202] It should be noted here that the linear relationship between β and the middle value of the target delay range may not be a strict linear relationship, that is, within a certain allowable range, the relationship between β and the middle value of the target delay range may be approximately a linear relationship.

[0203] like Figure 6 As shown, the accuracy of the target neural network in three different test sets, LFW, Age-DB and CFP-FP based on different β, corresponding to the line type of the circular icon, the line type of the square icon and the line type of the triangle icon, as well as the relationship between β and the delay value in the middle position of the target delay range (the line type of the pentagonal icon), the line type of the pentagonal icon can be approximately regarded as a linear relationship.

[0204] In face recognition business scenarios, such as Figure 7 As shown, the delay of MobileFaceNet running on Jetson (an edge device) is 22 milliseconds (ms), and the delays of ArcFaceResNet50 and ArcFaceResNet101 running on Jetson are 29ms and 52ms respectively. By adopting the method of the embodiment of the present application, the delay of the target neural network searched on Jetson is 15ms, which is more than 30% faster than the running speed of MobileFaceNet. Obviously, the data processing efficiency of the target neural network is higher.

[0205] In face recognition business scenarios, such as Figure 8As shown in the figure, the target neural network obtained by Jetson search has a high accuracy while reducing latency. The accuracy on the three test data sets of LFW, Age-DB, and CFP-FP reached 99.18%, 92.1%, and 94.9% respectively. The accuracy of MobileFaceNet, ArcFaceResNet50, and ArcFaceResNet101 on the three test data sets of LFW, Age-DB, and CFP-FP can be seen in Figure 8 .

[0206] In face recognition business scenarios, such as Fig. 9 As shown, the delay of MobileFaceNet running on a certain terminal device is 870ms, the delay of ArcFaceResNet50 and ArcFaceResNet101 running on the terminal device are 4916ms and 8926ms respectively. However, using the method provided in the embodiment of the present application, the delay of the target neural network on the terminal device is 767ms. The running speed of the target neural network is increased by more than 10% compared with MobileFaceNet. Obviously, the data processing efficiency of the target neural network is higher.

[0207] In face recognition business scenarios, such as Fig.10 As shown in the figure, the target neural network obtained by searching based on the terminal device has a low latency and a high accuracy. The accuracy rates on the three test data sets LFW, Age-DB, and CFP-FP reached 99.22%, 92.67%, and 95.63% respectively. The accuracy rates of MobileFaceNet, ArcFaceResNet50, and ArcFaceResNet101 on the three test data sets LFW, Age-DB, and CFP-FP can be found in Fig.10 .

[0208] like Fig.11 As shown, it is a schematic diagram of the speed of acquiring the target neural network using the neural network acquisition method provided in the embodiment of the present application and the multiple of the speed of acquiring the target neural network in the existing method. Here, the existing method of acquiring the target neural network is a hardware-in-the-loop method, that is, during the search process, the target neural network is deployed in real time to the target hardware platform to run and obtain the delay, and then the delay is returned for evaluation. Among them, the ordinate is the multiple of the speed of acquiring different target neural networks using the neural network acquisition method provided in the embodiment of the present application and the speed of acquiring the target neural network in the existing method, and the abscissa is the complexity of the target neural network. The complexity of the target neural network can be determined based on the actual delay of the target neural network. Generally, for the same hardware platform, the higher the complexity of the target neural network, the greater the actual delay of the target neural network on the hardware platform.

[0209] from Fig.11 It can be seen that for a target neural network of any complexity, the speed of acquiring the target neural network using the neural network acquisition method provided in the embodiment of the present application is faster than the speed of acquiring the target neural network with the existing method, and the speed of acquiring the target neural network using the neural network acquisition method provided in the embodiment of the present application is nearly 300 times faster than the speed of acquiring the target neural network with the existing method.

[0210] In the above description, the neural network acquisition device 100 determines the target neural network for the target hardware platform after acquiring the target hardware platform as an example. In a possible application scenario, the user may also only input the identification information of the target processing unit, that is, the user does not select or input the target hardware platform, but only inputs the identification information of the target processing unit. In this case, the neural network acquisition device 100 can determine the target neural network for the target processing unit. The way in which the neural network acquisition device 100 determines the target neural network for the target processing unit is similar to the way in which the neural network acquisition device 100 determines the target neural network for the target hardware platform, wherein, when the neural network acquisition device 100 determines the estimated delay of the candidate neural network based on the delay set, it can determine the delay of the target processing unit to execute each submodule constituting the candidate neural network from the delay set according to the identification information of the target processing unit (here, when the delay set records the corresponding relationship between the delay and the processing unit), sum these delays to obtain the estimated delay of the candidate neural network, and then determine the target neural network from the candidate neural network.

[0211] Based on the same inventive concept as the method embodiment, the embodiment of the present application further provides a neural network acquisition device for executing the method for executing the neural network acquisition method in the above method embodiment. The relevant features can be found in the above method embodiment and will not be repeated here. Fig.12 As shown, the device includes a determination module 1201 and an acquisition module 1202:

[0212] The determination module 1201 is used to determine the target hardware platform.

[0213] The acquisition module 1202 is used to obtain a target neural network for a target hardware platform, wherein the difference between an estimated delay of the target neural network on the target hardware platform and an actual delay of the target neural network on the target hardware platform is within a standard deviation interval, and the standard deviation interval is determined based on the distribution of actual delays of the target neural network measured multiple times on the target hardware platform.

[0214] As a possible implementation, when acquiring a target neural network for a target hardware platform, the acquisition module 1202 may first acquire a delay set, where the delay set includes delays of multiple sub-modules on different hardware platforms; thereafter, based on the delay set, the target neural network is acquired for the target hardware platform, where the target neural network is composed of part or all of the multiple sub-modules, and the estimated delay is determined based on the delay set.

[0215] As a possible implementation, the estimated delay is equal to the sum of the delays of the submodules constituting the target neural network in the delay set on the target hardware platform.

[0216] As a possible implementation, the number of layers of the target neural network conforms to the range of the number of layers, and the estimated delay is the neural network with the highest accuracy within the target delay range, and the target delay range is confirmed based on the user's input.

[0217] As a possible implementation, when determining the target hardware platform, the determination module 1201 may prompt the user to input or select the target hardware platform; after detecting the user's input or selection, the target hardware platform is determined according to the user's input or selection.

[0218] As a possible implementation, when acquiring the delay set, the acquisition module 1202 can randomly select sub-modules from multiple sub-modules to form a test neural network; then, run the test neural network on different hardware platforms to measure the delays of each sub-module in the test neural network on different hardware platforms; the acquisition module 1202 can use the measured delays of each sub-module in the test neural network on different hardware platforms to construct a delay set; or it can first calibrate the measured delays of each sub-module in the test neural network on different hardware platforms, and use the delays of each sub-module in the calibrated test neural network on different hardware platforms to construct a delay set.

[0219] As a possible implementation, for any hardware platform among different hardware platforms, the delay set records part or all of the following:

[0220] The correspondence between the delay of each submodule on the hardware platform and the identification information of the hardware platform;

[0221] The correspondence between the delay of each submodule on the hardware platform and the identification information of the processing unit in the hardware platform.

[0222] As a possible implementation, for any hardware platform among different hardware platforms, if the hardware platform includes multiple processing units of the same model, the delay of the sub-module on the hardware platform is equal to the time consumed by one processing unit in the hardware platform to execute the calculations involved in the sub-module.

[0223] If the hardware platform includes multiple processing units of different models, a delay of a sub-module on the hardware platform is the delay of one model of processing unit executing the sub-module, and the delay of one model of processing unit executing the sub-module is equal to the time consumed by the calculations involved in the execution of the sub-module by the one model of processing unit in the hardware platform.

[0224] As a possible implementation, if the target hardware platform includes multiple processing units of the same model, the estimated delay is equal to the sum of the delays of the various sub-modules in the delay set that constitute the target neural network on the target hardware platform.

[0225] As a possible implementation, if the target hardware platform includes multiple processing units of different models, the determination module 1201 can also determine the target processing unit in the target hardware platform based on the user's input; at this time, the estimated delay is equal to the sum of the delays of the target processing unit in the delay set executing each sub-module that constitutes the target neural network.

[0226] As a possible implementation, the estimated delay is equal to the sum of the delays of the various sub-modules constituting the target neural network in the delay set on the candidate hardware platform, wherein the candidate hardware platform is a hardware platform among different hardware platforms that includes the same model of processing unit as the target neural network.

[0227] As a possible implementation, the standard deviation space is {-0.6, 0.6}.

[0228] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. Each functional unit in the embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0230] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0231] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0232] The above is only a specific embodiment of the present invention. Those skilled in the art may conceive of changes or substitutions based on the specific embodiments provided by the present invention, which should all be included in the protection scope of the present invention.

Claims

1. A neural network acquisition method, characterized in that: The method comprises: Determine the target hardware platform; A target neural network is obtained for the target hardware platform, wherein a difference between an estimated delay of the target neural network on the target hardware platform and an actual delay of the target neural network on the target hardware platform is within a standard deviation interval, and the standard deviation interval is determined based on a distribution of actual delays of the target neural network measured multiple times on the target hardware platform.

2. The method according to claim 1, characterized in that The step of obtaining a target neural network for the target hardware platform includes: Based on a delay set, the target neural network is obtained for the target hardware platform, the delay set includes delays of multiple sub-modules on different hardware platforms, the target neural network is composed of part or all of the multiple sub-modules, and the estimated delay is determined based on the delay set.

3. The method according to any one of claims 1 to 2, characterized in that: The number of layers of the target neural network conforms to the range of the number of layers, and the estimated delay is a neural network with the highest accuracy within the target delay range, and the target delay range is confirmed based on the user's input.

4. The method according to claim 3, characterized in that Determining the target hardware platform includes: The target hardware platform is determined according to the input or selection of the user.

5. The method according to any one of claims 2 or 4, characterized in that: The obtaining of the delay set includes: Randomly selecting a submodule from the plurality of submodules to form a test neural network; Applying the test neural network to the different hardware platforms, and measuring the delay of each submodule in the test neural network on the different hardware platforms; The delay set is constructed by using the measured delays of each submodule in the test neural network on the different hardware platforms.

6. The method according to claim 2, characterized in that For any of the different hardware platforms, the delay set records part or all of the following: a correspondence between the delay of each of the submodules on the hardware platform and the identification information of the hardware platform; The correspondence between the delay of each of the submodules on the hardware platform and the identification information of the processing unit in the hardware platform.

7. The method according to claim 2, characterized in that For any hardware platform among the different hardware platforms, if the hardware platform includes multiple processing units of the same model, the delay of the submodule on the hardware platform is equal to the time consumed by one of the processing units in the hardware platform to execute the operation involved in the submodule; If the hardware platform includes multiple processing units of different models, a delay of the sub-module on the hardware platform is the delay of one model of processing unit to execute the sub-module, and the delay of one model of processing unit to execute the sub-module is equal to the time consumed by the one model of processing unit in the hardware platform to execute the operations involved in the sub-module.

8. The method according to any one of claims 2, 4, 6 and 7, characterized in that: If the target hardware platform includes multiple processing units of the same model, the estimated delay is equal to the sum of the delays of the processing units of one model in the delay set executing the various sub-modules that constitute the target neural network.

9. The method according to any one of claims 2, 4, 6 and 7, characterized in that: If the target hardware platform includes a plurality of processing units of different models, the method further includes: The target processing unit in the target hardware platform is determined according to the user's input; the estimated delay is equal to the sum of the delays of the target processing unit in the delay set executing various sub-modules constituting the target neural network.

10. The method according to any one of claims 2, 4, 6 and 7, characterized in that: The estimated delay is equal to the sum of the delays of the various sub-modules in the delay set that constitute the target neural network on the candidate hardware platform, wherein the candidate hardware platform is a hardware platform among the different hardware platforms that includes a processing unit of the same model as the target neural network.

11. The method according to any one of claims 1, 2, 4, 6 and 7, characterized in that: The standard deviation interval belongs to {-0.6, 0.6}.

12. A neural network acquisition device, characterized in that: The device comprises a determination module and an acquisition module: The determination module is used to determine the target hardware platform; The acquisition module is used to acquire a target neural network for the target hardware platform, wherein a difference between an estimated delay of the target neural network on the target hardware platform and an actual delay of the target neural network on the target hardware platform is within a standard deviation interval, and the standard deviation interval is determined based on a distribution of actual delays of the target neural network measured multiple times on the target hardware platform.

13. The device according to claim 12, characterized in that When the acquisition module acquires the target neural network for the target hardware platform, it is specifically used to: Based on a delay set, the target neural network is obtained for the target hardware platform, wherein the delay set includes delays of multiple sub-modules on different hardware platforms; the target neural network is composed of part or all of the multiple sub-modules, and the estimated delay is determined based on the delay set.

14. The device according to any one of claims 12 to 13, characterized in that: The number of layers of the target neural network conforms to the range of the number of layers, and the estimated delay is a neural network with the highest accuracy within the target delay range, and the target delay range is confirmed based on the user's input.

15. The device according to claim 14, characterized in that When determining the target hardware platform, the determination module is specifically used to: The target hardware platform is determined according to the input or selection of the user.

16. The device according to any one of claims 13 or 15, characterized in that: When acquiring the delay set, the acquisition module is specifically used to: Randomly selecting a submodule from the plurality of submodules to form a test neural network; Applying the test neural network to the different hardware platforms to measure the delay of each submodule in the test neural network on the different hardware platforms; The delay set is constructed by using the measured delays of each submodule in the test neural network on the different hardware platforms.

17. The device according to claim 13, characterized in that For any of the different hardware platforms, the delay set records part or all of the following: The correspondence between the delay of each submodule on the hardware platform and the identification information of the hardware platform; The correspondence between the delay of each of the submodules on the hardware platform and the identification information of the processing unit in the hardware platform.

18. The device according to claim 13, characterized in that For any hardware platform among the different hardware platforms, if the hardware platform includes multiple processing units of the same model, the delay of the submodule on the hardware platform is equal to the time consumed by one of the processing units in the hardware platform to execute the operation involved in the submodule; If the hardware platform includes multiple processing units of different models, a delay of the sub-module on the hardware platform is the delay of one model of processing unit to execute the sub-module, and the delay of one model of processing unit to execute the sub-module is equal to the time consumed by the one model of processing unit in the hardware platform to execute the operations involved in the sub-module.

19. The device according to any one of claims 13, 15, 17 and 18, characterized in that: If the target hardware platform includes multiple processing units of the same model, the estimated delay is equal to the sum of the delays of the processing units of one model in the delay set executing the various sub-modules that constitute the target neural network.

20. The device according to any one of claims 13, 15, 17 and 18, characterized in that: If the target hardware platform includes a plurality of processing units of different models, the determining module is further configured to: The target processing unit in the target hardware platform is determined according to the user's input; the estimated delay is equal to the sum of the delays of the target processing unit in the delay set executing various sub-modules constituting the target neural network.

21. The device according to any one of claims 13, 15, 17 and 18, characterized in that: The estimated delay is equal to the sum of the delays of the various sub-modules in the delay set that constitute the target neural network on the candidate hardware platform, wherein the candidate hardware platform is a hardware platform among the different hardware platforms that includes a processing unit of the same model as the target neural network.

22. The device according to any one of claims 12, 13, 15, 17 and 18, characterized in that: The standard deviation interval belongs to {-0.6, 0.6}.

23. A computing device, characterized in that The computing device includes a processor and a memory; The memory is used to store computer program instructions; The processor executes and calls the computer program instructions in the memory to perform the method according to any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 11.

Images