Hardware information-based neural network structure search method and system

Abstract

The application provides a neural network structure searching method and system based on hardware information, and relates to the technical field of neural network model processing, which comprises the following steps: selecting a target network meeting the set requirements according to an application scene task, and calculating the expressiveness; constructing a first network according to a search model space according to the number of layers of the target network, and calculating the expressiveness of the first network; wherein if the expressiveness of the first network is less than that of the target network, the depth is increased until the expressiveness of the first network is greater than that of the target network; a second network is constructed, and the expressiveness is calculated; the expressiveness of the second network is compared with that of the target network; if the expressiveness of the second network is greater than that of the target network, the number of layers is reduced and the network is reconstructed, and if the expressiveness of the second network is less than that of the target network, the number of layers is increased and the network is reconstructed until the difference between the expressiveness of the second network and that of the target network is within a set threshold range, and the second network is obtained; a search range is set, the network is searched based on an evolution algorithm and combined with hardware information according to the search range, and the network with the best performance is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural network model processing, in particular to a neural network structure search method and system based on hardware information. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the application recited in the claims. The description herein does not constitute admission that the prior art is prior art nor, that the description is prior art to a currently claimed application.

[0003] Neural network model search is currently a means of model design, which is mainly divided into two directions: the first is neural network model search based on reinforcement learning and evolutionary algorithm, and the second is neural network model search based on gradient descent. Differential neural network model search cannot feed back the chip bottom layer information to the model design layer.

[0004] Neural network model architecture search has a particularly large search space dimension due to the unfixed algorithmic operator of each layer, unfixed specification parameter, unfixed connection relationship, and unfixed network layer number, resulting in a particularly long network architecture search time. For example, the NAS-net based on reinforcement learning proposed by Google initially needs more than 1000 GPU days; the neural network based on gradient descent, taking Darts as an example, does not need a long search time, but the hardware information cannot be searched through the gradient, and the connection relationship between the search model and the hardware data model cannot be established; the current research on model expressiveness is an important direction, the model expressiveness is judged by the output according to the input data conforming to the Gaussian distribution, which can realize the characteristics of quickly searching the optimal network structure and searching in discrete space; however, the constraints of searching the network and the target task are lacking, and the constraints of hardware information are lacking.

[0005] Different network structures cannot directly and effectively feed back the network deployment to the inference time difference of specific chips using FLOPs; however, accurate hardware information needs to be fed back to the network search process, so that the searched network can run better on the deployed chip.

[0006] In summary, there is an urgent need for a technical solution that can overcome the above-mentioned defects and can quickly search neural networks based on hardware information. SUMMARY

[0007] To solve the problems in the prior art, the application provides a neural network structure searching method and system based on hardware information.

[0008] In a first aspect of the embodiment of the application, a neural network structure searching method based on hardware information is provided, which comprises the following steps:

[0009] According to the application scene task, a target network meeting the set requirements is selected, and the expressiveness of the target network is calculated.

[0010] According to the number of layers of the target network, a first network is constructed according to a search model space, and the expressiveness of the first network is calculated; if the expressiveness of the first network is less than the expressiveness of the target network, the depth of the first network is increased until the expressiveness of the first network is greater than the expressiveness of the target network.

[0011] According to the number of layers of the first network, a second network is constructed, and the expressiveness of the second network is calculated; the expressiveness of the second network is compared with the expressiveness of the target network; if the expressiveness of the second network is greater, the number of layers is reduced and the network is reconstructed; if the expressiveness of the second network is less, the number of layers is increased and the network is reconstructed, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and the second network is obtained.

[0012] According to the number of layers of the second network, a search range is set, and according to the search range, a network is searched based on an evolution algorithm and combined with hardware information, to obtain a network with the best performance.

[0013] In a second aspect of the embodiment of the application, a neural network structure searching system based on hardware information is provided, which comprises the following steps:

[0014] A target network processing module is configured to select a target network meeting the set requirements according to the application scene task, and calculate the expressiveness of the target network.

[0015] A first network processing module is configured to construct a first network according to a search model space according to the number of layers of the target network, and calculate the expressiveness of the first network; if the expressiveness of the first network is less than the expressiveness of the target network, the depth of the first network is increased until the expressiveness of the first network is greater than the expressiveness of the target network.

[0016] a second network construction module, configured to construct a second network according to the number of layers of the first network, and calculate the expressiveness of the second network; compare the expressiveness of the second network with the expressiveness of the target network; if greater, reduce the number of layers and reconstruct the network, if smaller, increase the number of layers and reconstruct the network, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and obtain the second network;

[0017] a network search module, configured to set a search range according to the number of layers of the second network, and search for a network based on an evolutionary algorithm and in combination with hardware information according to the search range, and obtain a network with the best performance.

[0018] In a third aspect of the embodiments of the present application, a computer device is provided, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the neural network structure search method based on hardware information when executing the computer program.

[0019] In a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the neural network structure search method based on hardware information when executed by a processor.

[0020] In a fifth aspect of the embodiments of the present application, a computer program product is provided, which comprises a computer program, and the computer program implements the neural network structure search method based on hardware information when executed by a processor.

[0021] The neural network structure search method and system based on hardware information provided by the present application determine the network layer number range through the number of layers of the target network and the expressiveness of the model, and then gradually fix the layer number range to search in a small range in combination with the hardware information, so as to quickly obtain a network with good performance for deployment on a chip, while ensuring high accuracy, effectively solving the problems of the prior art that the bottom layer information of the chip cannot be fed back to the model design and the model search is time-consuming. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 FIG. 1 is a flowchart of a method for searching a neural network structure based on hardware information according to an embodiment of the present application.

[0024] Figure 2 FIG. 2 is a flowchart of processing a target network according to an embodiment of the present application.

[0025] Figure 3 FIG. 3 is a structural diagram of a large network according to an embodiment of the present application.

[0026] Figure 4 FIG. 4 is a flowchart of constructing a new network according to an embodiment of the present application.

[0027] Figure 5 FIG. 5 is a flowchart of searching a network according to an embodiment of the present application.

[0028] Figure 6 FIG. 6 is a flowchart of a method for searching a neural network structure based on hardware information according to another embodiment of the present application.

[0029] Figure 7 FIG. 7 is a relational diagram of searching a neural network structure based on hardware information according to an embodiment of the present application.

[0030] Figure 8 FIG. 8 is a relational diagram of determining a number of layers of a searched network according to an embodiment of the present application.

[0031] Figure 9 FIG. 9 is a system architecture diagram of searching a neural network structure based on hardware information according to an embodiment of the present application.

[0032] Figure 10 FIG. 10 is an architecture diagram of a target network processing module according to an embodiment of the present application.

[0033] Figure 11 FIG. 11 is an architecture diagram of a second network constructing module according to an embodiment of the present application.

[0034] Figure 12 FIG. 12 is an architecture diagram of a network searching module according to an embodiment of the present application.

[0035] Figure 13 FIG. 13 is a system architecture diagram of searching a neural network structure based on hardware information according to another embodiment of the present application.

[0036] Figure 14 FIG. 14 is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The principles and spirits of the present application will be described below with reference to a number of exemplary embodiments. It should be understood that the embodiments are given only so that those skilled in the art can better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, the embodiments are provided so that the present disclosure is more thorough and complete, and the scope of the present disclosure is fully conveyed to those skilled in the art.

[0038] Those skilled in the art will appreciate that the embodiments of the present application can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure can be embodied in the form of an entirely hardware, an entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0039] According to the embodiments of the present application, a neural network structure search method and system based on hardware information are proposed, relating to the technical field of neural network model processing.

[0040] The principles and spirits of the present application will be described below with reference to a number of exemplary embodiments. It should be understood that the embodiments are given only so that those skilled in the art can better understand and implement the present application, and are not intended to limit the scope of the present application in any way. On the contrary, the embodiments are provided so that the present disclosure is more thorough and complete, and the scope of the present disclosure is fully conveyed to those skilled in the art.

[0041] Figure 1 is a neural network structure search method based on hardware information according to an embodiment of the present application. As shown in the figure, the method comprises: Figure 1

[0042] S1, selecting a target network that meets the set requirements according to the application scenario task, and calculating the expressiveness of the target network;

[0043] S2, constructing a first network according to the search model space according to the number of layers of the target network, and calculating the expressiveness of the first network; wherein if the expressiveness of the first network is less than the expressiveness of the target network, the depth of the first network is increased until the expressiveness of the first network is greater than the expressiveness of the target network;

[0044] S3, constructing a second network according to the number of layers of the first network, and calculating the expressiveness of the second network; comparing the expressiveness of the second network with the expressiveness of the target network; if greater, reducing the number of layers and reconstructing the network, if less, increasing the number of layers and reconstructing the network, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, obtaining the second network;

[0045] S4, setting a search range according to the number of layers of the second network, searching the network based on the evolutionary algorithm and combining the hardware information according to the search range, and obtaining the network with the best performance.

[0046] ​In order to more clearly explain the above hardware information-based neural network structure search method, the following will be described in detail in combination with each step.

[0047] In S1, referring to Figure 2 , a target network meeting the set requirements is selected according to the application scene task, and the specific process of calculating the expressiveness of the target network is as follows:

[0048] S101, a target network with a precision reaching a set value is selected according to the application scene task, and the number of layers of the target network is calculated.

[0049] Generally, the target network can be a neural network with better precision and suitable for the application scene task.

[0050] S102, the weights of the target network are replaced with normalized Gaussian distribution random numbers, and the expressiveness of the target network is calculated.

[0051] In S2, a first network is constructed according to the search model space according to the number of layers of the target network, and the expressiveness of the first network is calculated; if the expressiveness of the first network is less than the expressiveness of the target network, the depth of the first network is increased until the expressiveness of the first network is greater than the expressiveness of the target network.

[0052] As shown in Figure 3 , when constructing a large network (first network), all candidate operators of the search space are constructed in parallel for each layer; if the model expressiveness of the large network is less than the target network, the depth of the large network can be continuously increased until the model expressiveness of the constructed large network is greater than the target network.

[0053] In S3, referring to Figure 4 , a second network is constructed according to the number of layers of the first network, and the expressiveness of the second network is calculated; the expressiveness of the second network is compared with the expressiveness of the target network; if it is greater, the number of layers is reduced and the network is reconstructed, if it is less, the number of layers is increased and the network is reconstructed, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and the specific process of obtaining the second network is as follows:

[0054] S301, setting the upper limit as the number of layers of the first network and the lower limit as 0.

[0055] S302, constructing a second network with the number of layers being the average value of the upper and lower limits of the current interval according to the interval, and calculating the expressiveness of the second network.

[0056] S303, judging whether the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range.

[0057] If the difference is within the set threshold range, stop building;

[0058] If the difference is not within the set threshold range, S304, compare the expressiveness of the second network with the expressiveness of the target network;

[0059] If the expressiveness of the second network is greater than the expressiveness of the target network, set a new interval with an upper limit as the upper limit of the current interval and a lower limit as the average of the upper and lower limits of the current interval, and rebuild the second network with the number of layers as the average of the upper and lower limits of the new interval;

[0060] If the expressiveness of the second network is less than the expressiveness of the target network, set a new interval with an upper limit as the average of the upper and lower limits of the current interval and a lower limit as the lower limit of the current interval, and rebuild the second network with the number of layers as the average of the upper and lower limits of the new interval;

[0061] Repeat the process of comparing the expressiveness of the network and rebuilding the second network (S303, S304) until the difference between the expressiveness of the second network and the expressiveness of the target network is within the set threshold range, and stop building.

[0062] The present application can use binary method when building the second network, by continuously adjusting the number of layers, rebuilding the new network, comparing the expressiveness of the new network and the target network, narrowing down the interval of the number of layers that can be selected, so as to obtain a network with expressiveness adjacent to the target network.

[0063] Taking the number of layers of the first network as 80 layers as an example, the process of building a new network is as follows:

[0064] First, set the interval [0, 80].

[0065] In the first construction, a new network of 40 layers (the average of the upper and lower limits of the interval [0, 80]) is established; the expressiveness of the new network of 40 layers is calculated, and the difference between the expressiveness of the new network and the target network is compared;

[0066] If the difference is small, stop building;

[0067] If the difference is large and the expressiveness of the new network is greater than the target network, take 60 layers (i.e., the average of the upper and lower limits of the interval [40, 80], wherein the upper limit of the interval is the upper limit of the current interval [0, 80], and the lower limit is the average of the upper and lower limits of the current interval [0, 80]), and build a new network of 60 layers;

[0068] If the difference is large and the expressiveness of the new network is less than the target network, take 20 layers (i.e., the average of the upper and lower limits of the interval [0, 40], wherein the upper limit of the interval is the average of the upper and lower limits of the current interval [0, 80], and the lower limit is the lower limit of the current interval [0, 80]), and build a new network of 20 layers.

[0069] Further, after the new network is constructed, the expressiveness of the new network is recalculated, and the difference between the expressiveness of the new network and the target network is compared.

[0070] If the difference is small, the construction of the new network is stopped.

[0071] If the difference is large and the expressiveness of the new network is greater than that of the target network, 70 layers (the average of the upper and lower limits of the interval [60, 80]) are taken, and a new network with 70 layers is constructed.

[0072] If the difference is large and the expressiveness of the new network is less than that of the target network, 50 layers (the average of the upper and lower limits of the interval [40, 60]) are taken, and a new network with 50 layers is constructed.

[0073] If the difference is small, the construction of the new network is stopped.

[0074] If the difference is large and the expressiveness of the new network is greater than that of the target network, 30 layers (the average of the upper and lower limits of the interval [20, 40]) are taken, and a new network with 30 layers is constructed.

[0075] If the difference is large and the expressiveness of the new network is less than that of the target network, 10 layers (the average of the upper and lower limits of the interval [0, 20]) are taken, and a new network with 10 layers is constructed.

[0076] The above process is repeated, and the expressiveness of the new network is continuously adjusted until the difference between the expressiveness of the new network and the target network is small, and the construction is stopped.

[0077] In S4, with reference to Figure 5 , the search range is set according to the number of layers of the second network, and the search of the network is performed based on the evolutionary algorithm and combined with the hardware information according to the search range, and the specific process of obtaining the network with the best performance is as follows:

[0078] S401, according to the number of layers of the second network, the search range is set by increasing or decreasing the number of layers;

[0079] S402, according to the search range, the search of the network is performed based on the evolutionary algorithm;

[0080] S403, in the case that the expressiveness of the searched network meets the standard, the network whose hardware feedback time exceeds the set value is eliminated, the remaining networks are randomly combined and iteratively searched until the network with the shortest hardware feedback time is obtained.

[0081] Further, with reference to Figure 6 , the flowchart of the neural network structure search method based on hardware information of an embodiment of the present application is shown. As shown in Figure 6 , the method further includes:

[0082] S5, training the model by combining the application scene task with the best network to obtain a trained neural network model.

[0083] Reference Figure 7 The present application is a hardware information-based neural network structure search relationship diagram for a specific embodiment. The present application can search for a neural network model structure based on an evolutionary algorithm combined with hardware information, and in order to speed up the search speed, first search and locate the network layer based on the expressiveness of the target model, then search in detail based on the network layer range, and finally realize the fast acquisition of a network with good performance on the deployment chip while having relatively high precision. The present application has great value and significance for designing different efficient network structures for different hardware architectures to fully utilize the computing and storage resources of the underlying chip, and provides strong technical support for the research of neural network models.

[0084] It should be noted that although the operations of the method of the present application are described in a specific order in the above embodiments and drawings, this does not require or imply that the operations must be performed in that specific order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps.

[0085] In order to more clearly explain the above hardware information-based neural network structure search method, a specific embodiment will be described below.

[0086] Taking a classification task as an example:

[0087] S01, on a public dataset (cifar10), select the target network resnet50, replace the weight coefficients of the network with random numbers in a normalized Gaussian distribution, input random numbers also conforming to the normalized Gaussian distribution, and calculate the expressiveness of the target network.

[0088] The calculation method of the expressiveness used in the present application can refer to the published paper Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition. The present application calculates the expressiveness of the neural network structure by the above method, and then uses the expressiveness as a parameter for comparison between networks. It should be noted that the calculation method of the expressiveness is not improved in the overall process of the present application, and the specific expressiveness calculation method is not limited to this, and other calculation methods can also be used.

[0089] S02, a 50-layer large network is constructed according to the search space, and the expressiveness of the constructed large network is calculated, if the expressiveness of the large network is less than resnet50, the network is further expanded in the number of layers to construct a large network and calculate the expressiveness, until the expressiveness of the constructed large network is greater than resnet50.

[0090] S03, based on the number of layers of the large network, a new network is constructed, by adjusting the number of layers of the network, a network with an expressiveness close to that of the target network is obtained, so as to determine the number of layers of the search network.

[0091] In this embodiment, referring to Figure 8 The number of layers of the search network can be determined by using dichotomy.

[0092] As shown in Figure 8 , the interval [0, 50] is set, and 50 represents the number of layers of the large network; the average value (0+50) / 2 of the two ends of the interval [0, 50] is taken, a 25-layer network is constructed, and the model expressiveness of the new network is calculated.

[0093] According to the model expressiveness of the new network, it is judged whether the preset requirement is met: the difference between the model expressiveness of the new network and the expressiveness of the target network is less than the set threshold value;

[0094] If the difference is less than the set threshold value, it is considered that the preset requirement is met, and the construction is stopped and the number of layers of the search network is determined by the new network;

[0095] If the difference is greater than the set threshold value, it is considered that the preset requirement is not met, and the number of layers is continued to be adjusted.

[0096] In the case where the preset requirement is not met, the size relationship between the expressiveness of the new network and the target network is judged.

[0097] For example, if the 25-layer network does not meet the preset requirement, the size relationship between the expressiveness of the 25-layer network and the target network is judged; wherein,

[0098] If the expressiveness of the 25-layer network is less than the target network, the average value (25+50) / 2 of the two ends of the interval [25, 50] is taken, a 37-layer (rounded down) network is constructed;

[0099] If the expressiveness of the 25-layer network is greater than the target network, the average value (0+25) / 2 of the two ends of the interval [0, 25] is taken, a 12-layer (rounded down) network is constructed;

[0100] After the new network is constructed, the above process is repeated to judge whether the preset requirement is met.

[0101] If the preset requirement is still not met, the size relationship between the expressiveness of the new network and the target network is continued to be judged, the new network is re-constructed until the preset requirement is met.

[0102] For example, as shown in 8, if the 12-layer network still does not meet the preset requirement, the expressiveness of the 12-layer network is compared with that of the target network, if less than the target network, the average value of the two ends of the interval [12, 25] (12+25) / 2 is taken, and a 18-layer (downward rounding) network is constructed;

[0103] If the 18-layer network still does not meet the preset requirement, and the expressiveness of the 18-layer network is less than that of the target network, the average value of the two ends of the interval [18, 25] (18+25) / 2 is continued to be taken, and a 21-layer (downward rounding) network is constructed; at this time, if the difference between the expressiveness of the 21-layer network and that of the target network is small (less than a set threshold), the construction is stopped.

[0104] If the 18-layer network still does not meet the preset requirement, and the expressiveness of the 18-layer network is greater than that of the target network, the average value of the two ends of the interval [12, 18] (12+18) / 2 is continued to be taken, and a 15-layer (downward rounding) network is constructed, and then it is continued to be judged whether the preset requirement is met.

[0105] In Figure 8 The "greater" marked in the figure indicates that the network at the starting end of the arrow is greater than the expressiveness of the target network, and the "smaller" indicates that the network at the starting end of the arrow is smaller than the expressiveness of the target network.

[0106] S04, based on the number of layers determined by the constructed new network, a set number of layers is added or subtracted, for example, 1 layer is added or subtracted, and the search range of the number of layers is limited.

[0107] S05, according to the set search range of the number of layers, the network structure is searched based on the evolutionary algorithm; in the case that the expressiveness meets the standard, a plurality of network structures with the longest hardware feedback time are eliminated each time the search is iterated; then, the next round of iteration search is performed through random combination. Finally, the best network performance is selected, and then the model training is performed combined with the application task scene to obtain the trained model.

[0108] After introducing the method of the example embodiment of the present application, next, reference Figure 9 The neural network structure search system based on hardware information of the example embodiment of the present application is introduced.

[0109] The implementation of the neural network structure search system based on hardware information can refer to the implementation of the above method, and the repeated parts will not be described again. The term "module" or "unit" used below can be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably realized in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0110] Based on the same inventive concept, the present application also proposes a neural network structure search system based on hardware information, as shown in Figure 9As shown, the system comprises:

[0111] a target network processing module 110, configured to select a target network meeting a set requirement according to an application scenario task, and calculate expressiveness of the target network;

[0112] a first network processing module 120, configured to construct a first network according to a search model space according to a number of layers of the target network, and calculate expressiveness of the first network; if the expressiveness of the first network is less than the expressiveness of the target network, increase the depth of the first network until the expressiveness of the first network is greater than the expressiveness of the target network;

[0113] a second network constructing module 130, configured to construct a second network according to the number of layers of the first network, and calculate expressiveness of the second network; compare the expressiveness of the second network with the expressiveness of the target network; if greater, reduce the number of layers and reconstruct the network; if less, increase the number of layers and reconstruct the network, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and obtain the second network;

[0114] a network searching module 140, configured to set a search range according to the number of layers of the second network, search for a network based on an evolution algorithm and combined with hardware information according to the search range, and obtain a network with the best performance.

[0115] In this embodiment, referring to Figure 10 , the target network processing module 110 comprises a number of layers calculating unit 1101 and an expressiveness calculating unit 1102; wherein,

[0116] the number of layers calculating unit 1101 is configured to select a target network with a precision reaching a set value according to the application scenario task, and calculate the number of layers of the target network;

[0117] the expressiveness calculating unit 1102 is configured to replace weights of the target network with normalized Gaussian distribution random numbers, and calculate the expressiveness of the target network.

[0118] In this embodiment, referring to Figure 11 , the second network constructing module 130 comprises an interval setting unit 1301, a network constructing unit 1302, and a comparison unit 1303; wherein,

[0119] the interval setting unit 1301 is configured to set an interval with the upper limit being the number of layers of the first network and the lower limit being 0;

[0120] the network constructing unit 1302 is configured to construct a second network with the number of layers being the average value of the upper and lower limits of the current interval according to the interval, and calculate the expressiveness of the second network;

[0121] The comparison unit 1303 is configured to determine whether the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range.

[0122] If the difference is within the set threshold range, the construction is stopped.

[0123] If the difference is not within the set threshold range, the expressiveness of the second network is compared with the expressiveness of the target network.

[0124] If the expressiveness of the second network is greater than the expressiveness of the target network, a new interval with an upper limit being the upper limit of the current interval and a lower limit being the average of the upper and lower limits of the current interval is set, and a second network with a layer number being the average of the upper and lower limits of the new interval is re-constructed.

[0125] If the expressiveness of the second network is less than the expressiveness of the target network, a new interval with an upper limit being the average of the upper and lower limits of the current interval and a lower limit being the lower limit of the current interval is set, and a second network with a layer number being the average of the upper and lower limits of the new interval is re-constructed.

[0126] The process of comparing the expressiveness of the network and re-constructing the second network is repeated until the difference between the expressiveness of the second network and the expressiveness of the target network is within the set threshold range, and the construction is stopped.

[0127] In the embodiment, with reference to Figure 12 , the network searching module 140 comprises a search range setting unit 1401 and a network searching unit 1402; wherein,

[0128] The search range setting unit 1401 is configured to set a search range by increasing or decreasing a set layer number according to the layer number of the second network.

[0129] The network searching unit 1402 is configured to search for a network based on an evolutionary algorithm according to the search range.

[0130] In the case that the expressiveness of the searched network meets the requirement, the network with a hardware feedback time exceeding a set value is removed, and the remaining networks are randomly combined and iteratively searched until a network with the shortest hardware feedback time is obtained.

[0131] With reference to Figure 13 , the system of the present application further comprises a model training module 150; wherein,

[0132] The model training module 150 is configured to train a model by using the network with the best performance in combination with the application scene task to obtain a trained neural network model.

[0133] It should be noted that although several modules of the hardware information based neural network structure search system are mentioned in the foregoing detailed description, such a division is merely exemplary and not mandatory. Indeed, according to embodiments of the application, the features and functionalities of two or more modules described above can be embodied in one module. Conversely, the features and functionalities of one module described above can be further divided into embodiments by multiple modules.

[0134] Based on the foregoing inventive concept, as shown in Figure 14 The application further proposes a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the foregoing hardware information based neural network structure search method when executing the computer program.

[0135] Based on the foregoing inventive concept, the application proposes a computer readable storage medium storing a computer program, wherein the computer program is executable by a processor to implement the foregoing hardware information based neural network structure search method.

[0136] Based on the foregoing inventive concept, the application proposes a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the hardware information based neural network structure search method.

[0137] The hardware information based neural network structure search method and system proposed by the application determines the network layer number range by the number of layers of the target network and the expressiveness of the model, and then gradually fixes the layer number range for small range search in combination with hardware information, so as to quickly obtain a network with good performance for deployment on a chip, while ensuring high accuracy, effectively solving the problem that the prior art cannot feed back the chip bottom layer information to the model design and the model search is time-consuming.

[0138] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0139] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0140] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0141] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0142] Finally, it should be noted that the above-mentioned embodiments are merely intended to illustrate the technical solutions of the present application, rather than limit the technical solutions of the present application, and the protection scope of the present application is not limited to this. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any modifications or changes to the technical solutions described in the foregoing embodiments, or any equivalent replacements, can be made by those skilled in the art within the technical scope of the present application, and these modifications or changes, or replacements, do not cause the technical solutions to deviate from the spirit and scope of the embodiments of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for searching a neural network structure based on hardware information, characterized in that, The method determines a network layer number range according to a layer number of a target network and an expressiveness of a model, combines hardware information as a constraint, gradually fixes the layer number range for small-range searching, and realizes obtaining a network deployed on a chip with good performance. The network structure designed for different hardware architectures has high execution efficiency and meets the demand of application scene tasks, including: According to an application scene task, a target network meeting a set requirement is selected, and an expressiveness of the target network is calculated. According to a layer number of the target network, a first network is constructed according to a search model space, and an expressiveness of the first network is calculated. If the expressiveness of the first network is less than the expressiveness of the target network, the depth of the first network is increased until the expressiveness of the first network is greater than the expressiveness of the target network. According to the layer number of the first network, a second network is constructed, and an expressiveness of the second network is calculated. The expressiveness of the second network is compared with the expressiveness of the target network. If the expressiveness of the second network is greater, the layer number is reduced and the network is reconstructed. If the expressiveness of the second network is less, the layer number is increased and the network is reconstructed, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and the second network is obtained. According to the layer number of the second network, a search range is set, and the network is searched based on an evolutionary algorithm and combined with hardware information according to the search range, to obtain a network with the best performance. According to the layer number of the second network, a search range is set, and the network is searched based on an evolutionary algorithm and combined with hardware information according to the search range, to obtain a network with the best performance, including: According to the layer number of the second network, a search range is set by increasing or decreasing the set layer number. According to the search range, the network is searched based on the evolutionary algorithm. In the case that the expressiveness of the searched network meets the standard, the network whose hardware feedback time exceeds the set value is removed. The remaining networks are randomly combined and iteratively searched until the network with the shortest hardware feedback time is obtained.

2. The method of claim 1, wherein, According to the application scene task, a target network meeting a set requirement is selected, and an expressiveness of the target network is calculated, including: According to the application scene task, a target network with a set accuracy is selected, and a layer number of the target network is calculated. The weights of the target network are replaced by normalized Gaussian distribution random numbers, and the expressiveness of the target network is calculated.

3. The method of claim 1, wherein, According to the layer number of the first network, a second network is constructed, and an expressiveness of the second network is calculated. The expressiveness of the second network is compared with the expressiveness of the target network. If the expressiveness of the second network is greater, the layer number is reduced and the network is reconstructed. If the expressiveness of the second network is less, the layer number is increased and the network is reconstructed, until the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range, and the second network is obtained, including: An interval with an upper limit of the layer number of the first network and a lower limit of 0 is set. According to the interval, a second network with a layer number being an average value of the upper limit and the lower limit of the current interval is constructed, and an expressiveness of the second network is calculated. It is judged whether the difference between the expressiveness of the second network and the expressiveness of the target network is within a set threshold range. If the difference is within the set threshold range, the construction is stopped. If the difference is not within the set threshold range, compare the expressiveness of the second network with the expressiveness of the target network; If the expressiveness of the second network is greater than the expressiveness of the target network, set a new interval with an upper limit being the upper limit of the current interval and a lower limit being the average of the upper and lower limits of the current interval, and reconstruct the second network with the number of layers being the average of the upper and lower limits of the new interval; If the expressiveness of the second network is less than the expressiveness of the target network, set a new interval with an upper limit being the average of the upper and lower limits of the current interval and a lower limit being the lower limit of the current interval, and reconstruct the second network with the number of layers being the average of the upper and lower limits of the new interval; Repeat the process of comparing the expressiveness of the network and reconstructing the second network until the difference between the expressiveness of the second network and the expressiveness of the target network is within the set threshold range, and stop building.

4. The method of claim 1, wherein, Further comprising: Use the best-performing network to combine the application scenario task for model training to obtain a trained neural network model. 5.A hardware information based neural network structure search system, characterized by, The system determines the network layer number range according to the number of layers of the target network and the expressiveness of the model, combines the hardware information as a constraint, gradually fixes the layer number range for small-range search, realizes the acquisition of the network with better performance for deployment on the chip, designs the network structure with efficient execution for different hardware architectures, and meets the demand of the application scenario task, including: A target network processing module is configured to select a target network meeting the set requirements according to an application scenario task, and calculate the expressiveness of the target network; A first network processing module is configured to construct a first network according to a search model space according to the number of layers of the target network, and calculate the expressiveness of the first network; if the expressiveness of the first network is less than the expressiveness of the target network, increase the depth of the first network until the expressiveness of the first network is greater than the expressiveness of the target network; A second network constructing module is configured to construct a second network according to the number of layers of the first network, and calculate the expressiveness of the second network; compare the expressiveness of the second network with the expressiveness of the target network; if greater, reduce the number of layers and reconstruct the network; if less, increase the number of layers and reconstruct the network, until the difference between the expressiveness of the second network and the expressiveness of the target network is within the set threshold range, and obtain the second network; A network search module is configured to set a search range according to the number of layers of the second network, search the network according to the search range based on an evolutionary algorithm and combined with hardware information, and obtain the best-performing network; The network search module includes a search range setting unit and a network searching unit; wherein The search range setting unit is configured to set a search range by increasing or decreasing the set number of layers according to the number of layers of the second network; The network searching unit is configured to search the network based on an evolutionary algorithm according to the search range; In the case that the expressiveness of the searched network meets the requirements, eliminate the network whose hardware feedback time exceeds the set value, randomly combine the remaining networks and perform iterative search until the network with the shortest hardware feedback time is obtained.

6. The system of claim 5, wherein, The target network processing module includes a layer number calculating unit and an expressiveness calculating unit; wherein The layer number calculation unit is configured to select a target network with a precision reaching a set value according to the application scenario task, and calculate a layer number of the target network. The expressiveness calculation unit is configured to replace weights of the target network with normalized Gaussian distribution random numbers, and calculate expressiveness of the target network.

7. The system of claim 5, wherein, The second network construction module comprises an interval setting unit, a network construction unit and a comparison unit. The interval setting unit is configured to set an interval with an upper limit being a layer number of the first network and a lower limit being 0. The network construction unit is configured to construct a second network with a layer number being an average value of upper and lower limits of the current interval according to the interval, and calculate expressiveness of the second network. The comparison unit is configured to judge whether a difference between the expressiveness of the second network and expressiveness of the target network is within a set threshold range. If the difference is within the set threshold range, the construction is stopped. If the difference is not within the set threshold range, the expressiveness of the second network is compared with the expressiveness of the target network. If the expressiveness of the second network is greater than the expressiveness of the target network, a new interval with an upper limit being an upper limit of the current interval and a lower limit being an average value of upper and lower limits of the current interval is set, and a second network with a layer number being the average value of upper and lower limits of the new interval is reconstructed. If the expressiveness of the second network is less than the expressiveness of the target network, a new interval with an upper limit being the average value of upper and lower limits of the current interval and a lower limit being a lower limit of the current interval is set, and a second network with a layer number being the average value of upper and lower limits of the new interval is reconstructed. The process of comparing expressiveness of networks and reconstructing the second network is repeated until the difference between the expressiveness of the second network and the expressiveness of the target network is within the set threshold range, and the construction is stopped.

8. The system of claim 5, wherein, The model training module is configured to train a model by using the network with the best performance in combination with the application scenario task, and obtain a trained neural network model. The processor executes the computer program to implement the method of any one of claims 1 to 4. The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1 to 4.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, ​ 11. A computer program product, characterised in that, ​

Images