Search method, apparatus, device, and computer-readable storage medium

Abstract

Embodiments of the present application disclose a search method, device and equipment and a computer readable storage medium. The method comprises: selecting chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; determining a parent population based on the initial population and an initial target evolution direction, and performing mutation and crossover on the chromosomes included in the parent population to determine a target population; continuing to determine a next parent population based on the target population and a next target evolution direction, and performing mutation and crossover on next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, multi-objective evolution is realized, and a last target population is obtained; determining a target chromosome from the last target population to realize multi-objective architecture search and obtain a target sub-network corresponding to the target chromosome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a search method, device and equipment and computer readable storage medium. BACKGROUND

[0002] Neural Network Architecture Search (NAS) refers to that for a neural network model, an optimal architecture is automatically designed by an algorithm to replace manual design by an algorithm engineer. Hardware-aware Neural Network Architecture Search (HWNAS) refers to that on the basis of NAS, a hardware-aware function is added. In actual application, a neural network (NN) usually needs to be deployed on specific target hardware. For example, a chip needs to achieve optimal states of accuracy, delay, power consumption, storage and the like when deploying a specific scene NN. Therefore, when deploying the NN network on the target hardware, not only the performance of the model needs to be considered, but also the hardware indicators such as delay and power consumption of the model on the hardware need to be balanced, so that the product achieves the best effect.

[0003] In the related art, a fixed hardware indicator constraint is artificially set, and an Evolution Algorithm (EA) is used to obtain a Pareto Optimal Solution in architecture search.

[0004] However, in architecture search given a hardware indicator constraint c (for example, delay), there may be a model x searched out, and the performance Px of the model x is consistent with the performance Py of another model y: Px = Py, or the performance Px of the model x is only slightly greater than the performance Py of the model y, but the hardware constraint is obviously c > Cx > Cy, so that the model x searched out is not the most needed by the user. If there is a model with consistent or similar performance and lower delay, the user is often more willing to select the model with lower delay, that is, to select the model y. Therefore, the search method in the related art reduces the accuracy of the search result. SUMMARY

[0005] The embodiments of the present application provide a search method, device, equipment and computer readable storage medium, and the accuracy of the search result is improved.

[0006] The technical solution of the embodiments of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a search method, which comprises: selecting chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; determining a parent population based on the initial population and an initial target evolution direction, and performing mutation and crossover on chromosomes included in the parent population to determine a target population; continuing to determine a next parent population based on the target population and a next target evolution direction, and performing mutation and crossover on next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, realizing multi-objective evolution, and obtaining a last target population; determining a target chromosome from the last target population to realize multi-objective architecture search, and obtaining a target sub-network corresponding to the target chromosome.

[0008] In a second aspect, an embodiment of the present application provides a search device, which comprises: a population determination module configured to select chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; and determine a parent population based on the initial population and an initial target evolution direction; an evolution module configured to perform mutation and crossover on chromosomes included in the parent population to determine a target population; the population determination module is further configured to continue to determine a next parent population based on the target population and a next target evolution direction; the evolution module is further configured to perform mutation and crossover on next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, realize multi-objective evolution, and obtain a last target population; and a target sub-network determination module configured to determine a target chromosome from the last target population to realize multi-objective architecture search, and obtain a target sub-network corresponding to the target chromosome.

[0009] In a third aspect, an embodiment of the present application provides a search device, which comprises: a memory configured to store an executable computer program; and a processor configured to execute the executable computer program stored in the memory to realize the search method in the first aspect.

[0010] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, which is configured to be executed by a processor to realize the search method in the first aspect.

[0011] The embodiment of the present application provides a search method, device and equipment and a computer readable storage medium. According to the scheme provided by the embodiment of the present application, the method comprises the following steps: selecting a chromosome satisfying a fixed constraint in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; determining a parent population based on the initial population and an initial target evolution direction, wherein the initial chromosomes in the initial population all satisfy the fixed constraint, selecting a chromosome superior in the initial target evolution direction from the initial population, and constructing the parent population. The chromosomes included in the parent population are subjected to mutation and crossover to determine a target population, and the evolution process for the initial target evolution direction in the current round is completed. The next parent population is determined based on the target population and a next target evolution direction, and the next chromosomes included in the next parent population are subjected to mutation and crossover to determine a next target population, and the evolution process for the next target evolution direction in the next round is completed. Until the number of iterations is reached, the multi-target evolution is realized through the evolution processes for different target evolution directions in multiple rounds, the last target population is obtained, the target chromosome is determined from the last target population, and the architecture search of the multi-target is realized, so that the target sub-network corresponding to the target chromosome is obtained. Through the multiple target evolution directions in the scheme, the target sub-network searched can meet the hardware requirements and achieve better model performance. While meeting the search efficiency, the collaborative optimization of model accuracy, delay, power consumption and other hardware targets is realized, so that the multi-dimensional requirements in the actual scene are met, and the accuracy of the search result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 An exemplary schematic diagram of a Pareto optimal solution provided by the embodiment of the present application is shown in the following figure:

[0013] Figure 2 An exemplary schematic diagram of a single-target Pareto optimal solution provided by the embodiment of the present application is shown in the following figure:

[0014] Figure 3 An optional step flowchart of a search method provided by the embodiment of the present application is shown in the following figure:

[0015] Figure 4 An optional step flowchart of another search method provided by the embodiment of the present application is shown in the following figure:

[0016] Figure 5 An optional step flowchart of still another search method provided by the embodiment of the present application is shown in the following figure:

[0017] Figure 6 An optional step flowchart of yet another search method provided by the embodiment of the present application is shown in the following figure:

[0018] Figure 7An exemplary schematic diagram of a cross-mutation proportion self-adaptive adjustment strategy provided for an embodiment of the present application;

[0019] Figure 8 An optional step flowchart of a multi-objective evolutionary algorithm based on a greedy strategy provided for an embodiment of the present application;

[0020] Figure 9 An optional structural schematic diagram of a search device provided for an embodiment of the present application;

[0021] Figure 10 An exemplary structural schematic diagram of a search device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. It should be understood that some embodiments described herein are only used to explain the technical solutions of the present application, and do not limit the technical scope of the present application.

[0023] In order to better understand the search method provided in the embodiments of the present application, before introducing the technical solutions of the embodiments of the present application, the application background and related technologies are described.

[0024] The advantages of NAS mainly include the following aspects: first, it can greatly reduce the cost of manual network design, which is particularly obvious when the scene is complex and the number of tasks is huge; second, NAS can maximize the potential of the optimal model through automatic means; third, it can largely avoid the disadvantages brought by manual network design, such as reducing the deviation of human inherent knowledge system and cognition on model design. Therefore, NAS has been more and more widely applied. NAS mainly includes three contents: search space design, search strategy design, and model evaluation strategy.

[0025] Based on the NAS of HW-NAS, in order to consider the performance of the model, the delay of the model on the hardware, the power consumption and other hardware indicators at the same time, the HW-NAS automatically adjusts the search strategy through the performance feedback of the specific target hardware, so as to search the model with the best comprehensive indicators such as accuracy, delay, power consumption, etc. In the case of increasing demand for embedded devices, edge computing, etc., higher requirements are put forward for the inference delay, power consumption, etc. of the NN model, and HW-NAS and multi-objective search technology emerge as the times require.

[0026] Evolutionary Algorithm (EA) is a common heuristic algorithm in mathematics, which is derived from the evolution mechanism of biology, simulates the processes of reproduction, mutation, genetic recombination and natural selection in nature, and thus finds the individual most suitable for the environment from the candidate population. One of its outstanding advantages is that it can find a relatively optimal solution to the optimization problem to be solved within an acceptable time. Evolutionary algorithms include Genetic Algorithm (GA), Evolutionary Programming (EP), Genetic Programming (GP) and Evolution Strategy (ES). Genetic Algorithm (GA) is also widely used in HW-NAS.

[0027] Once-for-All (OFA) proposed in the related art is to find the best model of actual hardware perception through Genetic Algorithm (GA). It can realize one training and multiple hardware deployments in a short time. For example, OFA first trains a super network containing a large number of sub-networks, and then obtains the best sub-model meeting the requirements from the super network according to the hardware requirements (for example, latency) to be deployed for actual deployment.

[0028] In the network architecture search based on Genetic Algorithm such as OFA, the super network training and search process are usually decoupled. That is, the super network is trained before the search. The super network can be regarded as a network pool containing a large number of sub-networks. In the evolutionary algorithm of OFA, the target hardware constraint c needs to be given, and the search for optimal performance is performed under the condition of meeting the target hardware constraint c to obtain the model architecture of the optimal sub-network, that is, the Pareto optimal solution in the mathematical sense. The definition of the Pareto optimal solution is as follows: a solution x is called a Pareto optimal solution if and only if x is not dominated by other solutions. The definition of domination is as follows: x dominates y, denoted as x < y, if and only if f i (x)≤f i (y), and s.t.f j (x)<f j (y), s.t. represents the constraint condition.

[0029] Based on the definition of the Pareto optimal solution, the optimal model obtained in the architecture search (that is, the Pareto optimal solution) needs to meet the condition that there is no model having better effect on all targets. It is impossible to make the current target better without sacrificing other targets. That is, to further optimize the current target, it is necessary to cause loss to other targets. The search algorithm of OFA is as shown in Figure 1 .Figure 1 This is an exemplary schematic diagram of a Pareto optimal solution provided in an embodiment of this application. Figure 1 The horizontal axis represents the hardware metric c, the vertical axis represents the performance p, and the curve represents the Pareto front, given a hardware target constraint ( Figure 1 Given the hardware constraints (c), a performance-oriented search is performed. Figure 1 The OFA optimization approach treats a single hardware objective as a fixed constraint to obtain the optimal model.

[0030] Based on the above Figure 1 OFA treats a single hardware objective as a fixed, strong constraint, which makes it difficult to achieve Pareto optimality in multi-objective optimization scenarios. For example... Figure 2 As shown, Figure 2 This is an exemplary schematic diagram of a single-objective Pareto optimal solution provided in an embodiment of this application. Figure 2 The horizontal axis represents the hardware metric c, the vertical axis represents performance p, and the curve represents the Pareto front. Performance p will not continuously increase as the hardware metric c is relaxed; it will gradually slow down over time. From... Figure 2 As can be seen from this, in the architecture search given hardware constraint c, the optimal model x( Figure 2 The performance Px of the optimal model x obtained from OFA is similar to that of another model y (Py is slightly larger than Py), but model x has higher hardware requirements. Compared to model x, users may prefer to choose model y. Figure 2 (The model y that the user might need more). In other words, if multiple models are similar in terms of hardware specifications, OFA can easily find the best model that is not needed by the user.

[0031] However, in practical applications, the architecture search process can encounter some non-ideal situations. One situation is that relaxing hardware specifications doesn't necessarily lead to improved search performance. This is due to the influence of hypernetwork training and evaluation algorithms, as well as the randomness of model training. Another situation is that the improvement in model performance slows down as hardware specifications are relaxed. In such cases, users typically don't want to sacrifice too much hardware performance for a slight increase in accuracy; the optimization ratio between objectives depends on the specific needs of the scenario. Figure 2 This case illustrates that once hardware specifications exceed a certain value, performance increases become very slow. In scenarios where performance requirements are not extremely stringent, users typically prefer to sacrifice a small amount of performance in exchange for greater potential for hardware specification optimization.

[0032] The technical defects of the OFA-based architecture search method in the related art are as follows: (1) the OFA-based architecture search does not optimize hardware indicators sufficiently. The OFA treats hardware indicators as fixed constraints, which can be regarded as single-target optimization of performance under a certain hardware target feasible region. The OFA algorithm needs to manually set a fixed hardware indicator constraint, such as a delay constraint c. In the evolutionary algorithm process, the hardware indicator constraint of all candidate models generated by random generation, crossover, and mutation can meet the condition of being less than c. This logic leads to the fact that the searched model is not necessarily a Pareto optimal solution. The significance of seeking a Pareto optimal solution in architecture search is that the current target cannot be better without sacrificing other targets. That is, to further optimize the current target, some other target must be compromised. For example, in the architecture search given the hardware constraint c, there may be a model x searched out, the performance of which is consistent with that of another model y: Px=Py, or the performance of the model x is only slightly greater than that of the model y, but the hardware constraint is obviously c>Cx>Cy, so that the model x searched out is not the most needed by the user.

[0033] (2) Lack of multi-objective optimization scheme. In the OFA architecture search process, only the best model is searched for a single specific hardware constraint (i.e., single target). However, in actual application scenarios, the searched model needs to be able to achieve the best in precision, delay, power consumption, and other indicators (i.e., multi-target) at the same time, so as to meet the multi-element demand. Therefore, the HW-NAS needs to meet the demand of multi-objective optimization.

[0034] In summary, the technical problems of the architecture search performed by the evolutionary algorithm adopted by the OFA include: (1) the search result may not be a Pareto optimal solution; (2) the OFA does not support multi-target (such as performance, delay, and power consumption) architecture search task at the same time.

[0035] An embodiment of the present application provides a search method, as shown in Figure 3 The search method provided by an embodiment of the present application includes the following steps: Figure 3 The search method provided by an embodiment of the present application includes the following steps:

[0036] S101, selecting a chromosome satisfying a fixed constraint in a super network corresponding to an architecture search space to determine an initial population; the super network includes at least two sub-networks, and each chromosome corresponds to a sub-network.

[0037] In the embodiment of the present application, the trained super network includes at least two sub-networks, each sub-network is regarded as a chromosome, and chromosomes meeting the fixed constraint are screened from the chromosomes. The fixed constraint can be a parameter related to a model index, which is set by a person skilled in the art according to actual needs, and the model index includes but is not limited to model performance, delay, power consumption, calculation amount, storage space, etc. The model performance can be understood as the accuracy of the prediction result of the model, for example, for a network with classification function, the higher the classification result accuracy is, the better the model performance is. The fixed constraint can be a preset value related to a certain index or at least two or more indexes. The index information of each sub-network in the super network is predicted, and when the index information of the sub-network meets the fixed constraint, the sub-network is added to the initial population until the initial population size is met, thereby generating the initial population.

[0038] In the embodiment of the present application, the index information of each sub-network in the super network can be predicted by testing the hardware requirements of the sub-network through hardware testing, and testing the model performance of the sub-network through a test sample set. Alternatively, each sub-network can be input into a target predictor for prediction, and the hardware requirement parameters and model performance parameters of the sub-network are output.

[0039] In the embodiment of the present application, a person skilled in the art not only sets the fixed constraint, but also defines the evolution algorithm related parameters, which include but are not limited to: initial population size (for example, the initial population includes N initial chromosomes), iteration rounds (for example, iteration number E), crossover proportion coefficient, mutation proportion coefficient (for example, the mutation proportion coefficient is M / N, the crossover proportion coefficient is (N-M) / N, M times of mutation and (N-M) times of crossover are performed, thereby ensuring that the number of chromosomes in the population is N), mutation probability, etc.

[0040] In the embodiment of the present application, the super network in the architecture search space includes at least two sub-networks, and is randomly sampled in the architecture search space. A plurality of chromosomes (i.e. architecture configurations of the sub-networks) are generated, and the index information (which can also be understood as the target value under each target) under each index is calculated. Chromosomes meeting the fixed constraint are added to the initial population until the initial population size is met, thereby generating the initial population. If there is a fixed constraint under a certain index, the chromosome meeting the constraint is selected.

[0041] S102, based on the initial population and the initial target evolution direction, determine the parent population, and perform mutation and crossover on the chromosomes included in the parent population to determine the target population.

[0042] In the embodiment of the present application, the initial population includes N initial chromosomes, the index information of each chromosome is predicted for each index, the initial target evolution direction is any one index, or at least two indexes with correlation or conflict. The index information of the N initial chromosomes in the initial target evolution direction is sorted, the first k chromosomes are selected as the parent population. The chromosomes included in the parent population are crossed according to the cross ratio coefficient defined in the foregoing, and the chromosomes included in the parent population are mutated according to the mutation ratio coefficient defined in the foregoing. The chromosomes after crossing and mutation are added to the target population, so as to obtain the target population, so that the target population includes N evolution chromosomes. The target population can also be referred to as the elite population, and the evolution chromosomes can also be referred to as the elite chromosomes.

[0043] In the embodiment of the present application, the evolution process of the round is completed by selecting the optimal chromosome to generate the target population, and the evolution direction selection based on the greedy strategy. The evolution direction selection based on the greedy strategy includes: in each evolution process, the evolution direction is sampled, the target evolution direction sampled is any index, for different chromosomes, the index information corresponding to the target evolution direction (i.e. the target value corresponding to the selected evolution direction) is taken as the fitness score of the chromosome, and the chromosome with the highest fitness score is saved and added to the parent population. And the chromosomes in the parent population are mutated and crossed according to the cross-mutation ratio adaptive adjustment strategy to obtain a new population (i.e. the target population). The cross-mutation ratio adaptive adjustment strategy includes: the mutation ratio coefficient and the cross ratio times of the first round are default (which are defined by the person skilled in the art according to actual needs), and the second round and later are adjusted in real time according to the distribution law (for example, the mean and the variance) of the chromosomes in the parent population.

[0044] S103, continue to determine the next parent population based on the target population and the next target evolution direction, and mutate and cross the next chromosomes included in the next parent population to determine the next target population, until the iteration number is reached, realize multi-objective evolution, and obtain the last target population.

[0045] In the embodiment of the present application, the steps of generating the parent population based on the target evolution direction in S102, crossing and mutating to determine the target population are repeatedly executed to determine the next target population, and complete the evolution process of the next target evolution direction in the next round. In turn, cycle until the iteration number E is reached, and output the last target population (the last target population can also be referred to as the elite set).

[0046] The scheme proposes an evolutionary direction selection based on a greedy strategy. In the multi-objective evolutionary algorithm based on the greedy strategy, the target evolutionary direction needs to be selected in each round. The greedy strategy refers to selecting a single target as the optimization direction in each iteration, and realizing the optimization effect of multiple targets by constantly searching for the local optimum under a single target.

[0047] In the embodiments of the present application, a target evolutionary direction can be randomly selected in each evolutionary process, or the target evolutionary direction can be selected in turn. Each evolutionary process includes crossover and mutation, and the process of selection-mutation-crossover is repeated until the set iteration round (i.e., the number of iterations) is reached. If there is a fixed constraint under the index corresponding to a certain target evolutionary direction, the mutation process and the crossover process both select the chromosome that satisfies the fixed constraint. After multiple evolutionary processes, the multi-objective evolutionary direction selection based on the greedy strategy is completed, and the last target population is obtained.

[0048] In the embodiments of the present application, multiple target hardware constraints are added to realize multi-objective optimization, and a better balance (trade-off) between model performance and efficiency is achieved. While ensuring the model performance, the work efficiency is improved, for example, the delay is low, the power consumption is small, and the calculation amount is small.

[0049] In S104, a target chromosome is determined from the last target population to realize the architecture search of multiple targets, and a target subnetwork corresponding to the target chromosome is obtained.

[0050] In the embodiments of the present application, the last target population includes multiple candidate chromosomes, and the index information of the subnetwork corresponding to the multiple candidate chromosomes is predicted. The index information includes a model performance parameter and a hardware demand parameter (for example, delay, power consumption, calculation amount, storage space, etc.). After obtaining the last target population, a candidate chromosome with the optimal model performance parameter (for example, the highest accuracy) can be selected from the last target population as the target chromosome. A candidate chromosome with the optimal hardware demand parameter (any one of delay, power consumption, calculation amount, and storage space) can also be selected from the last target population as the target chromosome. A subnetwork that satisfies the fixed constraint can also be selected from the last target population as the target chromosome, and the number of target chromosomes is at least one. The selection mode and the number of target chromosomes are not limited in the embodiments of the present application, as long as the scheme of determining the last target population and then obtaining the target chromosome through multi-objective optimization is within the protection scope of the present application.

[0051] Exemplarily, taking the fixed constraint of delay being less than c as an example, the selection of the sub-network satisfying the fixed constraint in the last target population is described. Chromosomes with delay less than c in the index information in the last target population are taken as target chromosomes. Taking the fixed constraint of the model performance corresponding to the accuracy being greater than d as an example, chromosomes with accuracy greater than d in the index information in the last target population are taken as target chromosomes. In this way, the architecture search of multiple targets is implemented.

[0052] In the embodiment of the present application, the target chromosomes are determined from the last target population, and the search result is returned, that is, the chromosome with optimal model performance or hardware requirement is returned. The chromosome corresponds to the candidate model architecture of the sub-network and the corresponding weight parameters, so as to obtain the target sub-network.

[0053] The architecture search method in the embodiment of the present application is to replace the original evolution algorithm adopted by OFA with a multi-objective evolutionary algorithm based on a greedy strategy, and output the target model (corresponding to the target sub-network) searched. Through the multi-objective evolutionary algorithm based on the greedy strategy, the common optimization of multiple targets is realized. The single-target evolutionary algorithm of OFA is improved, and a hardware-aware multi-objective neural network architecture search method based on a greedy strategy is proposed, so that the architecture search of multiple targets is realized. Compared with the evolution algorithm adopted by OFA, the present scheme can realize more efficient compression of hardware constraints, find a sub-network with lower power consumption (or lower delay, smaller calculation amount) in the super network, and improve the accuracy of the search result.

[0054] According to the scheme provided in the embodiment of the present application, the method comprises: selecting chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; determining a parent population based on the initial population and an initial target evolution direction, wherein the initial chromosomes in the initial population all satisfy the fixed constraints, and the chromosomes in the initial population that are better in the initial target evolution direction are selected to construct the parent population. The chromosomes included in the parent population are mutated and crossed to determine a target population, and the evolution process for the initial target evolution direction in the current round is completed. The next parent population is determined based on the target population and a next target evolution direction, and the next chromosomes included in the next parent population are mutated and crossed to determine a next target population, and the evolution process for the next target evolution direction in the next round is completed. Until the number of iterations is reached, the multi-target evolution is realized through the evolution processes for different target evolution directions in multiple rounds, and the last target population is obtained. The target chromosome is determined from the last target population to realize the multi-target architecture search, and the target sub-network corresponding to the target chromosome is obtained. Through the multiple target evolution directions in the scheme, the target sub-network searched can meet the hardware requirements while achieving better model performance. While meeting the search efficiency, the collaborative optimization of model accuracy, delay, power consumption and other hardware targets is realized, thereby meeting the multi-dimensional requirements in actual scenarios and improving the accuracy of the search results.

[0055] In some embodiments, the S104 of determining the target chromosome from the last target population can also be implemented in the following manner. Figure 3

[0056] In the embodiment of the present application, the index information of the sub-network corresponding to each candidate chromosome in the last target population can be tested by testing the model performance of the sub-network on the test sample set, or each sub-network can be input into the target predictor for prediction to output the model performance parameters of the sub-network. Then, the chromosome with the best model performance (for example, the highest accuracy) in the last target population is determined as the target chromosome.

[0057] In the embodiment of the present application, the multi-target evolution is realized through the evolution processes for different target evolution directions in multiple rounds, which reduces the case that the sub-network is only optimal in a certain index while sacrificing other indexes, and thus the chromosome with the best model performance is selected in the final last target population, improving the accuracy of the search results.

[0058] In some embodiments, the S104 of determining the target chromosome from the last target population can also be implemented in the following manner. Figure 3 ​The step of determining the parent population in S102 can further include S1021-S1023. As shown in Figure 4 Figure 4 An optional step flowchart of another search method provided by an embodiment of the present application.

[0059] S1021, predicting each index of each initial chromosome in the initial population to obtain at least two index information of each initial chromosome; the indexes include model performance and hardware requirement of the sub-network.

[0060] In an embodiment of the present application, each initial chromosome corresponds to a sub-network, and the hardware requirement of the sub-network can be tested by hardware testing. The sub-network works on the hardware device, and the hardware requirement (for example, delay, power consumption, calculation amount, storage space, etc.) of the sub-network is tested. For example, the hardware index is obtained by testing on the target hardware, and the efficiency of the acceleration architecture search is improved. The model performance of the sub-network is tested by the test sample set. The test sample set is input into the sub-network, and the test accuracy is determined by comparing the label value of the test sample set with the prediction result output by the sub-network. Each sub-network can also be input into the target predictor for prediction, and the hardware requirement parameter and the model performance parameter of the sub-network are output.

[0061] In some embodiments, the above Figure 4 S1021 in the present application can also be implemented in the following manner. Based on an index prediction model, each index of each initial chromosome in the initial population is predicted to obtain at least two index information of each initial chromosome; wherein the index prediction model is used to predict each index of the model architecture of the sub-network, and the index prediction model includes a performance index prediction model and a hardware index prediction model.

[0062] In an embodiment of the present application, any index is taken as a target for optimization, and the index prediction model can also be understood as a target predictor. The target predictor is pre-trained, and the target predictor is used to predict the model performance and the hardware requirement of the model architecture of the sub-network. The target predictor includes a performance index predictor and a hardware index predictor. The predictor can be regressed by a machine learning method such as a neural network, so that a candidate model architecture can be input, and the prediction value of the related index of the candidate model can be output, replacing the actual evaluation strategy.

[0063] S1022, according to the index information corresponding to the initial target evolution direction in the at least two index information of each initial chromosome, sorting the initial chromosomes included in the initial population to determine a sorting result; the initial target evolution direction is any one index, or at least two indexes with correlation or conflict.

[0064] ​In the embodiment of the present application, the initial target evolution direction is any one index, and the initial chromosomes in the initial population are sorted in descending order (or in ascending order, or in high-to-low order, or in low-to-high order, or in optimal-to-poor order, or in poor-to-optimal order) according to the index information of the initial chromosomes in the index, and the sorting result is determined.

[0065] In the embodiment of the present application, for the technical problem of multi-objective optimization, the correlation or conflict between different objectives can be analyzed, and the algorithm can be further optimized. Other multi-objective optimization algorithm schemes can also be adopted. For example, a particle swarm algorithm, a multi-objective optimization method based on mutual relationship, etc.

[0066] In the embodiment of the present application, the initial target evolution direction is at least two indexes with correlation or conflict, the correlation value or conflict value of each initial chromosome in at least two indexes with correlation or conflict is calculated, the initial chromosomes in the initial population are sorted according to the size of the correlation value or conflict value, and the sorting result is determined.

[0067] S1023, according to the sorting result, selecting a first preset number of chromosomes in a preset order to construct a parent population.

[0068] In the embodiment of the present application, taking k as an example, when the sorting result is in descending order (or in high-to-low order, or in optimal-to-poor order), the chromosomes in the front k are selected to construct the parent population; when the sorting result is in ascending order (or in low-to-high order, or in poor-to-optimal order), the chromosomes in the back k are selected to construct the parent population. The chromosomes in the parent population are locally optimal, and the number of chromosomes in the parent population is less than the number of chromosomes in the initial population.

[0069] For example, each chromosome corresponds to at least two indexes, a plurality of evolution processes correspond to a plurality of target evolution directions (i.e. a plurality of indexes), and a plurality of targets f1, f2, f3…… exist in hardware-aware architecture search, x represents any subnetwork in the population, f i (x) represents the value of x on the target i (corresponding to the index information). In each iteration, any target is selected as the optimization direction of this round, for example, f t is the target evolution direction at the tthround, and the target values of all candidate models (corresponding to the chromosomes included in the parent population) on f t are used as the fitness of sorting. Based on the multi-objective fitness sorting method, the evolution algorithm can form a hardware-aware multi-objective neural network architecture search method based on the greedy strategy.

[0070] It should be noted that the greedy strategy can be improved in the evolutionary direction selection to improve the convergence of the algorithm. In addition to the greedy strategy, other methods can be tried to select the evolutionary direction, such as a recursive algorithm, a divide and conquer method, a back track method, a branch and bound method, and the like.

[0071] In some embodiments, the parent population includes a first preset number of chromosomes; the above Figure 3 The step of determining the target population in S102 can further include S1024-S1026. As Figure 5 shown, Figure 5 An optional step flowchart of another search method provided by the embodiments of the present application.

[0072] S1024, based on the first preset number of chromosomes, performing mutation according to an initial mutation proportion coefficient to determine a first number of mutated chromosomes; the first number is a product of the initial mutation proportion coefficient and a second preset number.

[0073] In the embodiments of the present application, one chromosome is randomly sampled from the first preset number of chromosomes, and one mutation is performed on the chromosome to obtain one mutated chromosome, and the process is repeated M times (the first number is M) to obtain the first number of mutated chromosomes.

[0074] In the embodiments of the present application, the initial mutation proportion coefficient and the initial crossover proportion coefficient also need to be set by the person skilled in the art according to the actual needs, the sum of the initial mutation proportion coefficient and the initial crossover proportion coefficient is 1, and the initial mutation proportion coefficient and the initial crossover proportion coefficient can also be default coefficients, which can be set by the person skilled in the art according to the actual needs through a large amount of experimental data, and the default coefficients can be adaptively adjusted in the subsequent evolution process.

[0075] For example, taking the first preset number k, the second preset number N, the initial mutation proportion coefficient M / N, and the initial crossover proportion coefficient (N-M) / N as an example, the first number is M / N×N=M, in the first round of evolution, k chromosomes are randomly sampled according to the default proportion coefficient, one mutation is performed on each sampled chromosome, and the process is repeated until M mutations are performed, and M mutated chromosomes (corresponding to the first number of mutated chromosomes) are obtained.

[0076] It should be noted that the mutation refers to gene mutation on a chromosome of a single parent candidate model to generate a new chromosome, and at this time, the new chromosome has genes of the single parent model, and part of the genes are replaced by new genes generated by the generation algorithm (such as random generation). Compared with the crossover, less information is retained. When random sampling is performed, the same chromosome may be sampled, but the mutation direction is inconsistent, and therefore, the mutated chromosomes are different.

[0077] In S1025, the first preset number of chromosomes are crossed based on the initial crossover proportion coefficient to determine a second number of crossover chromosomes; and the initial crossover proportion coefficient is a difference between 1 and the initial mutation proportion coefficient.

[0078] In the embodiment of the present application, two chromosomes are randomly sampled from the first preset number of chromosomes, and one crossover is performed on the two chromosomes to obtain one crossover chromosome, and the process is sequentially repeated (N-M) times (that is, the second number is N-M) to obtain the second number of crossover chromosomes.

[0079] For example, taking the first preset number k, the second preset number N, and the initial crossover proportion coefficient (N-M) / N as an example, the second number is (N-M) / N×N=N-M, in the first round of evolution process, k chromosomes are randomly sampled according to the default proportion coefficient, one crossover is performed each time a chromosome is sampled, and the process is sequentially repeated until (N-M) / N crossovers are performed to obtain (N-M) crossover chromosomes (corresponding to the second number of crossover chromosomes).

[0080] It should be noted that the crossover refers to the crossover of the chromosomes of two parent candidate models to generate a new chromosome, and at this time, the new chromosome retains more information of the genes of the two parent models. When random sampling is performed, the same two chromosomes may be sampled, but the genes sampled by the crossover are inconsistent, and therefore, the crossover chromosomes are different.

[0081] In S1026, the target population is determined according to the first number of mutated chromosomes and the second number of crossover chromosomes; and the target population includes the second preset number of evolution chromosomes.

[0082] In the embodiment of the present application, the first number of mutated chromosomes are added to the target population, and the second number of crossover chromosomes are added to the target population, and the target population is generated after the mutation and the crossover, the target population includes the second preset number of evolution chromosomes, and the number of chromosomes in the population is consistent (for example, N). The target population can also be referred to as an elite population, and the evolution chromosome can also be referred to as an elite chromosome.

[0083] In the embodiments of the present application, according to the initial mutation proportion coefficient and the initial crossover proportion coefficient, mutation and crossover are performed based on the first preset number of chromosomes, so as to generate a target population and complete a round of evolution process. The evolved chromosomes in the target population can be used for the next round of evolution process (selecting the optimal k as the parent to perform mutation and crossover in the next round), and the process is sequentially cycled until the number of iterations is reached, a last target population is obtained, and multi-objective evolution is realized.

[0084] In some embodiments, for the mutation proportion coefficient and the crossover proportion coefficient used in each round, the search method further includes a crossover-mutation proportion coefficient adaptive adjustment strategy, including S201-S205. As shown in Figure 6 Figure 6 An optional step flowchart of another search method provided by the embodiments of the present application.

[0085] S201, according to the index information of each current chromosome in the current parent population in the current target evolution direction, the mean and the variance are determined.

[0086] In the embodiments of the present application, for the parent population in each round of evolution process (corresponding to the current parent population), it includes a plurality of chromosomes (corresponding to the current chromosome), each chromosome corresponds to at least two index information, and the mean and the variance are calculated according to the index information of each chromosome in the target evolution direction (any index) of the round of evolution process. That is, the target value of the parent population is counted, including the mean and the variance.

[0087] In the embodiments of the present application, according to the calculated mean and variance, different crossover mutation proportions are taken, as shown in Figure 7 Figure 7 An exemplary schematic diagram of a crossover mutation proportion adaptive adjustment strategy provided by the embodiments of the present application; for the advantages and disadvantages of the mean, the mean in the parent statistical parameters of the last round can be compared, or the related target value of the maximum network or the minimum network in the super network can be compared to judge the advantages and disadvantages of the statistical parameters. In the strategy part, the crossover and mutation proportions can be given a default value (i.e. the initial mutation proportion coefficient and the initial crossover proportion coefficient), wherein, Figure 7 The high and low in the above formula refer to the adjustment relative to the default value.

[0088] S202, in the case that the mean is less than a preset mean and the variance is less than a preset variance, the initial mutation proportion coefficient is adjusted to a first coefficient; the first coefficient is greater than the initial mutation proportion coefficient.

[0089] In the embodiments of the present application, as shown in Figure 7 ​​As shown, when the mean is poor and the variance is low, it indicates that the target value of the parent model is poor (i.e., the indicator information of the current chromosomes included in the current parent population in the current target evolutionary direction is poor), and this has a certain degree of universality. The mutation ratio can be appropriately increased (i.e., the initial mutation ratio coefficient is adjusted to the first coefficient) to generate as many new, superior chromosomes as possible. Since the sum of the mutation ratio coefficient and the crossover ratio coefficient is 1, increasing the mutation ratio is equivalent to decreasing the crossover ratio.

[0090] S203. If the mean is greater than the preset mean and the variance is less than the preset variance, adjust the initial crossover ratio coefficient to the second coefficient; the second coefficient is greater than the initial crossover ratio coefficient.

[0091] In the embodiments of this application, such as Figure 7 As shown, when the mean is good and the variance is low, it indicates that the target value of the parent model is good (i.e., the indicator information of the current chromosomes included in the current parent population in the current target evolutionary direction is better) and has a certain degree of universality. The crossover ratio can be appropriately increased (that is, the initial crossover ratio coefficient is adjusted to the second coefficient) so that more excellent parent genes are retained. Since the sum of the mutation ratio coefficient and the crossover ratio coefficient is 1, increasing the crossover ratio is equivalent to reducing the mutation ratio.

[0092] S204. If the mean is less than the preset mean and the variance is greater than the preset variance, retain the initial crossover ratio coefficient and the initial variation ratio coefficient.

[0093] In the embodiments of this application, such as Figure 7 As shown, when the mean is poor and the variance is high, it indicates that the target value of the parent model is poor and its generality is weak. It may have some good chromosomes and can retain the default proportion of crossover variation.

[0094] S205. If the mean is greater than the preset mean and the variance is greater than the preset variance, retain the initial crossover ratio coefficient and the initial variation ratio coefficient.

[0095] In the embodiments of this application, such as Figure 7 As shown, when the mean is good and the variance is high, it indicates that the target value of the parent model is good and the generality is weak, so the default ratio of crossover variation can be retained.

[0096] It should be noted that the above Figure 6 S202-S205 are four parallel schemes with no specific execution order. During one round of evolution, one pair of scaling factors (mutation scaling factor and crossover scaling factor) can be selected for adjustment.

[0097] The adaptive crossover-mutation ratio adjustment strategy proposed in this application can further optimize the evolutionary algorithm and improve its convergence. Compared to setting a fixed crossover-mutation ratio, this scheme proposes an adaptive crossover-mutation ratio adjustment strategy to optimize the evolutionary algorithm and better achieve population optimization during the crossover and mutation process of the parent generation.

[0098] In some embodiments, the above Figure 6 The preset mean and preset variance in S202-S205 can be achieved by determining the preset mean and preset variance based on the index information of each previous chromosome in the previous parent population in the current target evolutionary direction.

[0099] In this embodiment, since the current parent population is obtained by selecting superior chromosomes from the target population obtained through crossover and mutation of the previous parent population, the distribution pattern (e.g., mean and variance) of chromosomes in the previous parent population in the current target evolutionary direction adopted in this round of evolution is relevant for this round of evolution. The parent population in the previous round of evolution (corresponding to the previous parent population) includes multiple chromosomes (corresponding to the previous chromosome), each chromosome corresponding to at least two indicator information. Based on the indicator information of each chromosome in the current target evolutionary direction (since statistical comparison is performed in this round, it is necessary to calculate the distribution pattern of each chromosome in the current target evolutionary direction in the previous round), the mean and variance are calculated and used as preset mean and preset variance.

[0100] In some embodiments, the above Figure 6 The preset mean and preset variance in S202-S205 can be achieved by determining the preset mean based on the index information of the largest or smallest network in the supernetwork in the current target evolution direction.

[0101] In this embodiment, the supernetwork includes multiple subnetworks. The size of each subnetwork can be determined based on at least one of the following parameters: resolution, dimension, number of repetitions per block, convolutional kernel size, etc. The largest network has a large number of parameters and good model performance, but its hardware requirements are poor, such as higher latency, higher power consumption, and higher computational cost. The smallest network has a small number of parameters and good hardware requirements, but its model performance is poor, such as lower accuracy.

[0102] In this embodiment, the quality of the mean can be judged by comparing it with the mean in the previous generation of parent statistical parameters, or by comparing it with the relevant target value of the largest or smallest network in the supernetwork, thereby improving the diversity of the conditions on which adaptive adjustment is based.

[0103] In some embodiments, the above Figure 3 The step of determining the next parent population in S103 can also be implemented in the following manner. The indicators of each evolution chromosome in the target population are predicted to obtain at least two indicator information of each evolution chromosome; the evolution chromosomes included in the target population are sorted according to the indicator information corresponding to the next target evolution direction in the at least two indicator information of each evolution chromosome to determine a next sorting result; the next target evolution direction is any one indicator, or at least two indicators with correlation or conflict; and the first preset number of next chromosomes are selected according to the preset order according to the next sorting result to construct the next parent population.

[0104] In the embodiments of the present application, the at least two indicator information of the predicted evolution chromosome can be implemented in the following manner. The indicators of each evolution chromosome in the target population are predicted based on the indicator prediction model to obtain at least two indicator information of each evolution chromosome.

[0105] In the embodiments of the present application, the selection of the evolution direction can be performed by random sampling, target-by-target loop traversal, etc. That is, for the target evolution direction in each round of evolution process, a target evolution direction can be randomly selected as the current target evolution direction, or the target evolution direction can be selected in turn. The evolution chromosomes in the target population are subjected to indicator prediction, sorting and selection to construct the next parent population. Figure 4 The principles of S1021-S1023 are similar to the above description of the indicator prediction, sorting and selection of the initial chromosomes in the initial population to construct the parent population, and the implementation method and the achieved technical effects can be referred to the above description.

[0106] In the embodiments of the present application, each evolution chromosome corresponds to a subnetwork, and the hardware requirements of the subnetwork can be tested by hardware testing. The subnetwork works on the hardware device to test its hardware requirements (such as delay, power consumption, calculation amount, storage space, etc.). The model performance of the subnetwork is tested by a test sample set. The test sample set is input into the subnetwork, and the test accuracy is determined by comparing the label value of the test sample set with the prediction result output by the subnetwork. Each subnetwork can also be input into a target predictor for prediction to output the hardware requirement parameters and model performance parameters of the subnetwork.

[0107] In the embodiments of the present application, when the next target evolution direction is any one index, the evolution chromosomes in the target population are sorted in descending order (or in ascending order, or in high-to-low order, or in low-to-high order, or in optimal-to-poor order, or in poor-to-optimal order) according to the index information of the evolution chromosomes in the index, and a sorting result is determined. When the next target evolution direction is at least two indexes with correlation or conflict, the correlation values or conflict values of each evolution chromosome in at least two indexes with correlation or conflict are calculated, the evolution chromosomes in the target population are sorted according to the sizes of the correlation values or conflict values, and a sorting result is determined. The k (corresponding to the first preset number) chromosomes with better performance in the sorting result are selected to construct the next parent population. The chromosomes in the next parent population are locally optimal, and the number of chromosomes in the next parent population is less than the number of chromosomes in the target population.

[0108] In some embodiments, the next parent population includes a first preset number of next chromosomes; the above Figure 3 The step of determining the next target population in S103 can also be implemented in the following manner. Based on the first preset number of next chromosomes, mutation is performed according to a next mutation proportion coefficient to determine a third number of mutated chromosomes; the third number is the product of the next mutation proportion coefficient and the second preset number; based on the first preset number of next chromosomes, crossover is performed according to a next crossover proportion coefficient to determine a fourth number of crossed chromosomes; the next crossover proportion coefficient is the difference between 1 and the next mutation proportion coefficient; the next target population is determined according to the third number of mutated chromosomes and the fourth number of crossed chromosomes; and the next target population includes a second preset number of next evolution chromosomes.

[0109] In the embodiments of the present application, the next chromosomes in the next parent population are crossed and mutated to generate the next target population, and the principle is similar to that of S1024-S1026 in the above Figure 5 The implementation method and the technical effects achieved by the above process of crossing and mutating the chromosomes in the parent population to generate the target population can be referred to the description of the above process. One chromosome is randomly sampled from the first preset number of next chromosomes, and is mutated once to obtain a mutated chromosome, and the process is repeated P times (the third number is P) to obtain the third number of mutated chromosomes. The third number can be consistent with the first number, or can be inconsistent. When the crossover proportion coefficient is the same, the third number is the same as the first number.

[0110] For example, taking the first preset number k, the second preset number N, the next mutation proportion coefficient P / N, and the next crossover proportion coefficient (N-P) / N as examples, the third number is P / N*N=P, and the fourth number is (N-P) / N*N=N-P. In the current round of evolution, the k chromosomes are randomly sampled according to the current round proportion coefficient, one chromosome is sampled each time, one mutation is performed, and the process is repeated until P mutations are performed, obtaining P mutated chromosomes (corresponding to the third number of mutated chromosomes). The k chromosomes are randomly sampled, one chromosome is sampled each time, one crossover is performed, and the process is repeated until (N-P) / N crossovers are performed, obtaining (N-P) crossover chromosomes (corresponding to the fourth number of crossover chromosomes).

[0111] In the embodiments of the present application, the third number of mutated chromosomes is added to the next target population, and the fourth number of crossover chromosomes is added to the next target population. After the mutation and crossover processes, the next target population is generated, and the next target population still includes the second preset number of evolution chromosomes, ensuring that the number of chromosomes in the population is consistent (for example, N), and realizing multi-objective evolution. The next target population can also be referred to as the next elite population, and the next evolution chromosome can also be referred to as the next elite chromosome.

[0112] In some embodiments, the search method further includes a training process of the super network, generating an initial super network based on preset parameters of the architecture search space; the initial super network includes at least two initial sub-networks; training the at least two initial sub-networks to determine the super network.

[0113] In the embodiments of the present application, the skilled person also needs to define the architecture search space of the super network, that is, set the preset parameters of the architecture search space of the super network, and then generate the initial super network according to the preset parameters. The defined architecture search space includes but is not limited to: resolution search space, convolution kernel search space per layer, width (convolution kernel channel number) search space, and depth (layer stacking number) search space. According to the preset parameters, a plurality of sub-networks with different levels (which can also be understood as layer stacking numbers), different convolution kernels, different channel numbers, and different image resolutions are generated, thereby obtaining the initial super network. The initial super network includes at least two initial sub-networks; and the initial sub-networks are trained to obtain the trained sub-networks.

[0114] In the embodiments of the present application, the super network is trained by using a sandwich rule and a progressive shrinkage. The training method of the sandwich rule is as follows: in each training stage, 1 largest sub-network and 1 smallest sub-network are randomly sampled, in each training iteration, the largest sub-network with the largest channel width, the smallest sub-network with the smallest channel width and two randomly sampled sub-networks are sampled and the accumulated gradient is obtained. The training method of the progressive shrinkage is as follows: a network with the largest kernel size, depth and width is first trained, the network is fine-tuned to support the sub-network, that is, a small sub-network is added to the sampling space, and a knowledge distillation method is used to guide the learning of the sub-network by the largest super network. The method for training the super network is not limited in the embodiments of the present application.

[0115] In some embodiments, the search method further includes an application process of the searched target sub-network, and the target sub-network is used to perform any one of the following processing on the to-be-processed data: generation, classification, clustering, prediction and identification, to obtain a data processing result.

[0116] In the embodiments of the present application, the super network is generated and trained according to the actual needs of the user, and therefore, after the target sub-network is searched by using the hardware-aware multi-objective neural network architecture search method based on the greedy strategy provided in the present application, the target sub-network can be used to perform data processing on the to-be-processed data in the application stage, and the data processing includes but is not limited to: generation, classification, clustering, prediction and identification. For example, if the demand of the user when constructing the super network is data classification, the target sub-network can be used to classify the to-be-processed data to obtain a classification result. Since the target sub-network not only meets the hardware requirement but also has better model performance, the data processing efficiency is improved and the accuracy of the data processing result is improved when the data processing is performed by using the target sub-network.

[0117] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0118] In the embodiments of the present application, as shown in Figure 8 , Figure 8 an optional step flowchart of the multi-objective evolutionary algorithm based on the greedy strategy provided in the embodiments of the present application includes S301-S309.

[0119] S301, an initial population is generated.

[0120] In the embodiments of the present application, several chromosomes (i.e., architecture configurations of subnetworks) are constantly generated by randomly sampling in the architecture search space and the target values of each target of the chromosomes are calculated, and the chromosomes meeting the fixed constraints are added to the initial population until the initial population size is met. If there is a fixed constraint under a certain target, the chromosome meeting the constraint is selected.

[0121] S302, input the subnetworks included in the population into the target indicator predictor to obtain each target indicator.

[0122] S303, determine whether the number of cycles reaches the iteration number E.

[0123] If yes, S309 is executed, and if no, S304 is executed.

[0124] S304, select the current optimization target t in the evolution direction based on the greedy strategy.

[0125] In the embodiments of the present application, the current target evolution direction is the t indicator, and the evolution direction selection based on the greedy strategy is as follows: the evolution direction is sampled, and for different chromosomes, the selected evolution direction, i.e., the selected target value, is used as the fitness score of the chromosome. The chromosome with the highest fitness score is saved and added to the elite population (corresponding to the target population).

[0126] The evolution direction selection based on the greedy strategy is as follows: in the multi-objective evolution algorithm based on the greedy strategy, the evolution direction needs to be selected in each round. The greedy strategy refers to selecting a single target as the optimization direction in each round of iteration, and realizing the optimization effect of multiple targets by constantly searching for local optimum under a single target. Assuming that there are multiple targets f1, f2, f3…… in hardware-aware architecture search, x represents any model in the population, and f i (x) represents the value of x on target i. In each round of iteration, a single target is selected as the optimization direction of this round, for example, f t is the evolution direction in the tthround, and the values of all candidate models in f t are used as the fitness for sorting. The evolution direction can be selected by random sampling, target-by-target loop traversal, etc.

[0127] S305, establish a new population.

[0128] The established new population is an empty set, which is used to store the chromosomes after crossover and the chromosomes after mutation.

[0129] S306, sort the population according to the t indicator, and take the top K as the parent population.

[0130] The parent population can also be referred to as a parent set.

[0131] S307, randomly sampling in the parent population and performing M times of mutation, and adding to the new population.

[0132] S308, randomly sampling in the parent population, performing N-M times of crossover, and adding to the new population.

[0133] In the embodiment of the application, after S308 is performed, S302 is performed, and S302-S308 are circularly performed until the number of iterations reaches the iteration number E, and S309 is performed.

[0134] According to the cross-mutation proportion adaptive adjustment strategy (corresponding to Figure 6 and Figure 7 ) proposed in the scheme, the elite is selected to perform mutation and crossover to obtain a new population. The selection-mutation-crossover process is repeated until the set iteration round. If there is a fixed constraint under a certain target, both the mutation process and the crossover process select the chromosome satisfying the constraint.

[0135] S309, returning the best candidate network in the population.

[0136] In the embodiment of the application, the optimal chromosome (the best candidate model architecture) and the corresponding weight parameter are returned.

[0137] In summary, the technical scheme of the scheme is as follows: the evolution direction selection based on the greedy strategy is combined with the cross-mutation proportion adaptive adjustment strategy, so as to obtain the multi-objective evolutionary algorithm based on the greedy strategy, and then the hardware-aware multi-objective neural network architecture search method based on the greedy strategy is obtained.

[0138] The scheme proposes a hardware-aware multi-objective neural network architecture search method based on the greedy strategy, which can search for a better performance model while considering more hardware target constraints for optimization. The scheme improves the evolutionary algorithm in OFA and proposes a multi-objective architecture search method based on the greedy strategy, which can realize the collaborative optimization of model accuracy, delay, power consumption and other multi-objectives while meeting the search efficiency, so as to achieve the best user experience in the actual scene.

[0139] In the embodiment of the application, if the hardware-aware multi-objective neural network architecture search method based on the greedy strategy is searched, the user needs to set the judgment logic of the target value; or the product manual and / or product document (PR) indicates that the "greedy strategy architecture search", "greedy multi-object NAS" and "hill climbing multi-object architecture search" information are used, which indicates that the search method provided in the embodiment of the application is applied.

[0140] To implement the search method of the embodiments of the present application, the embodiments of the present application further provide a search device. Figure 9 As shown in the figure, Figure 9 An optional structure diagram of a search device provided by the embodiments of the present application is shown in the figure. The search device 90 includes a population determination module 901 configured to select chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network includes at least two sub-networks, and each chromosome corresponds to a sub-network; determine a parent population based on the initial population and an initial target evolution direction; an evolution module 902 configured to mutate and cross chromosomes included in the parent population to determine a target population; the population determination module 901 is further configured to continue to determine a next parent population based on the target population and a next target evolution direction; the evolution module 902 is further configured to mutate and cross next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, multi-objective evolution is achieved, and a last target population is obtained; a target sub-network determination module 903 configured to determine a target chromosome from the last target population to achieve multi-objective architecture search and obtain a target sub-network corresponding to the target chromosome.

[0141] In some embodiments, the population determination module 901 is further configured to predict each index of each initial chromosome in the initial population to obtain at least two index information of the each initial chromosome; the indexes include model performance and hardware requirements of the sub-network; sort the initial chromosomes included in the initial population according to index information corresponding to the initial target evolution direction in the at least two index information of the each initial chromosome to determine a sorting result; the initial target evolution direction is any one index, or at least two indexes having correlation or conflict; and select a first preset number of chromosomes in a preset order according to the sorting result to construct the parent population.

[0142] In some embodiments, the parent population includes a first preset number of chromosomes;

[0143] The evolution module 902 is further configured to mutate the first preset number of chromosomes according to an initial mutation proportion coefficient to determine a first number of mutated chromosomes; the first number is a product of the initial mutation proportion coefficient and a second preset number; cross the first preset number of chromosomes according to an initial cross proportion coefficient to determine a second number of crossed chromosomes; the initial cross proportion coefficient is a difference between 1 and the initial mutation proportion coefficient; and determine the target population according to the first number of mutated chromosomes and the second number of crossed chromosomes; the target population includes the second preset number of evolved chromosomes.

[0144] In some embodiments, the search device 90 further comprises an adjusting module 904;

[0145] The adjusting module 904 is configured to determine a mean value and a variance according to the index information of each current chromosome in the current parent population in the current target evolution direction; adjust an initial mutation proportion coefficient to a first coefficient in a case where the mean value is less than a preset mean value and the variance is less than a preset variance; the first coefficient is greater than the initial mutation proportion coefficient; adjust an initial crossover proportion coefficient to a second coefficient in a case where the mean value is greater than the preset mean value and the variance is less than the preset variance; the second coefficient is greater than the initial crossover proportion coefficient; retain the initial crossover proportion coefficient and the initial mutation proportion coefficient in a case where the mean value is less than the preset mean value and the variance is greater than the preset variance; and retain the initial crossover proportion coefficient and the initial mutation proportion coefficient in a case where the mean value is greater than the preset mean value and the variance is greater than the preset variance.

[0146] In some embodiments, the adjusting module 904 is further configured to determine the preset mean value and the preset variance according to the index information of each previous chromosome in the previous parent population in the current target evolution direction.

[0147] In some embodiments, the adjusting module 904 is further configured to determine the preset mean value according to the index information of the maximum network or the minimum network in the current target evolution direction in the super network.

[0148] In some embodiments, the population determining module 901 is further configured to continue to predict each index of each evolution chromosome in the target population to obtain at least two index information of the each evolution chromosome; sort the evolution chromosomes included in the target population according to the index information corresponding to a next target evolution direction in the at least two index information of the each evolution chromosome to determine a next sorting result; the next target evolution direction is any one index, or at least two indexes with correlation or conflict; and select a first preset number of next chromosomes according to the next sorting result in a preset order to construct the next parent population.

[0149] In some embodiments, the next parent population comprises the first preset number of next chromosomes.

[0150] The evolution module 902 is further configured to determine a third number of mutated chromosomes by mutating the first preset number of next chromosomes according to a next mutation proportion coefficient; the third number is a product of the second preset number and the next mutation proportion coefficient; determine a fourth number of crossed chromosomes by crossing the first preset number of next chromosomes according to a next crossing proportion coefficient; the next crossing proportion coefficient is a difference between 1 and the next mutation proportion coefficient; and determine the next target population according to the third number of mutated chromosomes and the fourth number of crossed chromosomes; the next target population includes the second preset number of next evolved chromosomes.

[0151] In some embodiments, the population determination module 901 is further configured to predict each index of each initial chromosome in the initial population based on an index prediction model to obtain at least two index information of the each initial chromosome; the index prediction model is used to predict each index of a model architecture of a sub-network, and the index prediction model includes a performance index prediction model and a hardware index prediction model.

[0152] In some embodiments, the search device 90 further includes a training module 905.

[0153] The training module 905 is configured to generate an initial super network based on preset parameters of the architecture search space; the initial super network includes at least two initial sub-networks; and train the at least two initial sub-networks to determine the super network.

[0154] In some embodiments, the target sub-network determination module 903 is further configured to predict model performance of each candidate chromosome in the last target population to obtain model performance of the each candidate chromosome; and determine the target chromosome according to the model performance of the each candidate chromosome.

[0155] In some embodiments, the search device 90 further includes a processing module 906.

[0156] The processing module 906 is configured to use the target sub-network to perform any one of the following processing on to-be-processed data: generation, classification, clustering, prediction, and identification, to obtain a data processing result.

[0157] It should be noted that the search device provided in the above embodiment is only used as an example to illustrate the division of the above program modules, and in actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the search device and the search method provided in the above embodiment belong to the same concept, and the specific implementation process and beneficial effects are described in the method embodiment, which will not be repeated here. For technical details not disclosed in the present device embodiment, please refer to the description of the method embodiment of the present application for understanding.

[0158] In the embodiments of the present application, Figure 10 The search device structure provided in the embodiments of the present application is shown in the structure diagram as Figure 10 The search device 100 provided in the embodiments of the present application includes a processor 1001 and a memory 1002 storing executable computer programs. The processor 1001 is used to execute the executable computer programs stored in the memory 1002 to realize the search method provided in the embodiments of the present application.

[0159] In some embodiments, the search device 100 can further include a communication interface 1003, and a bus 1004 for connecting the processor 1001, the memory 1002 and the communication interface 1003.

[0160] In the embodiments of the present application, the above processor 1001 can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic devices used to realize the functions of the above processor can also be other devices, and the embodiments of the present application do not make specific limitations.

[0161] In the embodiments of the present application, the bus 1004 is used to connect the communication interface 1003, the processor 1001 and the memory 1002 to realize the mutual communication between these devices.

[0162] The memory 1002 is configured to store executable computer programs and data, and the executable computer programs include computer operation instructions. The memory 1002 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least two disk memories. In actual applications, the memory 1002 can be a volatile memory such as a random access memory (RAM), or a non-volatile memory such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state disk (SSD), or a combination of the above types of memories, and provides executable computer programs and data to the processor 1001.

[0163] In addition, each functional module in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional module.

[0164] When the integrated unit is realized in the form of a software functional module and is not sold or used as an independent product, the integrated unit can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiment can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method of the embodiment. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0165] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and is used for implementing the search method according to any one of the above embodiments when executed by a processor.

[0166] Exemplarily, the program instruction corresponding to the search method in the embodiment can be stored on a storage medium such as an optical disc, a hard disc, a U disc, etc. When the program instruction corresponding to the search method in the storage medium is read by an electronic device or executed, the search method as described in any of the above embodiments can be implemented.

[0167] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer-usable program code.

[0168] The present application is described with reference to the implementation flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram and the combination of the flows and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one block or multiple blocks.

[0169] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including instruction devices that implement the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one block or multiple blocks.

[0170] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flowchart and / or block diagram. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The functions specified in one block or multiple blocks.

[0171] The above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application.

Claims

1. A search method characterized by, The method comprises: selecting a chromosome satisfying a fixed constraint in an architecture search space corresponding to a super network to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; the fixed constraint comprises at least one of the following: a model performance parameter, a delay parameter, a power consumption parameter, a calculation amount parameter, and a storage space parameter; predicting each index of each initial chromosome in the initial population to obtain at least two index information of each initial chromosome; the index comprises a model performance and a hardware requirement of a sub-network; the hardware requirement of the sub-network is predicted through hardware testing or through an index prediction model; sorting the initial chromosomes included in the initial population according to index information corresponding to an initial target evolution direction in the at least two index information of each initial chromosome to determine a sorting result; the initial target evolution direction is any one index or at least two indexes having correlation or conflict; selecting a first preset number of chromosomes in a preset order according to the sorting result to construct a parent population, and performing mutation and crossover on the chromosomes included in the parent population to determine a target population; continuing to determine a next parent population based on the target population and a next target evolution direction, performing mutation and crossover on the next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, realizing multi-objective evolution, and obtaining a last target population; determining a target chromosome from the last target population to realize multi-objective architecture search and obtain a target sub-network corresponding to the target chromosome; using the target sub-network to perform any one of the following on the to-be-processed data: generation, classification, clustering, prediction, and identification to obtain a data processing result.

2. The method of claim 1, wherein, The parent population comprises a first preset number of chromosomes; The parent population comprises a first preset number of chromosomes; based on the first preset number of chromosomes, performing mutation according to an initial mutation proportion coefficient to determine a first number of mutated chromosomes; the first number is a product of the initial mutation proportion coefficient and a second preset number; based on the first preset number of chromosomes, performing crossover according to an initial crossover proportion coefficient to determine a second number of crossed chromosomes; the initial crossover proportion coefficient is a difference between 1 and the initial mutation proportion coefficient; determining the target population according to the first number of mutated chromosomes and the second number of crossed chromosomes; the target population comprises the second preset number of evolved chromosomes.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: determining a mean value and a variance according to index information of each current chromosome in a current parent population in a current target evolution direction; in a case where the mean value is less than a preset mean value and the variance is less than a preset variance, adjusting an initial mutation proportion coefficient to a first coefficient; the first coefficient is greater than the initial mutation proportion coefficient. adjust the initial crossover ratio coefficient to a second coefficient in a case where the mean value is greater than the preset mean value and the variance is less than the preset variance; the second coefficient is greater than the initial crossover ratio coefficient; retain the initial crossover ratio coefficient and the initial mutation ratio coefficient in a case where the mean value is less than the preset mean value and the variance is greater than the preset variance; retain the initial crossover ratio coefficient and the initial mutation ratio coefficient in a case where the mean value is greater than the preset mean value and the variance is greater than the preset variance.

4. The method of claim 3, wherein, The method further comprises: determining the preset mean value and the preset variance according to index information of each previous chromosome in a previous parent population in the current target evolution direction.

5. The method of claim 3, wherein, The method further comprises: determining the preset mean value according to index information of a maximum network or a minimum network in the current target evolution direction in the super network.

6. The method of claim 1 or 2, wherein, The method further comprises: continuing to determine a next parent population based on the target population and a next target evolution direction, comprising: continuing to predict each index of each evolution chromosome in the target population to obtain at least two index information of the each evolution chromosome; sorting the evolution chromosomes included in the target population according to index information corresponding to the next target evolution direction in the at least two index information of the each evolution chromosome to determine a next sorting result; the next target evolution direction is any one index, or at least two indexes with correlation or conflict; 7. The method of claim 1 or 2, wherein, selecting a first preset number of next chromosomes in a preset order according to the next sorting result to construct the next parent population. The next parent population includes a first preset number of next chromosomes. The method further comprises: determining a third number of mutation chromosomes by mutation according to a next mutation ratio coefficient based on the first preset number of next chromosomes; the third number is a product of the next mutation ratio coefficient and a second preset number; determining a fourth number of crossover chromosomes by crossover according to a next crossover ratio coefficient based on the first preset number of next chromosomes; the next crossover ratio coefficient is a difference between 1 and the next mutation ratio coefficient; 8. The method of claim 1, wherein, determining the next target population according to the third number of mutation chromosomes and the fourth number of crossover chromosomes; the next target population includes the second preset number of next evolution chromosomes. The method further comprises:

9. The method of claim 1 or 2, wherein, predicting each index of each initial chromosome in the initial population to obtain at least two index information of the each initial chromosome based on the index prediction model; wherein the index prediction model is used to predict each index of a model architecture of a sub network, and the index prediction model includes a performance index prediction model and a hardware index prediction model. The method further comprises: generate an initial super network based on preset parameters of the architecture search space; the initial super network comprises at least two initial sub-networks; train the at least two initial sub-networks to determine the super network.

10. The method of claim 1 or 2, wherein, The target chromosome is determined from the last target population, comprising: predicting the model performance of each candidate chromosome in the last target population to obtain the model performance of each candidate chromosome; determining the target chromosome according to the model performance of each candidate chromosome.

11. A search apparatus characterized by comprising: The device comprises: A population determination module is configured to select chromosomes satisfying fixed constraints in a super network corresponding to an architecture search space to determine an initial population; the super network comprises at least two sub-networks, and each chromosome corresponds to a sub-network; each index of each initial chromosome in the initial population is predicted to obtain at least two index information of each initial chromosome; the index includes the model performance and hardware requirement of the sub-network; the hardware requirement of the sub-network is predicted by hardware testing or by an index prediction model; the initial chromosomes included in the initial population are sorted according to index information corresponding to an initial target evolution direction in at least two index information of each initial chromosome to determine a sorting result; the initial target evolution direction is any one index or at least two indexes having correlation or conflict; a first preset number of chromosomes are selected according to a preset order based on the sorting result to construct a parent population; the fixed constraints include at least one of the following: a model performance parameter, a delay parameter, a power consumption parameter, a calculation amount parameter, and a storage space parameter; An evolution module is configured to mutate and cross the chromosomes included in the parent population to determine a target population; The population determination module is further configured to continue to determine a next parent population based on the target population and a next target evolution direction; The evolution module is further configured to mutate and cross the next chromosomes included in the next parent population to determine a next target population until a number of iterations is reached, realize multi-objective evolution, and obtain a last target population; A target sub-network determination module is configured to determine a target chromosome from the last target population to realize multi-objective architecture search and obtain a target sub-network corresponding to the target chromosome; A processing module is configured to use the target sub-network to perform any one of the following processing on to-be-processed data: generation, classification, clustering, prediction, and identification to obtain a data processing result.

12. A search apparatus, characterized by comprising: The device comprises: A memory is configured to store an executable computer program; A processor is configured to execute the executable computer program stored in the memory to implement the method of any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, A computer program is stored and configured to be executed by a processor to implement the method of any one of claims 1-10.

Images