Multi-target neural network architecture searching method and device based on classification model

Abstract

The invention discloses a multi-target neural network architecture search method and device based on a classification model, and the method comprises the steps: obtaining solution population data based on an external file, determining a progeny solution population according to the solution population data, and enabling the solution population data to comprise the solution population and a target function value corresponding to each solution in the solution population; classifying the offspring solution population by adopting a classification model, and determining a target solution set; evaluating solutions in the target solution set based on a multi-target optimization problem, determining target solution set data, and updating the external archive based on the target solution set data; and obtaining the number of evaluation times, and when the number of evaluation times is equal to a preset number of evaluation times, outputting the non-dominated solution in the external archive as the target neural network architecture. The problem that the calculation time needed in the algorithm optimization process is increased due to the fact that an existing multi-target neural network architecture search method needs a large number of function evaluation times to evaluate the quality of the searched candidate solutions is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a multi-objective neural network architecture search method and device based on a classification model. Background Art

[0002] In recent years, deep neural networks (DNNs) have been widely used in natural language processing, image recognition, image classification and other fields, and have achieved many remarkable results. Traditional neural network architectures are mainly designed manually. However, with the continuous development of deep neural networks, the structure of deep neural networks has become complex, and it is difficult to rely on manual architecture design. For this reason, deep neural network architecture search methods have been proposed to automatically design neural network architectures.

[0003] Multi-objective evolutionary algorithms have become one of the main methods for solving multi-objective optimization problems because they can search for a set of approximate optimal solutions to the problem within a given time. In recent years, they have also been applied to deep neural network architecture search. Multi-objective evolutionary algorithms usually require a large number of function evaluations to evaluate the quality of the searched candidate solutions during the problem optimization process. However, for optimization problems such as deep neural network architecture search where the objective function evaluation is expensive (computationally time-consuming), a large number of function evaluations will increase the computational time required during the algorithm optimization process to a certain extent, which poses a challenge to the use of traditional multi-objective optimization algorithms for deep neural network architecture search.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a multi-objective neural network architecture search method and device based on a classification model are provided, aiming to solve the problem that the existing multi-objective neural network architecture search method requires a large number of function evaluations to evaluate the quality of the searched candidate solutions, thereby increasing the computing time required in the algorithm optimization process.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a multi-objective neural network architecture search method based on a classification model, wherein the method comprises:

[0008] Acquire solution population data based on external archives, and determine a subsequent solution population according to the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population;

[0009] Using a classification model to classify the offspring solution population to determine a target solution set;

[0010] Evaluating solutions in the target solution set based on a multi-objective optimization problem, determining target solution set data, and updating the external archive based on the target solution set data;

[0011] Obtain the number of evaluations, and when the number of evaluations is equal to a preset number of evaluations, output the non-dominated solution in the external archive as the target neural network architecture.

[0012] In one implementation method, obtaining solution population data based on external archives includes:

[0013] Determining the fitness vector corresponding to each solution in the external archive according to each solution in the external archive and the objective function value corresponding to each solution;

[0014] The solution population data is selected from an external archive according to each of the fitness vectors.

[0015] In one implementation method, the method for obtaining population data further includes:

[0016] When the number of evaluations is 0, initializing the solution population in a predefined initialization space;

[0017] Evaluate the solution population based on a multi-objective optimization problem, and determine the objective function value corresponding to each solution in the solution population;

[0018] The solution population data is determined according to the solution population and each of the objective function values, and the external archive is updated according to the solution population data.

[0019] In one implementation method, determining a subsequent solution population according to the solution population data includes:

[0020] A differential evolution operator, a genetic operator and a distribution estimation operator are respectively used to generate N offspring solutions based on the solution population data, and the generated 3N offspring solutions are used as the offspring solution population.

[0021] In one implementation method, the training method of the classification model includes:

[0022] When the external archive is updated, the solution population data in the external archive is used as training data for the classification model;

[0023] The classification model is trained based on the training data.

[0024] In one implementation method, a classification model is used to classify the offspring solution population to determine a target solution set, including:

[0025] Using a classification model to classify the offspring solution population, determine a candidate solution population and a prediction value corresponding to each solution in the candidate solution population;

[0026] The target solution set is selected from the candidate solution population based on the Euclidean distance between the solutions in the candidate solution population and the predicted value corresponding to each solution.

[0027] In one implementation method, the method further includes:

[0028] When the number of evaluations is less than the preset number of evaluations, continue to execute the steps of acquiring the solution population data based on the external archive, determining the target solution set data based on the solution population data, and updating the external archive based on the target solution set data, until the number of evaluations is equal to the preset number of evaluations, and then output the non-dominated solutions in the external archive as the target neural network architecture.

[0029] In a second aspect, an embodiment of the present invention further provides a multi-objective neural network architecture search device based on a classification model, wherein the multi-objective neural network architecture search device based on a classification model comprises:

[0030] A descendant generation module, used for obtaining solution population data based on external archives, and determining a descendant solution population according to the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population;

[0031] A classification module, used to classify the offspring solution population using a classification model to determine a target solution set;

[0032] An evaluation module, used for evaluating solutions in the target solution set based on a multi-objective optimization problem, determining target solution set data, and updating the external archive based on the target solution set data;

[0033] The judgment module is used to obtain the number of evaluations, and when the number of evaluations is equal to the preset number of evaluations, output the non-dominated solution in the external archive as the target neural network architecture.

[0034] In a third aspect, an embodiment of the present invention further provides a terminal comprising a memory and one or more processors; the memory stores one or more programs; the program comprises instructions for executing a multi-objective neural network architecture search method based on a classification model as described above; and the processor is used to execute the program.

[0035] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a plurality of instructions are stored, wherein the instructions are suitable for being loaded and executed by a processor to implement any of the above-mentioned classification model-based multi-objective neural network architecture search methods.

[0036] Beneficial effects of the present invention: The embodiments of the present invention obtain solution population data based on external archives, determine the offspring solution population based on the solution population data, wherein the solution population data includes the solution population and the target function value corresponding to each solution in the solution population; classify the offspring solution population using a classification model to determine the target solution set; evaluate the solutions in the target solution set based on the multi-objective optimization problem, determine the target solution set data, and update the external archive based on the target solution set data; obtain the number of evaluations, and when the number of evaluations is equal to the preset number of evaluations, output the non-dominated solutions in the external archive as the target neural network architecture. This effectively solves the problem that the existing multi-objective neural network architecture search method requires a large number of function evaluations to evaluate the quality of the searched candidate solutions, which increases the computing time required in the algorithm optimization process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of a multi-objective neural network architecture search method based on a classification model provided in an embodiment of the present invention.

[0039] Figure 2 It is a schematic diagram of the implementation steps of the multi-objective neural network architecture search method based on the classification model provided in an embodiment of the present invention.

[0040] Figure 3 It is a schematic diagram of the internal modules of a multi-objective neural network architecture search device based on a classification model provided in an embodiment of the present invention.

[0041] Figure 4 It is a principle block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention discloses a multi-objective neural network architecture search method and device based on a classification model. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0044] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0045] Multi-objective evolutionary algorithms have become one of the main methods for solving multi-objective optimization problems because they can search for a set of approximate optimal solutions to the problem within a given time. In recent years, they have also been applied to deep neural network architecture search. Multi-objective evolutionary algorithms usually require a large number of function evaluations to evaluate the quality of the searched candidate solutions during the problem optimization process. However, for optimization problems such as deep neural network architecture search where the objective function evaluation is expensive (computationally time-consuming), a large number of function evaluations will increase the computational time required during the algorithm optimization process to a certain extent, which poses a challenge to the use of traditional multi-objective optimization algorithms for deep neural network architecture search.

[0046] In view of the above-mentioned defects of the prior art, the present invention provides a multi-objective neural network architecture search method based on a classification model, wherein the method obtains solution population data based on an external archive, determines a subsequent solution population based on the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population; classifies the subsequent solution population using a classification model to determine a target solution set; evaluates the solutions in the target solution set based on a multi-objective optimization problem, determines the target solution set data, and updates the external archive based on the target solution set data; obtains the number of evaluations, and when the number of evaluations is equal to a preset number of evaluations, outputs the non-dominated solutions in the external archive as the target neural network architecture. This method effectively solves the problem that the existing multi-objective neural network architecture search method requires a large number of function evaluations to evaluate the quality of the searched candidate solutions, which increases the computing time required in the algorithm optimization process.

[0047] Exemplary Methods

[0048] like Figure 1 As shown, the method includes:

[0049] Step S100: acquiring solution population data based on an external archive, and determining a subsequent solution population according to the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population.

[0050] The external archive is a set of solution population data. The data in the external archive comes from the data of each solution evaluated by the multi-objective optimization algorithm during the optimization process, including each solution and the objective function value corresponding to each solution. Considering that some historically evaluated solutions may also be potential good solutions, this embodiment obtains several evaluated solutions and the objective function values ​​corresponding to each solution as solution population data based on the external archive, and generates offspring solution population data based on the solution population data.

[0051] In one implementation, the method for obtaining population data further includes:

[0052] When the number of evaluations is 0, initializing the solution population in a predefined initialization space;

[0053] Evaluate the solution population based on a multi-objective optimization problem, and determine the objective function value corresponding to each solution in the solution population;

[0054] The solution population data is determined according to the solution population and each of the objective function values, and the external archive is updated according to the solution population data.

[0055] Specifically, Figure 2As shown, (1) when the multi-objective optimization algorithm has not started iteration, the multi-objective optimization algorithm has not evaluated the solution, and there is no solution population data saved in the external archive. In this embodiment, a solution population is initialized in a predefined initialization space, and each individual in the population represents a solution for the multi-objective neural network architecture search, that is, a neural network. Each network will be encoded as a solution in the population before joining the population.

[0056] (2) For each solution in the solution population, an evaluation is performed based on the multi-objective optimization problem to obtain the objective function value corresponding to the solution. The multi-objective optimization problem is represented as follows:

[0057] Given a data set D = {D train ,D valid ,D test} and hardware device set H = {H1,…,H J},A neural network architecture search problem can be defined as a multi-objective optimization problem as shown in formula (1).

[0058]

[0059] swt * (x)∈argnin L trai n(x;w),x∈Ω.

[0060] Where Ω is the search space, x is the encoded neural network architecture, i.e., the decision variable, w(x) is the weight corresponding to the decoded network structure, and w * (x) is in the training set D train The optimal weight to achieve the minimum loss, L train (x; w) is the loss function, f e The objective function represents the prediction error of the model, The objective function represents the performance related to the model complexity,

[0061]

[0062] The objective function represents the performance related to the hardware device.

[0063] In one implementation, due to the characteristics of discrete data decision variables, multiple optimization objectives, and multi-mode fitness surface in the deep neural network architecture search problem, some multi-objective evolutionary algorithms are no longer applicable. This embodiment uses a decomposition-based multi-objective evolutionary algorithm (MOEA / D) suitable for solving complex optimization problems such as super-multi-objectives as a benchmark optimization algorithm to solve the above multi-objective optimization problem.

[0064] (3) For high-cost multi-objective optimization problems such as deep neural network architecture search, the computational cost of objective function value evaluation is high. Therefore, during the algorithm optimization process, the number of objective function value evaluations allowed is very limited, so the number of solutions actually evaluated during the algorithm optimization process is also very limited. Some historically evaluated solutions may also be potential good solutions. Directly discarding these solutions may affect the optimization performance of the algorithm to a certain extent. In this embodiment, the target solution set data obtained after each evaluation of the multi-objective optimization problem is saved in an external archive, so that the solution population data can be generated based on the external archive for optimization iteration of the multi-objective optimization algorithm or training data can be generated for training of the classification model, etc.

[0065] In one implementation, the obtaining of solution population data based on external archives includes:

[0066] Determining the fitness vector corresponding to each solution in the external archive according to each solution in the external archive and the objective function value corresponding to each solution;

[0067] The solution population data is selected from an external archive according to each of the fitness vectors.

[0068] Specifically, fitness is a criterion for measuring the quality of each solution. In a multi-objective optimization problem, each individual has multiple target values, so fitness is not a single value, but a vector, and each element in the vector corresponds to the performance of a target. This embodiment selects several solutions and the target function values ​​corresponding to each solution from an external archive as solution population data based on each fitness vector.

[0069] In one implementation, solution population data is selected from an external archive using an environment selection strategy in a decomposition-based multi-objective evolutionary algorithm.

[0070] In one implementation, determining a subsequent solution population according to the solution population data includes:

[0071] A differential evolution operator, a genetic operator and a distribution estimation operator are respectively used to generate N offspring solutions based on the solution population data, and the generated 3N offspring solutions are used as the offspring solution population.

[0072] In short, this embodiment adopts a multi-operator strategy to generate offspring solutions. In each generation, differential evolution operators, genetic operators and distribution estimation operators are used simultaneously to generate new solutions. Each operator generates N (population size) solutions, and a total of 3N offspring solutions are generated.

[0073] like Figure 1 As shown, the method further comprises the following steps:

[0074] Step S200: classify the offspring solution population using a classification model to determine a target solution set.

[0075] Since the data decision variables in the deep neural network architecture search are discrete, classifiers such as classification regression trees and random forests that are suitable for processing discrete data are considered as models used in the algorithm. This embodiment uses a classification model to classify the offspring solution population, screens out potential good architectures (i.e., target solution sets) for evaluation and subsequent optimization processes, and directly discards poor architectures. It effectively reduces the number of objective function evaluations required in the process of using a multi-objective optimization algorithm to search for a neural network architecture, and improves the search efficiency of the multi-objective optimization algorithm.

[0076] In one implementation, the training method of the classification model includes:

[0077] When the external archive is updated, the solution population data in the external archive is used as training data for the classification model;

[0078] The classification model is trained based on the training data.

[0079] Specifically, in multi-objective optimization problems, there is no solution that can achieve the best results on multiple objectives at the same time due to conflicts between objectives. Therefore, a set of Pareto solution sets is defined to achieve the best balance on different objectives. At the same time, the dominance relationship is used to represent the relationship between solutions. Let x u and x v are two solutions in F, if f i (x u )≤f i (x v )and f i (x u ) <f i (x v ), then x u Dominate x v , denoted as A solution x u It is called a Pareto optimal solution if and only if The set of all Pareto optimal solutions is called the Pareto solution set in the decision space, and the mapping of the Pareto solution set in the target space is called the Pareto frontier. Based on the above definition, the solutions in the multi-objective optimization problem can be divided into dominant solutions and non-dominated solutions. To this end, this embodiment divides the training data into two categories based on the dominance relationship between the solutions, where the non-dominated solution is defined as the "good" class and the dominant solution is defined as the "bad" class.

[0080] In order to ensure the performance of the trained model, this embodiment uses all evaluated solutions saved in the external archive A as training data, wherein all non-dominated solutions in the external archive A are classified as the "good" class and all dominated solutions are classified as the "bad" class. These two types of solutions are used as two types of training data in the classification model to construct the classification model.

[0081] Since the training data of the classification model is obtained based on the external archive, each time the external archive is updated, the classification model is trained with the data in the external archive to improve the accuracy of the classification model.

[0082] In one implementation, a classification model is used to classify the offspring solution population to determine a target solution set, including:

[0083] Using a classification model to classify the offspring solution population, determine a candidate solution population and a prediction value corresponding to each solution in the candidate solution population;

[0084] The target solution set is selected from the candidate solution population based on the Euclidean distance between the solutions in the candidate solution population and the predicted value corresponding to each solution.

[0085] Specifically, the classification model is used to classify the offspring solution population, and the solutions predicted by the classification model as good are retained as candidate solution populations, and the solutions predicted by the model as bad are directly discarded. In addition, considering that the dominant relationship between solutions mainly represents the convergence of the solutions obtained in the optimization process of the multi-objective optimization algorithm, in multi-objective optimization, in addition to convergence, the diversity of solutions also needs to be considered. In order to ensure the diversity of solutions, after selecting good solutions using the classification model, this embodiment uses a distance-based selection method to select solutions with good diversity from the candidate solution population. The specific process is: first, according to the accuracy of the predicted value given by the classification model, the solution x1 with the highest accuracy is selected; then, the Euclidean distance from the remaining good solutions to x1 is calculated, and the solution x2 with the farthest distance from x1 is selected; again, the Euclidean distance from the remaining solutions to x2 except x1 is calculated, and the solution with the farthest distance from x2 is selected; and so on, until a preset number of solutions are selected as the target solution set. In this embodiment, the preset number is 5.

[0086] In one implementation, the classification model can be a single-classification model or a multi-classification model, and the solutions can have other types of dominance relationships, such as θ dominance relationship, α dominance relationship, etc. The classification model can also be constructed on each sub-problem after the problem is decomposed by an evolutionary multi-objective algorithm based on decomposition, and learn the advantages and disadvantages relationship between the solutions on each sub-problem.

[0087] like Figure 1 As shown, the method further comprises the following steps:

[0088] Step S300: Evaluate the solutions in the target solution set based on the multi-objective optimization problem, determine the target solution set data, and update the external archive based on the target solution set data.

[0089] In simple terms, the evaluation of each solution in the target solution set is transformed into a multi-objective optimization problem, and the multi-objective evolutionary algorithm based on decomposition is used to solve the multi-objective optimization problem to obtain the objective function value corresponding to each solution. The target solution set data is determined according to the target solution set and the objective function value corresponding to each solution in the target solution set. At the same time, the target solution set data obtained after the evaluation is saved to an external archive, and the user subsequently generates new solution population data and trains the classification model, thereby improving the accuracy of the classification model.

[0090] like Figure 1 As shown, the method further comprises the following steps:

[0091] Step S400, obtaining the number of evaluations, and when the number of evaluations is equal to the preset number of evaluations, outputting the non-dominated solution in the external archive as the target neural network architecture.

[0092] When the number of evaluations corresponding to the multi-objective optimization algorithm reaches the maximum number of evaluations, the non-dominated solutions in the external archive are sorted, and a preset number of sorted non-dominated solutions are output as the searched target neural network architecture.

[0093] In one implementation, the method further includes:

[0094] When the number of evaluations is less than the preset number of evaluations, continue to execute the steps of acquiring the solution population data based on the external archive, determining the target solution set data based on the solution population data, and updating the external archive based on the target solution set data, until the number of evaluations is equal to the preset number of evaluations, and then output the non-dominated solutions in the external archive as the target neural network architecture.

[0095] Simply put, when the number of evaluations does not reach the preset number, the iterative execution continues to obtain solution population data based on external archives, and determine the offspring solution population based on the solution population data; the classification model is used to classify the offspring solution population to determine the target solution set; the solutions in the target solution set are evaluated based on the multi-objective optimization problem, the target solution set data is determined, and the external archive is updated based on the target solution set data until the number of evaluations reaches the preset number of evaluations, and the non-dominated solution in the external archive is output as the target neural network architecture.

[0096] The multi-objective neural network architecture search method based on a classification model proposed in this embodiment has the following advantages: (1) By constructing a classification model by taking the non-dominated solutions and the dominated solutions among the solutions evaluated in the optimization process of the multi-objective optimization algorithm as two categories in the training data, the classification model can learn the dominance relationship between the solutions, effectively reducing the number of models that need to be constructed in the model-based multi-objective neural network architecture search, thereby effectively reducing the inaccuracy caused by the model and the time required for model training; (2) Using the classification model to evaluate the quality of new solutions generated in the optimization process of the evolutionary multi-objective algorithm, screen out potential good solutions for subsequent real evaluation and optimization process, and directly discard the bad solutions, thereby effectively reducing the number of solutions that need to be truly evaluated in the neural network architecture search process; (3) Using a distance-based selection method, a solution with good diversity is selected from the candidate good solutions screened out by the classification model for subsequent real evaluation, effectively ensuring the diversity of the solutions for real evaluation.

[0097] Based on the above embodiments, the present invention also provides a multi-objective neural network architecture search device based on a classification model, such as Figure 3 As shown, the device comprises:

[0098] Offspring generation module 01, used to obtain solution population data based on external archives, and determine offspring solution populations according to the solution population data, wherein the solution population data includes solution populations and objective function values ​​corresponding to each solution in the solution population;

[0099] A classification module 02 is used to classify the offspring solution population using a classification model to determine a target solution set;

[0100] Evaluation module 03, used for evaluating the solutions in the target solution set based on the multi-objective optimization problem, determining the target solution set data, and updating the external archive based on the target solution set data;

[0101] The judgment module 04 is used to obtain the number of evaluations, and when the number of evaluations is equal to the preset number of evaluations, output the non-dominated solution in the external archive as the target neural network architecture.

[0102] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Figure 4As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-objective neural network architecture search method based on a classification model is implemented. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0103] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] In one implementation, the terminal has one or more programs stored in its memory and is configured to be executed by one or more processors. The one or more programs include instructions for performing a multi-objective neural network architecture search method based on a classification model.

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0106] In summary, the present invention discloses a method and device for searching a multi-objective neural network architecture based on a classification model. The method obtains solution population data based on an external archive, determines a subsequent solution population based on the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population; classifies the subsequent solution population using a classification model to determine a target solution set; evaluates the solutions in the target solution set based on a multi-objective optimization problem, determines the target solution set data, and updates the external archive based on the target solution set data; obtains the number of evaluations, and when the number of evaluations is equal to a preset number of evaluations, outputs the non-dominated solutions in the external archive as the target neural network architecture. This method effectively solves the problem that the existing multi-objective neural network architecture search method requires a large number of function evaluations to evaluate the quality of the searched candidate solutions, which increases the computing time required in the algorithm optimization process.

[0107] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-objective neural network architecture search method based on a classification model, characterized in that: The method comprises: Acquire solution population data based on external archives, and determine a subsequent solution population according to the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population; Using a classification model to classify the offspring solution population to determine a target solution set; Evaluating solutions in the target solution set based on a multi-objective optimization problem, determining target solution set data, and updating the external archive based on the target solution set data; Obtain the number of evaluations, and when the number of evaluations is equal to a preset number of evaluations, output the non-dominated solution in the external archive as the target neural network architecture.

2. The multi-objective neural network architecture search method based on classification model according to claim 1 is characterized in that: The obtaining of solution population data based on external archives includes: Determining the fitness vector corresponding to each solution in the external archive according to each solution in the external archive and the objective function value corresponding to each solution; The solution population data is selected from the external archive according to each of the fitness vectors.

3. The multi-objective neural network architecture search method based on classification model according to claim 1 is characterized in that: The method for obtaining population data also includes: When the number of evaluations is 0, initializing the solution population in a predefined initialization space; Evaluate the solution population based on a multi-objective optimization problem, and determine the objective function value corresponding to each solution in the solution population; The solution population data is determined according to the solution population and each of the objective function values, and the external archive is updated according to the solution population data.

4. The multi-objective neural network architecture search method based on classification model according to claim 1 is characterized in that: Determining the offspring solution population according to the solution population data comprises: A differential evolution operator, a genetic operator and a distribution estimation operator are respectively used to generate N offspring solutions based on the solution population data, and the generated 3N offspring solutions are used as the offspring solution population.

5. The multi-objective neural network architecture search method based on classification model according to claim 1 is characterized in that: The training method of the classification model includes: When the external archive is updated, the solution population data in the external archive is used as training data for the classification model; The classification model is trained based on the training data.

6. The multi-objective neural network architecture search method based on classification model according to claim 1, characterized in that: The offspring solution population is classified using a classification model to determine a target solution set, including: Using a classification model to classify the offspring solution population, determine a candidate solution population and a prediction value corresponding to each solution in the candidate solution population; The target solution set is selected from the candidate solution population based on the Euclidean distance between the solutions in the candidate solution population and the predicted value corresponding to each solution.

7. The multi-objective neural network architecture search method based on classification model according to claim 1 is characterized in that: The method further comprises: When the number of evaluations is less than the preset number of evaluations, continue to execute the steps of acquiring the solution population data based on the external archive, determining the target solution set data based on the solution population data, and updating the external archive based on the target solution set data, until the number of evaluations is equal to the preset number of evaluations, and then output the non-dominated solutions in the external archive as the target neural network architecture.

8. A multi-objective neural network architecture search device based on a classification model, characterized in that: The device comprises: A descendant generation module, used for obtaining solution population data based on external archives, and determining a descendant solution population according to the solution population data, wherein the solution population data includes a solution population and an objective function value corresponding to each solution in the solution population; A classification module, used to classify the offspring solution population using a classification model to determine a target solution set; An evaluation module, used for evaluating solutions in the target solution set based on a multi-objective optimization problem, determining target solution set data, and updating the external archive based on the target solution set data; The judgment module is used to obtain the number of evaluations, and when the number of evaluations is equal to the preset number of evaluations, output the non-dominated solution in the external archive as the target neural network architecture.

9. A terminal, characterized in that: The terminal includes a memory and one or more processors; the memory stores one or more programs; the program contains instructions for executing the multi-objective neural network architecture search method based on a classification model as described in any one of claims 1-7; and the processor is used to execute the program.

10. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that: The instructions are suitable for being loaded and executed by a processor to implement the steps of the multi-objective neural network architecture search method based on a classification model as described in any one of claims 1 to 7 above.

Images