Neural network generation method and apparatus based on genetic algorithm, and electronic device

Abstract

The present invention relates to the technical field of neural networks, and disclosed are a neural network generation method and apparatus based on a genetic algorithm, and an electronic device. The method comprises: determining the number of parameters of a model; initializing a population, and randomly generating a group of initial neural network structures; performing fitness evaluation on each initial neural network structure, and evaluating the performance of the initial neural network structures on the basis of task-specific performance metrics; on the basis of the initial neural network structures, selecting some high-performance neural network structures to serve as parents for a next-generation population; and performing a genetic algorithm operation, wherein the genetic algorithm operation is performed on the selected neural network structures to generate new neural network structures. The technical solution of the present application uses the genetic algorithm to dynamically generate the neural network structures, allowing a neural network to adapt to the requirements of different tasks. The present application not only saves the time and effort required for manual design of the neural network structures but also enhances the performance and generalization ability of the neural network.

Description

A method, device and electronic device for generating neural network based on genetic algorithm Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to a neural network generation method, device and electronic equipment based on a genetic algorithm. Background Art

[0002] A genetic algorithm (GA) is an optimization and search algorithm inspired by biological genetics. It is used to find the optimal solution or a near-optimal solution to a problem, particularly in complex search spaces. Genetic algorithms operate by simulating the heredity and natural selection processes of biological evolution. The algorithm creates and evolves a set of individuals (often called a population), each representing a potential solution to the problem. Traditional genetic algorithms can be inefficient.

[0003] Artificial neural networks are widely used in machine learning and artificial intelligence, but their performance is often highly dependent on their structure. Traditional neural network architecture design often requires extensive experience and experimentation, and cannot effectively adapt to the variability of different tasks. Specifically, the training algorithm for an artificial neural network is a method used to adjust the weights and parameters in the network so that the network can correctly learn the mapping relationship between input and output. Among them, backpropagation is one of the most common and widely used neural network training algorithms. It uses gradient descent to minimize the error between the network output and the target. Backpropagation propagates the error from the output layer to the hidden layer and updates the weights and biases based on the error gradient to reduce the error. Overall, the backpropagation algorithm is a common method for training artificial neural networks, but it also has some drawbacks and limitations, such as local optimal solutions, gradient vanishing and gradient exploding, and hyperparameter sensitivity.

[0004] Summary of the Invention

[0005] The present application provides a neural network generation method, device and electronic device based on genetic algorithm to solve the above-mentioned technical problem that the neural network structure design in the prior art cannot effectively cope with the changes in different tasks.

[0006] According to one aspect of the present application, an embodiment provides a method for generating a neural network based on a genetic algorithm, comprising:

[0007] Determining the parameters of the model, wherein the model is the structure of the neural network;

[0008] Initialize the population and randomly generate a set of initial neural network structures;

[0009] Initially calculating fitness, wherein fitness evaluation is performed on each of the initial neural network structures, and the performance of the initial neural network structures is evaluated according to a performance metric for a specific task;

[0010] Selecting a neural network structure, wherein a portion of high-performance neural network structures is selected based on the initial neural network structure as the parent generation of the next generation population;

[0011] Genetic algorithm operation, wherein the genetic algorithm operation is performed on the selected neural network structure to generate a new neural network structure;

[0012] Calculating a new fitness, wherein a new fitness calculation is performed on the new neural network structure.

[0013] In one embodiment, the parameter amount is based on a parameter comprising at least one of a neuron type, a neuron size, and a number of neurons of the model; and / or,

[0014] The neuron type includes at least one of a linear neuron, a convolutional neuron, a gated recurrent unit, and a long short-term memory unit.

[0015] In one embodiment, the initial neural network structure includes at least one of different layers, numbers of neurons, and connection methods.

[0016] In one embodiment, the adaptability evaluation is a process of calculating an objective function, wherein the objective function represents the difference between the output of the neural network and the expected output; and / or,

[0017] The performance metric includes at least one of mean square error, cross entropy loss, log likelihood loss, and KL divergence.

[0018] In one embodiment, the high performance refers to a network with sufficiently good fitness, where the sufficiently good fitness means that the loss value of the objective function is sufficiently small.

[0019] In one embodiment, the genetic algorithm-based neural network generation method further includes: repeated iteration; wherein the steps of selecting the neural network structure to calculating the new fitness are repeatedly performed until a stopping condition is met.

[0020] In one embodiment, the stopping condition includes a maximum number of iterations and / or performance convergence; the performance convergence means that the calculation result of the objective function meets the requirements or no longer changes.

[0021] According to one aspect of the present application, an embodiment provides a neural network generation device based on a genetic algorithm, comprising:

[0022] A determination module, configured to determine the parameters of a model, wherein the model is a structure of a neural network;

[0023] Initialization module, used to initialize the population and randomly generate a set of initial neural network structures;

[0024] A first calculation module is configured to initially calculate fitness, wherein fitness evaluation is performed on each of the initial neural network structures, and the performance of the initial neural network structures is evaluated according to a performance metric for a specific task;

[0025] A selection module is used to select a neural network structure, wherein a portion of high-performance neural network structures are selected based on the initial neural network structure as parents of the next generation population;

[0026] a processing module for performing a genetic algorithm operation, wherein the genetic algorithm operation is performed on the selected neural network structure to generate a new neural network structure; and

[0027] The second calculation module is used to calculate the new fitness, wherein the new fitness calculation is performed on the new neural network structure.

[0028] According to one aspect of the present application, an embodiment provides an electronic device comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions; the one or more computer instructions are executed by the processor to implement the genetic algorithm-based neural network generation method as described in any one of the above.

[0029] According to one aspect of the present application, an embodiment provides a readable storage medium having computer instructions stored thereon; wherein, when the computer instructions are executed by a processor, the neural network generation method based on a genetic algorithm as described above is implemented.

[0030] The innovative highlight of the above-mentioned embodiments of this application lies in the use of a genetic algorithm to dynamically generate a neural network structure, enabling the neural network to adapt to the needs of different tasks. This genetic algorithm-based neural network generation method not only saves the time and effort of manually designing the neural network structure, but also improves the performance and generalization ability of the neural network.

[0031] This method can be widely used in various application areas, including image recognition, natural language processing, autonomous driving, medical diagnosis, etc. It provides engineers and researchers in different fields with an automated and efficient tool to improve the performance of neural networks in their applications.

[0032] In addition, the genetic algorithm-based neural network generation method of the present application can reduce the cost and time of neural network design and optimization while improving performance, which is expected to enhance the competitiveness of related industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a flow chart of a method for generating a neural network based on a genetic algorithm in one embodiment;

[0034] FIG2 is a flowchart of a method for generating a neural network based on a genetic algorithm in one embodiment;

[0035] FIG3 is a schematic structural diagram of a neural network generation device based on a genetic algorithm in one embodiment. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0038] It should be noted that the terms "first", "second" etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged in appropriate circumstances, so that the embodiments of the application described herein. In addition, the terms "comprise" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0039] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element or intervening elements may be present. Moreover, in this application, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.

[0040] Example 1

[0041] Referring to FIG1 , an embodiment provides a method for generating a neural network based on a genetic algorithm, comprising the following steps:

[0042] S1. Determine the parameters of the model, wherein the model is the structure of a neural network.

[0043] S2. Initialize the population and randomly generate a set of initial neural network structures.

[0044] S3. Initially calculate the fitness, wherein each of the initial neural network structures is evaluated for fitness, and the performance of the initial neural network structure is evaluated according to a performance metric for a specific task; in this step, the objective function is used as the fitness, wherein the calculation process of the objective function involves defining a mathematical function, comparing the output of the model with the true target value, and using this function to measure the effect of the model.

[0045] S4. Select a neural network structure, wherein a portion of high-performance neural network structures is selected based on the initial neural network structure as the parent generation of the next generation population.

[0046] S5. Genetic algorithm operation, wherein the genetic algorithm operation is performed on the selected neural network structure to generate a new neural network structure.

[0047] S6. Calculate a new fitness, wherein a new fitness calculation is performed on the new neural network structure.

[0048] S7. Repeat the iteration, wherein the steps from selecting a neural network structure to calculating a new fitness are repeated until a stopping condition is met. The stopping condition includes a maximum number of iterations and / or performance convergence; performance convergence means that the calculation result of the objective function meets the requirements or does not change. Performance convergence includes the fitness meeting the requirements, the fitness not changing, or reaching the time limit.

[0049] In one embodiment, regarding step S1, the parameter amount is based on a parameter including at least one of a neuron type, a neuron size, and a number of neurons of the model, wherein the neuron type includes at least one of a linear neuron, a convolutional neuron, a gated recurrent unit, and a long short-term memory unit.

[0050] In one embodiment, regarding step S2, the initial neural network structure includes at least one of different layers, numbers of neurons, and connection methods.

[0051] In one embodiment, regarding step S3, the adaptability evaluation is a process of calculating an objective function, which represents the difference between the output of the neural network and the expected output. Objective function (Objective Function), generally speaking, is sometimes also called loss function (Loss Function) or cost function (Cost Function), and is a core concept in machine learning and optimization problems. It is a mathematical function used to measure the performance of the model and the difference between the output of the evaluation model and the true target. The smaller the loss value here, the higher the fitness. However, there may be some academic deviations regarding the objective function, and the final implementation level may be the same expression. For example, "objective function" is a very broad name; generally, we first determine an "objective function" and then optimize it. For example, in different tasks, the "objective function" can be: maximizing the posterior probability MAP (such as naive Bayes); maximizing the fitness function (genetic algorithm); maximizing the reward / value function (reinforcement learning); maximizing information gain / reducing child node purity (CART decision tree classifier); minimizing the squared error cost (or loss) function (CART, decision tree regression, linear regression, linear adaptive neuron); maximizing log-similarity or minimizing the information entropy loss (or cost) function; minimizing the hinge loss function (support vector machine SVM).

[0052] In one embodiment, regarding step S3, the performance metric includes at least one of common evaluation indicators such as mean square error, cross entropy loss, log likelihood loss and KL divergence.

[0053] In one embodiment, regarding step S4, the high performance refers to a network with sufficiently good fitness, and the sufficiently good fitness means that the loss value of the objective function is sufficiently small.

[0054] In one embodiment, regarding the operation of the genetic algorithm, its main components include the following steps:

[0055] Initialize the population: Initially, a set of individuals is randomly generated, representing potential solutions; the properties of these individuals are determined by the characteristics of the problem and the parameters to be optimized;

[0056] Evaluate fitness: Calculate the fitness score of each individual, which is a measure of the quality of the solution. The fitness function varies depending on the problem type and goal, and can be the objective function value, cost, utility, etc.

[0057] Selection: Through a certain selection strategy, a part of individuals are selected as "parents", usually individuals with higher fitness are selected;

[0058] Crossover (mating): Perform a crossover operation on selected parent individuals to produce a set of "offspring" individuals; the crossover operation simulates gene recombination in the genetic process;

[0059] Mutation: Mutating some offspring individuals to increase population diversity by introducing randomness; the mutation operation simulates gene mutation;

[0060] Replacement: Based on the fitness score, a group of individuals are selected from the parent and offspring generations to form the next generation population;

[0061] Termination condition: The algorithm will repeatedly perform the selection, crossover, mutation, and replacement steps until the termination condition is met, such as reaching the maximum number of iterations, finding a satisfactory solution, or a certain amount of time has passed.

[0062] This application presents a genetic algorithm-based neural network generation method for automatically designing and optimizing neural network structures to meet the needs of diverse application areas. This innovative method combines the principles of neural network design and genetic algorithms, enabling the neural network structure to adapt to specific task requirements through a continuous process of iteration and optimization, thereby improving performance and applicability.

[0063] Example 2

[0064] 2 , an embodiment provides a method for generating a neural network based on a genetic algorithm, targeting all aspects of neural network design, including parameters, network topology, activation functions, etc. The core process of the genetic algorithm is divided into five steps.

[0065] The first step is to determine the model's parameters. The model is the structure of the neural network; the number of parameters is primarily determined by the model's neuron type (generally linear, convolutional, etc.), neuron size, and number of neurons. Neuron types include linear neurons, convolutional neurons, gated recurrent units, and long short-term memory units.

[0066] Step 2: Initialize the population, that is, randomly generate a set of initial neural network structures; these structures include different layers, number of neurons, connection methods, etc.

[0067] Step 3: Initial fitness calculation. Each initial neural network structure is evaluated for fitness, and its performance is assessed based on task-specific performance metrics. Fitness evaluation involves calculating the objective function, which represents the difference between the neural network's output and the expected output. The objective function is described in detail in Example 1. Performance metrics include mean squared error, cross entropy loss, log-likelihood loss, and KL divergence.

[0068] Step 4: Selection. Select a subset of high-performance neural network structures from Step 3 as the parents of the next generation population. High performance refers to networks with sufficiently good fitness, which means the loss value of the objective function is sufficiently low.

[0069] Step 5: Genetic algorithm operation. Genetic algorithm operation is performed on the selected neural network structure. Genetic algorithm operation includes crossover and mutation to generate a new neural network structure.

[0070] Step 6: Calculate the new fitness. Based on the genetic algorithm operation, the fitness of the new neural network structure is calculated.

[0071] Step 7: Repeat the iteration. Repeat steps 4 to 6 until the stopping condition is met (such as reaching the maximum number of iterations, fitness reaching the requirement, fitness no longer changing, or reaching the time limit, etc.).

[0072] In Figure 2, x is the original input data, and the input of any task can be understood as x.

[0073] The innovative feature of this application's genetic algorithm-based neural network generation method is its use of a genetic algorithm to dynamically generate neural network structures, enabling them to adapt to the needs of different tasks. This method not only saves time and effort in manually designing neural network structures, but also improves the performance and generalization capabilities of neural networks.

[0074] Example 3

[0075] Please refer to FIG3 . Based on the same inventive concept, an embodiment of the present application provides a neural network generation device based on a genetic algorithm, which takes the following form.

[0076] 1. The determination module 10 is used to determine the parameters of the model, wherein the model is the structure of the neural network;

[0077] 2. The initialization module 20 is used to initialize the population and randomly generate a set of initial neural network structures;

[0078] 3. The first calculation module 30 is used to initially calculate fitness, wherein the fitness of each of the initial neural network structures is evaluated and the performance of the initial neural network structure is evaluated according to a performance metric for a specific task;

[0079] 4. The selection module 40 is used to select a neural network structure, wherein a portion of high-performance neural network structures are selected based on the initial neural network structure as the parent generation of the next generation population;

[0080] 5. The processing module 50 is used for genetic algorithm operation, wherein the genetic algorithm operation is performed on the selected neural network structure to generate a new neural network structure;

[0081] 6. The second calculation module 60 is used to calculate the new fitness, wherein the new fitness calculation is performed on the new neural network structure.

[0082] In one embodiment, the neural network generation device based on a genetic algorithm further includes an iteration module 70; the iteration module 70 is configured to repeat the iteration, wherein the steps from selecting a neural network structure to calculating a new fitness are repeatedly executed until a stopping condition is met.

[0083] In one embodiment, the stopping condition includes a maximum number of iterations and / or performance convergence; the performance convergence means that the calculation result of the objective function meets the requirements or no longer changes.

[0084] The above-mentioned neural network generation device based on genetic algorithm is used to implement the neural network generation method based on genetic algorithm in the above-mentioned embodiments. Each module in the device corresponds to each step in the method and will not be described in detail.

[0085] Example 4

[0086] Based on the same inventive concept, an embodiment of the present application provides an electronic device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions; the one or more computer instructions are executed by the processor to implement the genetic algorithm-based neural network generation method described in any one of the above embodiments.

[0087] The one or more computer instructions mentioned above may form a program.

[0088] Example 5

[0089] Based on the same inventive concept, an embodiment of the present application provides a readable storage medium having computer instructions stored thereon; wherein, when the computer instructions are executed by a processor, the genetic algorithm-based neural network generation method described in any one of the above embodiments is implemented.

[0090] The one or more computer instructions mentioned above may form a program.

[0091] The above program can be executed in a processor or stored in a memory (or computer-readable medium). Computer-readable media includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0092] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram, and corresponding different steps can be implemented through different modules.

[0093] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A neural network generation method based on genetic algorithm, characterized in that Comprising: Determine the number of parameters of the model, where the model is the structure of a neural network; Initialize the population and randomly generate a set of initial neural network structures; Initially calculate the fitness, where each of the initial neural network structures is evaluated for adaptability and the performance of the initial neural network structures is evaluated according to the performance metric criteria for a specific task; Select neural network structures, where a portion of the high-performance neural network structures are selected based on the initial neural network structures as the parents of the next generation population; Genetic algorithm operations, where genetic algorithm operations are performed on the selected neural network structures to generate new neural network structures; Calculate the new fitness, where a new fitness calculation is performed on the new neural network structures.

2. The neural network generation method based on genetic algorithm according to claim 1, characterized in that The number of parameters is based on parameters including at least one of the neuron type, neuron size, and number of neurons of the model; and / or, The neuron type includes at least one of a linear neuron, a convolutional neuron, a gated recurrent unit, and a long short-term memory unit.

3. A neural network generation method based on a genetic algorithm according to claim 1, characterized in that, The initial neural network structures include at least one of different layers, number of neurons, and connection methods.

4. A neural network generation method based on a genetic algorithm according to claim 1, characterized in that The adaptability evaluation is a process of calculating an objective function, and the objective function represents the difference between the output of the neural network and the expected output; and / or, The performance metric criteria include at least one of mean squared error, cross-entropy loss, log-likelihood loss, and KL divergence.

5. A neural network generation method based on genetic algorithm according to claim 1, characterized in that The high performance refers to a network with good enough fitness, and good enough fitness means that the loss value of the objective function is small enough.

6. A neural network generation method based on genetic algorithm according to any one of claims 1-5, characterized in that, The neural network generation method based on a genetic algorithm further includes: repeating iterations; where the steps of selecting neural network structures to calculating the new fitness are repeatedly executed until a stop condition is met.

7. A neural network generation method based on a genetic algorithm according to claim 6, characterized in that, The stop condition includes a maximum number of iterations and / or performance convergence; the performance convergence means that the calculation result of the objective function meets the requirements or no longer changes.

8. A neural network generation device based on a genetic algorithm, characterized in that, Comprising: A determination module for determining the number of parameters of the model, where the model is the structure of a neural network; An initialization module for initializing the population and randomly generating a set of initial neural network structures; A first calculation module for initially calculating the fitness, where each of the initial neural network structures is evaluated for adaptability and the performance of the initial neural network structures is evaluated according to the performance metric criteria for a specific task; A selection module for selecting neural network structures, where a portion of the high-performance neural network structures are selected based on the initial neural network structures as the parents of the next generation population; A processing module for genetic algorithm operations, where genetic algorithm operations are performed on the selected neural network structures to generate new neural network structures; and A second calculation module for calculating the new fitness, where a new fitness calculation is performed on the new neural network structures.

9. An electronic device, characterized in that, Comprising: A memory, and A processor; Wherein, the memory is used to store one or more computer instructions; the one or more computer instructions are executed by the processor to implement the neural network generation method based on a genetic algorithm according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores computer instructions; wherein, the computer instructions are executed by a processor to implement the genetic algorithm-based neural network generation method according to any one of claims 1 to 7.

Images