Method and device for optimizing neural network structure, equipment and storage medium
By using genetic algorithms to optimize the neural network structure, the problems of trial-and-errorism, high cost and low efficiency in the optimization of hidden layer structure are solved, and high-quality neural network learning effects and structural parameters are optimized.
Patent Information
- Application Number
- CN202411581492.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2025-04-11
AI Technical Summary
When creating neural network models, the process of determining hidden layer structures is usually trial and error, with high cost and low efficiency, and under non-optimal structural parameters, the learning effect of neural networks is poor.
Genetic algorithms are used to optimize the neural network structure, and by constructing chromosomes to represent the number of hidden layers and the number of neurons, the selection, crossover and mutation operators of the genetic algorithm are used to gradually optimize the neural network structure to determine the optimal parameters.
The high-quality neural network learning effect is achieved. By optimizing the hidden layer structure, the performance and efficiency of the neural network are improved and the cost of the optimization process is reduced.
Smart Images
Figure CN119808875A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of neural network technology, and in particular to a method, device, equipment and storage medium for optimizing a neural network structure. Background Art
[0002] Neural Networks are a powerful class of models in machine learning, inspired by biological nervous systems. They are a core component of deep learning. Neural networks are composed of neurons (similar to biological neurons) that are connected to each other through connection weights to form a network. The goal of neural networks is to simulate complex nonlinear relationships by learning these connection weights in order to perform various tasks, such as face recognition, image recognition, speech recognition, video recognition, chat systems, etc.
[0003] When creating a neural network model to perform a given machine learning task, a key design decision that needs to be made is the network structure configuration. In the case of using a multilayer perceptron (MLP), the number of nodes in the input and output layers is determined by the characteristics of the problem to be processed, so the choice that needs to be made is the structure of the hidden layer, that is, how many layers there are and how many neurons there are in each layer. Currently, the process of determining the parameters of the hidden layer is a trial-and-error process, which is not only costly and inefficient, but also the learning effect of the neural network is not satisfactory under non-optimal structural parameters. Summary of the invention
[0004] The present disclosure provides a method, apparatus, device and storage medium for optimizing a neural network structure to determine a neural network under optimal structural parameters and achieve high-quality learning effects.
[0005] In a first aspect, the present disclosure provides a method for optimizing a neural network structure, comprising:
[0006] Constructing neural network structure;
[0007] Optimizing the neural network structure using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure;
[0008] Among them, the chromosome in the genetic algorithm is a chromosome of fixed length, the length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure, and when the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
[0009] In some embodiments, the optimizing the neural network structure using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure comprises:
[0010] In the process of optimizing the neural network structure using a genetic algorithm, the gene value of the chromosome represents the number of neurons in the corresponding hidden layer. The optimization is stopped when the gene value appears below 0 for the first time, and the number of genes before the gene value below 0 is determined as the number of hidden layers in the neural network structure, and the gene value before the gene value below 0 is determined as the number of neurons in the corresponding hidden layer.
[0011] In some embodiments, the first gene value of the chromosome is greater than 0.
[0012] In some embodiments, in the process of optimizing the neural network structure using a genetic algorithm, a strategy of retaining the elite and a strategy of retaining the chromosome with the worst fitness are used.
[0013] In some embodiments, in the process of optimizing the neural network structure using a genetic algorithm, the classification accuracy of the neural network structure corresponding to each chromosome after training is used as the individual fitness.
[0014] In some embodiments, the selection operator used by the genetic algorithm is one of proportional selection, Boltzmann selection, sorting selection, and league selection, and the crossover operator used by the genetic algorithm is one of single-point crossover and multi-point crossover.
[0015] In a second aspect, the present disclosure provides a device for optimizing a neural network structure, comprising:
[0016] Building blocks for building neural network structures;
[0017] An optimization module, used for optimizing the neural network structure by using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure;
[0018] Among them, the chromosome in the genetic algorithm is a chromosome of fixed length, the length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure, and when the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
[0019] In a third aspect, the present disclosure provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.
[0020] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.
[0021] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.
[0022] The present disclosure provides a method, device, equipment and storage medium for optimizing a neural network structure. The present disclosure provides a method, device, equipment and storage medium for optimizing a neural network structure. The constructed neural network structure is optimized using a genetic algorithm to determine the number of hidden layers in the neural network structure and the number of neurons in each hidden layer. The chromosome in the genetic algorithm is a chromosome of fixed length, and the length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure. When the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value to determine the neural network under the optimal structural parameters to achieve high-quality learning effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:
[0024] Figure 1 A flowchart of a method for optimizing a neural network structure provided in an embodiment of the present disclosure.
[0025] In the drawings, the same reference numerals are used for the same components and the drawings are not drawn to scale. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.
[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0028] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0029] Embodiment 1
[0030] Figure 1 A schematic diagram of a process for optimizing a neural network structure provided in this embodiment. Figure 1 As shown, a method for optimizing a neural network structure comprises:
[0031] Step S11, constructing a neural network structure;
[0032] Step S12: optimizing the neural network structure using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure;
[0033] Among them, the chromosomes in the genetic algorithm are chromosomes of fixed length. The length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure. When the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
[0034] In the method of this embodiment, the number of hidden layers and the number of neurons in each hidden layer in the neural network structure are measured by designing a special structure of chromosomes in the genetic algorithm, so as to obtain the optimal number of hidden layers and neurons as the optimal structural parameters, and obtain a neural network structure that can achieve high-quality learning effects.
[0035] In some embodiments, optimizing the neural network structure using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure includes:
[0036] Step S12a, in the process of optimizing the neural network structure by using the genetic algorithm, the gene value of the chromosome represents the number of neurons in the corresponding hidden layer. The optimization is stopped when the gene value appears below 0 for the first time, and the number of genes before the gene value below 0 is determined as the number of hidden layers in the neural network structure, and the gene value before the gene value below 0 is determined as the number of neurons in the corresponding hidden layer.
[0037] Since the structure of the neural network is mainly determined by the hidden layer configuration, this embodiment constructs a chromosome structure that can simultaneously represent the number of hidden layers and the number of nodes in each hidden layer. If a variable-length chromosome is used, it is necessary to design a special selection operator, crossover operator, and mutation operator separately, which will make the calculation very cumbersome and the versatility of the genetic algorithm is not strong. In order to use standard genetic operators, this embodiment uses a fixed-length representation method to predetermine the maximum number of hidden layers N of the neural network, that is, the upper limit of the number of hidden layers I N, so as to reduce the running time of the genetic algorithm when optimizing the neural network structure; at the same time, the chromosome represents all possible hidden layers, but the meaning represented by each chromosome gene is slightly different.
[0038] For example, if the upper limit of the number of hidden layers is N, then using Represents the chromosome structure of the genetic algorithm (GA), S1, S2, .., S N represents the N genes of the chromosome, is a 1×N dimensional row vector, S l is the gene at position l, then chromosome gene S l ∈{1,2,..,M}, that is, use natural numbers 1, 2,…,M to encode the number of neuron nodes in each hidden layer.
[0039] In order to control the actual number of hidden layers in the neural network and realize a fixed length of chromosomes, standard genetic operators can be used. In this embodiment, the values of some genes are allowed to be 0 or negative. When 0 or a negative number is encountered, it means that no more layers can be added to the neural network structure, which determines the number of hidden layers and the number of nodes in each hidden layer.
[0040] For example:
[0041] When the chromosome is {10,20,-5,15}, it means that the hidden layer of the neural network is 2 layers, the first layer has 10 neurons, and the second layer has 20 neurons;
[0042] When the chromosome is {10,0,-5,15}, it means that the network has 1 hidden layer and the first layer has 10 neurons;
[0043] When the chromosome is {10,20,5,-15}, it means that the network has 3 hidden layers, the first layer has 10 neurons, the second layer has 20 neurons, and the third layer has 5 neurons.
[0044] In some implementations, in order to ensure that there is at least one hidden layer, it is necessary to ensure that the first parameter of the chromosome is always greater than 0. That is, the first gene value of the chromosome is greater than 0.
[0045] In addition, after the chromosomes are acted upon by the selection operator, crossover operator, and mutation operator, they will become non-integer numbers (floating point numbers).
[0046] In many genetic algorithms, selection mechanisms based on fitness, such as the roulette wheel selection mechanism, random competition selection mechanism, and expected value model selection mechanism, are used. The number of times an individual is selected is proportional to its fitness. This selection mechanism may have the following two problems in the genetic algorithm optimization process:
[0047] (1) When there are a few or a few individuals with relatively high fitness in a population, this type of selection mechanism may cause a very small number of chromosomal genes to be rapidly inherited and spread in the population, thus causing local convergence or immature convergence.
[0048] (2) When the fitness of individuals in a group is very close to each other, this type of selection mechanism will tend to random selection, causing the convergence process to fall into a slow state. This situation often occurs in the later stages of genetic algorithm optimization.
[0049] Therefore, in the process of optimizing the neural network structure using the genetic algorithm, this embodiment uses the elite retention strategy and the strategy of retaining the chromosome with the worst fitness. On the one hand, the use of the elite retention strategy can achieve the global convergence of the algorithm to the optimal solution. On the other hand, the use of the chromosome with the worst fitness to achieve the diversity of all chromosomes to avoid premature algorithm maturation. The combination of the two strategies enables the genetic algorithm to more accurately evaluate the quality of chromosomes, so as to further obtain the optimal structural parameters of the neural network - the number of hidden layers and the number of neurons, and thus obtain high-quality learning effects.
[0050] In some embodiments, when optimizing the neural network structure using a genetic algorithm, the classification accuracy of the neural network structure corresponding to each chromosome after training is used as the individual fitness, so as to determine the optimal chromosome and obtain a neural network structure (hidden layer and number of neurons) that can achieve high-quality learning results.
[0051] In some implementations, the execution logic pseudo code of the genetic algorithm is as follows:
[0052]
[0053] In the above execution logic pseudocode, t represents the number of iterations, and the control parameters mainly include mutation rate, crossover rate, population size and number of iterations. Since the elite retention strategy and the strategy of retaining the chromosome with the worst fitness are adopted, when forming the core population, the population size formed is N-2 to retain the individuals with the best and worst fitness and enter the next round of iteration until the set upper limit of the number of iterations is reached.
[0054] In some implementations, the selection operator used by the genetic algorithm is one of proportional selection, Boltzmann selection, sorting selection, and league selection, and the crossover operator used by the genetic algorithm is one of single-point crossover and multi-point crossover.
[0055] Since a chromosome of fixed length N is used, the genetic algorithm in this embodiment solves the problem of optimizing the structure of a neural network with fixed boundaries, which facilitates efficient and accurate solution of various practical application problems, such as face recognition, license plate recognition, cluster analysis, image product classification, and other neural network application scenarios. At the same time, since a chromosome of fixed length is used and a special chromosome structure is designed to simultaneously represent the number of hidden layers and neurons, the genetic algorithm can use conventional standard operators, effectively reducing the running time of the algorithm during neural network optimization, and can quickly and efficiently determine the optimal structural parameters of the neural network.
[0056] Embodiment 2
[0057] Based on the above embodiment, this embodiment provides a device for optimizing a neural network structure, including:
[0058] Building blocks for building neural network structures;
[0059] An optimization module, used for optimizing the neural network structure by using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure;
[0060] Among them, the chromosomes in the genetic algorithm are chromosomes of fixed length. The length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure. When the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
[0061] By designing a special structure of chromosomes in the genetic algorithm to measure the number of hidden layers and the number of neurons in each hidden layer in the neural network structure, it is convenient to obtain the optimal number of hidden layers and neurons as the optimal structural parameters, and obtain a neural network structure that can achieve high-quality learning effects.
[0062] In some embodiments, a neural network structure is optimized using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure, including: in the process of optimizing the neural network structure using a genetic algorithm, the gene value of the chromosome represents the number of neurons in the corresponding hidden layer, the optimization is stopped when the gene value appears below 0 for the first time, the number of genes before the gene value below 0 is determined as the number of hidden layers in the neural network structure, and the gene value before the gene value below 0 is determined as the number of neurons in the corresponding hidden layer.
[0063] Since the structure of the neural network is mainly determined by the hidden layer configuration, this embodiment constructs a chromosome structure that can simultaneously represent the number of hidden layers and the number of nodes in each hidden layer. If a variable-length chromosome is used, it is necessary to design a special selection operator, crossover operator, and mutation operator separately, which will make the calculation very cumbersome and the versatility of the genetic algorithm is not strong. In order to use standard genetic operators, this embodiment uses a fixed-length representation method to predetermine the maximum number of hidden layers N of the neural network, that is, the upper limit of the number of hidden layers I N, so as to reduce the running time of the genetic algorithm when optimizing the neural network structure; at the same time, the chromosome represents all possible hidden layers, but the meaning represented by each chromosome gene is slightly different.
[0064] For example, if the upper limit of the number of hidden layers is N, then using Represents the chromosome structure of the genetic algorithm (GA), S1, S2, .., S N represents the N genes of the chromosome, is a 1×N dimensional row vector, S l is the gene at position l, then chromosome gene S l ∈{1,2,..,M}, that is, use natural numbers 1, 2,…,M to encode the number of neuron nodes in each hidden layer.
[0065] In order to control the actual number of hidden layers in the neural network and realize a fixed length of chromosomes, standard genetic operators can be used. In this embodiment, the values of some genes are allowed to be 0 or negative. When 0 or a negative number is encountered, it means that no more layers can be added to the neural network structure, which determines the number of hidden layers and the number of nodes in each hidden layer.
[0066] For example:
[0067] When the chromosome is {10,20,-5,15}, it means that the hidden layer of the neural network is 2 layers, the first layer has 10 neurons, and the second layer has 20 neurons;
[0068] When the chromosome is {10,0,-5,15}, it means that the network has 1 hidden layer and the first layer has 10 neurons;
[0069] When the chromosome is {10,20,5,-15}, it means that the network has 3 hidden layers, the first layer has 10 neurons, the second layer has 20 neurons, and the third layer has 5 neurons.
[0070] In some implementations, in order to ensure that there is at least one hidden layer, it is necessary to ensure that the first parameter of the chromosome is always greater than 0. That is, the first gene value of the chromosome is greater than 0.
[0071] In addition, after the chromosomes are acted upon by the selection operator, crossover operator, and mutation operator, they will become non-integer numbers (floating point numbers).
[0072] In many genetic algorithms, selection mechanisms based on fitness, such as the roulette wheel selection mechanism, random competition selection mechanism, and expected value model selection mechanism, are used. The number of times an individual is selected is proportional to its fitness. This selection mechanism may have the following two problems in the genetic algorithm optimization process:
[0073] (1) When there are a few or a few individuals with relatively high fitness in a population, this type of selection mechanism may cause a very small number of chromosomal genes to be rapidly inherited and spread in the population, thus causing local convergence or immature convergence.
[0074] (2) When the fitness of individuals in a group is very close to each other, this type of selection mechanism will tend to random selection, causing the convergence process to fall into a slow state. This situation often occurs in the later stages of genetic algorithm optimization.
[0075] Therefore, in the process of optimizing the neural network structure using the genetic algorithm, this embodiment uses the elite retention strategy and the strategy of retaining the chromosome with the worst fitness. On the one hand, the use of the elite retention strategy can achieve the global convergence of the algorithm to the optimal solution. On the other hand, the use of the chromosome with the worst fitness to achieve the diversity of all chromosomes to avoid premature algorithm maturation. The combination of the two strategies enables the genetic algorithm to more accurately evaluate the quality of chromosomes, so as to further obtain the optimal structural parameters of the neural network - the number of hidden layers and the number of neurons, and thus obtain high-quality learning effects.
[0076] In some embodiments, when optimizing the neural network structure using a genetic algorithm, the classification accuracy of the neural network structure corresponding to each chromosome after training is used as the individual fitness, so as to determine the optimal chromosome and obtain a neural network structure (hidden layer and number of neurons) that can achieve high-quality learning results.
[0077] In some implementations, the execution logic pseudo code of the genetic algorithm is as follows:
[0078]
[0079] In the above execution logic pseudocode, t represents the number of iterations, and the control parameters mainly include mutation rate, crossover rate, population size and number of iterations. Since the elite retention strategy and the strategy of retaining the chromosome with the worst fitness are adopted, when forming the core population, the population size formed is N-2 to retain the individuals with the best and worst fitness and enter the next round of iteration until the set upper limit of the number of iterations is reached.
[0080] In some implementations, the selection operator used by the genetic algorithm is one of proportional selection, Boltzmann selection, sorting selection, and league selection, and the crossover operator used by the genetic algorithm is one of single-point crossover and multi-point crossover.
[0081] Since a chromosome of fixed length N is used, the genetic algorithm in this embodiment solves the problem of optimizing the structure of a neural network with fixed boundaries, which facilitates efficient and accurate solution of various practical application problems, such as face recognition, license plate recognition, cluster analysis, image product classification, and other neural network application scenarios. At the same time, since a chromosome of fixed length is used and a special chromosome structure is designed to simultaneously represent the number of hidden layers and neurons, the genetic algorithm can use conventional standard operators, effectively reducing the running time of the algorithm during neural network optimization, and can quickly and efficiently determine the optimal structural parameters of the neural network.
[0082] Embodiment 3
[0083] On the basis of the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the method described in the above embodiments.
[0084] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.
[0085] In some implementations of this embodiment, a computer program product is provided, including a computer program / instructions, and when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.
[0086] The processor may include, but is not limited to, one or more processors or microprocessors, etc. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the methods in the above embodiments.
[0087] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the computer-readable storage medium may include but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).
[0088] The computer-readable storage medium may also store at least one computer executable program / instruction, which may be, for example, a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.
[0089] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.
[0090] The processor may communicate with external devices via an I / O bus via a wired or wireless network.
[0091] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.
[0092] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0093] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0094] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.
Claims
1. A method for optimizing a neural network structure, characterized in that: include: Constructing neural network structure; Optimizing the neural network structure using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure; Among them, the chromosome in the genetic algorithm is a chromosome of fixed length, the length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure, and when the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
2. The method according to claim 1, characterized in that The method of optimizing the neural network structure by using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure includes: In the process of optimizing the neural network structure using a genetic algorithm, the gene value of the chromosome represents the number of neurons in the corresponding hidden layer. The optimization is stopped when the gene value appears below 0 for the first time, and the number of genes before the gene value below 0 is determined as the number of hidden layers in the neural network structure, and the gene value before the gene value below 0 is determined as the number of neurons in the corresponding hidden layer.
3. The method according to claim 2, characterized in that The first gene value of the chromosome is greater than 0.
4. The method according to claim 1, characterized in that: In the process of optimizing the neural network structure by using genetic algorithm, the strategy of retaining the elite and the strategy of retaining the chromosome with the worst fitness are used.
5. The method according to claim 1, characterized in that In the process of optimizing the neural network structure by using a genetic algorithm, the classification accuracy of the neural network structure corresponding to each chromosome after training is used as the individual fitness.
6. The method according to claim 1, characterized in that The selection operator used by the genetic algorithm is one of proportional selection, Boltzmann selection, sorting selection, and league selection; the crossover operator used by the genetic algorithm is one of single-point crossover and multi-point crossover.
7. A device for optimizing a neural network structure, characterized in that: include: Building blocks for building neural network structures; An optimization module, used for optimizing the neural network structure by using a genetic algorithm to determine the number of hidden layers and the number of neurons in each hidden layer in the neural network structure; Among them, the chromosome in the genetic algorithm is a chromosome of fixed length, the length of the chromosome represents the upper limit of the number of hidden layers in the neural network structure, and when the gene value of the chromosome appears below 0 for the first time, the number of hidden layers in the neural network structure and the number of neurons in each hidden layer are determined based on the gene value.
8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.