Hybrid evolution neural network search method and device, equipment and storage medium

Abstract

The invention relates to a hybrid evolutionary neural network search method, which comprises the following steps of: endowing initial weights to candidate operations of a plurality of neural network architectures, and optimizing the initial weights through a gradient descent algorithm to form a preliminary search direction; based on the preliminary search direction, generating a plurality of neural network architecture individuals by using Gaussian distribution sampling, and constructing an initial population; and performing fitness evaluation on the initial population, selecting individuals with better performance, generating new individuals through crossover and mutation operations, and constructing a new population. And the mean vector and the covariance matrix of the population are updated by using the distribution information of the excellent individuals so as to optimize the population distribution direction, and the neural network model with the optimal performance on the verification set is finally selected through iterative optimization search and updating of the neural network architecture. According to the method, the problem that a single gradient descent method is easy to fall into local optimum is effectively avoided, and the efficiency of neural network architecture search and the model performance are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a hybrid evolutionary neural network search method, device, equipment and storage medium. Background Art

[0002] In the modern financial field, data-driven decision-making is becoming more and more common and important. With the development of big data and artificial intelligence technology, financial institutions are not only facing the challenge of massive data, but also need to make decisions quickly in the fierce market competition. Therefore, efficient and accurate model construction has become one of the key factors for financial institutions to improve their business capabilities and competitiveness.

[0003] Traditional financial models, such as rule-based risk control systems or statistical models, usually rely on the experience and rules of human experts. However, with the increasing complexity of data and the rapid changes in the market environment, these traditional models have gradually shown their limitations in dealing with nonlinear, heterogeneous and highly dynamic data. In order to meet these challenges, the financial industry has begun to introduce deep learning models, especially neural network models, on a large scale to improve the ability to process complex data. However, the performance of neural network models depends largely on the design of their architecture, which directly affects the adaptability and generalization ability of the model in different financial scenarios.

[0004] In this context, Neural Architecture Search (NAS) has become a core technology for optimizing the architecture of neural network models. NAS uses an automated search process to find the neural network architecture that best suits specific financial data sets and tasks to maximize the performance of the model. However, the existing NAS methods still have the following problems when applied to financial scenarios:

[0005] Complexity of search space: The diversity and complexity of financial data means that a wider search space is needed to cover possible model architectures. However, an overly large search space will significantly increase the computational complexity and time cost of the search algorithm.

[0006] Local optimality problem: In financial scenarios, using a single gradient descent method for NAS, although it can search efficiently, is prone to premature convergence, causing the search process to fall into a local optimality. This means that the neural network architecture finally selected may only be the local optimal solution in the search space, rather than the global optimal solution, resulting in the model performance not being fully utilized.

[0007] Resource consumption and time cost: If the NAS process falls into local optimality multiple times, financial institutions may need to repeatedly adjust and retrain the model, which not only consumes a lot of computing resources, but also delays the deployment time of the model, increasing manpower and time costs.

[0008] Therefore, in order to fully tap the potential of neural network models in financial business, it is necessary to introduce more advanced NAS methods, expand the breadth and depth of the search space through a variety of optimization strategies, avoid local optimal traps, and improve search efficiency, and ultimately build a high-performance neural network model that can accurately adapt to financial data sets. Summary of the invention

[0009] The main purpose of the present invention is to provide a hybrid evolutionary neural network search method, device, equipment and storage medium, aiming to solve the technical problems of premature convergence and falling into local optimal limitations that are prone to occur when using a single gradient descent method to search for a neural network architecture in the prior art.

[0010] To achieve the above object, the present invention provides a hybrid evolutionary neural network search method, comprising:

[0011] Assigning initial weights to candidate operations of multiple neural network architectures, and optimizing the initial weights using a gradient descent algorithm to form a preliminary search direction;

[0012] Based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling to construct an initial neural network architecture population;

[0013] Perform fitness evaluation on each individual in the initial neural network architecture population, select a preset number of individuals with top fitness values ​​as excellent individuals, generate new neural network architecture individuals based on the excellent individuals through crossover and mutation operations, and construct a new population;

[0014] Using the distribution information of the excellent individuals, the mean vector and the covariance matrix of the new population are updated to optimize the distribution direction of the new population, wherein the distribution direction is used to guide crossover and mutation operations;

[0015] Iteratively perform the operations of generating a new population, evaluating fitness, crossover, mutation, and updating parameters of the new population, iteratively search and optimize the neural network architecture design until the predetermined stopping condition is met, stop the iteration, and select the neural network architecture model with the best performance on the validation set from the finally generated optimized population.

[0016] In one embodiment, the initial weights are optimized using a gradient descent algorithm to form a preliminary search direction, including:

[0017] Using the training set, calculate the initial loss function for candidate operations of multiple neural network architectures;

[0018] Based on the initial loss function, using a gradient descent algorithm to iteratively update the initial weights for multiple times;

[0019] After each iterative update, the validation set is used to calculate the loss value of the neural network architecture on the validation set, and the learning rate is dynamically adjusted based on the loss value of the validation set;

[0020] Add momentum term during gradient descent optimization;

[0021] When updating the gradient, introduce noise or perform mutation operations to expand the search space;

[0022] Based on the loss value and performance evaluation results of the neural network architecture on the training set and the validation set, the optimized preliminary search direction is determined.

[0023] In one embodiment, initial weights are assigned to candidate operations of multiple neural network architectures, including:

[0024] Select the corresponding weight initialization method based on the activation function of each neural network architecture;

[0025] Determine the initial weight distribution parameters of each neural network architecture according to the selected weight initialization method, wherein the initial weight distribution parameters include mean and variance;

[0026] Based on the determined initial weight distribution parameters, generating initial weights for the candidate operations of each neural network architecture, wherein the initial weights are generated from corresponding random distributions according to the distribution parameters determined by the weight initialization method;

[0027] The generated initial weights are applied to each layer of each neural network architecture, wherein the weight matrix of each layer is generated according to the number of input nodes and the number of output nodes, and the bias value is initialized to zero or a set small constant value.

[0028] In one embodiment, based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling to construct an initial neural network architecture population, including:

[0029] Set a population size range based on the number of layers, number of parameters, and computational complexity of the loss function for each neural network architecture;

[0030] According to the preliminary search direction, the mean and variance of the Gaussian distribution are set so that the generated neural network architecture individuals are distributed around the preliminary search direction;

[0031] According to the set population size range, randomly sample and generate multiple neural network architecture individuals from Gaussian distribution;

[0032] During the Gaussian distribution sampling process, random jitter is applied to the initial weights of each individual neural network architecture;

[0033] After the initial neural network architecture population is generated, a fitness evaluation is performed, and the initial neural network architecture population is adjusted according to the evaluation result.

[0034] In one embodiment, the distribution information of the excellent individuals is used to update the mean vector and covariance matrix of the new population to optimize the distribution direction of the new population, including:

[0035] Calculate the mean vector of weights and bias values ​​of all excellent individuals, where the mean vector reflects the central position of the excellent individuals in the parameter space of the new population, where the parameter space includes the set of weights and bias values ​​of all neural network architectures in the new population;

[0036] Based on the difference between the weight and bias value of each excellent individual and the calculated mean vector, the covariance of each excellent individual relative to the mean vector is calculated, and the covariances of all excellent individuals are averaged to update the covariance matrix;

[0037] The mean vector and the updated covariance matrix are used to optimize the distribution direction of the new population and guide the generation of neural network architecture individuals in the new population.

[0038] In one embodiment, performing fitness evaluation on each individual in the initial neural network architecture population and selecting a preset number of individuals with top fitness values ​​as excellent individuals includes:

[0039] Create a fitness value list to store the fitness value of each neural network architecture individual;

[0040] Perform fitness evaluation on each individual in the initial neural network architecture population, obtain a fitness value by calculating the performance of each individual on a preset task, and store the fitness value of each individual in the fitness value list;

[0041] Sorting all neural network architecture individuals according to the values ​​in the fitness value list to identify the best performing individual;

[0042] A preset number of individuals at the top of the fitness value ranking are selected as excellent individuals to construct a new population. The unselected individuals are eliminated and no longer participate in subsequent iterations.

[0043] In one embodiment, iteratively searching and optimizing the neural network architecture design until a predetermined stopping condition is met and the iteration is stopped includes:

[0044] Initialize an iteration counter, record the number of iterations starting from 0, and determine the upper limit of the maximum number of iterations;

[0045] When the accuracy rate on the validation set changes less than a first preset threshold value in a first preset number of consecutive iterations, the iteration is stopped;

[0046] Monitor the change of the loss function, and when the change amplitude of the loss function value is lower than the second preset threshold value in the second preset number of consecutive iterations, confirm that the loss function has converged and stop the iteration;

[0047] When the number of iterations recorded by the iteration counter reaches the upper limit of the maximum number of iterations, the iteration is stopped.

[0048] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a hybrid evolutionary neural network search device, which includes a memory, a processor, and a hybrid evolutionary neural network search program stored in the memory and executable on the processor, and the hybrid evolutionary neural network search program, when executed by the processor, implements the steps of the hybrid evolutionary neural network search method as described above.

[0049] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a hybrid evolutionary neural network search program is stored, and when the hybrid evolutionary neural network search program is executed by a processor, the steps of the hybrid evolutionary neural network search method as described above are implemented.

[0050] Beneficial effects: The present invention relates to a hybrid evolutionary neural network search method, which assigns initial weights to candidate operations of multiple neural network architectures, optimizes the initial weights through a gradient descent algorithm, and forms a preliminary search direction. Based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling, and an initial population is constructed. The initial population is evaluated for fitness, individuals with better performance are selected, new individuals are generated through crossover and mutation operations, and a new population is constructed. The mean vector and covariance matrix of the population are updated using the distribution information of excellent individuals to optimize the distribution direction of the population, and the neural network architecture is searched and updated through iterative optimization, and finally the neural network model with the best performance on the validation set is selected. The present invention effectively avoids the problem that the single gradient descent method is prone to falling into local optimality, and improves the efficiency of neural network architecture search and model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0052] Figure 1 It is a flow chart of an embodiment of a hybrid evolutionary neural network search method of the present invention;

[0053] Figure 2 A schematic diagram of functional modules of a preferred embodiment of a hybrid evolutionary neural network search device of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of the device hardware operating environment involved in the embodiment of the hybrid evolutionary neural network search device of the present invention. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0056] In the modern financial field, data-driven decision-making is becoming more and more common and important. With the development of big data and artificial intelligence technology, financial institutions are not only facing the challenge of massive data, but also need to make decisions quickly in the fierce market competition. Therefore, efficient and accurate model construction has become one of the key factors for financial institutions to improve their business capabilities and competitiveness.

[0057] Traditional financial models, such as rule-based risk control systems or statistical models, usually rely on the experience and rules of human experts. However, with the increasing complexity of data and the rapid changes in the market environment, these traditional models have gradually shown their limitations in dealing with nonlinear, heterogeneous and highly dynamic data. In order to meet these challenges, the financial industry has begun to introduce deep learning models, especially neural network models, on a large scale to improve the ability to process complex data. However, the performance of neural network models depends largely on the design of their architecture, which directly affects the adaptability and generalization ability of the model in different financial scenarios.

[0058] In this context, Neural Architecture Search (NAS) has become a core technology for optimizing the architecture of neural network models. NAS uses an automated search process to find the neural network architecture that best suits specific financial data sets and tasks to maximize the performance of the model. However, the existing NAS methods still have the following problems when applied to financial scenarios:

[0059] Complexity of search space: The diversity and complexity of financial data means that a wider search space is needed to cover possible model architectures. However, an overly large search space will significantly increase the computational complexity and time cost of the search algorithm.

[0060] Local optimality problem: In financial scenarios, using a single gradient descent method for NAS, although it can search efficiently, is prone to premature convergence, causing the search process to fall into a local optimality. This means that the neural network architecture finally selected may only be the local optimal solution in the search space, rather than the global optimal solution, resulting in the model performance not being fully utilized.

[0061] Resource consumption and time cost: If the NAS process falls into local optimality multiple times, financial institutions may need to repeatedly adjust and retrain the model, which not only consumes a lot of computing resources, but also delays the deployment time of the model, increasing manpower and time costs.

[0062] Therefore, in order to fully tap the potential of neural network models in financial business, it is necessary to introduce more advanced NAS methods, expand the breadth and depth of the search space through a variety of optimization strategies, avoid local optimal traps, and improve search efficiency, and ultimately build a high-performance neural network model that can accurately adapt to financial data sets.

[0063] See also Figure 1 , Figure 1 A flow chart of an embodiment of a hybrid evolutionary neural network search method provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0064] like Figure 1 As shown, the hybrid evolutionary neural network search method proposed in the present invention includes the following steps:

[0065] S10, assigning initial weights to candidate operations of multiple neural network architectures, and optimizing the initial weights using a gradient descent algorithm to form a preliminary search direction;

[0066] In this embodiment, in the search process of the neural network architecture, candidate operations (such as convolution operations, pooling operations, attention mechanisms, etc.) need to be quantified and compared. To this end, these operations are first given initial weights so that the contribution of each operation can be effectively evaluated in the subsequent optimization process. The selection of initial weights can be set based on experience or generated by random distribution.

[0067] These initial weights are iteratively optimized using the gradient descent algorithm. The gradient descent algorithm updates the weights by calculating the gradient of the loss function, which gradually reduces the loss function. After each iteration, the weight value is adjusted based on the gradient information to better guide the optimization direction of the neural network architecture.

[0068] After multiple iterations, the gradient descent algorithm adjusts the initial weights to a better state. These optimized weights determine which candidate operations should be prioritized when building the neural network architecture, thus forming a preliminary search direction. This search direction is the basis for subsequent neural network architecture search and optimization.

[0069] In one specific implementation, in a financial fraud detection scenario, candidate operations include convolution operations, fully connected operations, and batch normalization operations. The initial weight assignment is based on the statistical characteristics of historical transaction data, with a particular focus on operations that have shown high sensitivity in past fraud detection. After the initial weight assignment, these weights are optimized using a gradient descent algorithm with a momentum term. During the optimization process, fast convergence is ensured by dynamically adjusting the learning rate, while random noise is introduced to avoid falling into local optimality. The final preliminary search direction will be fine-tuned by verification on a small portion of the transaction data set, so that the search direction shows higher accuracy and robustness in practical applications.

[0070] In another specific embodiment, in the construction of the credit scoring model, candidate operations are expanded to include residual connections, skip connections, and attention mechanisms. The assignment of initial weights not only takes into account the user's credit history data, but also combines the user's behavior patterns, such as credit card usage frequency and repayment record. During the weight optimization process, the L2 regularized gradient descent algorithm is used to prevent the model from overfitting, and is combined with an adaptive adjustment strategy for the dynamic learning rate. After optimization, the preliminary search direction formed will be verified on different credit score data subsets, and the weights will be fine-tuned based on the verification results to ensure the generalization ability of the model in different user groups.

[0071] By assigning initial weights to candidate operations of multiple neural network architectures and optimizing these initial weights using the gradient descent algorithm to form a preliminary search direction, the search efficiency and model performance of the neural network architecture are significantly improved. It can effectively avoid the local optimal problem common in the single gradient descent method and ensure rapid convergence to a better solution in a larger search space.

[0072] S20, based on the preliminary search direction, using Gaussian distribution sampling to generate multiple neural network architecture individuals, and constructing an initial neural network architecture population;

[0073] In this embodiment, the preliminary search direction is determined by the weight optimization result of the previous step, indicating which operations and structures have higher priority in the search space. This direction provides a center point around which sampling is performed to ensure that the generated architecture individuals can effectively explore high potential areas of the search space. Sampling using a Gaussian distribution means that the generated neural network architecture individuals will be randomly distributed around the preliminary search direction. The mean of the Gaussian distribution is set as the weight parameter in the preliminary search direction, and the variance controls the extended range of the sampling. A larger variance allows for wider exploration, while a smaller variance focuses on local optimization.

[0074] Multiple neural network architecture individuals are generated by Gaussian distribution sampling, each with different structures and parameter configurations. The diversity of these individuals is crucial for exploring the entire search space, preventing early convergence and increasing the chance of finding the global optimal solution. These generated individuals constitute the initial population as the basis for the subsequent evolutionary process. Each individual in the population will be gradually optimized in the future fitness evaluation and selection process, gradually approaching the global optimal architecture.

[0075] In one specific implementation, not only a single Gaussian distribution is used for sampling, but multiple Gaussian distributions are applied in parallel for sampling based on multiple preliminary search directions. These directions may come from multiple initial models or different feature weight configurations. By sampling the generated neural network architecture individuals on multiple Gaussian distributions, the search space can be more comprehensively covered to ensure the diversity of model architectures. This method is particularly suitable for situations where financial markets are volatile, and multiple sampling can be used to improve the adaptability of the model to different market conditions.

[0076] In other specific implementations, in the portfolio optimization scenario, an asymmetric Gaussian distribution is introduced for sampling in response to the asymmetric risk distribution in the market. On the basis of the traditional symmetric Gaussian distribution, an offset is added so that the generated neural network architecture individuals are more inclined to cover high-risk or high-return market scenarios. Through this asymmetric distribution sampling, the generated initial population can more accurately capture key changes in the market and optimize the risk-return ratio of the investment strategy.

[0077] By using Gaussian distribution sampling based on the initial search direction to generate multiple neural network architecture individuals and construct an initial neural network architecture population, the search space is effectively expanded to ensure that the model can explore more potential high-performance architectures, thereby improving the adaptability and robustness of the final model. At the same time, parameter adjustment during the sampling process allows flexible adjustment according to the needs of actual application scenarios, significantly improving the performance of the model in complex financial data processing.

[0078] S30, performing fitness evaluation on each individual in the initial neural network architecture population, selecting a preset number of individuals with top fitness values ​​as excellent individuals, generating new neural network architecture individuals based on the excellent individuals through crossover and mutation operations, and constructing a new population;

[0079] In this embodiment, fitness evaluation is to measure the performance of each neural network architecture individual when processing a specific task. Fitness can be measured by a variety of indicators, such as accuracy, error rate, AUC (area under the curve), etc. After evaluation, each individual will be ranked according to its fitness score. Among all the evaluated individuals, a preset number of individuals with top fitness values ​​are selected as "excellent individuals". These excellent individuals represent the best performing architectures in the current population, and they will serve as the basis for generating the next generation of populations.

[0080] Through the crossover operation, part of the structure or parameters of two or more excellent individuals are exchanged to generate a new neural network architecture individual. This operation draws on the crossover mechanism in the genetic algorithm, aiming to retain the excellent characteristics of excellent individuals and combine them to generate new individuals. After the crossover operation, some individuals are mutated. Mutation is a random modification of certain parts of an individual (such as the number of layers, number of nodes, weights, etc.) to introduce new features, increase the diversity of the population, and prevent premature convergence.

[0081] Based on the new individuals generated by these crossovers and mutations, a new population of neural network architectures is constructed. The new population will continue to perform the next round of fitness evaluation, selection, crossover and mutation operations to gradually optimize the model architecture.

[0082] In one specific implementation, in financial market forecasting, fitness assessment uses an adaptive method to dynamically adjust the weights of fitness calculations based on market fluctuations. For example, when the market fluctuates violently, more emphasis is placed on the reaction speed and adaptability of the model, while when the market is stable, more emphasis is placed on the accuracy of the prediction. According to this adaptive evaluation method, the best performing individuals are selected, and the time series processing layers of different individuals are combined through crossover operations to optimize the prediction effect. In the mutation operation, small-scale random adjustments are made to the hierarchy and weight distribution of individuals to ensure that the new population can effectively capture the key features of market changes.

[0083] In other specific embodiments, in anti-fraud detection, fitness evaluation introduces a dynamic adjustment strategy of a genetic algorithm to adjust the fitness calculation method after each iteration. For example, in the initial iteration, fitness focuses more on the generalization ability of the model, while in the later iterations, fitness gradually tends to balance accuracy and false alarm rate. Through this dynamic evaluation strategy, the best individual is selected, and a new individual is generated by combining different detection layers of multiple excellent individuals in a crossover operation. The mutation operation further enhances the detection ability and robustness of the new population by introducing a new feature extraction method or adjusting the activation function type.

[0084] By evaluating the fitness of each individual in the initial neural network architecture population and generating new neural network architecture individuals based on excellent individuals through crossover and mutation operations, the efficiency of population evolution and the effect of model optimization are significantly improved. Not only can the excellent characteristics of excellent individuals be retained and combined, but new architecture characteristics can also be introduced through mutation operations to increase the diversity of the population and prevent premature convergence. The final generated model shows higher accuracy and adaptability in complex financial scenarios.

[0085] S40, using the distribution information of the excellent individuals, updating the mean vector and covariance matrix of the new population to optimize the distribution direction of the new population, wherein the distribution direction is used to guide crossover and mutation operations;

[0086] In this embodiment, after fitness evaluation, the excellent individuals usually represent the neural network architecture with the best performance in the current population. By analyzing the parameters such as weights and bias values ​​of these excellent individuals, their distribution information is extracted. The distribution information usually includes the mean and variance of these parameters, which reflects the distribution characteristics of the architecture with better performance in the search space.

[0087] Calculate the mean vector of the weights and bias values ​​of all excellent individuals. The mean vector can be regarded as the "center point" of these excellent individuals, reflecting the central tendency of the current population in the parameter space. The updated mean vector will serve as the basis for the generation of a new population, making the new generation of individuals closer to the best performing individuals.

[0088] The covariance matrix is ​​generated by calculating the difference between the excellent individual parameter values ​​and the mean vector. The covariance matrix describes the relationship and distribution range between parameters. The updated covariance matrix can not only optimize the distribution of individuals, but also enhance the diversity of the population and avoid individuals in the population being too concentrated in a narrow range.

[0089] By updating the mean vector and covariance matrix, a new distribution direction is determined. This distribution direction guides the subsequent crossover and mutation operations, so that the newly generated individuals can be more reasonably distributed in the search space, improving the search efficiency and individual adaptability.

[0090] In one specific implementation, in the credit scoring model, the distribution information of excellent individuals is not only based on the parameters of the individuals themselves, but also combines the performance of these individuals in different subgroups. For example, when scoring credit for users of different age groups, the mean vectors of excellent individuals are weighted averaged according to the different characteristics of these subgroups. The update of the covariance matrix introduces adaptive weights so that the characteristics of different subgroups are fully reflected in the new population. The optimized distribution direction can better reflect the credit risk of different user groups, which helps to improve the accuracy of the credit scoring model.

[0091] In another specific implementation, in financial market forecasting, the distribution information of excellent individuals not only considers weights and bias values, but also introduces feature space analysis. For example, by analyzing the distribution of excellent individuals in different market characteristics (such as volatility, trading volume, etc.), a multidimensional mean vector is formed. The update of the covariance matrix enables the new population to maintain stable prediction capabilities under different market conditions by performing collaborative analysis on these multidimensional features. The crossover and mutation operations are guided by the distribution direction of this multidimensional feature, and the generated new individuals show higher robustness in market forecasting.

[0092] By using the distribution information of excellent individuals to update the mean vector and covariance matrix, the distribution direction of the new population can be effectively optimized to ensure that the individuals of the newly generated neural network architecture are reasonably distributed in the search space. The diversity and search efficiency of the new population are enhanced, so that the final model can show higher accuracy and robustness in complex financial scenarios, significantly improving the adaptability and stability of the model.

[0093] S50, iteratively executes the operations of generating a new population, evaluating fitness, crossover, mutation, and updating parameters of the new population, iteratively searches and optimizes the neural network architecture design until a predetermined stopping condition is met, stops the iteration, and selects the neural network architecture model that performs best on the validation set from the finally generated optimized population.

[0094] In this embodiment, the generation of the new population is based on the distribution information of excellent individuals, and new individuals are generated through Gaussian distribution sampling and other generation strategies. After each iteration, the new population will replace the old population and enter the next round of optimization process. The new individuals generated in each iteration will undergo fitness evaluation, and the fitness value is usually calculated by the performance on the validation set. Fitness evaluation ensures that those individuals who perform well in the validation set are retained and low fitness individuals are eliminated.

[0095] When generating new individuals, the crossover operation generates new individuals by mixing the features of two or more excellent individuals. The mutation operation randomly modifies certain parts of the new individuals (such as weights, biases, number of layers, etc.) to increase the diversity of the population. In each iteration, parameters such as the mean vector and covariance matrix are updated according to the new distribution of excellent individuals to ensure that the new population is more likely to cover the global optimal area. This process will continue, gradually adjusting and optimizing the neural network architecture until the predetermined stopping conditions are met. These conditions may include that the performance on the validation set is no longer significantly improved, the maximum number of iterations is reached, or a specific error rate requirement is met. When the stopping condition is reached, the iteration is stopped, and the neural network architecture model with the best performance in the validation set is finally selected as the final output to ensure that the model has good generalization ability and robustness in practical applications.

[0096] In one specific implementation, during the iteration of the credit scoring model, the fitness evaluation is divided into two stages: the first stage focuses on the identification of high-risk users, and the second stage focuses on the accuracy of low-risk users. After each stage, the excellent individuals in the two stages are cross-operated to generate a new population. The mutation operation is performed at the end of each stage, introducing fine-tuning for specific credit scoring features. Through this multi-stage evaluation and staged crossover approach, iterative search can gradually optimize the performance of the model in different user groups, and ultimately output a comprehensive credit scoring model.

[0097] By iteratively executing new population generation, fitness evaluation, crossover, mutation, and parameter update operations, the neural network architecture can be gradually optimized to ensure that the global optimal solution is found in a large-scale search space. The final output model not only performs well in the validation set, but also has high generalization ability and stability, and can maintain excellent performance in different financial application scenarios.

[0098] The present invention relates to a hybrid evolutionary neural network search method, which assigns initial weights to candidate operations of multiple neural network architectures, optimizes the initial weights through a gradient descent algorithm, and forms a preliminary search direction. Based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling, and an initial population is constructed. Fitness evaluation is performed on the initial population, individuals with better performance are selected, new individuals are generated through crossover and mutation operations, and a new population is constructed. The mean vector and covariance matrix of the population are updated using the distribution information of excellent individuals to optimize the distribution direction of the population, and the neural network architecture is searched and updated through iterative optimization, and finally the neural network model with the best performance on the verification set is selected. The present invention effectively avoids the problem that a single gradient descent method is prone to falling into local optimality, and improves the efficiency of neural network architecture search and model performance.

[0099] In one embodiment, in the above S10, the initial weight is optimized using a gradient descent algorithm to form a preliminary search direction, including:

[0100] S101, using the training set to calculate the initial loss function of the candidate operations of multiple neural network architectures;

[0101] S102, based on the initial loss function, using a gradient descent algorithm to iteratively update the initial weights for multiple times;

[0102] S103, after each iterative update, using the validation set to calculate the loss value of the neural network architecture on the validation set, and dynamically adjusting the learning rate based on the loss value of the validation set;

[0103] S104, adding a momentum term during the gradient descent optimization process;

[0104] S105, when updating the gradient, introducing noise or performing a mutation operation to expand the search space;

[0105] S106, determining the optimized preliminary search direction based on the loss value and performance evaluation results of the neural network architecture on the training set and the validation set.

[0106] In this embodiment, the candidate operation of the neural network architecture is applied to the training set under the initial weights, and its loss function is calculated. This initial loss function reflects the performance of the current model on the training set, which is usually measured by mean square error, cross entropy or other applicable loss functions.

[0107] Based on the initial loss function, the initial weights are updated multiple times using the gradient descent algorithm. In each iteration, the gradient descent algorithm adjusts the weights according to the gradient of the loss function relative to the weights to gradually reduce the loss value and optimize the neural network architecture.

[0108] After each iterative update, the loss value is calculated through the validation set to evaluate the effect of the current weight configuration in real time. If the loss value on the validation set converges slowly or changes insignificantly, the learning rate is dynamically adjusted, usually reducing the learning rate, to prevent the model from converging to the local optimal solution too early.

[0109] Adding a momentum term during the gradient descent process takes into account the gradient changes of the previous iterations, helping the model move quickly in a narrow loss valley and preventing stagnation at the local optimum.

[0110] When updating the gradient, introduce a small amount of random noise or perform mutation operations. These operations can expand the search space, break the model's convergence trend to the local optimal solution, increase the diversity of the search path, and ensure that the model continues to optimize towards the global optimal solution.

[0111] Through repeated iterations of the above steps, the weight parameters of the neural network architecture are gradually optimized. Finally, based on the loss values ​​and performance evaluation results of the training set and validation set, the preliminary search direction is determined, that is, the most promising path in the search space.

[0112] In one specific implementation, in financial market forecasting, the iterative update is divided into multiple stages during the gradient descent optimization process. Each stage is optimized for specific market conditions (such as high volatility period, low volatility period). In high volatility period, the gradient update speed is faster and the weight of the momentum term is higher to quickly capture market changes; while in low volatility period, the gradient update speed is slower and the weight of the momentum term is lower to prevent the model from being over-adjusted. Through this segmented gradient update method, the optimized preliminary search direction can adapt to the prediction needs under different market conditions.

[0113] This embodiment uses the gradient descent algorithm to optimize the initial weights and form a preliminary search direction, which effectively improves the search efficiency of the neural network architecture and the adaptability of the model. Dynamically adjusting the learning rate, introducing momentum terms and noise terms, and other operations further prevent the model from falling into the dilemma of local optimal solutions, ensuring that the model can maintain high robustness and stability in a complex data environment. The final preliminary search direction lays a solid foundation for the subsequent optimization process.

[0114] In one embodiment, in the above S10, initial weights are assigned to candidate operations of multiple neural network architectures, including:

[0115] S107, selecting a corresponding weight initialization method based on the activation function of each neural network architecture;

[0116] S108, determining initial weight distribution parameters of each neural network architecture according to the selected weight initialization method, wherein the initial weight distribution parameters include a mean and a variance;

[0117] S109, generating initial weights of candidate operations of each neural network architecture based on the determined initial weight distribution parameters, wherein the initial weights are generated from corresponding random distributions according to the distribution parameters determined by the weight initialization method;

[0118] S110, applying the generated initial weights to each layer of each neural network architecture, wherein the weight matrix of each layer is generated according to the number of input nodes and the number of output nodes, and the bias value is initialized to zero or a set small constant value.

[0119] In this embodiment, each neural network architecture may use different activation functions (such as ReLU, sigmoid, tanh, etc.), and different activation functions have different requirements for weight initialization. For example, for the ReLU activation function, the He initialization method is usually used, while for the tanh activation function, the Xavier initialization method is often used. This step ensures that the weight initialization method of each neural network architecture matches its activation function and optimizes the subsequent training effect. According to the selected weight initialization method, the initial weight distribution parameters of each neural network architecture are determined. The initial weight distribution parameters usually include mean and variance, which define the initial distribution characteristics of the weights. For example, in He initialization, the weights are usually sampled from a distribution with zero mean and high variance to adapt to the characteristics of the ReLU activation function. Based on the determined distribution parameters, the initial weights of the candidate operations of each neural network architecture are generated from the corresponding random distribution. These weights are obtained by sampling to ensure that each candidate operation is at a reasonable starting point at the beginning of training, and it is not easy to have the problem of gradient disappearance or fall into gradient explosion. The generated initial weights are applied to each layer of each neural network architecture, and the weight matrix is ​​adjusted according to the number of input nodes and the number of output nodes. At the same time, the bias value is usually initialized to zero or set to a small constant value to ensure that the network output is not affected by the bias in the early stages of training, thereby stabilizing the training process.

[0120] In another specific implementation, in financial market forecasting, in order to adapt to the complexity of the market, a multi-distribution mixed initialization strategy is used. Specifically, during the initialization process of each neural network architecture, not only a single random distribution is selected, but multiple distributions (such as Gaussian distribution and uniform distribution) are mixed according to the market volatility characteristics. This mixed strategy makes the initial weight distribution more diverse, increases the flexibility and robustness of the model in capturing market changes, and thus improves the accuracy of market forecasting.

[0121] This embodiment can significantly improve the stability and adaptability of the neural network model in the early stage of training by assigning initial weights to candidate operations of multiple neural network architectures. The appropriate weight initialization method is selected according to different activation functions to ensure the effectiveness of the model in a complex data environment. The reasonable distribution and application of initial weights not only reduces the risk of gradient vanishing and gradient exploding, but also enhances the robustness of the model on diversified data, ultimately improving the training effect and practical application performance of the model.

[0122] In one embodiment, the above S20 includes:

[0123] S201, setting a population size range based on the number of layers, number of parameters, and computational complexity of the loss function of each neural network architecture;

[0124] S202, according to the preliminary search direction, setting the mean and variance of the Gaussian distribution so that the generated neural network architecture individuals are distributed around the preliminary search direction;

[0125] S203, randomly sampling from a Gaussian distribution to generate a plurality of neural network architecture individuals according to the set population size range;

[0126] S204, during the Gaussian distribution sampling process, applying random jitter to the initial weights of each individual neural network architecture;

[0127] S205, after generating the initial neural network architecture population, perform fitness evaluation, and adjust the initial neural network architecture population according to the evaluation result.

[0128] In this embodiment, the size range of the population is determined according to the number of layers, the number of parameters and the computational complexity of the loss function of each neural network architecture. This step ensures that the generated population has sufficient diversity to cover the search space without wasting computing resources. The choice of population size directly affects the efficiency and effect of the search process. Based on the preliminary search direction, by setting the mean and variance of the Gaussian distribution, the newly generated neural network architecture individuals are controlled to be distributed around the preliminary search direction. The mean determines the center position of the population, while the variance determines the distribution breadth of the population. These parameters are key because they determine the search area and concentration of the new population. Random sampling is performed under the set Gaussian distribution to generate multiple neural network architecture individuals. Through random sampling, it is ensured that the individuals in the new population have a certain degree of randomness, and can explore multiple possibilities near the preliminary search direction, thereby increasing the probability of finding the global optimal solution. When generating individuals, a small random jitter is applied to the initial weight of each neural network architecture individual. This jitter increases the diversity of the population and avoids the newly generated individuals from being too concentrated in a narrow area, thereby improving the comprehensiveness of the search. After the initial neural network architecture population is generated, its fitness is evaluated. The purpose of fitness evaluation is to screen out individuals with excellent performance and eliminate or adjust unsuitable individuals to ensure that the population can evolve in a better direction in subsequent iterations.

[0129] In another specific embodiment, in the anti-fraud detection system, a multi-scale strategy is used when generating the initial neural network architecture population, that is, generating neural network architecture individuals of different scales (such as number of layers, parameter scale) in a population. Smaller-scale architectures are used to quickly detect simple fraudulent behaviors, while larger-scale architectures are used to deeply analyze complex fraud patterns. Through this multi-scale population generation strategy, the system can quickly identify common fraudulent behaviors in the early stages, while retaining complex models for subsequent in-depth analysis, significantly improving the overall detection efficiency of the system.

[0130] This embodiment generates multiple neural network architecture individuals by using Gaussian distribution sampling based on the preliminary search direction, which can effectively cover potential areas in the search space and increase the diversity of the population by applying random jitter. The fitness evaluation and adjustment steps further optimize the quality of the population, ensuring that the global optimal solution can be found more efficiently in subsequent iterations. The resulting neural network architecture can better adapt to complex application scenarios and significantly improve the practical application effect and stability of the model.

[0131] In one embodiment, in the above S40, using the distribution information of the excellent individuals, updating the mean vector and covariance matrix of the new population to optimize the distribution direction of the new population includes:

[0132] S401, calculating a mean vector of weights and bias values ​​of all excellent individuals, wherein the mean vector reflects the central position of the excellent individuals in the parameter space of the new population, wherein the parameter space includes a set of weights and bias values ​​of all neural network architectures in the new population;

[0133] S402, based on the difference between the weight and bias value of each excellent individual and the calculated mean vector, calculate the covariance of each excellent individual relative to the mean vector, average the covariances of all excellent individuals, and update the covariance matrix;

[0134] S403, using the mean vector and the updated covariance matrix, optimizing the distribution direction of the new population, and guiding the generation of neural network architecture individuals in the new population.

[0135] In this embodiment, the weights and bias values ​​of all excellent individuals in the new population are averaged to obtain a mean vector. This mean vector represents the central position of the excellent individuals in the parameter space and reflects the most representative features in the new population. This step is to ensure that the population generated subsequently can focus on a relatively stable area to avoid excessive divergence. By calculating the difference between the weight and bias value of each excellent individual and the mean vector, the covariance of each excellent individual relative to the mean vector is obtained. These covariances are then averaged to obtain an updated covariance matrix. The covariance matrix reflects the distribution range and direction of the new population individuals in the parameter space, ensuring that the new population can effectively cover important areas in the search space. The calculated mean vector and the updated covariance matrix are used to optimize the distribution direction of the new population. This means that when generating new neural network architecture individuals, these individuals will tend to be concentrated in the optimized parameter space, thereby improving search efficiency and effect.

[0136] In one specific implementation, in financial market forecasting, the calculation of the mean vector includes not only market prices and trading volumes, but also market sentiment indicators (such as news sentiment analysis scores). In order to highlight the importance of different indicators, a weighting mechanism is added to the calculation of the mean vector. The weight of each excellent individual is adjusted according to its performance under specific market conditions. Through this weighted calculation, the generated new population can more accurately capture the core trend of market changes in the optimized parameter space.

[0137] In other specific embodiments, in the anti-fraud detection system, the calculation of the mean vector and the covariance matrix not only considers the traditional fraud detection indicators, but also introduces the computational efficiency of the model as one of the optimization goals. Through multi-objective optimization, while maintaining the accuracy of fraud detection, it ensures that the generated new population is also more optimized in the use of computing resources. The distribution direction is thus adjusted, so that the new population can not only accurately detect fraudulent behavior, but also perform well in terms of efficiency.

[0138] This embodiment uses the distribution information of excellent individuals to update the mean vector and covariance matrix of the new population, which can effectively optimize the distribution direction of the new population and improve the search efficiency. The calculated mean vector is concentrated in a representative parameter space area, and the updated covariance matrix ensures the breadth and diversity of the search. Ultimately, these optimization measures make the newly generated neural network architecture individuals more competitive and improve the performance and robustness of the overall model.

[0139] In one embodiment, in the above S30, performing fitness evaluation on each individual in the initial neural network architecture population and selecting a preset number of individuals with top fitness values ​​as excellent individuals includes:

[0140] S301, creating a fitness value list for storing the fitness value of each neural network architecture individual;

[0141] S302, performing fitness evaluation on each individual in the initial neural network architecture population, obtaining a fitness value by calculating the performance of each individual on a preset specified task, and storing the fitness value of each individual in the fitness value list;

[0142] S303, sorting all neural network architecture individuals according to the values ​​in the fitness value list to identify the best performing individual;

[0143] S304, selecting a preset number of individuals at the top of the fitness value ranking as excellent individuals for constructing a new population, and eliminating the unselected individuals so that they will no longer participate in subsequent iterations.

[0144] In this embodiment, a fitness value is assigned to each neural network architecture individual to evaluate the performance of the individual in a specific task. The fitness value can be calculated based on various performance indicators (such as accuracy, precision, recall, etc.), and these values ​​are stored in a fitness value list for subsequent sorting and selection. The fitness evaluation of each individual is performed by calculating its performance on a specified task. This may include multiple aspects such as the performance of the neural network architecture when processing a certain data set, the consumption of computing resources, or the training time of the model. The fitness value of each individual reflects its comprehensive performance in these tasks. According to the fitness value list, all individuals are sorted. The purpose of sorting is to identify the best performing individuals. The selection of excellent individuals is based on the top few ranked after the fitness value is sorted, and these individuals will be used to build new populations in subsequent iterations. Individuals that are not selected are eliminated and no longer participate in subsequent iterations to improve search efficiency.

[0145] In one specific implementation, in credit risk management, fitness evaluation not only considers a single indicator (such as the accuracy of credit scoring), but also introduces multi-dimensional evaluation indicators, including the interpretability, stability and efficiency of computing resources of the model. For example, a neural network architecture not only performs well in accurately predicting the credit risk of users, but also provides clear analysis when explaining credit risk factors. This multi-dimensional fitness evaluation mechanism ensures that the selected excellent individuals perform evenly in multiple important indicators and is suitable for more complex credit risk analysis scenarios.

[0146] In other specific embodiments, in the anti-fraud detection system, fitness evaluation is combined with the idea of ​​ensemble learning, that is, the fitness of each neural network architecture individual is not only based on its individual performance, but also takes into account its performance in multiple ensemble models. For example, an individual may perform poorly in certain fraud detection tasks, but after being combined with other models, the overall performance is significantly improved. This excellent individual selection method based on ensemble learning ensures that the excellent individuals finally selected can provide the greatest contribution to the overall system, rather than just outstanding performance in individual tasks.

[0147] This embodiment can effectively screen out the neural network architecture that performs best in a specific task by evaluating the fitness of each individual in the initial neural network architecture population and selecting a preset number of individuals with top fitness values ​​as excellent individuals. This process ensures the quality of the new population, allowing the model to be gradually optimized after each iteration, improving the overall search efficiency and model performance, and thus being applicable to a variety of complex application scenarios.

[0148] In one embodiment, in the above S50, iteratively searching and optimizing the neural network architecture design until a predetermined stopping condition is met and the iteration is stopped includes:

[0149] S501, initializing an iteration counter, recording the number of iterations starting from 0, and determining the upper limit of the maximum number of iterations;

[0150] S502, when the accuracy rate on the validation set changes less than a first preset threshold value in a first preset number of consecutive iterations, stop the iteration;

[0151] S503, monitoring the change of the loss function, when the change amplitude of the loss function value in the second preset number of consecutive iterations is lower than the second preset threshold, confirming that the loss function has converged, and stopping the iteration;

[0152] S504: When the number of iterations recorded by the iteration counter reaches the upper limit of the maximum number of iterations, stop the iteration.

[0153] In this embodiment, an iteration counter is initialized to record the number of each iteration. The counter starts at 0 and increments after each iteration. The upper limit of the maximum number of iterations is usually set based on resource constraints (such as time or computing power) in actual applications. This ensures that the search process will not be carried out indefinitely, preventing waste of resources.

[0154] After each iteration, the performance of the model on the validation set is monitored. When the accuracy on the validation set changes less than the set threshold in several consecutive iterations, it indicates that the performance of the model has stabilized, and the iteration is stopped. This mechanism prevents overfitting and ensures that the model does not waste computing resources on subtle changes.

[0155] In addition to accuracy, the change in the loss function is also a key metric. If the change in the loss function in several consecutive iterations is lower than the preset threshold, it indicates that the model may have reached the optimal state and further optimization will not bring significant improvement. At this time, the iteration is stopped. The convergence of the loss function is an important criterion for judging whether the model training is complete.

[0156] Even if the accuracy or loss function of the validation set does not meet the predetermined convergence conditions, the iteration will stop when the number of iterations reaches the set upper limit. This restriction ensures that the search process is completed in a controllable time and avoids unlimited computational costs.

[0157] In one specific implementation, in financial market forecasting, not only the change of a single loss function is monitored, but the combined change of multiple loss functions (such as mean square error and root mean square error) is monitored simultaneously. Each loss function has its own convergence threshold, and the weight is adjusted according to the different characteristics of the market. When the combined change amplitude of these loss functions is less than the preset combined threshold in several consecutive iterations, the iteration is stopped. This combined monitoring mechanism can more comprehensively evaluate the convergence of the model and ensure that the final model has stronger predictive ability.

[0158] In another specific implementation, in credit risk assessment, the upper limit of the number of iterations is no longer a fixed value, but is dynamically adjusted according to the performance of the model. For example, in the early iteration stage of the model, the upper limit of the number of iterations is set low to quickly filter out inappropriate model structures; when the model is close to convergence, the upper limit of the number of iterations can be appropriately increased to further optimize the model. Under this dynamic adjustment strategy, the model can obtain the optimal upper limit of the number of iterations at different stages, improving the overall optimization efficiency.

[0159] In other specific embodiments, in the anti-fraud detection system, the early stopping strategy is not only based on a fixed threshold, but also combined with an adaptive threshold adjustment mechanism. This mechanism dynamically adjusts the accuracy threshold on the validation set according to the detection rate of fraudulent behavior. For example, if the fraud detection rate increases rapidly in certain iterations, the threshold will be automatically tightened, forcing the model to be optimized more carefully; if the detection rate does not change much, the threshold will be relaxed, allowing more iterations to explore a better architecture. This adaptive threshold adjustment mechanism can better control the iteration process of the model, prevent overfitting, and ensure the efficiency of the model.

[0160] This embodiment can ensure the efficient search and optimization process of the neural network architecture while effectively controlling resource consumption by setting an iteration counter, monitoring the accuracy of the validation set and the change of the loss function, and setting an upper limit on the maximum number of iterations. This multi-condition combination stop mechanism prevents overfitting and waste of computing resources, ensures that the model stops iterating at an appropriate time point, outputs the optimal neural network architecture model, and improves the effect and efficiency in practical applications.

[0161] The present invention also provides a hybrid evolutionary neural network search device, referring to Figure 2 , Figure 2 The following is a functional module diagram of a preferred embodiment of a hybrid evolutionary neural network search device of the present invention. The hybrid evolutionary neural network search device comprises:

[0162] A weight initialization and optimization module assigns initial weights to candidate operations of multiple neural network architectures, and uses a gradient descent algorithm to optimize the initial weights to form a preliminary search direction;

[0163] An initial population generation module, based on the preliminary search direction, generates multiple neural network architecture individuals using Gaussian distribution sampling to construct an initial neural network architecture population;

[0164] A new population generation module, which evaluates the fitness of each individual in the initial neural network architecture population, selects a preset number of individuals with top fitness values ​​as excellent individuals, generates new neural network architecture individuals based on the excellent individuals through crossover and mutation operations, and constructs a new population;

[0165] A new population optimization module, using the distribution information of the excellent individuals, updates the mean vector and covariance matrix of the new population to optimize the distribution direction of the new population, wherein the distribution direction is used to guide crossover and mutation operations;

[0166] The iterative optimization and model output module iteratively performs the operations of generating new populations, fitness evaluation, crossover, mutation, and updating parameters of new populations, iteratively searches and optimizes the neural network architecture design until the predetermined stopping condition is met, stops the iteration, and selects the neural network architecture model with the best performance on the validation set from the finally generated optimized population.

[0167] The specific implementation of the hybrid evolutionary neural network search device of the present invention is basically the same as the various embodiments of the hybrid evolutionary neural network search method described above, and will not be repeated here.

[0168] The present invention also provides a hybrid evolutionary neural network search device, such as Figure 3 As shown, the hybrid evolutionary neural network search device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0169] Those skilled in the art will understand that Figure 3 The hardware structure of the hybrid evolutionary neural network search device shown in the figure does not constitute a limitation of the hybrid evolutionary neural network search device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0170] like Figure 3As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a hybrid evolution neural network search program. The operating system is a program for managing and controlling the hybrid evolution neural network search device and software resources, and supports the operation of the network communication module, the user interface module, the hybrid evolution neural network search program, and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0171] exist Figure 3 In the hardware structure of the hybrid evolutionary neural network search device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; the processor 1001 can call the hybrid evolutionary neural network search program stored in the memory 1005, and perform the same operations as the hybrid evolutionary neural network search method.

[0172] The specific implementation of the hybrid evolutionary neural network search device of the present invention is basically the same as the various embodiments of the hybrid evolutionary neural network search method described above, and will not be repeated here.

[0173] In addition, an embodiment of the present invention further proposes a computer storage medium, on which a hybrid evolutionary neural network search program is stored. When the hybrid evolutionary neural network search program is executed by a processor, the steps of the hybrid evolutionary neural network search method as described above are implemented.

[0174] The specific implementation of the computer storage medium of the present invention is basically the same as the various embodiments of the hybrid evolutionary neural network search method described above, and will not be repeated here.

Claims

1. A hybrid evolutionary neural network search method, characterized in that: The following steps are involved: Assigning initial weights to candidate operations of multiple neural network architectures, and optimizing the initial weights using a gradient descent algorithm to form a preliminary search direction; Based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling to construct an initial neural network architecture population; Perform fitness evaluation on each individual in the initial neural network architecture population, select a preset number of individuals with top fitness values ​​as excellent individuals, generate new neural network architecture individuals based on the excellent individuals through crossover and mutation operations, and construct a new population; Using the distribution information of the excellent individuals, the mean vector and the covariance matrix of the new population are updated to optimize the distribution direction of the new population, wherein the distribution direction is used to guide crossover and mutation operations; Iteratively perform the operations of generating a new population, evaluating fitness, crossover, mutation, and updating parameters of the new population, iteratively search and optimize the neural network architecture design until the predetermined stopping condition is met, stop the iteration, and select the neural network architecture model with the best performance on the validation set from the finally generated optimized population.

2. The hybrid evolutionary neural network search method according to claim 1, characterized in that: The initial weights are optimized using a gradient descent algorithm to form a preliminary search direction, including: Using the training set, calculate the initial loss function for candidate operations of multiple neural network architectures; Based on the initial loss function, using a gradient descent algorithm to iteratively update the initial weights for multiple times; After each iterative update, the validation set is used to calculate the loss value of the neural network architecture on the validation set, and the learning rate is dynamically adjusted based on the loss value of the validation set; Add momentum term during gradient descent optimization; When updating the gradient, introduce noise or perform mutation operations to expand the search space; Based on the loss value and performance evaluation results of the neural network architecture on the training set and the validation set, the optimized preliminary search direction is determined.

3. The hybrid evolutionary neural network search method according to claim 1, characterized in that: Initial weights are assigned to candidate operations for multiple neural network architectures, including: Select the corresponding weight initialization method based on the activation function of each neural network architecture; Determine the initial weight distribution parameters of each neural network architecture according to the selected weight initialization method, wherein the initial weight distribution parameters include mean and variance; Based on the determined initial weight distribution parameters, generating initial weights for the candidate operations of each neural network architecture, wherein the initial weights are generated from corresponding random distributions according to the distribution parameters determined by the weight initialization method; The generated initial weights are applied to each layer of each neural network architecture, wherein the weight matrix of each layer is generated according to the number of input nodes and the number of output nodes, and the bias value is initialized to zero or a set small constant value.

4. The hybrid evolutionary neural network search method according to claim 1, characterized in that: Based on the preliminary search direction, multiple neural network architecture individuals are generated using Gaussian distribution sampling to construct an initial neural network architecture population, including: Set a population size range based on the number of layers, number of parameters, and computational complexity of the loss function for each neural network architecture; According to the preliminary search direction, the mean and variance of the Gaussian distribution are set so that the generated neural network architecture individuals are distributed around the preliminary search direction; According to the set population size range, randomly sample and generate multiple neural network architecture individuals from Gaussian distribution; During the Gaussian distribution sampling process, random jitter is applied to the initial weights of each individual neural network architecture; After the initial neural network architecture population is generated, a fitness evaluation is performed, and the initial neural network architecture population is adjusted according to the evaluation result.

5. The hybrid evolutionary neural network search method as claimed in claim 1, characterized in that: Using the distribution information of the excellent individuals, the mean vector and covariance matrix of the new population are updated to optimize the distribution direction of the new population, including: Calculate the mean vector of weights and bias values ​​of all excellent individuals, where the mean vector reflects the central position of the excellent individuals in the parameter space of the new population, where the parameter space includes the set of weights and bias values ​​of all neural network architectures in the new population; Based on the difference between the weight and bias value of each excellent individual and the calculated mean vector, the covariance of each excellent individual relative to the mean vector is calculated, and the covariances of all excellent individuals are averaged to update the covariance matrix; The mean vector and the updated covariance matrix are used to optimize the distribution direction of the new population and guide the generation of neural network architecture individuals in the new population.

6. The hybrid evolutionary neural network search method according to claim 1, characterized in that: Perform fitness evaluation on each individual in the initial neural network architecture population, and select a preset number of individuals with top fitness values ​​as excellent individuals, including: Create a fitness value list to store the fitness value of each neural network architecture individual; Perform fitness evaluation on each individual in the initial neural network architecture population, obtain a fitness value by calculating the performance of each individual on a preset task, and store the fitness value of each individual in the fitness value list; Sorting all neural network architecture individuals according to the values ​​in the fitness value list to identify the best performing individual; A preset number of individuals at the top of the fitness value ranking are selected as excellent individuals to construct a new population. The unselected individuals are eliminated and no longer participate in subsequent iterations.

7. The hybrid evolutionary neural network search method according to claim 1, characterized in that: Iterate the search and optimize the neural network architecture design until the predetermined stopping condition is met and the iteration is stopped, including: Initialize an iteration counter, record the number of iterations starting from 0, and determine the upper limit of the maximum number of iterations; When the accuracy rate on the validation set changes less than a first preset threshold value in a first preset number of consecutive iterations, the iteration is stopped; Monitor the change of the loss function, and when the change amplitude of the loss function value is lower than the second preset threshold value in the second preset number of consecutive iterations, confirm that the loss function has converged and stop the iteration; When the number of iterations recorded by the iteration counter reaches the upper limit of the maximum number of iterations, the iteration is stopped.

8. A hybrid evolutionary neural network search device, characterized in that: The hybrid evolutionary neural network search device comprises: A weight initialization and optimization module assigns initial weights to candidate operations of multiple neural network architectures, and uses a gradient descent algorithm to optimize the initial weights to form a preliminary search direction; An initial population generation module, based on the preliminary search direction, generates multiple neural network architecture individuals using Gaussian distribution sampling to construct an initial neural network architecture population; A new population generation module, which evaluates the fitness of each individual in the initial neural network architecture population, selects a preset number of individuals with top fitness values ​​as excellent individuals, generates new neural network architecture individuals based on the excellent individuals through crossover and mutation operations, and constructs a new population; A new population optimization module, using the distribution information of the excellent individuals, updates the mean vector and covariance matrix of the new population to optimize the distribution direction of the new population, wherein the distribution direction is used to guide crossover and mutation operations; The iterative optimization and model output module iteratively performs the operations of generating new populations, fitness evaluation, crossover, mutation, and updating parameters of new populations, iteratively searches and optimizes the neural network architecture design until the predetermined stopping condition is met, stops the iteration, and selects the neural network architecture model with the best performance on the validation set from the finally generated optimized population.

9. A hybrid evolutionary neural network search device, characterized in that: The hybrid evolutionary neural network search device includes a memory, a processor, and a hybrid evolutionary neural network search program stored in the memory and executable on the processor. When the hybrid evolutionary neural network search program is executed by the processor, the steps of the hybrid evolutionary neural network search method as described in any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that: The storage medium stores a hybrid evolutionary neural network search program, which, when executed by a processor, implements the steps of the hybrid evolutionary neural network search method according to any one of claims 1 to 7.

Images