Methods, devices, storage media, and electronic devices for weight tuning of convolutional neural networks

Abstract

This application discloses a weight optimization method for convolutional neural networks, comprising: forming a learning rate parameter group by grouping the learning rate parameters corresponding to each specified layer of a first convolutional neural network; using the learning rate parameter group as individuals in a genetic algorithm, and using the learning rate parameters as genes in the genetic algorithm; and sequentially processing all learning rate parameter groups of each generation using the genetic algorithm until the optimization termination condition of the genetic algorithm is met, thereby determining the optimal learning rate parameter group for generating a second convolutional neural network by training the first convolutional neural network. Applying this application, the optimal or locally optimal learning rate parameter group in transfer learning can be found without relying on any prior knowledge, thereby improving the performance of the target convolutional neural network in transfer learning.

Description

Technical Field

[0001] This application relates to neural network technology, and in particular to a method, apparatus, storage medium and electronic device for weight tuning of a convolutional neural network. Background Technology

[0002] With the advancement of deep learning and neural network technologies, convolutional neural network models have been increasingly widely used.

[0003] Deep learning, implemented using convolutional neural networks (CNNs), is a method in the field of machine learning. By building deep neural networks, it enables tasks such as image classification, object detection, image segmentation, and machine translation. The application process of CNNs typically consists of two phases: training and testing. The training phase uses large-scale data to train the CNN, while the testing phase applies the trained CNN to various specific tasks.

[0004] Transfer learning is a machine learning method that reuses a model developed for task A (often called a pre-trained base network) as a starting point in the process of developing a model for task B. Its main idea is to transfer labeled data or knowledge structures from a related domain (also called the source domain) to improve or enhance the learning performance in the target domain (also called the target domain) or task. There are several mainstream approaches to transfer learning: 1) Transfer Learning: Freeze all convolutional layers of the pre-trained model and train only the custom fully connected layers; 2) Extract Feature Vector: First, calculate the feature vectors of the convolutional layers of the pre-trained model with respect to all training and testing data, then discard the pre-trained model and train only the custom simplified fully connected network; 3) Fine-tune: Freeze some convolutional layers of the pre-trained model (usually the majority of convolutional layers near the input), and train the remaining convolutional layers (usually the convolutional layers near the output) and fully connected layers. Here, the source domain refers to a domain with a large amount of labeled data, and the target domain refers to a domain with a small amount of labeled data.

[0005] Weight tuning is a transfer learning strategy that studies how to transfer knowledge learned in the source domain to the target domain by changing the learning rate parameters of specified layers in the base network. Currently, the main way to change the learning rate parameters (also known as learning weights) is by manually setting the learning weights, generally including learning weight increment and learning weight decrement strategies. However, this method relies heavily on prior knowledge, is inefficient, and the effectiveness of the found learning rate parameter combination depends on the accuracy of the prior knowledge, which cannot guarantee the performance of the target convolutional neural network. Summary of the Invention

[0006] This application provides a method, apparatus, storage medium, and electronic device for weight tuning of convolutional neural networks, which can find the optimal or locally optimal learning rate parameter set in transfer learning without relying on any prior knowledge, thereby improving the performance of the target convolutional neural network in transfer learning.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] A method for weight optimization in a convolutional neural network, comprising:

[0009] The learning rate parameters corresponding to each specified layer of the first convolutional neural network are grouped into a learning rate parameter group, and the learning rate parameter group is used as an individual of the genetic algorithm, and the learning rate parameter is used as the gene of the genetic algorithm.

[0010] The genetic algorithm is used to process all learning rate parameter sets of each generation in turn until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network.

[0011] The processing of all learning rate parameter sets for each generation includes:

[0012] Using all the learning rate parameter sets of the current generation, the first convolutional neural network is trained separately to generate network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets. Performance evaluation is then performed on each network model. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, the genetic algorithm is used to determine all learning rate parameter sets for the next generation.

[0013] Preferably, the step of generating the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set, and performing performance evaluation on each network model, includes:

[0014] For all learning parameter groups of the current generation, the second convolutional neural network is trained and generated by transfer learning based on the first convolutional neural network using each learning rate parameter group, so as to obtain the network model corresponding to each learning rate parameter group.

[0015] The performance of the network model corresponding to each learning rate parameter group is evaluated separately, and the performance evaluation result of the network model corresponding to each learning rate parameter group is obtained as the performance evaluation result of each learning rate parameter group.

[0016] Preferably, the determination of whether the optimization termination condition is met includes:

[0017] If the performance evaluation result corresponding to the current optimal learning rate parameter group is better than the preset target performance, or if the number of iterations of the genetic algorithm reaches the preset first maximum number of iterations, then it is determined that the optimization termination condition is met; otherwise, it is determined that the optimization termination condition is not met; wherein, the performance evaluation result corresponding to each learning rate parameter group is the performance evaluation result of the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter group.

[0018] Preferably, methods for determining the current optimal learning rate parameter set and its corresponding performance evaluation results include:

[0019] If the current generation is the first generation, then select the learning rate parameter group with the highest performance evaluation result from all the learning rate parameter groups in the first generation as the current best learning rate parameter group;

[0020] If the current generation is not the first generation, then among all the learning rate parameter sets in the current generation, if the performance evaluation result corresponding to any learning rate parameter set is better than the performance evaluation result corresponding to the current best learning rate parameter set before the current generation, then that learning rate parameter set is taken as the latest current best learning rate parameter set; otherwise, the current best learning rate parameter set remains unchanged.

[0021] Preferably, determining all learning rate parameter sets for the next generation using the genetic algorithm includes:

[0022] For each learning rate parameter in each learning rate parameter group of the current generation, determine whether a mutation has occurred. If so, update the corresponding learning rate parameter of the next generation; otherwise, ensure that the corresponding learning rate parameter of the next generation remains unchanged.

[0023] Preferably, the corresponding learning rate parameter for updating the next generation includes:

[0024] For the i-th learning rate parameter in each learning rate parameter group, update the corresponding learning rate parameter of the next generation to max(min(P[index[0]]). k +0.5*(P[index[1]] k -P[index[2]] k ),up_bound),low_bound); among them, index[i] k=random.sample(range(Np),3) means randomly sampling the k-th learning rate parameter value of the i-th learning rate parameter group from the Np learning rate parameter groups of the current generation, where 3≤Np, low_bound is the preset minimum value of the learning rate parameter, and up_bound is the maximum value of the learning rate parameter.

[0025] Preferably, after determining that the optimization termination condition is not met, the method further includes:

[0026] Determine whether the number of learning rate parameters in the current optimal learning rate parameter group that are higher than the first set value is greater than the set threshold value. If so, increase the maximum value of the learning rate parameter.

[0027] Preferably, the first set value is the midpoint of the learning rate parameter's value range; and / or,

[0028] The set threshold value is total_modules * high_value_ratio; where total_modules is the number of convolutional groups in the first convolutional neural network, and high_value_ratio is the proportion of learning rate parameters greater than the second set value in the learning rate parameter combination that corresponds to the best performance evaluation result among all learning rate parameter combinations in the current generation.

[0029] Preferably, the step of training the first convolutional neural network to generate a network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set includes:

[0030] Training is performed on the first convolutional neural network. When the number of training iterations reaches the set second maximum number of iterations, the training ends, and the trained network model is used as the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set.

[0031] A weight tuning device for a convolutional neural network includes: a setting unit and a genetic algorithm processing unit;

[0032] The setting unit is used to form a learning rate parameter group from the learning rate parameters corresponding to each specified layer of the first convolutional neural network, and to use the learning rate parameter group as an individual of the genetic algorithm, and the learning rate parameter as the gene of the genetic algorithm.

[0033] The genetic algorithm processing unit is used to process all learning rate parameter sets of each generation sequentially using the genetic algorithm until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network.

[0034] Specifically, in the genetic algorithm processing unit, processing all learning rate parameter sets for each generation includes:

[0035] Using all the learning rate parameter sets of the current generation, the first convolutional neural network is trained separately to generate network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets. Performance evaluation is then performed on each network model. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, the genetic algorithm is used to determine all learning rate parameter sets for the next generation.

[0036] Preferably, in the genetic algorithm processing unit, the step of generating network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets, and performing performance evaluation on each network model, includes:

[0037] For all learning parameter groups of the current generation, the second convolutional neural network is trained and generated by transfer learning based on the first convolutional neural network using each learning rate parameter group, so as to obtain the network model corresponding to each learning rate parameter group.

[0038] The performance of the network model corresponding to each learning rate parameter group is evaluated separately, and the performance evaluation result of the network model corresponding to each learning rate parameter group is obtained as the performance evaluation result of each learning rate parameter group.

[0039] Preferably, in the genetic algorithm processing unit, determining whether the optimization termination condition is met includes:

[0040] If the performance evaluation result corresponding to the current optimal learning rate parameter set is better than the preset target performance, or if the number of iterations of the genetic algorithm reaches the preset first maximum number of iterations, then the optimization termination condition is determined to be met; otherwise, the optimization termination condition is determined not to be met.

[0041] Preferably, in the genetic algorithm processing unit, determining the current optimal learning rate parameter set and its corresponding performance evaluation results includes:

[0042] If the current generation is the first generation, then select the learning rate parameter group with the highest performance evaluation result from all the learning rate parameter groups in the first generation as the current best learning rate parameter group;

[0043] If the current generation is not the first generation, then among all the learning rate parameter sets in the current generation, if the performance evaluation result corresponding to any learning rate parameter set is better than the performance evaluation result corresponding to the current best learning rate parameter set before the current generation, then that learning rate parameter set is taken as the latest current best learning rate parameter set; otherwise, the current best learning rate parameter set remains unchanged.

[0044] Preferably, in the genetic algorithm processing unit, determining all learning rate parameter sets for the next generation using the genetic algorithm includes:

[0045] For each learning rate parameter in each learning rate parameter group of the current generation, determine whether a mutation has occurred. If so, update the corresponding learning rate parameter of the next generation; otherwise, ensure that the corresponding learning rate parameter of the next generation remains unchanged.

[0046] Preferably, in the genetic algorithm processing unit, the corresponding learning rate parameter for updating the next generation includes:

[0047] For the i-th learning rate parameter in each learning rate parameter group, update the corresponding learning rate parameter of the next generation to max(min(P[index[0]]). k +0.5*(P[index[1]] k -P[index[2]] k ),up_bound),low_bound); among them, index[i] k =random.sample(range(Np),3) means randomly sampling the k-th learning rate parameter value of the i-th learning rate parameter group from the Np learning rate parameter groups of the current generation, where 3≤Np, low_bound is the preset minimum value of the learning rate parameter, and up_bound is the maximum value of the learning rate parameter.

[0048] Preferably, after determining that the optimization termination condition is not met, the genetic algorithm processing unit is further used to determine whether the number of learning rate parameters in the current optimal learning rate parameter group that are higher than the first set value is greater than the set threshold value. If so, the maximum value of the learning rate parameter is increased.

[0049] Preferably, the first set value is the midpoint of the learning rate parameter's value range; and / or,

[0050] The set threshold value is total_modules * high_value_ratio; where total_modules is the number of convolutional groups in the first convolutional neural network, and high_value_ratio is the proportion of the learning rate parameter in the first convolutional neural network that is greater than the second set value.

[0051] Preferably, in the genetic algorithm processing unit, the step of training the first convolutional neural network to generate a network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set includes:

[0052] Training is performed on the first convolutional neural network. When the number of training iterations reaches the set second maximum number of iterations, the training ends, and the trained network model is used as the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set.

[0053] A computer-readable storage medium storing computer instructions thereon, characterized in that, when the instructions are executed by a processor, they can implement the convolutional neural network weight tuning method described above.

[0054] An electronic device, comprising at least a computer-readable storage medium and a processor;

[0055] The processor is configured to read the executable instructions from the computer-readable storage medium and execute the instructions to implement the convolutional neural network weight tuning method described above.

[0056] As can be seen from the above technical solution, in this application, the learning rate parameters corresponding to each specified layer of the first convolutional neural network in the source domain are grouped into learning rate parameter sets. These learning rate parameter sets are used as individuals in a genetic algorithm, and the learning rate parameters are used as genes in the genetic algorithm. The genetic algorithm is used to optimize the learning rate parameter sets composed of each learning rate parameter. During the optimization process, the performance evaluation results of the second convolutional neural network model trained using each learning rate parameter set are used as the evaluation criteria for the corresponding learning rate parameter set for optimization. This method of optimizing learning rate parameters can improve the performance of the target convolutional neural network by continuously updating and optimizing the learning rate parameters in a continuous search space without relying on any prior knowledge.

[0057] Furthermore, when determining the next generation of learning rate combinations, if a certain proportion of the learning rate parameters in the current optimal learning rate parameter combination are closer to the upper limit of the current learning rate parameter combination's range, then the maximum value of the learning rate parameters can be increased to find the optimal or locally optimal learning rate parameter combination within a larger range. Compared with the method of fixing the learning rate parameter range (usually 0-1), this approach can find a better learning rate parameter combination, thereby further improving the performance of the target convolutional neural network. Attached Figure Description

[0058] Figure 1 This is a schematic diagram illustrating the basic process of the weight tuning method for the convolutional neural network in this application;

[0059] Figure 2 Here is the logical flowchart of the genetic algorithm;

[0060] Figure 3 This is a schematic diagram illustrating the specific process of the weight tuning method for a convolutional neural network in a specific embodiment of this application.

[0061] Figure 4 This is a schematic diagram of the basic structure of the weight tuning device for the convolutional neural network in this application;

[0062] Figure 5 This is a schematic diagram of the basic structure of an electronic device provided in this application. Detailed Implementation

[0063] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0064] Currently, a common strategy for weight optimization (i.e., finding the optimal learning rate parameter set) is to freeze the lower layers and release the upper layers. However, some literature has shown through experiments that the opposite strategy can also achieve good results. That is, the optimal learning rate parameter distribution may also be a fluctuating form. In short, in transfer learning, there is no consensus strategy to guide how the learning rate parameters of each layer should be set. Currently, the main way to change the learning rate parameters is through manual setting methods (learning weight increment strategy and learning weight decrement strategy). This manual adjustment method often relies on prior knowledge from literature experimental results, and then verifies the effect through a large number of experiments. This not only leads to low efficiency and is time-consuming and labor-intensive, but also makes it difficult to find the optimal solution.

[0065] Considering the problems with the above-mentioned learning rate parameter setting methods, the basic idea of ​​this application is to use a genetic algorithm to optimize the weights in transfer learning, thereby obtaining the optimal or locally optimal combination of learning rate parameters in transfer learning and improving the performance of the target convolutional neural network.

[0066] Figure 1 This is a basic flowchart illustrating the weight tuning method for a convolutional neural network in this application embodiment, as shown below. Figure 1 As shown, the method includes:

[0067] Step 101: The learning rate parameters corresponding to each specified layer of the first convolutional neural network are combined into a learning rate parameter group, and the learning rate parameter group is used as an individual of the genetic algorithm, and the learning rate parameter is used as the gene of the genetic algorithm.

[0068] In this application, the convolutional neural network in the source domain (i.e., the pre-trained convolutional neural network) used in transfer learning is referred to as the first convolutional neural network. Based on this first convolutional neural network, a second convolutional neural network suitable for the target domain, i.e., the target convolutional neural network, is trained and generated by setting the learning rate parameter.

[0069] Based on the above settings, learning rate parameters are set for each specified layer of the first convolutional neural network. All learning rate parameters are combined to form a learning rate parameter set. The specified layers can be set as needed, typically the layers that the second convolutional neural network needs to learn from the first convolutional neural network; for example, it could be all convolutional layers. Each learning rate parameter is used as the gene in a genetic algorithm, and the learning rate parameter set is used as an individual in the genetic algorithm. Thus, in step 102 below, the genetic algorithm can find the optimal learning rate parameter set, and using this optimal learning rate parameter set, the second convolutional neural network can be trained and generated based on the first convolutional neural network.

[0070] Step 102: Use a genetic algorithm to process all learning rate parameter sets of each generation in turn until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network.

[0071] Figure 2 This is a schematic diagram of the logical flow of a genetic algorithm. In a genetic algorithm, each generation has multiple individuals, and each individual contains multiple genes. For each generation of individuals, some genes mutate and are updated to new gene values, while some genes do not mutate and their gene values ​​remain unchanged (that is, the genes are inherited by the next generation). These inherited or mutated genes then form the next generation of individuals.

[0072] When using a genetic algorithm to optimize the learning rate parameter, all learning rate parameter sets of each generation are processed sequentially, and after processing each generation, it is determined whether the optimization termination condition of the genetic algorithm is met. The genetic algorithm process ends when the learning rate parameter set of a certain generation meets the optimization termination condition.

[0073] This involves processing all learning rate parameter sets for each generation, specifically including:

[0074] Using all the learning rate parameter sets of the current generation, each convolutional neural network is trained on the basis of the first convolutional neural network to generate a network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set. The performance of each network model is then evaluated. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, a genetic algorithm is used to determine all the learning rate parameter sets for the next generation.

[0075] In the above process, the learning rate parameter set of the currently being processed generation is referred to as the learning rate parameter set of the current generation. For all learning rate parameter sets of the current generation, the neural network is trained on a unit basis. Specifically, taking learning rate parameter set A as an example, each learning rate parameter in learning rate parameter set A is assigned to the corresponding designated layer of the first convolutional neural network as the learning rate parameter of that layer. Then, the neural network is trained based on the first convolutional neural network to obtain a second convolutional neural network, which can also be called a network model of the second convolutional neural network (hereinafter referred to as network model B). This network model B corresponds to learning rate parameter set A. Next, the performance of network model B is evaluated, and the performance evaluation result is used as the evaluation standard of learning rate parameter set A in the genetic algorithm. The processing of each learning rate parameter set is the same as that of learning rate parameter set A. In this way, a corresponding network model can be trained and generated for each learning rate parameter set, and the performance evaluation result of the network model can be used as the evaluation standard of the corresponding learning rate parameter set. After obtaining the corresponding network model and its performance evaluation results for all learning rate parameter sets of the current generation, the current optimal learning rate parameter set is determined, and it is determined whether the current set meets the optimization termination condition of the genetic algorithm. If it does, the iteration of the genetic algorithm ends, and the current optimal learning rate parameter set is taken as the optimization result, i.e., the optimal learning rate parameter set. If it does not meet the condition, the genetic algorithm is used to determine all learning rate parameter sets for the next generation, and the processing of the learning rate parameter sets for the next generation continues until the optimization termination condition of the genetic algorithm is met.

[0076] At this point, Figure 1 The basic process shown is now complete. As can be seen from the above processing, by using the learning rate parameter as the "gene" of the genetic algorithm and the learning rate parameter set as the "individual" of the genetic algorithm, the optimization of the learning rate parameter set is implemented through the genetic algorithm. In this way, without relying on any prior knowledge, the genetic algorithm can continuously update and optimize the learning rate parameter in a continuous search space, thereby improving the performance of the target convolutional neural network.

[0077] The specific implementation of the weight tuning method for convolutional neural networks in this application is illustrated below through specific embodiments.

[0078] Figure 3 This is a schematic diagram illustrating the specific process of the convolutional neural network weight tuning method in a particular embodiment of this application.

[0079] like Figure 3 As shown, the method includes:

[0080] Step 301: Collect and divide the training set and validation set of the target domain.

[0081] Collecting and dividing the target domain training set and validation set refers to collecting the training data and validation data required by the deep learning network in the target task scenario of transfer learning, and dividing them according to a certain ratio; in this embodiment, the collected training data is divided into training set and validation set in a 1:1 ratio.

[0082] Step 302: Initialize the hyperparameters of the genetic algorithm.

[0083] Genetic algorithms require the setting of necessary hyperparameters. In this embodiment, the following hyperparameters are set:

[0084] 1. The number of genes in an individual, popu_element;

[0085] As mentioned above, in this application, the learning rate parameter is the gene in the genetic algorithm, and the learning rate parameter group is the individual in the genetic algorithm. Then, the number of genes in the individual, popu_element, refers to the number of learning rate parameters contained in the learning rate parameter group. Since a learning rate parameter is set for each specified layer of the first convolutional neural network in this application, popu_element is the number of specified layers, which is usually the number of learnable layers. For example, when a learning rate parameter is set for each convolutional layer, then popu_element is the number of convolutional layers.

[0086] 2. The total number of convolutional groups in the first convolutional neural network: total_modules;

[0087] This hyperparameter is optional. When the maximum value of the learning rate parameter can be updated, this hyperparameter is used to determine whether the maximum value of the learning rate parameter needs to be updated. This hyperparameter is introduced specifically for the scenario of this application and is not a hyperparameter of a traditional genetic algorithm. In this embodiment, the first convolutional neural network implements cross-layer connections of the convolutional layers through residual modules. Therefore, in this embodiment, the number of convolutional groups, total_modules, is the number of residual modules.

[0088] 3. The probability of gene mutation (CR);

[0089] When determining the learning rate parameter set for the next generation, according to the genetic algorithm, some genes (i.e., learning rate parameters) may mutate, and the probability CR is used to control the likelihood of gene mutation. Typically, a fixed value can be preset according to actual needs.

[0090] 4. The number of individuals in each generation, Np;

[0091] In this application, the number of individuals Np is the same for different generations, such as Figure 2 As shown, the settings can be adjusted according to actual needs. Generally, the larger the size of the first convolutional neural network, the more individuals (Np) there are in each generation, thus enabling a more accurate finding of the optimal learning rate parameter set. In this embodiment, Np is assumed to be 6.

[0092] 5. The maximum and minimum values ​​that the learning rate parameter can take;

[0093] The maximum and minimum values ​​(also called the minimum values) of the learning rate parameter define its range. In this embodiment, the minimum value (low_bound) remains constant and is typically 0. To find a better learning rate parameter, the maximum value is set to be updatable; the update process will be described in detail in step 308. However, considering limitations in processing power and resources, or network performance requirements, the maximum value can be fixed to simplify the process, although this will inevitably affect the performance of the target convolutional neural network to some extent.

[0094] 6. The maximum number of iterations N in a genetic algorithm max ;

[0095] In genetic algorithms, the processing of each generation of individuals is equivalent to one iteration process. This can be achieved by setting a maximum number of iterations N. max Limit the total number of generations in the genetic algorithm.

[0096] Step 303: Set the number of iterations (ITERATION_NUM) for each training session of the convolutional neural network.

[0097] To ensure fairness when training convolutional neural networks using different learning rate parameter sets, the weight tuning method in this application iterates the convolutional neural network ITERATION_NUM times each time before terminating the training. In other words, this application no longer uses the loss function as the termination condition for training the convolutional neural network; instead, it uniformly uses the number of iterations ITERATION_NUM as the termination condition. This ensures that for different learning rate parameter sets, the corresponding network model is obtained after ITERATION_NUM iterations, and then the network model's performance is evaluated. This guarantees fairness when comparing performance evaluation results. ITERATION_NUM is a preset constant.

[0098] The aforementioned steps 301-303 constitute the initialization process of the weight tuning method in this embodiment. The order of these three steps can be arbitrarily adjusted, or they can be performed simultaneously. This application does not impose any limitations on this; in this embodiment, only the sequential execution of the three steps is used as an example. Next, steps 304-308 constitute the execution part of the genetic algorithm, which iteratively processes the learning rate parameter set for each generation. The learning rate parameter set of the generation currently being processed is referred to as the current generation's learning rate parameter set. A detailed description will be given using the processing of the current generation's learning rate parameter set as an example.

[0099] Step 304: For all learning rate parameter sets of the current generation, use each learning rate parameter set to train and generate a second convolutional neural network based on the first convolutional neural network through transfer learning, thereby obtaining the network model corresponding to each learning rate parameter set.

[0100] This step involves training the convolutional neural network. Specifically, for each set of learning rate parameters in the current generation, the convolutional neural network is trained once, generating a second convolutional neural network model corresponding to each set of learning rate parameters.

[0101] If the current generation is the first generation of the genetic algorithm, then the learning rate parameter values ​​for all learning rate parameter groups in the first generation need to be calculated first, which is the initial value of the learning rate parameter.

[0102] In this embodiment, in any learning rate parameter group of the first generation, all learning rate parameter values ​​in that group are the same. For the i-th learning rate parameter group of the first generation, the value of the learning rate parameter is population. i The calculation can be performed according to the following formula (1):

[0103]

[0104] If the current generation is not the first generation of the genetic algorithm, then the values ​​of each learning rate parameter in the learning rate parameter group of the current generation are determined after the previous generation has finished processing, which will be introduced in step 308.

[0105] After determining the values ​​of each learning rate parameter in all learning rate parameter sets for the current generation, the convolutional neural network can be trained one by one for each learning rate parameter set. The following explanation uses the learning rate parameter set x as an example.

[0106] Each learning rate parameter in the learning rate parameter group x is assigned to a specific layer of the first convolutional neural network. Based on this first convolutional neural network, the second convolutional neural network is trained using the training set partitioned in step 301, generating a network model y. During the training of the convolutional neural network, training ends when the number of iterations reaches ITERATION_NUM set in step 303, resulting in network model y, to ensure fairness across different learning rate parameter groups. This network model y corresponds to the learning rate parameter group x. After this step, a corresponding network model is trained for each learning rate parameter group in the current generation.

[0107] Additionally, it should be noted that each time the process returns to this step after step 308 to train the convolutional neural network, the training is restarted based on the first convolutional neural network.

[0108] Step 305: For each learning rate parameter group of the current generation, the network model corresponding to each learning rate parameter group is evaluated separately to obtain the performance evaluation result of the network model corresponding to each learning rate parameter group, which is used as the performance evaluation result of the corresponding learning rate parameter group.

[0109] After processing in step 304, a corresponding network model is trained for each learning rate parameter group of the current generation. In this embodiment, Np is 6 for each generation, so 6 network models can be obtained. For each of these obtained network models, the network model performance is evaluated using the test set determined in step 301 to obtain the corresponding performance evaluation results. Specifically, the evaluation metrics used for performance evaluation can be selected according to the task requirements, such as recall, precision, F1, etc. The F1 metric is calculated according to the calculation formula (2), where precision refers to precision and recall refers to recall. Precision and recall can be determined in the existing way.

[0110]

[0111] Step 306: Determine the current optimal learning rate parameter set based on the performance evaluation results of all learning rate parameter sets in the current generation.

[0112] This step involves determining the best individual based on the genetic algorithm. Specifically, if the current generation is the first generation, the learning rate parameter set with the highest performance evaluation result among all learning rate parameter sets in the first generation is selected as the current best learning rate parameter set.

[0113] If the current generation is not the first generation, then among all learning rate parameter sets in the current generation, if the performance evaluation result of a certain learning rate parameter set 'a' is better than the performance evaluation result of the latest current best learning rate parameter set, then learning rate parameter set 'a' is taken as the latest current best learning rate parameter set; otherwise, the latest current best learning rate parameter set remains unchanged. Of course, in practical applications, different implementation methods can be used to determine the current best learning rate parameter set. For example, one can first select the learning rate parameter set with the best performance evaluation result from all learning rate parameter sets in the current generation, compare its performance evaluation result with the performance evaluation result of the latest current best learning rate parameter set, and take the learning rate parameter set with the better performance evaluation result as the latest current best learning rate parameter set; alternatively, one can compare the performance evaluation result of each learning rate parameter set in the current generation with the latest learning rate parameter set one by one, and select the learning rate parameter set with the better performance evaluation result as the latest learning rate parameter set, until all learning rate parameter sets in the current generation have been compared, thus determining the current best learning rate parameter set.

[0114] Step 307: Determine whether the optimization termination condition of the genetic algorithm is met. If not, proceed to step 308; otherwise, proceed to step 309.

[0115] After processing all learning rate parameter sets for each generation in steps 304-306, this step determines whether to end the iteration of the genetic algorithm.

[0116] The specific method for determining whether the optimization termination condition is met can be:

[0117] If the performance evaluation result F corresponding to the current optimal learning rate parameter set is better than the preset target performance F... target Alternatively, if the number of iterations of the genetic algorithm reaches the preset first maximum number of iterations N. max If the target performance F is satisfied, then the optimization termination condition of the genetic algorithm is met; otherwise, the optimization termination condition of the genetic algorithm is not met. target These are pre-set performance metrics that the convolutional neural network is expected to achieve.

[0118] Once it is determined that the optimization termination condition is not met, the specific values ​​of the next generation learning rate parameter set are determined using a genetic algorithm through the processing in step 308.

[0119] Once the optimization termination condition is met, the iteration of the genetic algorithm ends, and the optimal learning rate parameter is determined in step 309.

[0120] Step 308: Use a genetic algorithm to determine the next generation learning rate parameter set and the maximum value of the learning rate parameter.

[0121] As mentioned earlier, in genetic algorithms, genes may mutate when determining the next generation of individuals and genes, meaning the learning rate parameters in this application may need to be updated. This step is used to determine the specific values ​​of each learning rate parameter in all learning rate parameter groups for the next generation. The method for determining the specific value of each learning rate parameter can be the same; here, we will use the k-th learning rate parameter in a learning rate parameter group b as an example.

[0122] The value of the k-th learning rate parameter in the next generation's learning rate parameter group b is determined based on whether the learning rate parameter has mutated. Specifically, if the k-th learning rate parameter mutates, its value is updated, and the specific method for determining the updated value can be set as needed; if the k-th learning rate parameter does not mutate, its value remains unchanged, that is, it is the same as the corresponding learning rate parameter in the current generation.

[0123] The method for determining whether the learning rate parameter has mutated can be as follows: determine whether the random number generated for the corresponding learning rate parameter is less than the mutation probability CR set in step 302. If it is, then it is determined that mutation has occurred; otherwise, it is determined that no mutation has occurred.

[0124] This embodiment provides a method for updating the learning rate parameter value when a mutation occurs, namely, determining the value of the k-th learning rate parameter according to the following formula (3). k :

[0125]

[0126] Where, index[i] k =random.sample(range(Np),COUNT),index[i] k This indicates that COUNT learning rate parameter sets are randomly selected from the Np learning rate parameter sets of the current generation, where the k-th learning rate parameter of the i-th learning rate parameter set is index[i]. k COUNT = 3 ≤ Np, low_bound is the minimum value of the preset learning rate parameter, and up_bound is the maximum value of the learning rate parameter; random.random() represents a random number between 0 and 1 generated for the k-th learning rate parameter.

[0127] For each learning rate parameter, the value of the next generation learning rate parameter can be determined in the manner described above.

[0128] In addition, to further select a better combination of learning rate parameters, this embodiment may optionally determine in step 308 whether to update the maximum value of the learning rate parameters.

[0129] Specifically, in existing weight tuning methods, the range of the learning rate parameter is fixed. In this embodiment, when determining the next generation learning rate combination each time, if a certain proportion of the learning rate parameters in the current optimal learning rate parameter combination are closer to the upper limit of the current learning rate parameter combination's range, then the maximum value of the learning rate parameter can be increased to find the optimal learning rate parameter combination within a wider range.

[0130] More specifically, in this embodiment, it is determined whether the number of learning rate parameters in the current optimal learning rate parameter set that are higher than a first preset value is greater than a preset threshold. If so, the maximum value of the learning rate parameter is increased; otherwise, the maximum value of the learning rate parameter remains unchanged. The first preset value can be the midpoint of the learning rate parameter value range, i.e., (up_bound + low_bound) / 2.0. The preset threshold can be total_modules * high_value_ratio, where total_modules is the number of convolutional groups in the first convolutional neural network (in this embodiment, it is the number of residual modules in the first convolutional neural network), and high_value_ratio is the proportion of learning rate parameters greater than a second preset value in the learning rate parameter combination corresponding to the best performance evaluation result among all learning rate parameter combinations in the current generation.

[0131] This step yields the values ​​of all learning rate parameter sets for the next generation, as well as the latest maximum value of the learning rate parameter. Next, we return to step 304 for the next iteration of the genetic algorithm.

[0132] Step 309: Take the current best learning rate parameter set as the optimal learning rate parameter set, and use the network model trained with this learning rate parameter set as the trained second convolutional neural network.

[0133] When this step is executed, the genetic algorithm iteration ends and the optimal learning rate parameter set is obtained. Then, the network model corresponding to the optimal learning rate parameter set obtained by training in step 304 is the trained second convolutional neural network.

[0134] At this point, Figure 3The process shown ends here. Through the processing described in this application, the optimal or locally optimal learning rate parameter set in transfer learning can be accurately found, improving the performance of the target convolutional neural network. Furthermore, when determining the next generation learning rate combination each time, if a certain proportion of the learning rate parameters in the current optimal learning rate parameter combination are closer to the upper limit of the current learning rate parameter combination's range, then the maximum value of the learning rate parameters can be increased to find the optimal or locally optimal learning rate parameter combination within a larger range. Compared with the method of fixing the learning rate parameter range (usually 0-1), this processing method can find a better learning rate parameter combination, thereby further improving the performance of the target convolutional neural network.

[0135] The above describes a specific implementation of the convolutional neural network weight tuning method in this application. This application also provides a convolutional neural network weight tuning device, which can be used to implement the above-described weight tuning method. Figure 4 This is a schematic diagram of the basic structure of the convolutional neural network weight tuning device in this application, as shown below. Figure 4 As shown, the device includes a setting unit and a genetic algorithm processing unit.

[0136] The setting unit is used to form a learning rate parameter group from the learning rate parameters corresponding to each specified layer of the first convolutional neural network, and to use the learning rate parameter group as an individual of the genetic algorithm, and the learning rate parameter as the gene of the genetic algorithm.

[0137] The genetic algorithm processing unit is used to process all learning rate parameter sets of each generation sequentially using the genetic algorithm until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network.

[0138] More specifically, in the genetic algorithm processing unit, all learning rate parameter sets for each generation are processed, which may specifically include:

[0139] Using all the learning rate parameter sets of the current generation, each convolutional neural network is trained on the basis of the first convolutional neural network to generate a second convolutional neural network model corresponding to the corresponding learning rate parameter set. The performance of each network model is then evaluated. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, a genetic algorithm is used to determine all the learning rate parameter sets for the next generation.

[0140] Optionally, in the genetic algorithm processing unit, a network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set is generated, and performance evaluation is performed on each network model, including:

[0141] For all learning parameter sets of the current generation, a second convolutional neural network is trained and generated based on the first convolutional neural network using each learning rate parameter set through transfer learning, thus obtaining the network model corresponding to each learning rate parameter set.

[0142] The performance of the network model corresponding to each learning rate parameter group is evaluated separately, and the performance evaluation result of the network model corresponding to each learning rate parameter group is used as the performance evaluation result of each learning rate parameter group.

[0143] Optionally, the process of determining whether the optimization termination condition is met in the genetic algorithm processing unit may specifically include:

[0144] If the performance evaluation result corresponding to the current optimal learning rate parameter group is better than the preset target performance, or if the number of iterations of the genetic algorithm reaches the preset first maximum number of iterations, then it is determined that the optimization termination condition is met; otherwise, it is determined that the optimization termination condition is not met; wherein, the performance evaluation result corresponding to each learning rate parameter group is the performance evaluation result of the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter group.

[0145] Optionally, the method for determining the current optimal learning rate parameter set and its corresponding performance evaluation results in the genetic algorithm processing unit may specifically include:

[0146] If the current generation is the first generation, then select the learning rate parameter group with the highest performance evaluation result from all the learning rate parameter groups in the first generation as the current best learning rate parameter group;

[0147] If the current generation is not the first generation, then among all the learning rate parameter sets in the current generation, if the performance evaluation result corresponding to any learning rate parameter set is better than the performance evaluation result corresponding to the current best learning rate parameter set, then that learning rate parameter set will be taken as the latest current best learning rate parameter set; otherwise, the current best learning rate parameter set will remain unchanged.

[0148] Optionally, the process of determining all learning rate parameter sets for the next generation using a genetic algorithm in the genetic algorithm processing unit may specifically include:

[0149] For each learning rate parameter in each learning rate parameter group of the current generation, determine whether a mutation has occurred. If so, update the corresponding learning rate parameter of the next generation; otherwise, ensure that the corresponding learning rate parameter of the next generation remains unchanged.

[0150] Optionally, the processing of updating the corresponding learning rate parameters of the next generation in the genetic algorithm processing unit may specifically include:

[0151] For the i-th learning rate parameter in each learning rate parameter group, update the corresponding learning rate parameter for the next generation to max(min(P[index[0]]). k +0.5*(P[index[1]] k -P[index[2]] k ),up_bound),low_bound); among them, index[i] k =random.sample(range(Np),3) means randomly sampling the k-th learning rate parameter value of the i-th learning rate parameter group from the Np learning rate parameter groups of the current generation, where 3≤Np, low_bound is the preset minimum value of the learning rate parameter, and up_bound is the maximum value of the learning rate parameter.

[0152] Optionally, after determining that the optimization termination condition is not met, the genetic algorithm processing unit further determines whether the number of learning rate parameters in the current optimal learning rate parameter group that are higher than the first set value is greater than the set threshold value. If so, the maximum value of the learning rate parameter is increased.

[0153] Optionally, the first set value can be the midpoint of the learning rate parameter's range; and / or,

[0154] The threshold value can be set as total_modules * high_value_ratio; where total_modules can be the number of convolutional groups in the first convolutional neural network, and high_value_ratio can be the proportion of learning rate parameters greater than the second set value in the learning rate parameter combination that corresponds to the best performance evaluation result among all learning rate parameter combinations in the current generation.

[0155] Optionally, the genetic algorithm processing unit may include the following steps: Training a second convolutional neural network model based on the first convolutional neural network to generate a model corresponding to the corresponding learning rate parameter set.

[0156] Training is performed on the first convolutional neural network. When the number of training iterations reaches the set second maximum number of iterations, the training ends, and the trained network model is used as the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set.

[0157] This application also provides a computer-readable storage medium that stores instructions, which, when executed by a processor, can perform the steps in the convolutional neural network weight tuning method described above. In practical applications, the computer-readable medium may be included in the devices / apparatus / systems of the above embodiments, or it may exist independently and not assembled into the device / apparatus / system. The instructions stored in the computer-readable storage medium, when executed by a processor, can perform the steps in the convolutional neural network weight tuning method described above.

[0158] According to the embodiments disclosed in this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof, but not intended to limit the scope of protection of this application. In the embodiments disclosed in this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0159] Figure 5 An electronic device is also provided for this application. For example... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0160] The electronic device may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 502 is executed, a weight tuning method for a convolutional neural network can be implemented.

[0161] Specifically, in practical applications, this electronic device may also include components such as a power supply 503 and an input / output unit 504. Those skilled in the art will understand that... Figure 5 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0162] The processor 501 is the control center of the electronic device. It connects various parts of the electronic device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, it performs various functions of the server and processes data, thereby monitoring the electronic device as a whole.

[0163] Memory 502 can be used to store software programs and modules, i.e., the aforementioned computer-readable storage medium. Processor 501 executes various functional applications and data processing by running the software programs and modules stored in memory 502. Memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 502 may also include a memory controller to provide processor 501 with access to memory 502.

[0164] The electronic device also includes a power supply 503 that supplies power to the various components. This power supply can be logically connected to the processor 501 via a power management system, enabling functions such as charging, discharging, and power consumption management. The power supply 503 may also include one or more DC or AC power supplies, a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components.

[0165] The electronic device may also include an input / output unit 504, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit output 504 can also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof.

[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for weight optimization in a convolutional neural network, characterized in that, The convolutional neural network is used to perform tasks such as image classification, object detection, image segmentation, or machine translation, including: The learning rate parameters corresponding to each specified layer of the first convolutional neural network are grouped into a learning rate parameter group, and the learning rate parameter group is used as an individual of the genetic algorithm, and the learning rate parameter is used as the gene of the genetic algorithm. The genetic algorithm is used to process all learning rate parameter sets of each generation in turn until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network. The processing of all learning rate parameter sets for each generation includes: Using all the learning rate parameter sets of the current generation, the first convolutional neural network is trained separately to generate network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets. Performance evaluation is then performed on each network model. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, the genetic algorithm is used to determine all learning rate parameter sets for the next generation. The training data used to generate the second convolutional neural network was collected in the target task scenario.

2. The weight optimization method according to claim 1, characterized in that, The process of generating network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets, and evaluating the performance of each network model, includes: For all learning parameter groups of the current generation, the second convolutional neural network is trained and generated by transfer learning based on the first convolutional neural network using each learning rate parameter group, so as to obtain the network model corresponding to each learning rate parameter group. The performance of the network model corresponding to each learning rate parameter group is evaluated separately, and the performance evaluation result of the network model corresponding to each learning rate parameter group is obtained as the performance evaluation result of each learning rate parameter group.

3. The method according to claim 1 or 2, characterized in that, The determination of whether the optimization termination condition is met includes: If the performance evaluation result corresponding to the current optimal learning rate parameter group is better than the preset target performance, or if the number of iterations of the genetic algorithm reaches the preset first maximum number of iterations, then it is determined that the optimization termination condition is met; otherwise, it is determined that the optimization termination condition is not met; wherein, the performance evaluation result corresponding to each learning rate parameter group is the performance evaluation result of the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter group.

4. The weight optimization method according to claim 3, characterized in that, Methods for determining the current optimal learning rate parameter set and its corresponding performance evaluation results include: If the current generation is the first generation, then select the learning rate parameter group with the highest performance evaluation result from all the learning rate parameter groups in the first generation as the current best learning rate parameter group; If the current generation is not the first generation, then among all the learning rate parameter sets in the current generation, if the performance evaluation result corresponding to any learning rate parameter set is better than the performance evaluation result corresponding to the current best learning rate parameter set, then that learning rate parameter set is taken as the latest current best learning rate parameter set; otherwise, the current best learning rate parameter set remains unchanged.

5. The weight optimization method according to claim 1, characterized in that, The process of determining all learning rate parameter sets for the next generation using the genetic algorithm includes: For each learning rate parameter in each learning rate parameter group of the current generation, determine whether a mutation has occurred. If so, update the corresponding learning rate parameter of the next generation; otherwise, ensure that the corresponding learning rate parameter of the next generation remains unchanged.

6. The weight optimization method according to claim 5, characterized in that, The corresponding learning rate parameters for updating the next generation include: For the i-th learning rate parameter in each learning rate parameter group, update the corresponding learning rate parameter of the next generation as follows: ;in, Indicates from the current generation The k-th learning rate parameter value of the i-th learning rate parameter group randomly selected from three learning rate parameter groups, 3≤ `low_bound` is the minimum value of the preset learning rate parameter, and `up_bound` is the maximum value of the learning rate parameter.

7. The weight optimization method according to claim 1, characterized in that, After determining that the optimization termination condition is not met, the method further includes: Determine whether the number of learning rate parameters in the current optimal learning rate parameter group that are higher than the first set value is greater than the set threshold value. If so, increase the maximum value of the learning rate parameter.

8. The weight optimization method according to claim 7, characterized in that, The first set value is the midpoint of the learning rate parameter's range; and / or, The set threshold value is Where, total_modules is the number of convolutional groups in the first convolutional neural network, and high_value_ratio is the proportion of learning rate parameters greater than the second set value in the learning rate parameter combination that corresponds to the best performance evaluation result among all learning rate parameter combinations in the current generation.

9. The weight optimization method according to claim 1, characterized in that, The step of training the first convolutional neural network to generate a network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set includes: Training is performed on the first convolutional neural network. When the number of training iterations reaches the set second maximum number of iterations, the training ends, and the trained network model is used as the network model of the second convolutional neural network corresponding to the corresponding learning rate parameter set.

10. A weight tuning device for a convolutional neural network, characterized in that, Applied to image classification, object detection, image segmentation and / or machine translation tasks, it includes: a setting unit and a genetic algorithm processing unit; The setting unit is used to form a learning rate parameter group from the learning rate parameters corresponding to each specified layer of the first convolutional neural network, and to use the learning rate parameter group as an individual of the genetic algorithm, and the learning rate parameter as the gene of the genetic algorithm. The genetic algorithm processing unit is used to process all learning rate parameter sets of each generation sequentially using the genetic algorithm until the optimization termination condition of the genetic algorithm is met, so as to determine the optimal learning rate parameter set for generating the second convolutional neural network by training the first convolutional neural network. The processing of all learning rate parameter sets for each generation includes: Using all the learning rate parameter sets of the current generation, the first convolutional neural network is trained separately to generate network models of the second convolutional neural network corresponding to the corresponding learning rate parameter sets. Performance evaluation is then performed on each network model. The performance evaluation results of the network models are used as the evaluation criteria for the corresponding learning rate parameter sets to determine the current optimal learning rate parameter set. It is then determined whether the optimization termination condition is met. If the optimization termination condition is met, the current optimal learning rate parameter set is taken as the optimal learning rate parameter set, and the network model trained using this learning rate parameter set is taken as the trained second convolutional neural network. If the optimization termination condition is not met, the genetic algorithm is used to determine all learning rate parameter sets for the next generation. The training data used to generate the second convolutional neural network was collected in the target task scenario.

11. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instruction is executed by the processor, it can implement the weight tuning method of the convolutional neural network according to any one of claims 1 to 9.

12. An electronic device, characterized in that, The electronic device includes at least a computer-readable storage medium and a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the weight tuning method for the convolutional neural network according to any one of claims 1 to 9.

Images