Model training parameter determination method and device, medium and electronic equipment
Through iterative search and the use of parameter matrix, the neural network model training parameters are automatically determined, which solves the problem of parameter settings affecting model performance in the existing technology, and achieves more efficient and accurate model training parameter settings.
Patent Information
- Application Number
- CN202311308117.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-10
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, model training parameters are usually set as default values or set by human experience, resulting in the impact of model performance.
By obtaining the optional parameter value of each pending parameter, the parameter matrix is determined based on the optional parameter value of multiple pending parameters, and iterative search of parameter values is performed based on the matrix, and the target model is trained in combination with business data until the fitness data reaches a stable state, and the target parameter value of the pending parameter is determined.
It improves the accuracy and efficiency of model training parameter settings, reduces the cost of model training parameters, and thus improves model performance.
Smart Images
Figure CN119808865A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep learning technology, and in particular to a method, device, medium and electronic device for determining model training parameters. Background Art
[0002] Neural network models have very powerful computing capabilities and are widely used in various fields. The application process of neural network models includes training and derivation stages. In the training stage, the selected neural network model needs to be trained through model training parameters to obtain a trained neural network model. In the derivation stage, the trained neural network model is used to derive the data to be processed.
[0003] In the process of implementing the present invention, it is found that there are at least the following technical problems in the prior art: in the training stage, the model training parameters are generally set as default values, or are set according to human experience, and the neural network model is trained by the model training parameters determined by the above method, which affects the model performance. Summary of the invention
[0004] The present invention provides a method, device, medium and electronic device for determining model training parameters, so as to determine model training parameters suitable for a target model and improve model performance.
[0005] According to one aspect of the present invention, a method for determining model training parameters is provided, comprising:
[0006] Obtaining an optional parameter value for each undetermined parameter, and determining a parameter matrix based on a plurality of optional parameter values for the undetermined parameters;
[0007] An iterative search for parameter values is performed on the multiple pending parameters based on the parameter matrix, and a target model is trained based on the parameter group and business data obtained in each iteration to obtain fitness data corresponding to the parameter group. When the fitness data reaches a stable state, target parameter values of the multiple pending parameters are determined based on the parameter group in the stable state.
[0008] Optionally, an optional parameter value of each pending parameter is obtained, and a parameter matrix is determined based on the optional parameter values of the multiple pending parameters, including: for each pending parameter, based on the step size of the pending parameter, setting multiple optional parameter values; normalizing the optional parameter value of each pending parameter to obtain a normalized parameter value of the pending parameter, and forming a parameter matrix based on the normalized parameter values of the multiple pending parameters.
[0009] Optionally, determining target parameter values of multiple pending parameters based on the parameter group in the stable state includes: determining optional parameter values corresponding to parameter values in the parameter group based on a mapping relationship between optional parameter values of the pending parameters and normalized parameter values in the parameter matrix, and determining the optional parameter values corresponding to the parameter values in the parameter group as target parameter values.
[0010] Optionally, the iterative search for parameter values of the multiple pending parameters based on the parameter matrix includes: creating multiple parameter search points, determining the initial positions of the parameter search points in the parameter matrix, iteratively performing the following search process to update the positions of the parameter search points in the parameter matrix: determining an update strategy for the parameter search points, updating the current positions of the parameter search points in the parameter matrix based on the update strategy; and randomly updating the positions of the updated local parameter search points.
[0011] Optionally, the parameter search point includes a first search point and a second search point that satisfy a preset ratio;
[0012] The update strategy of the first search point includes: when a first warning condition is met, a global search is performed on the current position of the first search point in the parameter matrix, and the nearest parameter position of the searched position information in the parameter matrix is determined as the updated position; when a second warning condition is met, the current position of the first search point is updated based on a normal distribution.
[0013] Optionally, the update strategy of the second search point includes: when the second search point satisfies a first condition, determining an update position of the second search point based on a current position of the second search point and a position corresponding to historical minimum fitness data; and when the second search point satisfies a second condition, updating the position of the second search point to be within the update position range of the first search point.
[0014] Optionally, the position of the updated local parameter search point is randomly updated, including: updating the position of the local parameter search point to a random position in the parameter matrix; or, based on the current position of the local parameter search point and the corresponding position of the historical maximum fitness data, determining the updated position of the local parameter search point.
[0015] According to another aspect of the present invention, there is provided a device for determining model training parameters, comprising:
[0016] An optional parameter processing module, used for obtaining an optional parameter value of each undetermined parameter, and determining a parameter matrix based on the optional parameter values of the undetermined parameters;
[0017] A parameter search module is used to iteratively search for parameter values of the multiple undetermined parameters based on the parameter matrix, train the target model based on the parameter group and business data obtained in each iteration, obtain fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, determine the target parameter values of the multiple undetermined parameters based on the parameter group in the stable state.
[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0019] at least one processor; and
[0020] a memory communicatively connected to the at least one processor; wherein,
[0021] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the model training parameters described in any embodiment of the present invention.
[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining model training parameters described in any embodiment of the present invention when executed.
[0023] The technical solution of the embodiment of the present invention sets the optional parameter values of the pending model training parameters, and performs parameter search in the optional parameter values through the group intelligence algorithm to obtain the target parameter values of each model training parameter suitable for the target model. By replacing the manual setting of model training parameters with the automatic search method, the accuracy and efficiency of model training parameter setting are improved, and the setting cost of model training parameters is reduced.
[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0026] Figure 1 is a flow chart of a method for determining model training parameters provided by an embodiment of the present invention;
[0027] Figure 2 is a flow chart of a method for determining model training parameters provided by an embodiment of the present invention;
[0028] Figure 3 It is a structural schematic diagram of a device for determining model training parameters provided by an embodiment of the present invention;
[0029] Figure 4 It is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0032] Figure 1 This is a flow chart of a method for determining model training parameters provided by an embodiment of the present invention. This embodiment is applicable to the case where model training parameters applicable to a target model are determined by parameter search. This method can be executed by a device for determining model training parameters. The device for determining model training parameters can be implemented in the form of hardware and / or software. The device for determining model training parameters can be configured in electronic devices such as mobile phones, computers, servers, etc. Figure 1 As shown, the method includes:
[0033] S110: Obtain an optional parameter value for each undetermined parameter, and determine a parameter matrix based on a plurality of optional parameter values for the undetermined parameters.
[0034] S120, performing an iterative search for parameter values of the multiple pending parameters based on the parameter matrix, training the target model based on the parameter group and business data obtained in each iteration, obtaining fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, determining target parameter values of the multiple pending parameters based on the parameter group in the stable state.
[0035] In this embodiment, the undetermined parameters are undetermined model training parameters, which are hyperparameters required in the neural network training process, but are not limited to learning rate, batch size, cycle, regularization strength, etc. Optionally, for different target models, the target parameter values of the undetermined parameters applicable to the target model can be determined respectively, so as to perform model training on different target models and obtain high-performance target models. Optionally, for each target model, the target parameter values of the undetermined parameters applicable to the target model are determined based on the business data in different application scenarios, so as to perform model training on the target model in different application scenarios, so as to train high-performance target models in different application scenarios.
[0036] In some embodiments, a generation interface of model training parameters is displayed, and the generation interface of model training parameters includes a target model selection area, and multiple model types can be displayed in the target model selection area. In response to the selection operation of the model type, the model type of the target model can be determined, and the pre-stored target model can be called according to the selected model type. Among them, the model type of the target model includes but is not limited to convolutional neural network CNN, recurrent neural network RNN, long short-term memory neural network LSTM, generative adversarial network GAN, support vector machine SVM, etc. The model type of the target model can be updated according to demand, such as adding a new model, etc. The new model can be a model built by the user according to business needs.
[0037] The model training parameter generation interface also includes a parameter selection area, in which multiple pending parameters can be displayed, and in response to the selection operation of the pending parameters, the pending parameters for parameter search are determined. Among them, the parameter selection area can include a select all control, which is convenient for one-click selection of all pending parameters in the parameter selection area.
[0038] Each undetermined parameter is displayed in the generation interface of the model training parameters, and a step size setting area is displayed at the corresponding position of each undetermined parameter, for example, below or to the right of the undetermined parameter. The step size setting area may display a default step size of each undetermined parameter, which may be pre-set. In response to the editing operation of the initial step size, the default step size is updated. The step sizes of different undetermined parameters may be different.
[0039] For each undetermined parameter, a plurality of optional parameter values are set based on the step size of the undetermined parameter, wherein the number of optional parameter values of different undetermined parameters is the same, and a target parameter value of the undetermined parameter is searched among the plurality of optional parameter values.
[0040] Since the step sizes and data ranges of different parameters to be determined are different, the optional parameter values of each parameter to be determined are normalized to obtain the normalized parameter values of the parameter to be determined, and a parameter matrix is formed based on the normalized parameter values of the multiple parameters to be determined. The optional parameter values of each parameter to be determined are normalized and converted into normalized parameter values within the range of 0-1. The normalized parameter values of each parameter to be determined are used as matrix rows of the parameter matrix to form a parameter matrix.
[0041] Exemplarily, the following matrix includes optional parameter values for undetermined parameters:
[0042] Among them, the pending parameters are A, B...G, the step size of the pending parameter A is x, the step size of the pending parameter B is y, and so on.
[0043] By normalizing the optional parameter values of each undetermined parameter, the obtained parameter matrix is as follows:
[0044] Where d is the number of optional parameter values and n is the number of undetermined parameters.
[0045] Through the swarm intelligence search algorithm, parameter search is performed in the above parameter matrix. In some embodiments, multiple swarm intelligence algorithms can be pre-packaged and stored, and different swarm intelligence algorithms can be called to perform parameter search according to parameter search requirements. Among them, swarm intelligence algorithms include ant colony algorithm, bee colony algorithm, sparrow search algorithm, etc. Correspondingly, the generation interface of model training parameters can also include an algorithm selection operation. In response to the algorithm selection operation, parameter search is performed according to the selected swarm intelligence algorithm to determine the target parameter value of the parameter to be determined.
[0046] The above parameter search process is an iterative search process. After each iteration, a set of parameter values is obtained, which includes multiple normalized parameter values. Based on the parameter group, it is determined whether the search end condition is met. If the search end condition is met, the iterative process is terminated, and the target parameter value of the pending parameter is determined based on the parameter group. If the search end condition is not met, the next iterative process is continued.
[0047] Among them, after each iteration, the fitness data is calculated based on the obtained parameter group, and accordingly, the search end condition fitness data reaches a stable state. The stable state is determined by the convergence state of the fitness data. When the fitness data reaches the convergence state, it is determined that the stable state is reached. The fitness data can be determined by: obtaining the parameter group obtained in the current iteration, determining the optional parameter value corresponding to the pending parameter determined by the parameter group in the current iteration, and using it as the intermediate parameter value of the pending parameter in the current iteration. The target model is iteratively trained based on the intermediate parameter value of the pending parameter and the business data. During the training process of the target model, it is determined whether the target model has reached a convergence state based on the loss function. When the convergence state is reached, the convergence data is determined as the fitness data corresponding to the parameter group obtained in the current iteration.
[0048] Based on the above method, in each iteration process, a parameter group and fitness data corresponding to the parameter group are obtained. When the fitness data of multiple iterations reaches a converged state, that is, a stable state, the iteration is terminated. Accordingly, the target parameter values of the multiple parameters to be determined are determined based on the parameter group in the stable state, wherein the intermediate parameter values corresponding to the parameter group and the target parameter values corresponding to the parameter group are determined in the same manner. Taking the target parameter value as an example, the method for determining the target parameter value includes: determining the optional parameter value corresponding to the parameter value in the parameter group based on the mapping relationship between the optional parameter value of the parameter to be determined and the normalized parameter value in the parameter matrix, and determining the optional parameter value corresponding to the parameter value in the parameter group as the target parameter value.
[0049] Taking the matrix formed by the optional parameter values and the parameter matrix formed by the normalized parameters as an example, the optional parameter values and the normalized parameters at the same position have a mapping relationship, such as A0 and x 1,1 The corresponding optional parameter value is determined according to the above mapping relationship and the normalized parameter in the parameter group as the target parameter value.
[0050] The target parameter value corresponding to each undetermined parameter can be determined as a model training parameter suitable for the target model and / or the application scenario of the business data. The target parameter value of the model training parameter, the type of the target model and the application scenario of the business data can be stored in association, and before the subsequent model training, the type of the target model and / or the application scenario can be queried to obtain the corresponding model training parameter, without the need for repeated model training parameter determination process, thereby improving the performance of the trained model.
[0051] The technical solution provided in this embodiment sets the optional parameter values of the pending model training parameters, and searches the parameters in the optional parameter values through the group intelligence algorithm to obtain the target parameter values of each model training parameter suitable for the target model. By replacing the manual setting of model training parameters with the automatic search method, the accuracy and efficiency of model training parameter setting are improved, and the setting cost of model training parameters is reduced.
[0052] Figure 2 It is a flowchart of a method for determining model training parameters provided by an embodiment of the present invention. On the basis of the above embodiment, the iterative search process of parameter values is refined. Optionally, the iterative search of parameter values for the multiple undetermined parameters based on the parameter matrix includes: creating multiple parameter search points, determining the initial position of the parameter search point in the parameter matrix, iteratively performing the following search process to update the position of the parameter search point in the parameter matrix: determining the update strategy of the parameter search point, updating the current position of the parameter search point in the parameter matrix based on the update strategy; and randomly updating the position of the updated local parameter search point. Accordingly, the method specifically includes:
[0053] S210: Obtain an optional parameter value for each undetermined parameter, and determine a parameter matrix based on the optional parameter values of the undetermined parameters.
[0054] S220: Create multiple parameter search points, and determine initial positions of the parameter search points in the parameter matrix.
[0055] S230: Determine an update strategy for the parameter search point, and update a current position of the parameter search point in the parameter matrix based on the update strategy.
[0056] S240: Randomly update the position of the updated local parameter search point.
[0057] S250. Train the target model based on the parameter group and business data obtained in the current iteration to obtain fitness data corresponding to the parameter group. When the fitness data reaches a stable state, execute step S260. When the fitness data does not reach a stable state, execute step S230 to perform the next iteration.
[0058] S260: Determine target parameter values of a plurality of the to-be-determined parameters based on the parameter group in the stable state.
[0059] In this embodiment, multiple parameter search points are created, and during the iterative search process, the parameter search points are changed in position in the parameter matrix, wherein the parameter values corresponding to the determined positions of the multiple parameter search points at each iteration form a parameter group. The number of parameter search points may be the same as the number of parameters to be determined.
[0060] A plurality of update strategies and conditions corresponding to each update strategy are pre-set, and in each iteration, the applicable update strategy is determined by the conditions, and the update strategy is executed to perform parameter update. Optionally, the above conditions include but are not limited to early warning conditions, search point determination conditions, etc.
[0061] The parameter search points include a first search point and a second search point that satisfy a preset ratio; wherein the first search point and the second search point have different corresponding update strategies, and the preset ratio of the first search point and the second search point may be 1:4. The number of the first search points and the number of the second search points are determined according to the preset ratio and the number of the parameter search points. The first search point is randomly determined among all the parameter search points based on the number of the first search points, and the remaining parameter search points may be used as the second search points.
[0062] Optionally, the update strategy for the first search point includes: when a first warning condition is met, performing a global search for the current position of the first search point in the parameter matrix, and determining the nearest parameter position of the searched position information in the parameter matrix as the updated position; when a second warning condition is met, updating the current position of the first search point based on a normal distribution.
[0063] In each iteration, a warning value is randomly determined within the warning value range, and a safety value is randomly determined within the safety value range, where the warning value range can be [0,1], and the safety value range can be [0.5,1]. The warning value is compared with the safety value. When the warning value is less than the safety value, the first warning condition is met, and when the warning value is greater than or equal to the safety value, the second warning condition is met.
[0064] Under the first warning condition, a global search is performed in the parameter matrix based on the current position of the first search point, wherein a calculation method of the global search is preset. Since the position obtained by searching the current position through the global search falls between the parameter values of the parameter matrix, but does not fall at the position where the parameter value of the parameter matrix is located, for the above situation, the nearest parameter position of the position information searched in the parameter matrix is determined, and the nearest parameter position is determined as the position of the first search point after being updated in the current iteration. For example, the distance between the searched position information and each parameter position in the parameter matrix can be calculated, the distances can be sorted, and the parameter position corresponding to the minimum distance can be determined as the nearest parameter position.
[0065] Under the second warning condition, the current position of the first search point is updated by using a random number in a normal distribution and a preset one-dimensional vector. For example, the update strategy of the first search point is as follows:
[0066]
[0067] in, is the position of the i-th first search point in the j-th dimension in the t-th iteration. is a global search method, where m is the current iteration number, iterMAX is the maximum iteration number, and α is a random number in the range [0,1]. Q is a random number that follows a normal distribution, L is a 1×d one-dimensional matrix, R is the warning value, and ST is the safety value. mindis(X x±d,j±d ) characterization will be through The determined position is updated as the latest parameter position. It can be understood that the above iterative process is applicable to the first search point.
[0068] Optionally, the update strategy for the second search point includes: when the second search point satisfies a first condition, determining an update position of the second search point based on a current position of the second search point and a position corresponding to historical minimum fitness data; and when the second search point satisfies a second condition, updating the position of the second search point to be within the update position range of the first search point.
[0069] The attribute parameter i of the second search point is obtained, and the attribute parameter may be a sequence number of the second search point. When the attribute parameter i is greater than a preset threshold, such as n / 2, it is determined that the first condition is satisfied, and when the attribute parameter i is less than or equal to the preset threshold, it is determined that the second condition is satisfied, wherein n is the number of parameter search points.
[0070] Under the first condition, the position corresponding to the historical minimum fitness data of the second search point is determined. The current iteration number is t+1, and the corresponding historical iteration number is 1-t. The fitness data of the historical iteration numbers are compared to determine the minimum fitness data. The parameter group corresponding to the iteration number of the minimum fitness data is obtained, and the parameter value of the second search point is determined in the above parameter group, that is, the position corresponding to the historical minimum fitness data of the second search point.
[0071] The update strategy for the first condition is to determine the difference between the historical minimum fitness data and the current position of the second search point, determine the ratio data of the above difference and the square of the current number of iterations, and determine the updated position of the second search point based on the product of a normally distributed random number and the above ratio data.
[0072] Under the second condition, the update position range of the first search point in the current iteration is determined, and the update position range of the first search point may be a preset range centered on the update position of the first search point, and the update position of the second search point is determined as a random position within the update position range of the first search point. Optionally, in the case where there are multiple first search points, the first search point closest to the current position of the second search point is determined based on the update position of the first search point, and the update position of the second search point is determined to be within the update position range of the closest first search point. Optionally, in the case where there are multiple first search points, the first search point that matches each second search point is determined based on a polling method, and the update position of the second search point is determined to be within the update position range of the matching first search point.
[0073] Exemplarily, the update strategy of the second search point may be the following formula:
[0074]
[0075] Among them, X worst is the position corresponding to the historical minimum fitness data, is the position of the i-th second search point in the j-th dimension in the t-th iteration, and Q is a random number that conforms to the normal distribution.
[0076] On the basis of the above embodiment, in order to prevent the undetermined parameters from falling into the local optimum during the iterative search process, the updated local parameter search points are randomly updated in position to improve the randomness and generalization of the parameter search process.
[0077] Optionally, the local parameter search point may be a preset proportion of parameter search points, for example, the preset proportion may be 20%. The preset proportion of parameter search points is randomly determined among all parameter search points as the local parameter search point. Optionally, the local parameter search point may be a preset proportion of second search points, that is, a preset proportion of second search points is randomly determined among all second search points as the local parameter search point. Optionally, the distance between the second search point and the first search point (any first search point or the nearest first search point) may be determined, the second search points are sorted based on the distance, and the preset proportion of second search points are selected as the local parameter search point based on the sorting.
[0078] Optionally, the position of the updated local parameter search point is randomly updated, including: updating the position of the local parameter search point to a random position in the parameter matrix; or, based on the current position of the local parameter search point and the corresponding position of the historical maximum fitness data, determining the updated position of the local parameter search point.
[0079] In this embodiment, the position of the local parameter search point is updated twice, wherein the second update is used to increase the randomness of the position. Optionally, a random position is determined within the range of the parameter matrix as the position of the local parameter search point in the current iteration. Optionally, the position corresponding to the historical maximum fitness data of the local parameter search point is determined, the current iteration number is t+1, and correspondingly, the historical iteration number is 1 to t, and the fitness data of the historical iteration numbers are compared to determine the maximum fitness data. Obtain a parameter group corresponding to the iteration number of the maximum fitness data, and determine the parameter value of the local parameter search point in the above parameter group, that is, the position corresponding to the historical minimum fitness data of the second search point. Determine the absolute value of the difference between the current position of the local parameter search point and the position corresponding to the historical maximum fitness data, and determine the sum of the position corresponding to the historical maximum fitness data and the above absolute value of the difference as the position of the local parameter search point in the current iteration.
[0080] In some embodiments, the position of the local parameter search point can be randomly updated based on any method. In some embodiments, the local parameter search point can be divided into two parts, i.e., the first local parameter search point and the second local parameter search point, by the distance between them and the first search point. The first local parameter search point and the second local parameter search point are randomly updated in different sampling methods to further increase randomness. The number of the first local parameter search point and the second local parameter search point can be the same.
[0081] For example, the random location update method may be as follows:
[0082]
[0083] in, is the position corresponding to the historical maximum fitness data, β is a random number, random(X 00, ,X nd, ) is the parameter matrix range. 10 is the first local parameter search point, l 10～20 is the second local parameter search point, wherein, when the local parameter search point accounts for 20%, the first local parameter search point and the second local parameter search point account for 10%.
[0084] The technical solution of this embodiment sets optional parameter values of the pending model training parameters, performs parameter search among the optional parameter values, and obtains target parameter values of each model training parameter suitable for the target model. In the process of parameter search, different update strategies are used for parameter search, and some parameter search points are further subjected to position random processing to improve the accuracy of parameter search and avoid local optimal situations. By replacing the manual setting of model training parameters with automatic search, the accuracy and efficiency of model training parameter setting are improved, and the setting cost of model training parameters is reduced.
[0085] Figure 3 Schematic diagram of a device for determining model training parameters provided by an embodiment of the present invention. Figure 3 As shown, the device comprises:
[0086] An optional parameter processing module 310 is used to obtain an optional parameter value of each undetermined parameter and determine a parameter matrix based on the optional parameter values of the undetermined parameters;
[0087] The parameter search module 320 is used to iteratively search for parameter values of the multiple pending parameters based on the parameter matrix, train the target model based on the parameter group and business data obtained in each iteration, obtain the fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, determine the target parameter values of the multiple pending parameters based on the parameter group in the stable state.
[0088] The technical solution of this embodiment sets the optional parameter values of the pending model training parameters, and performs parameter search in the optional parameter values through the group intelligence algorithm to obtain the target parameter values of each model training parameter suitable for the target model. By replacing the manual setting of model training parameters with the automatic search method, the accuracy and efficiency of model training parameter setting are improved, and the setting cost of model training parameters is reduced.
[0089] On the basis of the above embodiment, optionally, the optional parameter processing module 310 is used to: for each undetermined parameter, set multiple optional parameter values based on the step size of the undetermined parameter; normalize the optional parameter value of each undetermined parameter to obtain the normalized parameter value of the undetermined parameter, and form a parameter matrix based on the normalized parameter values of the multiple undetermined parameters.
[0090] On the basis of the above embodiment, optionally, the parameter search module 320 is used to: determine the optional parameter value corresponding to the parameter value in the parameter group based on the mapping relationship between the optional parameter value of the to-be-determined parameter and the normalized parameter value in the parameter matrix, and determine the optional parameter value corresponding to the parameter value in the parameter group as the target parameter value.
[0091] Based on the above embodiment, optionally, the parameter search module 320 is further used to:
[0092] Create multiple parameter search points, determine the initial positions of the parameter search points in the parameter matrix, and iteratively perform the following search process to update the positions of the parameter search points in the parameter matrix: determine an update strategy for the parameter search points, and update the current positions of the parameter search points in the parameter matrix based on the update strategy; and randomly update the positions of the updated local parameter search points.
[0093] Optionally, the parameter search point includes a first search point and a second search point that satisfy a preset ratio.
[0094] Optionally, the parameter search module 320 is also used to: when the first warning condition is met, perform a global search for the current position of the first search point in the parameter matrix, and determine the nearest parameter position of the searched position information in the parameter matrix as the updated position; when the second warning condition is met, update the current position of the first search point based on the normal distribution.
[0095] Optionally, the parameter search module 320 is also used to: when the second search point satisfies the first condition, determine the update position of the second search point based on the current position of the second search point and the corresponding position of the historical minimum fitness data; when the second search point satisfies the second condition, update the position of the second search point to be within the update position range of the first search point.
[0096] Optionally, the parameter search module 320 is further used to: update the position of the local parameter search point to a random position in the parameter matrix; or determine the update position of the local parameter search point based on the current position of the local parameter search point and the corresponding position of the historical maximum fitness data.
[0097] The device for determining model training parameters provided in the embodiment of the present invention can execute the method for determining model training parameters provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0098] Figure 41 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0099] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0100] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0101] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a method for determining model training parameters.
[0102] In some embodiments, the method for determining model training parameters may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining model training parameters described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for determining model training parameters in any other appropriate manner (e.g., by means of firmware).
[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0104] The computer program for implementing the method for determining the model training parameters of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0105] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a processor to execute a method for determining a model training parameter, the method comprising:
[0106] Obtain optional parameter values for each pending parameter, and determine a parameter matrix based on the optional parameter values of the plurality of pending parameters; perform iterative search for parameter values for the plurality of pending parameters based on the parameter matrix, train a target model based on the parameter group and business data obtained in each iteration, and obtain fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, determine target parameter values for the plurality of pending parameters based on the parameter group in the stable state.
[0107] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0108] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0109] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0110] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0111] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0112] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for determining model training parameters, characterized in that: include: Obtaining an optional parameter value for each undetermined parameter, and determining a parameter matrix based on a plurality of optional parameter values for the undetermined parameters; An iterative search for parameter values of the plurality of pending parameters is performed based on the parameter matrix, and a target model is trained based on the parameter group and business data obtained in each iteration to obtain fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, target parameter values of the plurality of pending parameters are determined based on the parameter group in the stable state.
2. The method according to claim 1, characterized in that: Obtaining an optional parameter value for each undetermined parameter, and determining a parameter matrix based on a plurality of optional parameter values for the undetermined parameters, including: For each undetermined parameter, setting a plurality of optional parameter values based on the step size of the undetermined parameter; Normalizing the optional parameter value of each of the undetermined parameters to obtain a normalized parameter value of the undetermined parameter, and forming a parameter matrix based on a plurality of normalized parameter values of the undetermined parameters.
3. The method according to claim 2, characterized in that The step of determining target parameter values of a plurality of the to-be-determined parameters based on the parameter group in the stable state comprises: Based on the mapping relationship between the optional parameter value of the undetermined parameter and the normalized parameter value in the parameter matrix, the optional parameter value corresponding to the parameter value in the parameter group is determined, and the optional parameter value corresponding to the parameter value in the parameter group is determined as the target parameter value.
4. The method according to claim 1, characterized in that: The iterative search of parameter values for the plurality of undetermined parameters based on the parameter matrix comprises: Create multiple parameter search points, determine the initial positions of the parameter search points in the parameter matrix, and iteratively perform the following search process to update the positions of the parameter search points in the parameter matrix: Determine an update strategy for the parameter search point, and update a current position of the parameter search point in the parameter matrix based on the update strategy; The updated local parameter search points are randomly updated in position.
5. The method according to claim 4, characterized in that The parameter search point includes a first search point and a second search point that satisfy a preset ratio; and an update strategy for the first search point includes: In the case where the first warning condition is met, a global search is performed on the current position of the first search point in the parameter matrix, and the nearest parameter position of the searched position information in the parameter matrix is determined as the updated position; When the second warning condition is met, the current position of the first search point is updated based on the normal distribution.
6. The method according to claim 5, characterized in that The updating strategy of the second search point includes: In the case where the second search point satisfies the first condition, determining an updated position of the second search point based on a current position of the second search point and a position corresponding to historical minimum fitness data; When the second search point satisfies the second condition, the position of the second search point is updated to be within the update position range of the first search point.
7. The method according to claim 4, characterized in that The updated local parameter search point is randomly updated, including: updating the position of the local parameter search point to a random position in the parameter matrix; or, Based on the current position of the local parameter search point and the corresponding position of the historical maximum fitness data, an update position of the local parameter search point is determined.
8. A device for determining model training parameters, characterized in that: include: An optional parameter processing module, used for obtaining an optional parameter value of each undetermined parameter, and determining a parameter matrix based on the optional parameter values of the undetermined parameters; A parameter search module is used to iteratively search for parameter values of the multiple undetermined parameters based on the parameter matrix, train the target model based on the parameter group and business data obtained in each iteration, obtain the fitness data corresponding to the parameter group, and when the fitness data reaches a stable state, determine the target parameter values of the multiple undetermined parameters based on the parameter group in the stable state.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining model training parameters according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for determining model training parameters according to any one of claims 1 to 7 when executed.