Evaluation model parameter weight optimization method and system
By constructing the evaluation model and adjusting the weights using reinforcement learning algorithms, the subjectivity and instability problems of parameter weight determination in the prior art are solved, and the accuracy and reliability of the evaluation model are improved.
Patent Information
- Application Number
- CN202411771213.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-11-21
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-25
AI Technical Summary
Existing evaluation models have subjectivity and instability in determining parameter weights, resulting in inaccuracies of evaluation results and injustice in decision-making, and lack the ability to dynamically adjust weights.
By building an evaluation model, defining the weight probability distribution, and adding error terms to construct a likelihood function, the weight is adjusted using reinforcement learning algorithm to maximize the likelihood of the likelihood function, thereby dynamically optimizing the parameter weights.
Improve the accuracy and reliability of the evaluation model, reduce subjectivity and instability, and can robustly obtain the optimal solution in an uncertain environment.
Smart Images
Figure CN119691589A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of testing technology, and in particular to a method and system for optimizing the weights of evaluation model parameters. Background Art
[0002] In the field of test evaluation, it is crucial to accurately evaluate and compare the performance of different products and systems. With the development of technology and the intensification of market competition, companies are increasingly in need of a scientific and objective method to evaluate and compare the performance of their products or systems. This demand has driven the development of evaluation models, which usually consist of a set of parameters and corresponding weights, and are designed to quantify the performance of the evaluation object.
[0003] Existing evaluation models evaluate the performance of products or systems by integrating multiple parameters. These parameters may include but are not limited to response time, stability, user satisfaction, cost-effectiveness, etc. Each parameter is intended to capture different aspects of the evaluation object, and the weight is used to reflect the relative importance of these parameters in the overall evaluation. For example, for an online trading platform, response time and user satisfaction may be two key parameters. For another example, for robot intelligence evaluation, robot response time and action accuracy may be two key parameters. The weight of key parameters in the evaluation model may be higher than other parameters.
[0004] Although existing evaluation models provide a means of quantitative evaluation to a certain extent, there are still challenges in determining parameter weights. The determination of weights usually relies on expert experience or simple statistical methods, which may lead to subjectivity and instability in the evaluation results. Different weight allocations may lead to completely different evaluation results, thus affecting the accuracy and fairness of decision-making. In addition, with the development of technology and changes in the market environment, the importance of parameters may change, and existing evaluation models often lack the ability to dynamically adjust weights, which affects the evaluation accuracy of the products to be evaluated. Summary of the invention
[0005] Based on the technical problems existing in the background technology, the present invention proposes a method and system for optimizing the weights of evaluation model parameters, which improves the evaluation accuracy.
[0006] The present invention proposes a method for optimizing the weight of evaluation model parameters, which arranges the trained evaluation model in a robot intelligence evaluation platform to obtain a robot intelligence evaluation;
[0007] The construction and training process of the evaluation model are as follows:
[0008] Step 1: Preliminary model construction: Select evaluation indicators based on the evaluation object, and then build an evaluation model to quantify the evaluation indicators and assign different weights;
[0009] Step 2: Further model construction: define the weight probability distribution of the evaluation model, add an error term to the evaluation model based on the deviation of the sample data and the overall noise of the evaluation model, and define a likelihood function based on the evaluation model with the error term added to represent the joint probability distribution of the evaluation index and weight under the given sample data;
[0010] Step 3: Model training: Use the reinforcement learning algorithm to adjust the weight of the evaluation model with the error term added to maximize the likelihood function value. The output evaluation model weight value when the likelihood function value is maximized is used as the optimal weight value.
[0011] Furthermore, in step one, specifically:
[0012] In the robot intelligence evaluation task, the evaluation index is constructed from the perspective of OODA loop;
[0013] The evaluation indicators are quantified and assigned different weights. A linear weighted approach is used to construct an evaluation model for the comprehensive evaluation of robot intelligence.
[0014] Furthermore, in step 2, it is assumed that the evaluation model has n evaluation indicators, i.e., prediction variables, which are represented as x1, x2, …, x n , and there are m sample data for evaluation, each evaluation indicator corresponds to a weight, expressed as W t =w1,w2,…,w n ; The likelihood function is constructed as follows:
[0015] Define the weight probability distribution of the evaluation model. The weights meet two conditions: the range of any weight is [0,1], and the sum of all weights is 1;
[0016] An error term is added to the evaluation model based on the deviation of the observed data and the overall noise of the evaluation model to obtain an evaluation model with the error term added;
[0017] y=x1*(w1+∈1)+x2*(w2+∈2)+…+x n *(w n +∈ n )+ε;
[0018] Among them, y is the comprehensive score with the error term added, that is, the response variable, ∈ i is the deviation of the i-th sample data, that is, the error term of the predictor variable, ε is the overall noise of the evaluation model, that is, the error term of the response variable, i∈[1,n];
[0019] ε and each ∈ i Assuming independent and identically distributed standard normal random variables, the likelihood function of the evaluation model is obtained.
[0020] Further, in step three, the reinforcement learning environment includes a state space, an action space, and a reward function;
[0021] The state space is defined as the configuration values of the current weights; the action space is defined as the incremental adjustments to the current weights; and the reward function is defined as the amount of improvement based on the likelihood function.
[0022] Furthermore, in step 3, the training process of the evaluation model with the error term added is as follows:
[0023] A sample data set constructed based on the m sample data in step 2 to train the evaluation model;
[0024] Define an exploration strategy to strike a balance between exploring new weight configurations and exploiting the current best weights;
[0025] The Monte Carlo method is used to sample the weight space. At each time t, an action A is randomly selected according to the current exploration strategy. t , that is, the amount of adjustment of the weight;
[0026] Execute the action and change the selected action A t Apply to the current weight and update the weight configuration W t+1 =W t +A t , W t is the weight at time t, W t+1 is the weight at time t+1;
[0027] Calculate the reward, R(w t+1 )=L(w t+1 )-L(w t ), R(w t+1 ) is the reward at time t+1, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively;
[0028] The exploration strategy is updated according to the reward feedback, and the weights are gradually updated to maximize the likelihood function value.
[0029] A system for optimizing the weight of evaluation model parameters, which places the trained evaluation model in a robot intelligence evaluation platform, thereby obtaining an intelligence evaluation of the product to be evaluated;
[0030] The construction and training process of the evaluation model are as follows:
[0031] Step 1: Select evaluation indicators based on the evaluation object, and then build an evaluation model to quantify the evaluation indicators and assign different weights;
[0032] Step 2: Define the weight probability distribution of the evaluation model, add an error term to the evaluation model based on the deviation of the sample data and the overall noise of the evaluation model, and define a likelihood function based on the evaluation model with the error term added to express the joint probability distribution of the evaluation index and the weight under the given sample data;
[0033] Step 3: Use the reinforcement learning algorithm to adjust the weight of the evaluation model with the error term added to maximize the likelihood of the likelihood function, and take the output evaluation model weight value when the likelihood is maximized as the optimal weight value.
[0034] Further, in step three, the reinforcement learning environment includes a state space, an action space, and a reward function;
[0035] The state space is defined as the configuration values of the current weights; the action space is defined as the incremental adjustments to the current weights; and the reward function is defined as the amount of improvement based on the likelihood function.
[0036] Furthermore, in step 3, the training process of the evaluation model with the error term added is as follows:
[0037] A sample data set constructed based on the m sample data in step 2 to train the evaluation model;
[0038] Define an exploration strategy to strike a balance between exploring new weight configurations and exploiting the current best weights;
[0039] The Monte Carlo method is used to sample the weight space. At each time t, an action A is randomly selected according to the current exploration strategy. t , that is, the amount of adjustment of the weight;
[0040] Execute the action and change the selected action A t Apply to the current weight and update the weight configuration W t+1 =W t +A t , W t is the weight at time t, W t+1 is the weight at time t+1;
[0041] Calculate the reward, R(w t+1 )=L(w t+1 )-L(w t ), R(w t+1 ) is the reward at time t+1, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively;
[0042] The exploration strategy is updated according to the reward feedback, and the weights are gradually updated to obtain the likelihood of maximizing the likelihood function.
[0043] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for optimizing the weights of evaluation model parameters when executing the computer program.
[0044] A computer-readable storage medium stores a plurality of classification programs, wherein the plurality of classification programs are used to be called by a processor and execute the evaluation model parameter weight optimization method as described above.
[0045] The advantages of the method and system for optimizing the weights of evaluation model parameters provided by the present invention are: the method and system for optimizing the weights of evaluation model parameters provided in the structure of the present invention apply the trial and error mechanism of reinforcement learning to the process of solving parameter estimation, that is, a likelihood function of the weights of evaluation model parameters is constructed, and then the trial and error mechanism of reinforcement learning is used to obtain the optimal parameter weights, thereby maintaining the nonlinear relationship between parameters and robustly obtaining the optimal solution in an uncertain environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a structural schematic diagram of the present invention; DETAILED DESCRIPTION
[0047] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.
[0048] like Figure 1 As shown, the present invention proposes a method for optimizing the weight of evaluation model parameters, which arranges the trained evaluation model in a robot intelligence evaluation platform to obtain a robot intelligence evaluation;
[0049] The construction and training process of the evaluation model are as follows:
[0050] Step 1: Preliminary model construction: Select evaluation indicators based on the evaluation object, and then build an evaluation model to quantify the evaluation indicators and assign different weights;
[0051] The construction of the evaluation model framework is to select appropriate evaluation indicators based on the evaluation object, and then build a mathematical model to combine the selected indicators to form a comprehensive scoring system.
[0052] The first step is to analyze and obtain the corresponding evaluation indicators based on the characteristics of the evaluation object. Specific indicators may include multiple dimensions such as performance parameters, stability, user satisfaction, and security. Evaluation indicators should be combined with specific application tasks and comprehensively cover all relevant aspects of the evaluation object to ensure the comprehensiveness and accuracy of the evaluation results.
[0053] The second step is to build a mathematical model for the evaluation. The main task is to quantify the evaluation indicators and assign different weights to reflect their relative importance in the overall intelligence evaluation. For example, the weighted sum method is used to calculate the comprehensive score:
[0054] Comprehensive score = w1*x1+w2*x2+…+w n *x n ;
[0055] Among them, x1,x2,…,x n is the evaluation index, w1, w2, w3, w4 are the weights corresponding to the respective indexes. How to solve these weights is the problem that this embodiment aims to solve.
[0056] Step 2: Further model construction: define the weight probability distribution of the evaluation model, add an error term to the evaluation model based on the deviation of the sample data and the overall noise of the evaluation model, and define a likelihood function based on the evaluation model with the error term added to express the joint probability distribution of the evaluation index and weight under the given sample data;
[0057] The likelihood function is an important concept in statistics, which is used to quantify the probability of observation data under a model parameter. The purpose of applying the likelihood function to the evaluation model is to maintain the consistency between the predicted value of the evaluation model and the actual observed value. In this embodiment, the value of the weight is estimated by maximizing the likelihood function, so as to obtain the most appropriate comprehensive score.
[0058] Assume that the evaluation model has n evaluation indicators, i.e., predictor variables, represented as x1, x2, …, x n , and there are m sample data for evaluation, each evaluation indicator corresponds to a weight, expressed as W t =w1,w2,…,w n ; The construction of likelihood function is as follows (1) to (3):
[0059] (1) Define the weight probability distribution of the evaluation model. The weight of the evaluation model is a multivariate random variable, expressed as W = w1, w2, …, w n , the probability distribution of weights can be established according to the actual problem, such as multivariate normal distribution or multivariate Beta distribution. In addition, the weights meet two conditions: the range of any weight value is [0,1], and the sum of all weights is 1.
[0060] (2) adding an error term to the evaluation model based on the deviation of the observed data and the overall noise of the evaluation model to obtain an evaluation model with the error term added;
[0061] The error term of the likelihood function comes from two aspects: on the one hand, the score of the evaluation model is a linear combination of weights and indicators, and the likelihood function is related to the deviation of the observed data and weights; on the other hand, for the overall uncertainty and noise of the evaluation model, it is necessary to use the likelihood function to add an overall error term. The evaluation model with the error term is as follows:
[0062] y=x1*(w1+∈1)+x2*(w2+∈2)+ε+x n *(w n +∈ n )+ε;
[0063] Among them, y is the comprehensive score with the error term added, that is, the response variable, ∈ i is the deviation of the i-th sample data, that is, the error term of the predictor variable, ε is the overall noise of the evaluation model, that is, the error term of the response variable, i∈[1,n];
[0064] (3) Define the likelihood function. The likelihood function L is given the evaluation index x1, x2, …, x n Next, the parameters w1, w2, …, w n The form of the likelihood function depends on the assumption of the weight distribution, which represents the joint probability distribution of the evaluation index and the weight under given sample data. Therefore, the likelihood function can be expressed as L(w1,w2,…,w n |x), where x is the sample data of the evaluation metric.
[0065] Step 3: Model training: Use the reinforcement learning algorithm to adjust the weight of the evaluation model with the error term added to maximize the likelihood of the likelihood function, and take the evaluation model weight value output when the likelihood is maximized as the optimal weight value.
[0066] Solving the weights in the likelihood function using reinforcement learning can be viewed as an optimization problem: the agent learns how to adjust the weights of the evaluation model through interaction with the environment to maximize the likelihood of the likelihood function. The following are the general steps for applying reinforcement learning to weight solving:
[0067] Weight constraints
[0068] For weight constraints, first ensure that all weights are non-negative and their sum is 1. Secondly, at the beginning, the same weight can be assigned to each indicator, that is, w i = 1 / n, where n is the total number of evaluation metrics. In addition, if there is prior knowledge about the importance of the metrics, an initial weight can be assigned to each metric based on this information.
[0069] Designing reinforcement learning environments
[0070] Define the reinforcement learning environment, including the state space (current weight configuration), action space (weight adjustment), and reward function (improvement of likelihood function value);
[0071] The state space is defined as the configuration values of the current weights:
[0072] W t =(w 1t ,w 2t ,…,w nt );
[0073] The action space, defined as the incremental adjustment of the current weights:
[0074] A t =(Δw 1t ,Δw 2t ,…,Δw nt );
[0075] Among them, Δw it is the weight w i The amount of change at time t.
[0076] The reward function is defined as the improvement based on the likelihood function, that is,
[0077] R(w t+1 )=L(w t+1 )-L(w t );
[0078] Among them, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively.
[0079] Choosing a reinforcement learning algorithm
[0080] When choosing a reinforcement learning algorithm, you can choose some classic mainstream algorithms, such as Q-learning and deep reinforcement learning. Among them, Q-learning selects the optimal action by learning the action value function, which is applicable to discrete action spaces; deep reinforcement learning (such as DQN or PPO) uses deep neural networks to approximate Q functions or policy functions, which is applicable to complex or high-dimensional state and action spaces.
[0081] The training process of the evaluation model with the error term added is as follows (3-1) to (3-5):
[0082] (3-1) Define an exploration strategy to strike a balance between exploring new weight configurations and utilizing the current optimal weights; the exploration strategy can be selected based on the specific situation, such as selecting the ε-greedy algorithm.
[0083] (3-2) Monte Carlo sampling: use the Monte Carlo method to sample the weight space. At each time t, randomly select an action A according to the current exploration strategy. t , that is, the amount of adjustment of the weight;
[0084] (3-3) Execute the action and change the selected action A t Apply to the current weight and update the weight configuration W t+1 =W t +a t , W t is the weight at time t, W t+1 is the weight at time t+1;
[0085] (3-4) Calculate the reward, R(w t+1 )=L(w t+1 )-L(w t ), R(w t+1 ) is the reward at time t+1, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively;
[0086] (3-5) Update the exploration strategy according to the reward feedback and gradually update the weights to maximize the likelihood of the likelihood function. When the likelihood function value is maximized, the weight value output by the evaluation model with the error term added is the optimal weight value.
[0087] According to steps 1 to 3, this embodiment constructs a likelihood function of the parameter weights of the evaluation model, and then uses the trial and error mechanism of reinforcement learning to obtain the optimal weights. This method not only considers the relationship between the parameters, but also dynamically optimizes the parameter weights through the dynamic adjustment mechanism of the reinforcement learning algorithm, thereby improving the accuracy and reliability of the evaluation model.
[0088] As an example: Robot intelligence evaluation
[0089] (a1) Building an intelligent assessment framework
[0090] First, in the robot intelligence evaluation task, the evaluation index is constructed from the perspective of OODA loop (observation, orientation, decision-making, execution). Among them, observation is to evaluate the accuracy of the robot's perception of the environment and tasks; orientation is to evaluate the robot's understanding of information and situational awareness; decision-making is to evaluate the robot's ability to make decisions; action is to evaluate the robot's ability to execute decisions.
[0091] Then, a mathematical model for comprehensive evaluation of robot intelligence is constructed using a linear weighted approach:
[0092] Comprehensive score = w1×observation score + w2×positioning score + w3×decision-making score + w4×execution score;
[0093] Among them, w1, w2, w3, and w4 correspond to the weights of observation, positioning, decision-making, and execution.
[0094] (a2) Constructing the likelihood function for intelligence evaluation
[0095] By adding the error term to the evaluation model, we can get:
[0096] y=x1*(w1+∈1)+x2*(w2+∈2)+x3*(w3+∈3)+x4*(w4+
[0097] ∈4)+ε;
[0098] Among them, y is the response variable, x1, x2, x3, x4 are predictor variables, w1, w2, w3, w4 are the corresponding weights, ∈1, ∈2, ∈3, ∈4 are the error terms of the predictor variables, and ε is the error term of the response variable.
[0099] Each error term is assumed to be an independent and identically distributed standard normal random variable, that is and ε~N(0,σ 2 ). Therefore, we can get the normal likelihood function of the evaluation model in the robot intelligence evaluation, which is the parameter w1,w2,w3,w4, given data y,x1,x2,x3,x4 σ 2 The function is recorded as:
[0100]
[0101] For each sample data i, it is part of the likelihood function and can be expressed as:
[0102]
[0103] Among them, y i is the observed value, is the predicted value, that is, ∑x i *(w i +∈ i );
[0104] The likelihood function for the response variable y is:
[0105]
[0106] in, is all x i The weighted sum of plus the error term ε.
[0107] Because the error terms are independently distributed, the likelihood function of the entire sample data set can be obtained:
[0108]
[0109] It can be further written as:
[0110]
[0111] In order to solve the maximum likelihood, it can be further converted to log likelihood:
[0112]
[0113] The log-likelihood is then used to solve for the parameter values.
[0114] (a3) Solving parameters using reinforcement learning
[0115] Reinforcement learning is used to solve the parameters of the likelihood function, mainly using its trial and error mechanism to adjust the weight W t The updates are performed sequentially, and finally the weight value with the maximum likelihood is output. Implementation examples are shown in (b1) to (b4):
[0116] (b1) First, initialize the weights W0 = (0.25, 0.25, 0.25, 0.25), and set a fixed step size for weight update, for example, 0.01 for each update; after each update, normalize the weight vector to ensure that the sum of all weights is 1 to maintain the rationality and consistency of the weights.
[0117] (b2) Secondly, using the current weight W t Calculate the likelihood function value (i.e., likelihood) of the evaluation model, that is, the current weight of the quantitative indicator of the degree of fit of the evaluation model to the observed data, and calculate the likelihood function value. According to the feedback of the likelihood, use the defined exploration strategy (e.g., ε-greedy algorithm) to adjust the weight.
[0118] (b3) Then, the steps of calculating the likelihood function value (i.e., likelihood) and updating the weights are repeated until a stopping condition is met. Specifically, the likelihood function value (i.e., likelihood) no longer increases significantly or reaches a preset number of iterations.
[0119] (b4) Finally, the weight corresponding to the maximum likelihood function value (i.e., likelihood) in all iterations is output as the optimal weight. The robot is evaluated based on the evaluation model with the optimal weight, accurately quantifying the performance of different systems and providing more valuable reference information for decision makers.
[0120] Therefore, this embodiment can maintain the nonlinear relationship between parameters and eliminate the deviation caused by human factors by constructing the likelihood function of the evaluation model; secondly, the method based on reinforcement learning can always obtain the optimal solution by using the trial and error mechanism, solving the problems of non-differentiability and gradient disappearance.
[0121] In addition, this embodiment combines the task of robot intelligence evaluation and accurately quantifies the performance of different systems through this method, providing more valuable reference information for decision makers.
[0122] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for optimizing the weights of evaluation model parameters, characterized in that: The trained evaluation model is placed in the robot intelligence evaluation platform to obtain the robot intelligence evaluation; The construction and training process of the evaluation model are as follows: Step 1: Preliminary model construction: Select evaluation indicators based on the evaluation object, and then build an evaluation model to quantify the evaluation indicators and assign different weights; Step 2: Further model construction: define the weight probability distribution of the evaluation model, add an error term to the evaluation model based on the deviation of the sample data and the overall noise of the evaluation model, and define a likelihood function based on the evaluation model with the error term added to represent the joint probability distribution of the evaluation index and weight under the given sample data; Step 3: Model training: Use the reinforcement learning algorithm to adjust the weight of the evaluation model with the error term added to maximize the likelihood function value. The output evaluation model weight value when the likelihood function value is maximized is used as the optimal weight value.
2. The method for optimizing the weights of evaluation model parameters according to claim 1, characterized in that: In step one, specifically: In the robot intelligence evaluation task, the evaluation index is constructed from the perspective of OODA loop; The evaluation indicators are quantified and assigned different weights. A linear weighted approach is used to construct an evaluation model for the comprehensive evaluation of robot intelligence.
3. The method for optimizing the weights of evaluation model parameters according to claim 1, characterized in that: In step 2, it is assumed that the evaluation model has n evaluation indicators, i.e., predictor variables, which are represented as x1, x2, …, x n , and there are m sample data for evaluation, each evaluation indicator corresponds to a weight, expressed as W t =w1,w2,…,w n ; The likelihood function is constructed as follows: Define the weight probability distribution of the evaluation model. The weights meet two conditions: the range of any weight is [0,1], and the sum of all weights is 1; An error term is added to the evaluation model based on the deviation of the observed data and the overall noise of the evaluation model to obtain an evaluation model with the error term added; y=x1*(w1+∈1)+x2*(w2+∈2)+…+x n *(w n +∈ n )+ε; Among them, y is the comprehensive score with the error term added, that is, the response variable, ∈ i is the deviation of the i-th sample data, that is, the error term of the predictor variable, ε is the overall noise of the evaluation model, that is, the error term of the response variable, i∈[1,n]; ε and each ∈ i Assuming independent and identically distributed standard normal random variables, the likelihood function of the evaluation model is obtained.
4. The method for optimizing the weights of evaluation model parameters according to claim 3, characterized in that: In step 3, the reinforcement learning environment includes the state space, action space, and reward function; The state space is defined as the configuration values of the current weights; the action space is defined as the incremental adjustments to the current weights; and the reward function is defined as the amount of improvement based on the likelihood function.
5. The method for optimizing the weights of evaluation model parameters according to claim 4, characterized in that: In step 3, the training process of the evaluation model with the error term added is as follows: A sample data set constructed based on the m sample data in step 2 to train the evaluation model; Define an exploration strategy to strike a balance between exploring new weight configurations and exploiting the current best weights; The Monte Carlo method is used to sample the weight space. At each time t, an action A is randomly selected according to the current exploration strategy. t , that is, the amount of adjustment of the weight; Execute the action and change the selected action A t Apply to the current weight and update the weight configuration W t+1 =W t +A t , W t is the weight at time t, W t+1 is the weight at time t+1; Calculate the reward, R(w t+1 )=L(w t+1 )-L(w t ), R(w t+1 ) is the reward at time t+1, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively; The exploration strategy is updated according to the reward feedback, and the weights are gradually updated to maximize the likelihood function value.
6. A system for optimizing the weights of evaluation model parameters, characterized in that: The trained evaluation model is placed in the robot intelligence evaluation platform to obtain the robot intelligence evaluation; The construction and training process of the evaluation model are as follows: Step 1: Select evaluation indicators based on the evaluation object, and then build an evaluation model to quantify the evaluation indicators and assign different weights; Step 2: Define the weight probability distribution of the evaluation model, add an error term to the evaluation model based on the deviation of the sample data and the overall noise of the evaluation model, and define a likelihood function based on the evaluation model with the error term added to express the joint probability distribution of the evaluation index and the weight under the given sample data; Step 3: Use the reinforcement learning algorithm to adjust the weight of the evaluation model with the error term added to maximize the likelihood of the likelihood function, and take the output evaluation model weight value when the likelihood is maximized as the optimal weight value.
7. The evaluation model parameter weight optimization system according to claim 6, characterized in that: In step 3, the reinforcement learning environment includes the state space, action space, and reward function; The state space is defined as the configuration values of the current weights; the action space is defined as the incremental adjustments to the current weights; and the reward function is defined as the amount of improvement based on the likelihood function.
8. The evaluation model parameter weight optimization system according to claim 7, characterized in that: In step 3, the training process of the evaluation model with the error term added is as follows: A sample data set constructed based on the m sample data in step 2 to train the evaluation model; Define an exploration strategy to strike a balance between exploring new weight configurations and exploiting the current best weights; The Monte Carlo method is used to sample the weight space. At each time t, an action A is randomly selected according to the current exploration strategy. t , that is, the amount of adjustment of the weight; Execute the action and change the selected action A t Apply to the current weight and update the weight configuration W t+1 =W t +A t , W t is the weight at time t, W t+1 is the weight at time t+1; Calculate the reward, R(w t+1 )=L(w t+1 )-L(w t ), R(w t+1 ) is the reward at time t+1, L(w t+1 ) and L(w t ) are the likelihood function values at time t+1 and t respectively; The exploration strategy is updated according to the reward feedback, and the weights are gradually updated to obtain the likelihood of maximizing the likelihood function.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for optimizing the weights of evaluation model parameters according to any one of claims 1 to 5 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the evaluation model parameter weight optimization method as described in any one of claims 1-5.