Intelligent agent multi-index comprehensive evaluation method based on result-oriented reward function

Abstract

An agent multi-index comprehensive evaluation method based on a result-oriented reward function relates to the field of reinforcement learning agent performance evaluation, and mainly comprises the following steps: constructing a test case set and collecting data; selecting representative test cases, running the intelligent agent and recording a reward value of each test case; calculating an original value of the core evaluation index according to the reward value data and a preset failure threshold value; performing index hundred-differentiation treatment; and evaluating and visualizing the comprehensive capability and the like. According to the method, the strong correlation characteristic of reward value-task result is fully utilized, failure and non-failure cases are divided through failure threshold values, each evaluation index is pertinently set and calculated, it is ensured that the evaluation result is highly matched with the actual task completion effect of the intelligent agent, the problems that an existing method is single in index and poor in scene adaptability are solved, and the method has good application prospects. And the comprehensive capability of the intelligent agent can be reflected more accurately.

Description

Technical Field

[0001] This invention relates to the field of performance evaluation technology for reinforcement learning agents, and in particular to a multi-index comprehensive evaluation method for agents based on a result-oriented reward function. Background Technology

[0002] In the current era of rapid development in artificial intelligence, reinforcement learning, as an important machine learning paradigm, continuously optimizes its strategies through trial and error in the interaction between intelligent agents and the environment to achieve goals. It has been widely applied in various fields such as games, robot control, path planning, and industrial optimization. Performance evaluation of reinforcement learning agents is a core step in determining the effectiveness of algorithms, guiding iterative policy improvements, and ensuring the reliable operation of intelligent agents in real-world scenarios.

[0003] However, current evaluation methods for reinforcement learning agents still have many shortcomings and are difficult to meet the evaluation requirements of outcome-oriented reward function tasks: Evaluation metrics are too simplistic and one-sided: Most existing methods rely on only a few metrics such as average reward value and task success rate to evaluate agent performance. For example, in game-like tasks, judging an agent's performance solely by its average score cannot fully reflect the differences in the agent's performance in different test scenarios, such as failure scenarios in complex environments and long-term operational stability.

[0004] Insufficient scenario adaptability: Existing evaluation methods do not differentiate between reward function types. For result-oriented reward function tasks (such as scoring points by eating dots in Pac-Man, or reward values ​​for target capture in target tracking tasks, where the reward value directly reflects the final task completion effect), existing methods cannot fully utilize the strong correlation between "reward value - task result," leading to a disconnect between evaluation results and the agent's actual task completion ability.

[0005] The evaluation results lack visualization: Existing evaluation methods mostly output scattered numerical values, lacking an intuitive and comprehensive representation of capabilities. Technical personnel struggle to quickly and clearly grasp the agent's strengths and weaknesses across various dimensions, and are unable to efficiently compare performance differences between different agents, causing inconvenience for the iterative optimization of reinforcement learning algorithms and the selection and application of agents.

[0006] Therefore, for reinforcement learning tasks that employ outcome-oriented reward functions, there is an urgent need for a method that can comprehensively, accurately, and intuitively evaluate the overall capabilities of an agent in order to address the aforementioned shortcomings of existing technologies. Summary of the Invention

[0007] The purpose of this disclosure is to overcome the problems of single indicators, poor scenario adaptability, and low degree of result visualization in existing reinforcement learning agent evaluation methods. It provides a comprehensive evaluation method for agents based on result-oriented reward functions, which can achieve a comprehensive quantitative evaluation and intuitive visualization of the comprehensive capabilities of reinforcement learning agents in result-oriented reward function tasks, and provide a reliable basis for agent performance analysis and algorithm optimization.

[0008] This disclosure provides a multi-index comprehensive evaluation method for intelligent agents based on a result-oriented reward function, comprising the following steps: S1, Test case set construction; S2, obtain the final reward value of the reinforcement learning agent to be evaluated under each test case; S3, Preset reward value invalidation threshold T ; S4. Based on the reward value data of the test case set and the preset failure threshold, calculate the evaluation indicators, which include: Average agent performance is used to evaluate the overall average performance level of the agent across all test scenarios. Completion quality is used to evaluate the performance quality of an agent in test scenarios where it successfully completes a task; Failure severity is used to assess the degree of failure of an agent in a test scenario where the task is not completed; Failure risk is used to assess the potential risk of an agent failing in a test scenario; Stability is used to evaluate the stability of the agent's reward value under different test scenarios; Failure rate percentage is used to evaluate the proportion of times an agent fails to achieve the expected results in a test scenario; S5, Standardization and comprehensive evaluation of evaluation indicators.

[0009] Compared with the prior art, the beneficial effects of this disclosure are: (1) Comprehensive evaluation indicators: By constructing six core indicators, namely, average performance of the agent, completion quality, failure degree, failure risk, stability, and failure rate percentage, the performance of the agent is comprehensively evaluated from six dimensions: overall performance, quality of effective scenarios, severity of failure scenarios, potential risks, performance stability, and task effectiveness. This overcomes the shortcomings of existing methods with single indicators and can more accurately reflect the comprehensive capabilities of the agent.

[0010] (2) Precise scenario adaptability: It is designed specifically for reinforcement learning tasks with result-oriented reward functions. It makes full use of the strong correlation between "reward value and task result", divides failed and non-failed use cases by failure threshold, and calculates each index in a targeted manner to ensure that the evaluation results are highly consistent with the actual task completion effect of the agent, thus solving the problem of poor scenario adaptability of existing methods.

[0011] (3) Visualizing the evaluation results: With the help of the six-dimensional radar chart, the scattered index values ​​are transformed into intuitive graphical displays, which allows technicians to quickly grasp the performance advantages and disadvantages of the agent. At the same time, the average capability value realizes the quantification of comprehensive performance, which facilitates the performance comparison and classification of different agents, and provides a clear and efficient decision-making basis for algorithm optimization and agent selection.

[0012] (4) The method has strong versatility and scalability: parameters such as failure threshold, test case set, theoretical maximum / minimum reward value, etc. can be flexibly adjusted according to the specific result-oriented task requirements, and are applicable to various result-oriented reinforcement learning scenarios such as games, robot control, and logistics scheduling; in addition, if new evaluation indicators (such as task completion time) need to be added, they can be easily extended based on the existing radar chart and average capability value calculation logic, which has strong flexibility and scalability. Attached Figure Description

[0013] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments of this disclosure taken in conjunction with the accompanying drawings, in which the same reference numerals generally represent the same components.

[0014] Figure 1 The flowchart shows a multi-index comprehensive evaluation method for agents based on a result-oriented reward function. Figure 2 This is an example radar chart. Detailed Implementation

[0015] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0016] This disclosure provides a multi-index comprehensive evaluation method for intelligent agents based on a result-oriented reward function. By constructing six core evaluation indicators, and combining indicator percentage processing, radar chart visualization, and average capability value calculation, a complete comprehensive evaluation system for intelligent agents is formed. The process is attached. Figure 1 As shown.

[0017] In one exemplary implementation, the specific technical solution mainly includes the following steps: 1. Assessment Prerequisites and Basic Definitions Application scenario definition: This method is only applicable to scenarios where the reward function is result-oriented in reinforcement learning tasks, that is, the reward value of the agent in the task directly corresponds to the final completion effect of the task. The higher the reward value, the better the task completion effect (e.g., in game tasks, a higher score means a higher degree of game completion, and in logistics scheduling tasks, a higher reward value means higher scheduling efficiency).

[0018] Test case set construction: Based on the specific outcome-oriented reinforcement learning task, select representative test cases. N Each test scenario forms a test case set, with each test case corresponding to an independent task scenario. Under the same environmental parameters and initial conditions, the reinforcement learning agent to be evaluated runs independently in each test case, and the final reward value of the agent under each test case is recorded, denoted as . R i (in i = 1,2,..., N , N (Total number of test cases).

[0019] Failure threshold setting: Based on the specific task requirements and objectives, a reward value failure threshold is preset. T The reward value of the agent in a certain test case. R i < T When the test case is deemed a "failed test case," it indicates that the agent has not achieved the expected task completion effect in this test scenario; when R i > T The test case is determined to be a "non-failed test case," indicating that the agent has achieved the expected task completion effect in this test scenario. 。

[0020] 2. Calculation of core evaluation indicators Based on the reward value data of the above test case set and the preset failure threshold, the following six core evaluation indicators are calculated respectively: (1) Average performance of agents Evaluation target: Test cases that are on the verge of failure.

[0021] Calculation principle: This metric is used to measure the overall average performance level of an agent across all test scenarios. It is achieved by calculating the normalized distance from the average reward value of all test cases to the failure threshold. The normalization process can eliminate the impact of differences in the magnitude of reward values ​​in different tasks, making the metric more universal and comparable.

[0022] Detailed calculation steps:

[0023] in, This represents the average performance index of the agent. The average reward value for all test cases T is the preset failure threshold. This is the theoretical maximum reward value for the reinforcement learning task (if the theoretical maximum reward value is difficult to obtain, the maximum reward value actually observed in the test case set can be used instead). The range of values ​​is ,when A positive value and the larger it is, the better the overall average performance of the agent; when... When this occurs, it indicates that the overall average reward value of the agent is lower than or equal to the failure threshold, and the overall performance is poor.

[0024] (2) Finished quality Evaluation target: Reward value of all non-failed use cases.

[0025] Calculation principle: This metric focuses on the performance quality of the agent in test scenarios where the task is successfully completed (non-failed test cases). It measures the performance level of the agent in scenarios where the task is effectively completed by calculating the normalized distance from the average reward value of non-failed test cases to the failure threshold.

[0026] Specific calculation formula:

[0027] in, To achieve the quality target values, The average of the reward values ​​for all non-failed use cases (assuming the number of non-failed use cases is ). ,but , T The failure threshold This is the theoretical maximum reward value (or the actual observed maximum reward value). The value range is (0,1]. The larger the value, the higher the average reward value of the agent in non-failed use cases, and the better the quality of task completion.

[0028] (3) Degree of failure Evaluation object: The reward value of all failed use cases.

[0029] Calculation principle: This metric is used to evaluate the severity of failure of an agent in test scenarios where the task is not completed (failed test cases). By calculating the normalized distance from the average reward value of the failed test cases to the failure threshold, it determines the degree to which the agent deviates from the expected performance in the failed scenario.

[0030] Specific calculation formula:

[0031] in, This is a failure level index value. The average reward value for all failed test cases (assuming the number of failed test cases is ). ,but , T The failure threshold This is the theoretical minimum reward value for the reinforcement learning task (if the theoretical minimum reward value is difficult to obtain, the minimum reward value actually observed in the test case set can be used instead). The value range is [0,1). The closer the value is to 0, the closer the average reward value of the failed use cases is to the failure threshold, and the less severe the failure. The closer it is to 1, the closer the average reward value of the failed use cases is to the theoretical minimum reward value, and the more severe the failure.

[0032] (4) Failure risk Evaluation target: Test cases that are on the verge of failure.

[0033] Calculation principle: This metric is used to measure the potential risk of failure faced by an agent in a test scenario. It identifies test cases whose reward value falls within the failure edge range of "the lowest reward value among non-failed test cases - the highest reward value among failed test cases" and calculates the proportion of their number to the total number of test cases to evaluate the stability and risk resistance of the agent in critical scenarios.

[0034] Detailed calculation steps: ① Determine the failure edge range: First, select the minimum reward value from all non-failed use cases, denoted as . (Right now Then, select the maximum reward value from all failed test cases, and denote it as... (Right now The failure margin interval is then... .

[0035] ② Count the number of edge test cases: Count the number of test cases whose reward value falls within the above-mentioned failure edge range, and record them as follows: .

[0036] ③ Calculate the failure risk ratio:

[0037] in, This is the failure risk index value. This represents the number of test cases within the failure edge range. NThis represents the total number of test cases. The value range of is [0,1]. The smaller the value, the fewer test cases are on the verge of failure, and the lower the risk of failure faced by the agent. The larger the value, the higher the risk of failure as the agent operates in more critical scenarios.

[0038] (5) Stability Evaluation target: The fluctuation of reward values ​​for all test cases.

[0039] Calculation principle: This metric is used to measure the stability of the reward value of an agent under different test scenarios. By calculating the normalized variance of the reward values ​​of all test cases, the influence of the difference in the magnitude of reward values ​​on the evaluation of the degree of fluctuation is eliminated, reflecting the consistency of the agent's performance.

[0040] Specific calculation formula:

[0041] in, This is a stability index value. The variance of the reward value for all test cases ( ), The theoretical maximum variance of the reward value for this reinforcement learning task (if the theoretical maximum variance is difficult to calculate, the maximum variance observed in a large amount of historical test data can be used as a substitute, or by ( Approximate estimation). The value range of is [0,1]. The closer it is to 1, the smaller the variance of the reward value, the smaller the performance fluctuation of the agent under different test scenarios, and the better the stability. The closer the value is to 0, the greater the fluctuation in reward value and the worse the stability of the agent.

[0042] (6) Failure rate percentage Evaluation object: The proportion of failed test cases in the total number of test cases.

[0043] Calculation principle: This metric is a direct indicator of the effectiveness of an agent in completing a task. By calculating the percentage of failed test cases out of the total number of test cases, it intuitively reflects the proportion of times the agent fails to achieve the expected results in the test scenario.

[0044] Specific calculation formula:

[0045] in, The failure rate percentage is the indicator value. The number of failed test cases. N This represents the total number of test cases. The range of values ​​is , The smaller the value, the fewer test scenarios where the agent fails, and the higher the effectiveness of task completion. The larger the value, the less effective the agent is, indicating that it cannot achieve the expected results in more test scenarios.

[0046] 3. Indicator Standardization and Comprehensive Evaluation (1) Percentage treatment of indicators Because the original value ranges and meanings of the above six core indicators are different (for some indicators, higher values ​​indicate better performance, while for others, lower values ​​indicate better performance), in order to facilitate comprehensive evaluation and radar chart visualization, all indicators need to be uniformly mapped to a percentage range of 0-100 points, and it should be ensured that the percentage-based indicators all satisfy the logic that "the higher the score, the better the corresponding performance": For the average performance of the agent ( ), completion quality ( ),stability( If the original index value If the percentage score is 100, then the score is 100. If the percentage is 0, then the score is 0; if 0 < S If the value is less than 1, then the percentage score is .

[0047] Regarding the degree of failure ( Since the closer the original index value is to 0, the less severe the failure (the better the performance), the percentage score is ( (when When =0, you get 100 points; when When the score is 1, you get 0 points.

[0048] For failure risk ( The smaller the original index value, the lower the failure risk (the better the performance), therefore the percentage score is ( points (when When =0, you get 100 points; when When the score is 1, you get 0 points.

[0049] For failure rate percentage ( The smaller the original metric value, the higher the effectiveness of task completion (better performance). Therefore, the percentage score is... 100 points (when When, you get 100 points; when (0 points)

[0050] Let the six index values ​​after percentage division be as follows: , , , , , (All scores are in the 0-100 range).

[0051] (2) Radar chart drawing of the comprehensive capabilities of the intelligent agent Using the subplots function from Python's Matplotlib library, a six-dimensional radar chart is constructed to visualize the comprehensive capabilities of an intelligent agent. ① Set the dimensional labels for the radar chart: Use the six percentage-based indicators (average agent performance, completion quality, failure degree, failure risk, stability, and failure rate percentage) as the six dimensional labels for the radar chart, and distribute them evenly on the circumference of the radar chart.

[0052] ② Determine the coordinate system of the radar chart: Take the center point of the radar chart as the origin, and set a scale of 0-100 points outward (the interval can be set according to the needs, such as every 20 points is a scale). The coordinate axis of each dimension extends from the origin to the circumference, corresponding to the score range of 0-100 points.

[0053] ③ Plot the agent performance curve: Calculate the six index values ​​after percentage division. - As six coordinate points on the radar chart, the coordinate points are connected sequentially to form a closed curve. The inside of the curve can be filled with color (such as semi-transparent blue) as needed to clearly distinguish the performance curves of different agents.

[0054] ④ Radar Chart Enhancement and Annotation: Add a title (e.g., "Radar Chart of Comprehensive Capabilities of Reinforcement Learning Agents"), legend (if comparing multiple agents), grid lines, etc., to ensure the readability and aesthetics of the radar chart. Through the radar chart, the performance strengths and weaknesses of an agent in various dimensions can be intuitively observed, quickly identifying its advantageous dimensions (dimensions where the curve extends outwards more significantly) and its weaker dimensions (dimensions where the curve contracts inwards more noticeably).

[0055] (3) Calculation of the average ability value of the agent To quantitatively evaluate the overall performance level of the agent, the average capability value of the agent is obtained by calculating the average of all percentage indicators:

[0056] in, This represents the average ability score of the agent, ranging from 0 to 100. A higher average ability score indicates better overall performance of the agent. Based on the average ability score, the agent's overall ability can be graded (e.g., 90 points and above is "Excellent", 80-89 points is "Good", 70-79 points is "Medium", 60-69 points is "Pass", and below 60 points is "Needs Improvement"), providing a clear grading standard for agent performance evaluation.

[0057] 4. Evaluation Process and Conclusion Output Based on the above steps, the multi-index comprehensive evaluation process of the intelligent agent in this embodiment can be summarized as follows: 1) Determine the type of evaluation task: Determine whether the reward function of the reinforcement learning task is outcome-oriented. If it is outcome-oriented, proceed to the next evaluation step; if it is process-oriented, this method is not applicable.

[0058] 2) Constructing test case sets and data collection: Select representative test cases, run the agent and record the reward value of each test case.

[0059] 3) Calculation of core indicators: Based on the reward value data and the preset failure threshold, calculate the original values ​​of the six core evaluation indicators.

[0060] 4) Indicator percentage processing: The original values ​​of the six core indicators are uniformly mapped to a percentage range of 0-100 points to ensure the consistency of indicator logic.

[0061] 5) Comprehensive evaluation and visualization: Draw a radar chart of the agent's comprehensive capabilities and calculate the average capability value.

[0062] 6) Evaluation Conclusion Output: Combining the "shape characteristics" of the radar chart (such as whether the performance of each dimension is balanced and whether there are obvious shortcomings) and the average capability value, a detailed analysis of the agent's comprehensive capabilities is conducted to form an evaluation report, clarifying the agent's performance level, strengths and weaknesses, and proposing targeted optimization suggestions (such as for agents with a high failure rate percentage, it is recommended to optimize the strategy to improve the effectiveness of task completion; for agents with poor stability, it is recommended to improve the algorithm to reduce performance fluctuations).

[0063] Application Examples Taking the "Pac-Man game reinforcement learning agent" as an example, the reward function of this task is a typical result-oriented one: the agent gets 10 points for eating a regular bean, 50 points for eating a power bean, and 100 points for eating a ghost (while the power bean is active). The total score at the end of the game is the reward value. The higher the reward value, the better the agent's performance in completing the game task (the more beans and ghosts it eats).

[0064] (1) Setting evaluation parameters 1) Test Case Set: 100 Pac-Man game scenarios with different map layouts were selected as the test case set. N =100), the map size, number of beans (regular beans and energy beans), number of ghosts and initial positions are different for each scene to ensure the representativeness and diversity of test cases.

[0065] 2) Failure threshold: Based on the difficulty and expected goal of the Pac-Man game, the preset failure threshold T = 500 points, which is the game score (reward value) of the agent in a certain test case. In time-sharing, the use case is determined to be a failed use case; If the score is >500, it is determined to be a valid test case.

[0066] 3) Theoretical reward range: The maximum theoretical score for each map in this Pac-Man game scenario (assuming all dots and ghosts are eaten) is: =2000 points, the minimum theoretical score is =0 points (did not eat any beans or ghosts).

[0067] 4) Theoretical maximum variance: using The theoretical maximum variance of the reward value .

[0068] (2) Data collection and indicator calculation Run the Pac-Man game agent to be evaluated sequentially through 100 test cases, and record the reward value for each test case. The reward values ​​for some test cases are shown in the table below (the complete data includes 100 test cases):

[0069] Based on the above data, calculate the raw values ​​of the six core indicators: 1) Average performance of agents ( ): Calculate the average reward value for all 100 test cases.

[0070] Substitute into the formula:

[0071] 2) Completed quality ( ): Statistics on non-failed use cases ( Quantity >500 points =65 cases, calculate the average reward value of non-failed test cases;

[0072] Substitute into the formula:

[0073] 3) Degree of failure ( ): Statistical failure test cases ( ) quantity =35 cases, calculate the average reward value for failed test cases:

[0074] Substitute into the formula:

[0075] 4) Failure risk ( ): From 2) Quality Completion: Statistics on non-failed use cases ( Quantity >500 points =65 cases, calculate the average reward value of non-failed test cases.

[0076] Substitute into the formula:

[0077] 5) Degree of failure ( ): Statistical failure test cases ( ) quantity =35 cases, calculate the average reward value for failed test cases:

[0078] Substitute into the formula:

[0079] 6) Failure risk ( ): Filter out the minimum reward value from the never-failed test cases =520 points (the reward value for a non-failed test case), select the maximum reward value from the failed test cases. =490 points (the reward value for a certain failed test case), then the failure edge range is [490, 520] points.

[0080] Count the number of test cases within this interval. =8 (i.e., the number of use cases with reward values ​​between 490 and 520 points). Substitute into the formula:

[0081] 7) Stability ): Calculate the variance of the reward values ​​for all 100 test cases.

[0082] Substitute into the formula:

[0083] 8) Failure rate percentage ( ): Number of known failure test cases =35, total number of test cases N=100.

[0084] Substitute into the formula:

[0085] (3) Indicator percentage and comprehensive evaluation 1) Indicator percentage processing Average performance of agents ; Finished quality ; Failure degree point; Failure risk point; stability point; Failure rate percentage point.

[0086] The percentage of the index values ​​is as follows: point, point, point, point, point, point.

[0087] 2) Radar chart drawing A six-dimensional radar chart was created using Matplotlib's `subplots` function, with dimension labels "Average Agent Performance," "Completion Quality," "Failure Level," "Failure Risk," "Stability," and "Failure Rate Percentage," and scales set to 0, 20, 40, 60, 80, and 100 points. These percentage values ​​were used as coordinate points for each dimension, connected to form a closed curve, and filled with semi-transparent blue. The completed radar chart shows that the agent performed best in the "Failure Risk" dimension (92 points, the curve extends the longest outwards), performed worst in the "Average Agent Performance" dimension (45.3 points, the curve contracts significantly inwards), and performed at a moderate level in other dimensions.

[0088] 3) Calculation of average ability value

[0089] The average ability score of this Pac-Man game agent is approximately 69.18 points. According to the preset level classification standard (below 70 points is "qualified"), its overall ability level is "qualified".

[0090] (4) Evaluation conclusions Based on the analysis of radar charts and average ability scores, the overall ability of this Pac-Man game reinforcement learning agent is at a "passable" level. The specific conclusions are as follows: Strengths: Outstanding failure risk control capability (92 points), indicating that the agent has good stability in critical scenarios and is less likely to be on the verge of failure; relatively mild failure degree (76 points), even in failure scenarios, the agent's reward value is close to the failure threshold and no serious failure occurs.

[0091] Weaknesses: The agent's average performance was poor (45.3 points), and the overall average reward value was low, reflecting that the agent's overall performance needs to be improved in most test scenarios; the failure rate percentage (65 points) and completion quality (68 points) were at a medium level, indicating some room for optimization.

[0092] Optimization recommendations: Focus on optimizing the agent's overall strategy to improve the average reward value in most test scenarios (e.g., improve the target exploration algorithm to increase the capture efficiency of beans and ghosts); at the same time, appropriately reduce the failure rate percentage (e.g., optimize the agent's decision-making logic in complex maps based on the characteristics of failure scenarios) to further improve the effectiveness of task completion.

[0093] The above technical solutions are merely exemplary embodiments of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the specific embodiments of the present invention. Therefore, the methods described above are only preferred and not restrictive.

Claims

1. A multi-index comprehensive evaluation method for intelligent agents based on a result-oriented reward function, characterized in that, Includes the following steps: S1, Test case set construction; S2, obtain the final reward value of the reinforcement learning agent to be evaluated under each test case; S3, Preset reward value invalidation threshold T ; S4. Based on the reward value data of the test case set and the preset failure threshold, calculate the evaluation indicators, which include: Average agent performance is used to evaluate the overall average performance level of the agent across all test scenarios. Completion quality is used to evaluate the performance quality of an agent in test scenarios where it successfully completes a task; Failure severity is used to assess the degree of failure of an agent in a test scenario where the task is not completed; Failure risk is used to assess the potential risk of an agent failing in a test scenario; Stability is used to evaluate the stability of the agent's reward value under different test scenarios; Failure rate percentage is used to evaluate the proportion of times an agent fails to achieve the expected results in a test scenario; S5, Standardization and comprehensive evaluation of evaluation indicators.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: Based on the specific outcome-oriented reinforcement learning task, select representative examples. N Each test scenario is used as a test case set, and each test case corresponds to an independent task scenario.

3. The method according to claim 2, characterized in that, Step S2 specifically includes: Under the same environmental parameters and initial conditions, the reinforcement learning agent to be evaluated is run independently in each test case, and the final reward value of the agent in each test case is recorded, denoted as . R i ,in i = 1,2,..., N .

4. The method according to claim 3, characterized in that, In step S3: When the agent's reward value in a certain test case R i < T When this happens, the test case is determined to be a "failed test case," indicating that the agent did not achieve the expected task completion effect in this test scenario; when R i > T If the test case is determined to be a "non-failed test case", it means that the agent has achieved the expected task completion effect in this test scenario.

5. The method according to claim 3, characterized in that, In step S4, the calculation method for the evaluation index includes: 1) Average performance of agents in, This represents the average performance index of the agent. The average reward value for all test cases T is the preset failure threshold. This refers to the theoretical maximum reward value for the reinforcement learning task or the maximum reward value actually observed in the test case set. 2) Completion Quality in, To achieve the quality target values, The average of the reward values ​​for all non-failed use cases. T The failure threshold This refers to either the theoretical maximum reward value or the actual observed maximum reward value. 3) Degree of failure in, This is a failure level index value. The average of the reward values ​​for all failed use cases. T The failure threshold This is the theoretical minimum reward value for the reinforcement learning task or the minimum reward value actually observed in the test case set; 4) Failure risk Detailed calculation steps: ① Determine the failure edge range: First, select the minimum reward value from all non-failed use cases, denoted as . ,Right now Then, select the maximum reward value from all failed test cases, and denote it as... ,Right now The failure margin interval is then... ; ② Count the number of edge test cases: Count the number of test cases whose reward value falls within the above-mentioned failure edge range, and record them as follows: ; ③ Calculate the failure risk ratio: in, This is the failure risk index value. This represents the number of test cases within the failure edge range. N This represents the total number of test cases; 5) Stability in, This is a stability index value; The variance of reward values ​​for all test cases ; Replace it with the theoretical maximum variance or the maximum variance observed in historical test data for the reward value of this reinforcement learning task, or by ( Approximate estimation; 6) Failure rate percentage in, The failure rate percentage is the indicator value. The number of failed test cases. N This represents the total number of test cases.

6. The method according to any one of claims 1-5, characterized in that, Step S5 specifically includes: S51, percentage-based processing of indicators; For the agent's average performance, completion quality, and stability: if the original index values... If the percentage score is 100, then the score is 100. If the percentage is 0, then the score is 0; if 0 < S If the value is less than 1, then the percentage score is ; Regarding the degree of failure The percentage score is ( ; Regarding the risk of failure The percentage score is ( point; For failure rate percentage Percentage score 100 points; Let the six index values ​​after percentage division be as follows: , , , , , ; S52, radar chart plotting of the comprehensive capabilities of intelligent agents; S53, Calculation of the average capability value of the agent: in, This represents the average ability value of the agent.

Images