Intelligent agent evaluation method and system

Abstract

The invention relates to the technical field of data processing, and discloses an intelligent agent evaluation method and system, and the method comprises the steps: analyzing key factors and performance indexes of an intelligent agent in different combat environments, determining evaluation indexes, and constructing an evaluation index system; setting parameters and algorithms of the evaluation model, and configuring an association relationship between the evaluation model and the evaluation index system; gaming deduction data and agent decision data are collected, and evaluation calculation data are generated and stored; in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording an evaluation calculation result; and in response to the evaluation calculation result, generating an evaluation report framework, integrating the evaluation calculation data and the corresponding evaluation calculation result into the evaluation report framework in a visual form, and outputting an evaluation report. According to the invention, standardization and systematization of the evaluation process are realized, scientificity and accuracy of evaluation are improved, and efficiency and accuracy of data processing are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an intelligent agent evaluation method, system, electronic device and storage medium. Background Art

[0002] At present, the application of intelligent agent technology is becoming more and more extensive, especially in the red-blue confrontation simulation of unmanned combat platforms. The comprehensive evaluation of intelligent agents is the key to ensure their effective execution of tasks in complex battlefield environments.

[0003] However, existing evaluation methods often rely on qualitative analysis and expert experience, lacking systematic and standardized evaluation processes and tools. In addition, the evaluation data collection, processing and analysis process of the intelligent agent is cumbersome and inefficient, making it difficult to meet the needs of the rapidly changing battlefield.

[0004] Therefore, the present application provides an agent evaluation method to solve the above technical problems. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent agent evaluation method, system, electronic device and storage medium to solve the technical problems that the evaluation method in the prior art lacks a standardized evaluation process, has low evaluation data processing efficiency and cannot meet battlefield requirements.

[0006] In order to solve the above technical problems, the present invention provides an agent evaluation method, comprising:

[0007] The step of constructing an agent evaluation index system includes analyzing key factors and performance indicators of the agent in different combat environments, determining evaluation indicators and constructing the evaluation index system;

[0008] The step of constructing an agent evaluation model includes setting parameters and algorithms of the evaluation model in response to the evaluation index system, configuring the association relationship between the evaluation model and the evaluation index system, and verifying the validity of the evaluation model;

[0009] The step of collecting and processing evaluation data includes preprocessing the collected game deduction data and agent decision data, generating evaluation calculation data and storing the data;

[0010] An evaluation calculation execution step includes, in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording the evaluation calculation results;

[0011] The step of generating and displaying an evaluation report includes generating an evaluation report framework in response to the evaluation calculation results, integrating the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in a visual form, and outputting the evaluation report.

[0012] In some specific embodiments, the step of constructing an agent evaluation index system includes analyzing key factors and performance indicators of the agent in different combat environments, determining evaluation indicators and constructing the evaluation index system, and further includes:

[0013] Analyzing the behavior and performance of the intelligent agent in the specific combat environment to determine the evaluation index;

[0014] A structured method is used to determine the hierarchical relationship between the evaluation indicators and to construct a multi-level evaluation indicator system;

[0015] Based on the multi-level evaluation indicator system, a logical structure diagram of the evaluation indicator system is constructed and visualized;

[0016] Based on a relational database, the multi-level evaluation index system is stored and managed.

[0017] In some specific embodiments, the step of constructing an agent evaluation model includes responding to the evaluation index system, setting parameters and algorithms of the evaluation model, configuring the association relationship between the evaluation model and the evaluation index system, and verifying the validity of the evaluation model, and further includes:

[0018] In response to the evaluation requirements and the evaluation indicator system, construct the evaluation model;

[0019] Setting the input parameters, weight coefficients and algorithm flow of the evaluation model to achieve quantitative evaluation of the performance of the agent;

[0020] Configuring the corresponding relationship between the evaluation model and the evaluation indicator system to realize the automatic calculation of the evaluation model;

[0021] The evaluation model is verified using test data to ensure the accuracy and reliability of the evaluation model.

[0022] In some specific embodiments, the step of collecting and processing evaluation data includes preprocessing in response to the collected game deduction data and agent decision data, generating evaluation calculation data and storing it, and further includes:

[0023] Determine the needs and sources of data collection, including operational data of the agent and environmental data;

[0024] Preprocessing the collected game deduction data and agent decision data, including data cleaning, format conversion and normalization, to improve data quality;

[0025] The evaluation calculation data is generated based on the preprocessed data and is organized and stored in multiple dimensions.

[0026] In some specific embodiments, the evaluation calculation execution step includes, in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording the evaluation calculation results, further including:

[0027] According to the requirements of the evaluation model, the evaluation calculation parameters are configured, wherein the evaluation calculation parameters include algorithm parameters and calculation accuracy;

[0028] Calling the evaluation model to calculate the evaluation calculation data to obtain the evaluation calculation result of the agent;

[0029] Real-time monitoring and evaluation of the calculation process, including logging and anomaly detection methods to ensure the correctness of the calculation;

[0030] The evaluation calculation results are recorded and stored.

[0031] In some specific embodiments, the step of generating and displaying an evaluation report includes generating an evaluation report framework in response to the evaluation calculation result, integrating the evaluation calculation data and the corresponding evaluation calculation result into the evaluation report framework in a visual form, and outputting the evaluation report, further including:

[0032] In response to the evaluation calculation results, generating the evaluation report framework, including chapters and elements, according to the evaluation purpose and requirements;

[0033] Integrate the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in the form of charts, graphics, text or a combination thereof to generate the evaluation report;

[0034] The evaluation report is output in the form of an electronic document.

[0035] In some specific embodiments, it further comprises:

[0036] Analyzing the evaluation report and predicting the future performance of the agent;

[0037] According to the prediction results, an optimization algorithm is used to optimize and improve the performance of the intelligent agent;

[0038] Using a decision support method, providing decision support for improving the performance of the intelligent agent;

[0039] Through continuous integration and continuous deployment methods, rapid iteration and deployment of the agent performance improvement measures are achieved.

[0040] Based on the same concept, the present invention also provides an agent evaluation system, comprising:

[0041] An agent evaluation index system construction module, configured to analyze key factors and performance indicators of the agent in different combat environments, determine evaluation indicators and construct the evaluation index system;

[0042] An agent evaluation model building module is configured to respond to the evaluation index system, set parameters and algorithms of the evaluation model, configure the association relationship between the evaluation model and the evaluation index system, and verify the validity of the evaluation model;

[0043] The module for collecting and processing evaluation data is configured to perform preprocessing in response to the collected game deduction data and agent decision data, generate evaluation calculation data and store them;

[0044] An evaluation calculation execution module is configured to respond to the evaluation calculation data, configure evaluation calculation parameters, call the evaluation model to calculate the evaluation calculation data, and record the evaluation calculation results;

[0045] The module for generating and displaying an evaluation report is configured to generate an evaluation report framework in response to the evaluation calculation results, integrate the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in a visual form, and output the evaluation report.

[0046] Based on the same concept, the present invention also provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the intelligent agent evaluation method.

[0047] Based on the same concept, the present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the agent evaluation method.

[0048] Compared with the prior art, the invention has the following beneficial effects:

[0049] The present invention discloses an intelligent agent evaluation method and system, which realizes the standardization and systematization of the evaluation process, improves the scientificity and accuracy of the evaluation, reduces the tediousness of manual operation, improves the efficiency and accuracy of data processing, shortens the evaluation cycle, and improves the real-time nature of the evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of an agent evaluation method of the present invention in some specific embodiments;

[0051] Figure 2 It is a schematic diagram of the architecture of an intelligent agent evaluation method of the present invention in some applications;

[0052] Figure 3 It is a schematic diagram of the composition of an agent resource library in some applications of an agent evaluation method of the present invention;

[0053] Figure 4 It is a system flow diagram of an intelligent agent evaluation method of the present invention in some applications;

[0054] Figure 5 It is a schematic diagram of the internal and external interfaces of an intelligent agent evaluation method of the present invention in some applications;

[0055] Figure 6 It is a schematic diagram of an indicator calculation architecture of an intelligent agent evaluation method of the present invention in some applications;

[0056] Figure 7 It is a schematic diagram of a calculation flow of a real-time confrontation process evaluation of an intelligent agent evaluation method in some applications of the present invention;

[0057] Figure 8 It is a schematic diagram of the calculation process of reviewing and revisiting the global evaluation of an agent evaluation method of the present invention in some applications;

[0058] Fig. 9 It is a schematic diagram showing evaluation data of an intelligent agent evaluation method in some applications of the present invention;

[0059] Fig.10 It is a schematic diagram of the structure of an intelligent agent evaluation system of the present invention in some specific embodiments;

[0060] Fig.11 It is a schematic diagram of the structure of an electronic device in some specific embodiments of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0063] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0064] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0065] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0066] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0067] It should be particularly noted that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0068] Reference Figure 1 , an agent evaluation method, comprising:

[0069] S101, a step of constructing an agent evaluation index system, including analyzing key factors and performance indicators of the agent in different combat environments, determining evaluation indicators and constructing the evaluation index system;

[0070] It is understandable that in the red-blue confrontation simulation, the evaluation index system of the agent is used to measure its performance and combat effectiveness. First, all combat environments where the agent is deployed are identified;

[0071] For example: at sea, in the air, electronic warfare, etc. Taking the unmanned combat platform at sea as an example, its operating characteristics in high sea conditions and complex electromagnetic environments are analyzed.

[0072] Identify key factors affecting agent performance through historical data analysis and simulation experiments.

[0073] For example, in a maritime environment, key factors include: autonomous obstacle avoidance capability, target recognition accuracy, and mission execution efficiency.

[0074] Define specific performance indicators for each key factor.

[0075] For example, autonomous obstacle avoidance capability can be measured by two indicators: “obstacle avoidance reaction time” and “obstacle avoidance success rate”.

[0076] The analytic hierarchy process (AHP) was used to determine the weight of each indicator and construct a multi-level indicator system.

[0077] For example, "mission execution efficiency" is used as the top-level indicator, with secondary indicators such as "route planning time" and "target strike accuracy" set up under it.

[0078] Use professional software (such as MATLAB or Excel) to visualize the indicator system. Create an indicator system structure diagram and present it in a tree diagram for easy understanding and operation.

[0079] Collect the actual operation data of the intelligent agent in the simulation environment.

[0080] For example, the simulation platform can record the "route planning time" and "target strike accuracy" of the unmanned combat platform in simulated naval battles.

[0081] Conduct actual tests on the indicator system to verify its effectiveness and accuracy.

[0082] Adjust indicator weights and system architecture based on actual test results.

[0083] In some of these applications, the key factors and performance indicators of intelligent agents in different combat environments are analyzed, the evaluation indicators are determined and the evaluation indicator system is constructed. The behavior and performance of the intelligent agents in specific combat environments are analyzed to determine the evaluation indicators; a structured method is used to determine the hierarchical relationship between the evaluation indicators and to construct a multi-level evaluation indicator system; based on the multi-level evaluation indicator system, a logical structure diagram of the evaluation indicator system is constructed and visualized; based on a relational database, the multi-level evaluation indicator system is stored and managed.

[0084] It is understandable that in complex combat environments, the behavior and performance evaluation of intelligent agents is used to optimize their combat effectiveness and decision-making. This application analyzes the behavior and performance of intelligent agents, establishes and maintains a multi-level evaluation indicator system, and ensures the systematic and efficient evaluation process.

[0085] For example, to evaluate the combat effectiveness of an unmanned surface vessel (USV), we first need to identify the key factors of the USV in the maritime combat environment, including its stability in complex sea conditions, and its ability to identify and strike enemy targets.

[0086] Key factors analysis:

[0087] Stability: The ability of a USV to maintain its course in sea conditions of level 5.

[0088] Target Identification: The ability of a USV to identify enemy vessels through radar and optical sensors.

[0089] Strike capability: How effectively a USV can launch missiles or perform electronic jamming missions.

[0090] Performance indicator definition:

[0091] Stability: defined as "sea resistance level" and "course keeping deviation".

[0092] Target recognition: defined as “target recognition time” and “recognition accuracy”.

[0093] Strike capability: defined as “missile launch preparation time” and “electronic jamming success rate”.

[0094] Construction of indicator system:

[0095] Build a system with three levels of indicators:

[0096] Top-level metric: operational effectiveness.

[0097] Secondary indicators: stability, target identification, and strike capability.

[0098] Level 3 indicators: wave resistance level, route keeping deviation, target identification time, identification accuracy, missile launch preparation time, and electronic jamming success rate.

[0099] Weight distribution:

[0100] The weight of each secondary indicator is determined through the AHP method, such as stability (30%), target recognition (40%), and strike capability (30%).

[0101] Visual design:

[0102] Use MATLAB software to draw the indicator system structure diagram to display the indicators at all levels and their weights.

[0103] Data collection and testing:

[0104] In the simulation environment, the performance of the USV in the simulated naval battle was recorded, including the course keeping capability and target identification accuracy under specific sea conditions.

[0105] Verification of indicator system:

[0106] According to the test results, the indicator system is reviewed to confirm its rationality and effectiveness. If the weight distribution of a certain indicator is found to be unreasonable, appropriate adjustments are made.

[0107] S102, a step of constructing an agent evaluation model, including setting parameters and algorithms of the evaluation model in response to the evaluation index system, configuring an association relationship between the evaluation model and the evaluation index system, and verifying the validity of the evaluation model;

[0108] It is understandable that the agent evaluation model is used to measure its performance in a specific combat environment. This step responds to the evaluation index system, sets the parameters and algorithms of the evaluation model, configures the relationship between the evaluation model and the evaluation index system, and verifies the effectiveness of the evaluation model.

[0109] Based on the constructed multi-level evaluation indicator system, the specific intelligent agent for evaluation is determined.

[0110] Define necessary parameters for the model, such as weights, thresholds, performance metric benchmarks, etc.

[0111] Select the corresponding algorithm, such as analytic hierarchy process (AHP), fuzzy logic, data envelopment analysis (DEA), etc.

[0112] Write algorithms in Python or MATLAB to implement evaluation logic.

[0113] Configure the relationship between the evaluation model and the evaluation indicator system:

[0114] Mapping indicators to models: Map each indicator in the evaluation indicator system to the parameters of the evaluation model.

[0115] Weight allocation: Assign weights to each indicator based on historical data.

[0116] Model integration: Ensure that all indicators and parameters are correctly integrated in the model to form a complete evaluation system.

[0117] Verify the effectiveness of the evaluation model:

[0118] Test data preparation: Collect data on the agent’s performance under different conditions.

[0119] Model testing: Input test data into the model and perform the evaluation process.

[0120] Results Analysis: Analyze model output and compare it with expected results and actual performance.

[0121] In some of the applications, in response to the evaluation index system, the parameters and algorithms of the evaluation model are set, the correlation between the evaluation model and the evaluation index system is configured, and the effectiveness of the evaluation model is verified. In response to the evaluation needs and the evaluation index system, an evaluation model is constructed; the input parameters, weight coefficients and algorithm flow of the evaluation model are set to achieve quantitative evaluation of the performance of the intelligent agent; the correspondence between the evaluation model and the evaluation index system is configured to achieve automatic calculation of the evaluation model; and the evaluation model is verified using test data to ensure the accuracy and reliability of the evaluation model.

[0122] It is understandable that in the simulation of the Red-Blue Challenge, the performance evaluation of the intelligent agent is used to optimize its algorithms and strategies. This application builds an evaluation model, sets model parameters, configures the relationship between the model and the evaluation index system, and verifies the accuracy and reliability of the model.

[0123] According to the requirements of the Red-Blue Challenge, determine the key performance indicators that the evaluation model needs to cover, such as situation analysis, target allocation, task planning, force coordination, etc.

[0124] Select an evaluation model, such as a multi-attribute decision analysis model, to conduct an evaluation based on multiple indicators.

[0125] Set the input parameters, weight coefficients and algorithm flow of the evaluation model:

[0126] Input parameter definition: Define the input parameters of the model, such as the agent’s response time, target recognition accuracy, task completion rate, etc.

[0127] Weight coefficient setting: Set the weight coefficient according to the importance of each indicator. For example, the response time weight is 0.3, the target recognition accuracy weight is 0.5, and the task completion rate weight is 0.2.

[0128] Algorithm process design: Design algorithm processes, such as using the weighted summation method to calculate the comprehensive performance score of the intelligent agent.

[0129] The correspondence between the configuration evaluation model and the evaluation indicator system

[0130] Indicator mapping: Map each indicator in the evaluation indicator system to the input parameters of the evaluation model.

[0131] Automated calculation configuration: Automated calculation of the evaluation model is achieved through programming, such as using Python to write scripts to automatically calculate the performance score of the agent based on the input parameters and weight coefficients.

[0132] Use test data to validate the evaluation model:

[0133] Test data preparation: Collect performance data of the agent in different scenarios, such as response time and task completion in simulated red-blue confrontation.

[0134] Model validation: Run the evaluation model using the test data to check whether the output performance score is consistent with the expected one.

[0135] Accuracy and reliability analysis: Analyze the accuracy and reliability of the model by comparing it with actual performance.

[0136] For example, in the Red-Blue Challenge, to evaluate the performance of intelligent drones, a multi-attribute decision analysis model is selected, which can comprehensively consider multiple performance indicators of drones. Define three input parameters, namely response time (RT), target recognition accuracy (TAR), and task completion rate (TCR). Set the weight coefficients to 0.3 for response time, 0.5 for target recognition accuracy, and 0.2 for task completion rate. Design a weighted summation algorithm with the calculation formula:

[0137] [Comprehensive score = (RT × 0.3) + (TAR × 0.5) + (TCR × 0.2)]

[0138] Map the response time, target recognition accuracy, and task completion rate to the model's input parameters, and configure the automated calculation in the Python script. Use test data to verify the evaluation model. The drone's response time is 45 seconds, the target recognition accuracy is 92%, and the task completion rate is 100%. Enter the above data into the model and get a comprehensive score of 85 points. Verify the accuracy and reliability of the model by comparing it with the actual task performance.

[0139] S103, a step of collecting and processing evaluation data, including preprocessing in response to the collected game deduction data and agent decision data, generating evaluation calculation data and storing it;

[0140] It can be understood that in this step, the source of data collection is determined, including the game deduction data of the simulation platform and the real-time decision data of the intelligent agent.

[0141] Data preprocessing:

[0142] Data cleaning: Remove invalid or erroneous data records, such as eliminating values ​​outside the normal range or records with obvious errors.

[0143] Data formatting: Convert data into a unified format to facilitate subsequent processing and analysis.

[0144] Data normalization: Normalize the data to eliminate the dimensional effects between different indicators, such as converting all indicator values ​​to the range of 0 to 1.

[0145] Generate evaluation calculation data:

[0146] Feature extraction: Extract features from preprocessed data that are useful for evaluating the model.

[0147] Data aggregation: Perform aggregation operations on data as needed, such as calculating the average, maximum, minimum, etc.

[0148] Data Storage:

[0149] Select a database system for data storage, such as the relational database MySQL or the non-relational database MongoDB. Store the processed data according to the designed database model.

[0150] In some of these applications, in response to the collected game theory data and agent decision data, preprocessing is performed to generate evaluation calculation data and store it, and the needs and sources of data collection are determined, including the agent's operating data and environmental data; the collected game theory data and agent decision data are preprocessed, including data cleaning, format conversion and normalization to improve data quality; evaluation calculation data is generated based on the preprocessed data and is organized and stored in multiple dimensions.

[0151] It is understandable that the needs and sources of data collection are determined, and the types of data required to analyze and evaluate the performance of the agent are analyzed, including the agent's operating data (such as task completion time, resource consumption) and environmental data (such as battlefield terrain, weather conditions). Identify data sources, such as simulation platforms, sensors, log files, etc. According to the needs and sources, formulate a data collection strategy. For example, obtain agent operation data from the simulation platform API and obtain environmental data from the environmental monitoring system.

[0152] Data preprocessing to remove invalid or erroneous data records. For example, remove values ​​outside the normal operating range or records with obvious errors. Convert data to a unified format for subsequent processing. For example, unify timestamps from multiple formats to ISO 8601 standards. Normalize data to eliminate dimensional effects between different indicators. For example, convert all numerical indicators to a range of 0 to 1.

[0153] Extract features from preprocessed data, such as calculating the average response time and success rate of the agent. Aggregate data, such as calculating the average, maximum, and minimum values ​​over a period of time.

[0154] Design database models, including tables, fields, and relationships, to support multi-dimensional data storage. Store pre-processed and generated data into the database, ensuring data security and accessibility.

[0155] S104, an evaluation calculation execution step, including, in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording the evaluation calculation results;

[0156] It is understood that, according to the requirements of the evaluation model, define the calculation parameters, such as weights, thresholds, algorithm selection, etc. Set specific values ​​for each parameter, which can be based on historical data analysis or previous evaluation results. According to the evaluation objectives and available data, select the evaluation model, such as multi-attribute decision analysis model, machine learning model, etc. The model call is implemented through programming to ensure that the model can receive preprocessed data as input. Input the preprocessed evaluation calculation data into the evaluation model. Execute the calculation logic of the model, including mathematical operations, statistical analysis, machine learning reasoning, etc. Capture the evaluation results output by the model, including scores, grades, probabilities, etc. Store the evaluation results in a data storage system, such as a database or data warehouse.

[0157] In some of the applications, in response to evaluation calculation data, evaluation calculation parameters are configured, an evaluation model is called to calculate the evaluation calculation data, and the evaluation calculation results are recorded. According to the requirements of the evaluation model, the evaluation calculation parameters are configured, and the evaluation calculation parameters include algorithm parameters and calculation accuracy; the evaluation model is called to calculate the evaluation calculation data to obtain the evaluation calculation results of the intelligent agent; the evaluation calculation process is monitored in real time, including log recording and anomaly detection methods to ensure the correctness of the calculation; and the evaluation calculation results are recorded and stored.

[0158] It is understood that according to the type of evaluation model (such as regression analysis, decision tree, etc.), set algorithm-specific parameters, such as learning rate, tree depth, etc. Set accuracy requirements for the calculation process, such as the number of digits retained after the decimal point or the threshold of calculation error. Call the evaluation model through programming and input preprocessed data. Execute the calculation logic of the model to obtain the evaluation result of the intelligent agent. Record the detailed process of the evaluation calculation, including input data, intermediate results and final output. Implement anomaly detection mechanisms, such as range checking, comparison of historical data, etc., to identify and handle anomalies in the calculation process. Record the evaluation results in a log file or database. Store the evaluation results in a database for subsequent analysis and use.

[0159] For example, set the depth of the decision tree model to 5 and the learning rate to 0.01. Set the calculation result to retain four decimal places. Use Python to call the decision tree model, and the function is defined as decision_tree_model(data,depth=5,learning_rate=0.01).

[0160] Input the agent's operating data, such as response time and task completion rate, to obtain the evaluation score. Record the input data, model parameters, and output results of each calculation. Set threshold detection. If the evaluation score exceeds 10% of the historical highest score, it is marked as abnormal and further checked. Record the calculated evaluation score in the evaluation log. Store the evaluation score and other related information in the assessment_results table of the MySQL database. The table structure includes fields such as agent ID, evaluation score, and calculation time.

[0161] S105, the step of generating and displaying an evaluation report, includes generating an evaluation report framework in response to the evaluation calculation result, integrating the evaluation calculation data and the corresponding evaluation calculation result into the evaluation report framework in a visual form, and outputting the evaluation report.

[0162] Understandably, design the basic structure of the report, including sections such as the abstract, methodology, results, discussion, and conclusions. Use document editing software (such as Microsoft Word or LaTeX) to create a report template to ensure that the report has a uniform and professional format. Select relevant evaluation calculation data and results based on the purpose of the report and the needs of the readers. Design charts and graphs to present the data and results, such as using bar charts to show the comparison of performance indicators, line charts to show trends, etc. Insert the designed charts into the corresponding sections of the report. Add necessary text descriptions next to or below the charts to explain the content of the charts and the significance of the evaluation results. Perform a final edit and proofreading of the report to ensure the accuracy and completeness of the information. Convert the report into a format suitable for sharing, such as PDF or HTML, for easy distribution and reading.

[0163] In some of the applications, in response to the evaluation calculation results, an evaluation report framework is generated, the evaluation calculation data and the corresponding evaluation calculation results are integrated into the evaluation report framework in a visual form, and the evaluation report is output; in response to the evaluation calculation results, an evaluation report framework is generated according to the evaluation purpose and needs, including chapters and elements; the evaluation calculation data and the corresponding evaluation calculation results are integrated into the evaluation report framework in the form of charts, graphics, text or a combination of charts, graphics, text or a combination of charts, to generate an evaluation report; and the evaluation report is output in the form of an electronic document.

[0164] It is understood that in this application, for example, it is determined that the report includes sections such as summary, introduction, methodology, results, discussion, conclusion, and appendix. Create a report template with the required sections and format. Select data on the agent's response time, target recognition accuracy, and task completion rate. Design three charts: a bar chart to show the response time of different agents, a pie chart to show the target recognition accuracy, and a line chart to show the change of task completion rate over time. Integrate the designed charts and graphs and necessary text descriptions into the corresponding parts of the report.

[0165] Report Contents:

[0166] Abstract: Summarizes the main findings and conclusions of the intelligent agent in the red-blue confrontation simulation;

[0167] Introduction: This describes the background and purpose of the assessment;

[0168] Methodology: describe the evaluation approach, the evaluation model used, and the data sources;

[0169] Results: charts and graphs showing evaluation calculation data and results;

[0170] Discussion: Analyze the significance and possible impact of the evaluation results;

[0171] Conclusion: Summarizes the main findings of the evaluation and provides recommendations;

[0172] Output the evaluation report.

[0173] Convert reports to PDF format, ensuring compatibility on different devices and operating systems.

[0174] In the report, present the following assessment results:

[0175] Response time: average 75 seconds, standard deviation 5 seconds.

[0176] Target recognition accuracy: average 92%, standard deviation 3%.

[0177] Mission Completion Rate: Increased from 85% to 95% during simulation.

[0178] These data are presented in the following charts:

[0179] The bar chart shows the response time of the agent in five simulations. Each bar represents a simulation, and the color depth indicates the speed of the response time.

[0180] The pie chart shows the distribution of object recognition accuracy, with the 92% area colored to indicate high accuracy.

[0181] The line chart shows the improvement trend of task completion rate over time, with a rising line indicating improved performance.

[0182] In some of these applications, evaluation reports are analyzed to predict the future performance of the agent; based on the prediction results, optimization algorithms are used to optimize and improve the performance of the agent; decision support methods are used to provide decision support for the performance improvement of the agent; and through continuous integration and continuous deployment methods, rapid iteration and deployment of measures to improve the performance of the agent are achieved.

[0183] Understandably, review performance data and trends in evaluation reports, such as response time, object recognition accuracy, etc. Use time series analysis or machine learning models to predict the agent’s performance in future tasks. Based on the prediction results, select an optimization algorithm, such as a genetic algorithm. Adjust the agent’s parameters or algorithm to improve its performance. Build decision models, such as decision trees, Bayesian networks, etc., to support performance improvement decisions. Evaluate the effectiveness and feasibility of different optimization solutions. Implement automated testing processes to ensure that new improvements perform as expected. Integrate new code and improvements through continuous integration tools, such as Jenkins. Implement continuous deployment to quickly deploy improvements to the agent system.

[0184] Combine the following Figures 2 to 9 The following describes some examples of the agent evaluation method of the present invention in some applications:

[0185] like Figure 2 As shown, the architecture of this embodiment includes:

[0186] Basic platform layer: including operating system, database management system and red-blue challenge simulation game platform. The database management system uses MYSQL. The red-blue challenge simulation game platform is used for intelligent agent simulation game situation display and data recording after transformation, and is provided to the evaluation system for comprehensive evaluation of intelligent agents.

[0187] Evaluation resource management layer: used to store various resources required for performance evaluation, divided into four libraries. The first is the evaluation operator library, which mainly stores various basic models and operators prepared for evaluation modeling. By combining them, a composite evaluation model can be realized; the second is the game deduction database, which mainly stores the game deduction data required for evaluation calculations and various intermediate data generated by evaluation preprocessing. These data are the basis for analyzing evaluation results; the third is the evaluation indicator system library, which mainly stores various evaluation indicator systems prepared for system performance evaluation. Users can establish multiple evaluation indicator systems based on research results; the fourth is the intelligent agent resource library, such as Figure 3 As shown, the agent resource library mainly stores competition information and agent resource information to facilitate agent resource management.

[0188] Evaluation function layer: It is divided into four parts: agent resources, evaluation indicators and models, evaluation data loading and preprocessing, and evaluation calculation execution. The specific composition of each part is as follows:

[0189] Agent resources: mainly provide game event management and agent full life cycle management, including adding, deleting, editing and other management functions for agents, and providing the function of searching for agents; support viewing the information of each version of agents and the upgrade log of each agent; provide agent event management, including adding, deleting, editing and other management functions for agents, and providing the function of searching for events.

[0190] Evaluation indicators and models: mainly provide a visual way to create and edit the evaluation indicator system of effectiveness, combat effectiveness and blue team scene interference performance, set the evaluation calculation model for the indicator system node, and perform necessary parameter configuration on the evaluation model, etc.

[0191] Data loading and preprocessing: mainly used to read the game deduction process and result data required for performance evaluation, and provide input for evaluation and data preprocessing, etc. Including reading database data, formatting file data, and preprocessing such as variance, range, standard deviation, etc.;

[0192] Evaluation calculation execution: It mainly provides the calculation execution function of the executable evaluation calculation model, as well as the process data recording function used to support the evaluation calculation execution. In addition, it can also perform necessary configuration for the evaluation calculation.

[0193] User interface layer: mainly provides a unified interface for various functional components of various assessments to interact with users. This system establishes an extensible multi-view user interface framework. In this framework, various functional components such as assessment modeling, assessment calculation, and result presentation can use views to implement their own interfaces for interaction with users, and are integrated together under the unified organization and management of the framework.

[0194] like Figure 4 and Figure 5 As shown in the figure, in order to build an evaluation index system for unmanned platform agents, it is necessary to analyze and count the key factors and indicators of unmanned system collaborative operations such as collaborative defense, collaborative attack, collaborative support, and collaborative suppression of platform agents such as drones, unmanned boats, and unmanned submarines in combat environments such as surface, underwater, air, nearshore, and information warfare, sort out the correlation between them, and determine the system indicators. The relevant indicator system is constructed through the indicator system construction tool. The indicator system construction creates, modifies, deletes, and saves the indicator model in a visual and graphical way, or loads the indicator system file that has been constructed.

[0195] Users can customize and build and display the evaluation indicator system in a visual way according to the specific research object. This includes creating, saving, opening, modifying, and deleting evaluation indicators and indicator systems, and supports the reuse and expansion of evaluation indicators and indicator system models. Data collection requirements for the indicator system can be generated.

[0196] The evaluation index system of the hierarchical structure is created in a visual way, which is expressed as the index structure creation and modification of the tree structure diagram. The existing index system template can be loaded and the index system structure currently created can be exported. The index system structure information elements include the index name, weight, remarks, etc. The index system is stored as an XML file, and its format is the structure customized by this software.

[0197] You can directly load the existing indicator system file from the indicator system library and display it in the current indicator system editing window, which can be further edited as needed. You can also save the currently edited indicator system to the indicator system library for subsequent reuse.

[0198] According to the evaluation needs, an evaluation calculation model is built for each indicator system node. The evaluation model is organized and designed in the form of operators. Users can complete the setting and editing of each node evaluation model by associating the indicator system node with the evaluation model and then setting the weight.

[0199] Evaluation operators are divided into control operators, data processing operators, basic operators, normalization operators and evaluation operators according to their calculation characteristics. Different operators are applicable to different categories of indicator nodes.

[0200] Construct a hierarchical analysis method, hierarchize the indicator system, build a judgment matrix, calculate the weights, and finally arrive at the optimal decision method.

[0201] Specific algorithm flow of the model:

[0202] Divide the index system into different levels and determine the relationship between the levels. For each level of elements, compare them two by two to get their relative importance and build a judgment matrix. By performing eigenvalue decomposition on the judgment matrix, get the weight of each element, that is, its importance. By calculating the consistency index, determine whether the constructed judgment matrix is ​​consistent. Combine the weights of each level to get the optimal decision.

[0203] The ADC method is used to determine the dependency and contribution between indicator design elements. Through the ADC method, the mutual influence between indicator design elements can be analyzed, key elements can be found, and indicator design and optimization can be guided.

[0204] Specific algorithm flow of the model:

[0205] Determine the design elements of the indicator and use them as attributes. Determine the correlation between attributes through experimental data. Construct the influence matrix between attributes based on the correlation between attributes. By calculating the eigenvalues ​​and eigenvectors of the influence matrix, obtain the contribution of each attribute, that is, its influence on the indicator performance. Determine the key elements based on the contribution, and design and optimize the indicator.

[0206] After building the indicator system, calculation system and setting relevant weights, you can process and import the data to be evaluated, match the model and data, and preview, verify, modify, and save the data.

[0207] The ZMQC li ent class is modified through python to transmit the decision data of the intelligent agent and the decision data of the game platform to the project message component. The data is pre-processed through the calculation function of the message component itself, and the processed data is stored in the process.

[0208] like Figure 6 As shown in the figure, the indicator evaluation calculation adopts the eKu i per open source technology, and the intelligent agent effectiveness evaluation system is built based on eKu i per to meet the needs of intelligent agent combat capability evaluation, carry out combat intelligent agent capability dimension composition analysis for heterogeneous tasks in typical combat scenarios, combat intelligent agent capability evaluation based on multi-dimensional evaluation, etc., to achieve the correlation connection between the quantitative evaluation of intelligent agent capability and combat effectiveness;

[0209] Data source layer:

[0210] The performance evaluation system supports access to real-time source MQ and Http data to realize real-time evaluation needs of intelligent entities; it supports access to F ile and SQL data to realize the review evaluation needs of intelligent entities.

[0211] Data preprocessing layer:

[0212] Data preprocessing is to clean / filter and standardize the received data, and apply the standard data to indicator calculations. It can also store the processed intermediate data. It comes with dozens of data processing functions and supports custom extensions using Go lang or Python.

[0213] Data storage layer:

[0214] The performance evaluation system supports sending calculation results through MQ and REST; it supports storing calculation results in File and SQL.

[0215] Result presentation layer:

[0216] The performance evaluation system can display the evaluation result data in graphical form, including pie charts, dot charts, curve charts, tables and other forms; the evaluation result data can also be exported as Word or PDF files.

[0217] like Figure 7 and Figure 8As shown, by transforming the ZMQC li ent, the agent decision data, game platform situation data, log data, etc. are sent to the message queue, the original data is stored and the streaming computing capability of the message component library is used to call the corresponding constructed indicator system and computing system model, and the evaluation calculation quantitative results are stored in the result database and generate an evaluation report.

[0218] The historical raw data stored in the message queue is used to call the established indicator system and calculation system model, and batch calculation is used to review, evaluate and analyze the entire data and store the results.

[0219] like Fig. 9 As shown, a Word version of the editable report document is generated for the evaluation results of the current solution. The evaluation process data and result data are stored in the Word document in the form of graphs and tables, and it also supports saving in PDF format. At the same time, the document records the indicator system architecture diagram, operator system architecture diagram, indicator weight table, solution configuration condition table, and evaluation results. The evaluation results in the report are displayed in graphics (histograms, pie charts, or curve charts, etc.), tabular display, etc.

[0220] For the method steps disclosed in the above embodiments, the method steps are described as a series of action combinations for the purpose of simple description, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0221] like Fig.10 As shown, the present invention also provides an intelligent agent evaluation system, comprising:

[0222] An agent evaluation index system construction module 201 is configured to analyze key factors and performance indicators of the agent in different combat environments, determine evaluation indicators and construct the evaluation index system;

[0223] An agent evaluation model building module 202 is configured to set parameters and algorithms of the evaluation model in response to the evaluation index system, configure the association relationship between the evaluation model and the evaluation index system, and verify the validity of the evaluation model;

[0224] The module 203 for collecting and processing evaluation data is configured to perform preprocessing in response to the collected game deduction data and agent decision data, generate evaluation calculation data and store them;

[0225] The evaluation calculation execution module 204 is configured to respond to the evaluation calculation data, configure evaluation calculation parameters, call the evaluation model to calculate the evaluation calculation data, and record the evaluation calculation results;

[0226] The evaluation report generation and display module 205 is configured to generate an evaluation report framework in response to the evaluation calculation results, integrate the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in a visual form, and output the evaluation report.

[0227] It is worth noting that although only some basic functional modules are disclosed in the embodiments of the present invention, it does not mean that the composition of the present system is limited to the above basic functional modules. On the contrary, what the present embodiment wants to express is that on the basis of the above basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. In other words, the present system is open rather than closed. Just because the present embodiment only discloses individual basic functional modules, it cannot be considered that the protection scope of the claims of the present invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described in terms of functions and are described separately in various units and modules. Of course, when implementing the present invention, the functions of each unit and module can be implemented in the same or one or more software and / or hardware.

[0228] like Fig.11 As shown, the present invention also provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the intelligent agent evaluation method.

[0229] Fig.11 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig.11 The electronic device provided in the embodiment of the present invention includes: one or more processors 710 and a storage device 720; the processor 710 in the electronic device may be one or more, Fig.11 A processor 710 is taken as an example; the storage device 720 is used to store one or more programs; the one or more programs are executed by the one or more processors 710, so that the one or more processors 710 implement the intelligent agent evaluation method as described in any one of the embodiments of the present invention.

[0230] The electronic device may further include: an input device 730 and an output device 740 .

[0231] The processor 710, storage device 720, input device 730 and output device 740 in the electronic device can be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0232] The storage device 720 in the electronic device is a computer-readable storage medium that can be used to store one or more programs, which can be software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the agent evaluation method provided in the embodiment of the present invention. The processor 710 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the storage device 720, that is, implementing the agent evaluation method in the above method embodiment.

[0233] The storage device 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the storage device 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the storage device 720 may further include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0234] The input device 730 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.

[0235] The present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the intelligent agent evaluation method.

[0236] Specifically, the computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0237] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An agent evaluation method, characterized in that: include: The step of constructing an agent evaluation index system includes analyzing key factors and performance indicators of the agent in different combat environments, determining evaluation indicators and constructing the evaluation index system; The step of constructing an agent evaluation model includes setting parameters and algorithms of the evaluation model in response to the evaluation index system, configuring the association relationship between the evaluation model and the evaluation index system, and verifying the validity of the evaluation model; The step of collecting and processing evaluation data includes preprocessing the collected game deduction data and agent decision data, generating evaluation calculation data and storing the data; An evaluation calculation execution step includes, in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording the evaluation calculation results; The step of generating and displaying an evaluation report includes generating an evaluation report framework in response to the evaluation calculation results, integrating the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in a visual form, and outputting the evaluation report.

2. The agent evaluation method according to claim 1, characterized in that: The step of constructing an agent evaluation index system includes analyzing key factors and performance indicators of the agent in different combat environments, determining evaluation indicators and constructing the evaluation index system, and further includes: Analyzing the behavior and performance of the intelligent agent in the specific combat environment to determine the evaluation index; A structured method is used to determine the hierarchical relationship between the evaluation indicators and to construct a multi-level evaluation indicator system; Based on the multi-level evaluation indicator system, a logical structure diagram of the evaluation indicator system is constructed and visualized; Based on a relational database, the multi-level evaluation index system is stored and managed.

3. The agent evaluation method according to claim 1, characterized in that: The step of constructing an agent evaluation model includes setting parameters and algorithms of the evaluation model in response to the evaluation index system, configuring the association relationship between the evaluation model and the evaluation index system, and verifying the validity of the evaluation model, further including: In response to the evaluation requirements and the evaluation indicator system, construct the evaluation model; Setting the input parameters, weight coefficients and algorithm flow of the evaluation model to achieve quantitative evaluation of the performance of the agent; Configuring the corresponding relationship between the evaluation model and the evaluation indicator system to realize the automatic calculation of the evaluation model; The evaluation model is verified using test data to ensure the accuracy and reliability of the evaluation model.

4. The agent evaluation method according to claim 1, characterized in that: The step of collecting and processing evaluation data includes preprocessing the collected game deduction data and agent decision data, generating evaluation calculation data and storing the data, and further includes: Determine the needs and sources of data collection, including operational data of the agent and environmental data; Preprocessing the collected game deduction data and agent decision data, including data cleaning, format conversion and normalization, to improve data quality; The evaluation calculation data is generated based on the preprocessed data and is organized and stored in multiple dimensions.

5. The agent evaluation method according to claim 1, characterized in that: The evaluation calculation execution step includes, in response to the evaluation calculation data, configuring evaluation calculation parameters, calling the evaluation model to calculate the evaluation calculation data, and recording the evaluation calculation results, further including: According to the requirements of the evaluation model, the evaluation calculation parameters are configured, wherein the evaluation calculation parameters include algorithm parameters and calculation accuracy; Calling the evaluation model to calculate the evaluation calculation data to obtain the evaluation calculation result of the agent; Real-time monitoring and evaluation of the calculation process, including logging and anomaly detection methods to ensure the correctness of the calculation; The evaluation calculation results are recorded and stored.

6. The agent evaluation method according to claim 1, characterized in that: The step of generating and displaying an evaluation report includes generating an evaluation report framework in response to the evaluation calculation result, integrating the evaluation calculation data and the corresponding evaluation calculation result into the evaluation report framework in a visual form, and outputting the evaluation report, further including: In response to the evaluation calculation results, generating the evaluation report framework, including chapters and elements, according to the evaluation purpose and requirements; Integrate the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in the form of charts, graphics, text or a combination thereof to generate the evaluation report; The evaluation report is output in the form of an electronic document.

7. The agent evaluation method according to claim 1, characterized in that: Further including: Analyzing the evaluation report and predicting the future performance of the agent; According to the prediction results, an optimization algorithm is used to optimize and improve the performance of the intelligent agent; Using a decision support method, providing decision support for improving the performance of the intelligent agent; Through continuous integration and continuous deployment methods, rapid iteration and deployment of the agent performance improvement measures are achieved.

8. An agent evaluation system, characterized in that: include: An agent evaluation index system construction module, configured to analyze key factors and performance indicators of the agent in different combat environments, determine evaluation indicators and construct the evaluation index system; An agent evaluation model building module is configured to respond to the evaluation index system, set parameters and algorithms of the evaluation model, configure the association relationship between the evaluation model and the evaluation index system, and verify the validity of the evaluation model; The module for collecting and processing evaluation data is configured to perform preprocessing in response to the collected game deduction data and agent decision data, generate evaluation calculation data and store them; An evaluation calculation execution module is configured to respond to the evaluation calculation data, configure evaluation calculation parameters, call the evaluation model to calculate the evaluation calculation data, and record the evaluation calculation results; The module for generating and displaying an evaluation report is configured to generate an evaluation report framework in response to the evaluation calculation results, integrate the evaluation calculation data and the corresponding evaluation calculation results into the evaluation report framework in a visual form, and output the evaluation report.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the method described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: It stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method described in any one of claims 1 to 8.

Images