Evaluation methods, devices, computer equipment, media and products for unmanned systems

Abstract

The present invention relates to the technical field of unmanned systems, and discloses an evaluation method, device, computer equipment, medium and product for unmanned systems. The evaluation method for unmanned systems includes: constructing an evaluation task in response to an evaluation request; the evaluation request includes an indicator tree identifier; according to the indicator tree identifier, querying a target indicator tree matching the evaluation task in multiple indicator trees in a database; the target indicator tree includes multiple evaluation indicators used to perform the evaluation task and the logical relationship between the multiple evaluation indicators; based on multiple evaluation indicators and logical relationships, creating a behavior tree, each evaluation indicator corresponds to a child node, and the logical relationship corresponds to the relationship between the multiple child nodes; using multiple child nodes on the behavior tree to process simulation test data, and obtain a target evaluation result. The present invention applies behavior trees to the evaluation of unmanned systems, improves the flexibility of the evaluation process, and improves the evaluation efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned systems, and in particular to an evaluation method, device, computer equipment, medium and product for an unmanned system. Background Art

[0002] With the development of artificial intelligence and machine learning technologies, unmanned systems are becoming more and more automated and intelligent. Unmanned systems refer to systems that can autonomously perform specific tasks in a certain environment without direct human involvement, such as unmanned equipment such as drones, unmanned ground vehicles, unmanned surface vessels or unmanned submarines. Intelligent algorithms are the core driving force of unmanned systems. Therefore, it is crucial to accurately and efficiently evaluate the performance of intelligent algorithms in unmanned systems.

[0003] At present, fixed rules or hierarchical indicator systems are usually used to evaluate the intelligent algorithms of unmanned systems. These solutions usually use preset evaluation indicators and rules to quantitatively calculate and analyze the performance of intelligent algorithms. However, there are defects in the way of preset evaluation indicators and rules. Due to the fixedness of preset evaluation indicators and rules, the evaluation scenarios are fixed, and the preset evaluation indicators are difficult to apply to new evaluation scenarios, resulting in low flexibility and low evaluation efficiency in the evaluation process. Summary of the invention

[0004] In view of this, the present invention provides an unmanned system evaluation method, apparatus, computer equipment, medium and product to solve the problems of low flexibility and low evaluation efficiency in the evaluation process.

[0005] In a first aspect, the present invention provides an evaluation method for an unmanned system, comprising: in response to an evaluation request, constructing an evaluation task; the evaluation request includes an indicator tree identifier; according to the indicator tree identifier, querying a target indicator tree matching the evaluation task in multiple indicator trees in a database; the target indicator tree includes multiple evaluation indicators used to perform the evaluation task and a logical relationship between the multiple evaluation indicators; based on the multiple evaluation indicators and the logical relationship, creating a behavior tree including a root node and multiple child nodes, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator; using the multiple child nodes on the behavior tree to process the simulation test data to obtain the target evaluation result.

[0006] The present invention responds to an evaluation request, constructs an evaluation task, queries a target indicator tree including multiple evaluation indicators used to perform the evaluation task and the logical relationship between the multiple evaluation indicators in multiple indicator trees in a database according to an indicator tree identifier in the evaluation request, creates a behavior tree including a root node and multiple child nodes based on multiple evaluation indicators and logical relationships, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child nodes are configured with the execution logic of the corresponding evaluation indicator, that is, the multiple child nodes process the simulation test data by executing the execution logic of the corresponding evaluation indicator to obtain the target evaluation result. The present invention applies the behavior tree to the evaluation task, and the behavior tree has a flexible node structure and multiple control functions. Compared with the related art, which usually performs quantitative calculation and analysis on the advantages and disadvantages of the intelligent algorithm by presetting the evaluation indicators and rules, the evaluation logic of multiple evaluation indicators is flexibly configured to the execution logic of multiple child nodes. Even if the scene of the unmanned system changes or the data is updated, it only needs to update the data processed by the multiple child nodes, and the behavior tree can still be used to perform the evaluation task. In the case of a change in the scene of the unmanned system, there is no need to redesign the code for evaluation, which reduces the risk of code intrusion, improves the flexibility of the evaluation process, and improves the evaluation efficiency.

[0007] In an optional implementation, based on multiple evaluation indicators and logical relationships, creating a behavior tree including a root node and multiple child nodes includes: using multiple evaluation indicators to create a root node and multiple child nodes; connecting the root node and the multiple child nodes according to the logical relationship to obtain an initial behavior tree; and configuring execution logic for the multiple child nodes according to the evaluation logic of the multiple evaluation indicators to obtain the behavior tree.

[0008] The present invention constructs a behavior tree that is the same as the multiple evaluation indicators and logical relationships of the indicator tree according to the multiple evaluation indicators and logical relationships of the indicator tree, and configures execution logic for multiple child nodes according to the evaluation logic of the multiple evaluation indicators, so that the multiple child nodes of the behavior tree are evaluated according to the execution logic, with high flexibility, simple execution process, and low error rate.

[0009] In an optional implementation, multiple child nodes on a behavior tree are used to process simulation test data to obtain a target evaluation result, including: executing execution logic corresponding to multiple child nodes on the behavior tree to process the simulation test data to obtain multiple execution results; and the multiple execution results form a target evaluation result.

[0010] In an optional implementation, multiple indicator trees and multiple evaluation indicators are stored in a database in the form of a preset table, and different evaluation tasks call corresponding indicator trees.

[0011] The multiple indicator trees of the present invention are pre-set, and different evaluation tasks call corresponding indicator trees, that is, any evaluation task can be completed by calling the corresponding indicator tree, which increases the number of completeable evaluation tasks. When setting the indicator tree, one evaluation indicator can be called by multiple indicator trees, realizing the reuse of evaluation indicators.

[0012] In an optional implementation, the evaluation request includes priority information; and the simulation test data is processed using multiple child nodes on the behavior tree, including: adding the evaluation task to the task execution queue according to the priority information; and controlling the multiple child nodes on the behavior tree corresponding to the evaluation task to process the simulation test data according to the task execution order in the task execution queue.

[0013] The present invention determines the execution priorities of different evaluation tasks according to the priority information, realizes the control of the urgency of the evaluation tasks, and improves the flexibility of the unmanned system evaluation.

[0014] In an optional implementation, the multiple sub-nodes include a timer node; the timer node is used to start timing when the simulation test data is processed using the multiple sub-nodes on the behavior tree until a preset time is reached, thereby terminating the evaluation task.

[0015] The timer node of the present invention is used to start timing when multiple child nodes on the behavior tree are used to process simulation test data until a preset time is reached, and then terminate the evaluation task, thereby preventing resource waste caused by long-term execution of an evaluation task and improving the utilization rate of computing resources and memory resources.

[0016] In an optional embodiment, the evaluation request includes an evaluation mode; the evaluation mode includes a real-time evaluation mode, a post-evaluation mode and an emergency evaluation mode; the real-time evaluation mode is a mode in which multiple child nodes on a behavior tree are used to process real-time simulation test data during the operation of the unmanned system; the post-evaluation mode is a mode in which multiple child nodes on a behavior tree are used to process simulation test data generated during the operation of the unmanned system after the operation of the unmanned system ends; the emergency evaluation mode is a mode in which, after the operation of the unmanned system ends, if the evaluation logic changes, the execution logic of the child nodes related to the evaluation indicators affected by the change in the evaluation logic is updated, and the historical simulation test data is processed using the updated multiple child nodes, or the historical simulation test data is processed in response to an immediate evaluation instruction.

[0017] The real-time evaluation mode of the present invention can instantly feedback the current state or performance of the unmanned system, realize real-time evaluation during the operation of the unmanned system scenario, greatly reduce the evaluation cycle, and can quickly adjust or optimize the intelligent algorithm of the unmanned system according to the real-time evaluation results. It can instantly reflect the algorithm effect during the execution of the unmanned system scenario, and synchronously display the overall score at the end of the unmanned system scenario operation, avoiding the situation of waiting for the unmanned system scenario to run completely before scoring, greatly improving the timeliness of the evaluation task. The post-evaluation mode of the present invention reviews and summarizes the unmanned system scenario operation that has occurred, which is convenient for analyzing the effect of the intelligent algorithm of the unmanned system. The emergency evaluation mode of the present invention can quickly evaluate the effect of the intelligent algorithm in emergencies or emergency situations. The three evaluation modes cover all-round needs from immediate feedback to long-term summary to emergency response, so as to adapt to different application scenarios, making the evaluation of unmanned systems more flexible and comprehensive. Whether it is daily response or emergency response, a suitable evaluation mode can be found to improve the effectiveness and pertinence of the unmanned system evaluation work.

[0018] In a second aspect, the present invention provides an evaluation device for an unmanned system, comprising: a task creation unit, for constructing an evaluation task in response to an evaluation request; the evaluation request comprises an indicator tree identifier; a target indicator tree query unit, for querying a target indicator tree matching the evaluation task from multiple indicator trees in a database according to the indicator tree identifier; the target indicator tree comprises multiple evaluation indicators used to perform the evaluation task and a logical relationship between the multiple evaluation indicators; a behavior tree creation unit, for creating a behavior tree comprising a root node and multiple child nodes based on multiple evaluation indicators and logical relationships, each evaluation indicator corresponding to a child node, the logical relationship corresponding to the relationship between the multiple child nodes, and the child node being configured with the execution logic of the corresponding evaluation indicator; a target evaluation result determination unit, for processing simulation test data using the multiple child nodes on the behavior tree to obtain a target evaluation result.

[0019] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the unmanned system evaluation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the unmanned system evaluation method of the first aspect or any corresponding embodiment thereof.

[0021] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the unmanned system evaluation method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a flow chart of an evaluation method for an unmanned system according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the evaluation index structure in a dynamic obstacle avoidance scenario according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of a behavior tree structure in a dynamic obstacle avoidance scenario according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of the operation flow of a behavior tree according to an embodiment of the present invention;

[0027] Figure 5 is a flow chart of another unmanned system evaluation method according to an embodiment of the present invention;

[0028] Figure 6 is a flow chart of another unmanned system evaluation method according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of the structure of an evaluation system for an unmanned system according to an embodiment of the present invention;

[0030] Figure 8 is a structural block diagram of an evaluation device for an unmanned system according to an embodiment of the present invention;

[0031] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] With the development of artificial intelligence and machine learning technologies, unmanned systems are becoming more and more automated and intelligent. Intelligent algorithms are the core driving force of unmanned systems. In this context, it is crucial to accurately and efficiently evaluate the performance of intelligent algorithms of unmanned systems.

[0034] At present, related technologies usually use fixed rules or hierarchical indicator systems to evaluate the intelligent algorithms of unmanned systems. These solutions usually use preset evaluation indicators and rules to quantify and calculate the pros and cons of intelligent algorithms. However, there are some defects in the method of presetting evaluation indicators and rules. First, the evaluation flexibility is low. Due to the fixedness of the preset evaluation indicators and rules, the evaluation scenario is fixed, and the preset evaluation indicators are difficult to use in new evaluation scenarios, resulting in low flexibility and low evaluation efficiency in the evaluation process. Secondly, the evaluation process is rigid, and the preset evaluation indicators and rules are difficult to adapt to the changing algorithm operation environment and evaluation requirements. Finally, the evaluation efficiency is low. Usually, the evaluation can only be carried out after the intelligent algorithm of the unmanned system has finished running, which is time-consuming and inefficient.

[0035] The embodiment of the present invention provides an evaluation method for an unmanned system, which improves the flexibility of the evaluation process and the evaluation efficiency by applying a behavior tree to the evaluation task.

[0036] According to an embodiment of the present invention, an embodiment of an evaluation method for an unmanned system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In this embodiment, an evaluation method for an unmanned system is provided, which can be used for computer equipment. Figure 1 is a flow chart of an evaluation method for an unmanned system according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0038] Step S101, constructing an evaluation task in response to an evaluation request; the evaluation request includes an indicator tree identifier.

[0039] Among them, the user sends or inputs the demand information to the back-end service module configured in the computer device, and the evaluation request is generated by the back-end service module according to the demand information; the computer device is also configured with an evaluation service module that is in communication with the back-end service module, and the evaluation service module receives the evaluation request sent by the back-end service module, triggering the construction process of the evaluation task. The evaluation task refers to the task of evaluating the specific tasks performed on the object under test in the simulation test project. For example, the simulation test project can be an unmanned vehicle simulation test project, and the evaluation task is a task evaluated based on the simulation test project. The simulation test project can specifically include an unmanned vehicle passing through an intersection test project, an unmanned vehicle overtaking test project, etc.

[0040] In some optional implementations, the evaluation request includes information such as an indicator tree identifier, a simulation task identifier, a round to be evaluated, a callback instruction, an evaluation type, and a task priority.

[0041] Among them, the metric tree identifier (Identity Document, ID) is the ID of the metric tree stored in the database. Exemplarily, the ID of the metric tree can be metric_tree_id, and the ID of each metric tree is different. The simulation task identifier is the ID (Identity Document, ID) of the simulation test task of the intelligent algorithm of the unmanned system to be evaluated. The round required for evaluation is the round of the simulation test task required for evaluation. The callback instruction is the instruction that needs to be called when returning the indicator evaluation result, such as the URL (Uniform Resource Locator) instruction. The evaluation type is the evaluation task mode, including real-time evaluation mode, post-evaluation mode and emergency evaluation mode. The task priority is used to judge the importance and execution order of the evaluation task. For example, the smaller the priority value, the higher the priority.

[0042] Exemplarily, as shown in Table 1, the parameter attributes of the evaluation request are illustrated.

[0043] Table 1: Parameter attributes of the evaluation request.

[0044]

[0045] Step S102, according to the indicator tree identifier, searching for a target indicator tree matching the evaluation task in a plurality of indicator trees in a database; the target indicator tree includes a plurality of evaluation indicators used to perform the evaluation task and the logical relationship between the plurality of evaluation indicators.

[0046] Among them, the database stores multiple indicator trees for executing different evaluation tasks, the target indicator tree is the indicator tree corresponding to the evaluation task among the multiple indicator trees in the database, each indicator tree includes multiple evaluation indicators used when executing the corresponding evaluation task and the logical relationship between the multiple evaluation indicators; according to the indicator tree identifier, the target indicator tree matching the evaluation task is queried in the multiple indicator trees.

[0047] In some optional implementations, the evaluation index refers to the smallest unit in the evaluation system, which is used to quantify a specific performance and is stored in a database, as shown in Table 2, which illustrates the properties of the evaluation index.

[0048] Table 2: Properties of evaluation metrics.

[0049]

[0050] For example, for the evaluation index of the unmanned system intelligent algorithm, when the unmanned system is an unmanned fighter and the operation scenario is a dynamic obstacle avoidance scenario, Figure 2 As shown in the figure, it is a schematic diagram of the evaluation index structure in the dynamic obstacle avoidance scenario. The evaluation index in the dynamic obstacle avoidance scenario includes two evaluation indicators: task difficulty and scene complexity. Furthermore, the task difficulty is calculated by the two evaluation indicators of task attribute and external threat, and the scene complexity is calculated by the complexity of obstacles. Furthermore, the task attribute is calculated by the two indicators of task time requirement and cargo carrying requirement, the external threat is calculated by the number of interference firepower, and the obstacle complexity is calculated by the obstacle density and obstacle speed. Among them, the dynamic obstacle avoidance scenario represents the scenario of avoiding obstacles during the flight of the unmanned fighter; the task time requirement represents the time requirement for completing the dynamic obstacle avoidance task; the cargo carrying requirement represents the weight and quantity of cargo that the unmanned fighter needs to carry; the number of interference firepower represents the number of firepower output by the enemy's unmanned equipment; the obstacle density represents the density of obstacles that need to be avoided; and the obstacle speed represents the running speed of obstacles that need to be avoided.

[0051] Exemplarily, as shown in Table 3, the attributes of the evaluation indicator task difficulty in a dynamic obstacle avoidance scenario are illustrated.

[0052] Table 3: Attributes of the task difficulty of evaluation indicators in the dynamic obstacle avoidance scenario.

[0053]

[0054] In some optional implementations, the indicator tree is a hierarchical structure composed of multiple evaluation indicators, which is used to represent the logical relationship between the evaluation indicators, as shown in Table 4, which illustrates the properties of the indicator tree.

[0055] Table 4: Properties of the indicator tree.

[0056]

[0057] Among them, the indicator node (MetricNodes) is the key component of the indicator tree, which constitutes the various nodes of the indicator tree. Each node refers to a specific evaluation indicator through the indicator object reference (MetricObjectRef) attribute contained in the MetricNode attribute, which points to the unique identifier of the evaluation indicator. MetricNodes defines the set of all nodes in the indicator tree, which contains one or more MetricNode elements, which represent each node in the indicator tree. Exemplarily, MetricNodes can be leaf nodes (leaf nodes are nodes without child nodes), directly referencing an evaluation indicator, or non-leaf nodes, serving as containers for other evaluation indicators.

[0058] Exemplarily, as shown in Table 5, the properties of the target indicator tree in the dynamic obstacle avoidance scenario are illustrated.

[0059] Table 5: Attributes of the target indicator tree in the dynamic obstacle avoidance scenario.

[0060]

[0061] In Table 5, the meaning of the nested structure after MetricNodes is that the task difficulty and scene complexity are referenced through the MetricObjectRef attribute contained in the MetricNode attribute; under the task difficulty, the task attributes and external threats are referenced through the MetricObjectRef attribute; under the task attributes, the task time requirements and cargo carrying requirements are referenced through the MetricObjectRef attribute; under the external threat attribute, the number of interference firepower is referenced through the MetricObjectRef attribute; under the scene complexity, the obstacle complexity is referenced through the MetricObjectRef attribute; under the obstacle complexity, the obstacle density and obstacle speed are referenced through the MetricObjectRef attribute.

[0062] In some optional implementations, multiple indicator trees and multiple evaluation indicators are stored in a database in the form of a preset table, and different evaluation tasks call corresponding indicator trees.

[0063] The preset tables are in the form of tables such as Table 4 and Table 5. The embodiment of the present invention defines the evaluation index and the index tree through a file in XML format, and the evaluation index and the index tree can be flexibly reconfigured through the XML file, which is convenient for visualization and user reference and modification.

[0064] In some optional implementations, the execution logic of the class name (ClassName) evaluation indicator is updated to obtain a new behavior tree.

[0065] The embodiment of the present invention updates the execution logic of the evaluation indicator in ClassName (the class name of the specific logic implementation of the evaluation indicator). It only needs to be re-evaluated using the post-evaluation mode after the change to take effect, and there is no need to re-run the simulation process. The multiple indicator trees of the embodiment of the present invention are pre-set, and different evaluation tasks call the corresponding indicator trees, that is, for any evaluation task, it can be completed by calling the corresponding indicator tree, which increases the number of complete evaluation tasks. When setting the indicator tree, one evaluation indicator can be called by multiple indicator trees, realizing the reuse of evaluation indicators.

[0066] Step S103, based on multiple evaluation indicators and logical relationships, create a behavior tree including a root node and multiple child nodes, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator.

[0067] The behavior tree includes a root node and multiple child nodes. Except for the root node, all other nodes are child nodes. Based on multiple evaluation indicators and logical relationships, each evaluation indicator corresponds to a child node. Multiple child nodes are connected according to the logical relationship, and the execution logic of the corresponding evaluation indicator is configured on the child node to obtain the behavior tree.

[0068] For example, a behavior tree for an evaluation task is constructed, starting from the root node, and the tree structure is designed based on multiple evaluation indicators and logical relationships. Specific execution logic is written for each child node, which can be to check environmental conditions or perform an action. The behavior tree is iterated in a loop. In each iteration, starting from the root node, each child node is updated in parallel until all child nodes return success or failure information.

[0069] In some optional implementations, control logics such as sequence, selector, and parallel may be used to organize multiple sub-nodes. For example, at least two of the sub-nodes are controlled to run in parallel.

[0070] In some optional implementations, each child node includes three states: initialization state, update state, and termination state. The initialization state is called only once in the life cycle of each node to initialize the resources and states required by the child node. The update state is the core state of each child node. Each iteration of the behavior tree will update the data processed by the child node and return success or failure information based on the execution result. When the evaluation index changes, the execution logic of the child node is updated accordingly.

[0071] In some optional implementations, the multiple sub-nodes include a timer node; the timer node is used to start timing when the simulation test data is processed using the multiple sub-nodes on the behavior tree until a preset time is reached, thereby terminating the evaluation task.

[0072] The preset duration may be a default duration or may be customized according to actual conditions.

[0073] The timer node in the embodiment of the present invention is used to start timing when multiple child nodes on the behavior tree are used to process simulation test data until a preset time is reached, terminating the evaluation task, preventing resource waste caused by long-term execution of an evaluation task, and improving the utilization rate of computing resources and memory resources.

[0074] For example, Figure 3 As shown, it is a schematic diagram of the behavior tree structure in a dynamic obstacle avoidance scenario, wherein the root node is an evaluation behavior tree, i.e., a behavior tree of an embodiment of the present invention, and the child nodes include a timer node, an indicator node, a task time requirement node, a cargo carrying node, an interference fire node, an obstacle density node, and an obstacle speed node, wherein the timer node and the indicator node are parallel nodes, i.e., the timer node and the indicator node execute calculation logic in parallel, and the task time requirement node, the cargo carrying node, the interference fire node, the obstacle density node, and the obstacle speed node are parallel nodes, i.e., the task time requirement node, the cargo carrying node, the interference fire node, the obstacle density node, and the obstacle speed node execute calculation logic in parallel.

[0075] Step S104, using multiple child nodes on the behavior tree to process the simulation test data to obtain a target evaluation result.

[0076] Among them, the simulation test data is the data used and obtained for the simulation test of the intelligent algorithm of the unmanned system; for example, the simulation test data can be the data related to the unmanned vehicle simulation test project, and the simulation test data is the operation data and task-related data generated during the unmanned vehicle simulation test.

[0077] In some optional implementations, the simulation test data is processed using multiple child nodes on the behavior tree, including: extracting target data used by the execution logic of each child node in the simulation test data, processing the target data according to the execution logic, and obtaining a target evaluation result.

[0078] Exemplarily, the execution logic of the child node is to compare the target data with the evaluation data.

[0079] In some optional implementations, the simulation test data includes database data and file data, the database data is stored in a database, and the file data is stored in a file storage system, and is shared with the evaluation task through a shared directory.

[0080] The database data includes general data and test data. General data is data related to the evaluation task, including but not limited to task operation data, task event information, etc. For example, general data includes task information, main vehicle attributes and other data. General data can be placed on a blackboard that can be shared by child nodes. The blackboard is a mechanism for sharing data based on the behavior tree, allowing each part of the behavior tree to access and modify global variables. In the embodiment of the present invention, the blackboard is used for child nodes to call general data, so as to realize the reuse of general data in all indicator nodes in a round of evaluation; test data includes real-time operation data and real-time event data of simulation test.

[0081] Exemplarily, as shown in Table 6, an example of real-time operation data is illustrated.

[0082] Table 6: Example of real-time run data.

[0083]

[0084] Exemplarily, as shown in Table 7, an example of real-time event data is illustrated.

[0085] Table 7: Example of real-time event data.

[0086]

[0087] In an embodiment of the present invention, file data includes but is not limited to a road network file, a simulation operation log file, etc. Exemplarily, the road network file may be an OpenDRIVE file, which is a standardized format for describing a road network. Static traffic network elements such as the geometry of roads, lanes, and intersections are described in detail. OpenDRIVE files use the XML format (file extension .xodr) to store data, which can be used to simulate a real road traffic environment in simulation software. The simulation operation log file may be a carlalog (simulator log) file, which is usually created by a simulation simulator to record relevant information of the current round of simulation operation, such as sensor data, the position, speed, acceleration, etc. of the vehicle under test.

[0088] In some optional implementations, each time the behavior tree is iterated, the evaluation service module calls simulation test data from the database or file storage system to ensure that updated road network information, scenario information, etc. are applied to this evaluation task.

[0089] In some optional implementations, after the simulation test data is processed using multiple child nodes on the behavior tree to obtain a target evaluation result, the unmanned system evaluation method further includes: sending the target evaluation result to a backend service module according to a callback instruction.

[0090] Among them, the callback instruction is the instruction that needs to be called when returning the indicator evaluation result, such as the URL (Uniform Resource Locator) instruction. The parameters in the callback instruction include: task identifier, indicator tree identifier, indicator evaluation task result and indicator result, etc., as shown in Table 8, which illustrates the callback instruction parameters.

[0091] Table 8: Callback instruction parameters.

[0092]

[0093] like Figure 4 The figure shows a schematic diagram of the operation process of the behavior tree. The indicator tree definition table and the evaluation indicator definition table written by the backend service module are stored in the database, and the real-time operation data and real-time events, task information and static data written by the scenario service are stored in the database; the road network file and the log file written by the scenario service are stored in the file storage system; the external application calls the indicator evaluation service, and the evaluation manager calls the data in the database to build the behavior tree, and runs the behavior tree based on the data in the database and the file storage system, summarizes the results of each round of indicator evaluation, obtains the target evaluation result, and returns the target evaluation result to the external application.

[0094] The unmanned system evaluation method provided in this embodiment is a method for evaluating the performance of an unmanned system intelligent algorithm. The execution steps of the method include: in response to an evaluation request, constructing an evaluation task; querying a target indicator tree including multiple evaluation indicators used to execute the evaluation task and the logical relationship between the multiple evaluation indicators in multiple indicator trees in a database according to the indicator tree identifier in the evaluation request; based on the multiple evaluation indicators and the logical relationship, creating a behavior tree including a root node and multiple child nodes, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator, that is, the multiple child nodes process the simulation test data by executing the execution logic of the corresponding evaluation indicator to obtain the target evaluation result. The embodiment of the present invention applies a behavior tree to an evaluation task. The behavior tree has a flexible node structure and multiple control functions. Compared with the related art that usually performs quantitative calculation and analysis on the pros and cons of an intelligent algorithm through preset evaluation indicators and rules, the evaluation logic of multiple evaluation indicators is flexibly configured to the execution logic of multiple sub-nodes. Even if the scenario of the unmanned system changes or the data is updated, it is only necessary to update the data processed by the multiple sub-nodes to still use the behavior tree to perform the evaluation task. When the scenario of the unmanned system changes, there is no need to redesign the code for evaluation, which reduces the risk of code intrusion, improves the flexibility of the evaluation process, and improves the evaluation efficiency.

[0095] In this embodiment, an evaluation method for an unmanned system is provided, which can be used for computer equipment. Figure 5 is a flow chart of another unmanned system evaluation method according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0096] Step S501: In response to an evaluation request, construct an evaluation task; the evaluation request includes an indicator tree identifier. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0097] Step S502: According to the indicator tree identifier, a target indicator tree matching the evaluation task is searched in a database of multiple indicator trees; the target indicator tree includes multiple evaluation indicators used to perform the evaluation task and the logical relationship between the multiple evaluation indicators. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0098] Step S503, based on multiple evaluation indicators and logical relationships, create a behavior tree including a root node and multiple child nodes, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator.

[0099] Specifically, the above step S503 includes:

[0100] Step S5031, using multiple evaluation indicators, create a root node and multiple child nodes.

[0101] Step S5032: Connect the root node and multiple child nodes according to the logical relationship to obtain an initial behavior tree.

[0102] Step S5033: configuring execution logic for multiple sub-nodes according to the evaluation logic of multiple evaluation indicators to obtain a behavior tree.

[0103] Step S504: Process the simulation test data using multiple child nodes on the behavior tree to obtain a target evaluation result.

[0104] Specifically, the above step S504 includes:

[0105] Step S5041, executing execution logics corresponding to multiple child nodes on the behavior tree to process simulation test data to obtain multiple execution results.

[0106] Step S5042, multiple execution results form a target evaluation result.

[0107] In some optional implementations, forming a target evaluation result from a plurality of execution results further includes: combining the plurality of execution results according to weights of the plurality of execution results to obtain the target evaluation result.

[0108] In some optional implementations, the evaluation request includes priority information; and the simulation test data is processed using multiple child nodes on the behavior tree, including: adding the evaluation task to the task execution queue according to the priority information; and controlling the multiple child nodes on the behavior tree corresponding to the evaluation task to process the simulation test data according to the task execution order in the task execution queue.

[0109] Exemplarily, the priority attribute is searched from the evaluation request to obtain the priority information, and the evaluation task is added to the task execution queue according to the priority information. With the help of the task execution queue, the evaluation service module can intelligently determine the execution order of the evaluation tasks, and control the multiple sub-nodes on the behavior tree corresponding to the evaluation task to process the simulation test data according to the task execution order in the task execution queue, so as to ensure that the high-priority tasks are executed first. The evaluation service module takes out the evaluation task from the task execution queue, and can execute multiple evaluation tasks in parallel, each evaluation task corresponding to the calculation of a behavior tree. The simulation test process in the embodiment of the present invention includes one or more evaluation tasks. During the simulation test operation, new evaluation tasks can be added at any time according to demand, so as to realize the control of the urgency of the evaluation tasks and improve the flexibility of the unmanned system evaluation.

[0110] In some optional embodiments, the evaluation request includes an evaluation mode; the evaluation modes include real-time evaluation mode, post-evaluation mode and emergency evaluation mode; the real-time evaluation mode is a mode in which real-time simulation test data is processed using multiple child nodes on a behavior tree during the operation of the unmanned system; the post-evaluation mode is a mode in which simulation test data generated during the operation of the unmanned system is processed using multiple child nodes on a behavior tree after the operation of the unmanned system ends; the emergency evaluation mode is a mode in which, after the operation of the unmanned system ends, if the evaluation logic changes, the execution logic of the child nodes related to the evaluation indicators affected by the change in the evaluation logic is updated, and the historical simulation test data is processed using the updated multiple child nodes, or the historical simulation test data is processed in response to an immediate evaluation instruction.

[0111] Among them, the real-time evaluation mode generates real-time evaluation results during the simulation test of the unmanned system, which can accurately capture the performance of the intelligent algorithm during the simulation process. For specific indicators (such as patrol area), calculations can be performed during the simulation test, so that the evaluation results can be produced immediately after the simulation test ends, thereby greatly saving evaluation time. In addition, the indicator data can be presented in real time during the simulation test, which is convenient for immediate adjustment of the algorithm.

[0112] The post-evaluation mode is an evaluation conducted after the simulation test run is completed, which evaluates the overall performance of the intelligent algorithm to obtain a comprehensive performance of the intelligent algorithm.

[0113] The emergency assessment mode is an assessment of historical simulation test data. It can quickly re-evaluate historical data after updating the assessment indicators, avoiding repeated running of simulation tests, or immediately conduct an assessment in response to an immediate assessment instruction. When an immediate assessment instruction is received, the assessment task of the unmanned system is started without waiting in line, which significantly improves the assessment efficiency.

[0114] In some optional implementations, when the evaluation mode is a post-evaluation mode or an emergency evaluation mode, the evaluation service module assembles the evaluation results into a form consistent with the indicator tree after the evaluation is completed and returns them to the backend service module through a callback instruction.

[0115] The evaluation method for unmanned systems provided in this embodiment, the real-time evaluation mode can immediately feedback the current state or performance of the unmanned system, realize the real-time evaluation during the operation of the unmanned system scenario, greatly reduce the evaluation cycle, and can quickly adjust or optimize the intelligent algorithm of the unmanned system according to the real-time evaluation results, and can immediately reflect the algorithm effect during the execution of the unmanned system scenario, and synchronously display the overall score at the end of the unmanned system scenario operation, avoiding the situation of waiting for the unmanned system scenario to run completely before scoring, greatly improving the timeliness of the evaluation task. The post-evaluation mode of the present invention reviews and summarizes the unmanned system scenario operation that has occurred, which is convenient for analyzing the effect of the intelligent algorithm of the unmanned system. The emergency evaluation mode of the present invention can quickly evaluate the effect of the intelligent algorithm in emergencies or emergency situations. The three evaluation modes cover the full range of needs from immediate feedback to long-term summary to emergency response, so as to adapt to different application scenarios, making the evaluation of unmanned systems more flexible and comprehensive. Whether it is daily response or emergency response, a suitable evaluation mode can be found to improve the effectiveness and pertinence of the unmanned system evaluation work.

[0116] In this embodiment, an evaluation method for an unmanned system is provided, which can be used for computer equipment. Figure 6 is a flow chart of another unmanned system evaluation method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0117] Step S601, constructing a standard format file of evaluation indicators and indicator tree according to the evaluation system, and storing it in a database.

[0118] Among them, indicators and indicator trees are defined and managed through the XML standard format. Evaluation indicators refer to the smallest unit in the evaluation system, which are used to quantify specific performance and exist in the database.

[0119] In some optional implementations, the indicator tree is a hierarchical structure composed of multiple evaluation indicators, which is used to represent the logical relationship between the evaluation indicators.

[0120] Step S602, sending an evaluation request to the evaluation service module, the evaluation request includes a simulation task identifier, a round to be evaluated, an indicator tree identifier, a callback instruction, an evaluation type and a task priority, etc.

[0121] Step S603, construct an evaluation task according to the evaluation request, search for the corresponding indicator tree, and determine the execution order of the evaluation task.

[0122] Among them, the database is queried according to the indicator tree identifier parameter in the evaluation request to find the corresponding indicator tree.

[0123] Step S604: construct a behavior tree, including a timer node and an indicator node, where each evaluation indicator corresponds to an indicator node in the behavior tree.

[0124] Among them, the indicator node in the embodiment of the present invention is a child node.

[0125] In some optional implementations, a behavior tree for the evaluation task is constructed, starting from a root node, the tree structure is designed according to the evaluation task to be performed, and specific execution logic is written for each child node, which may be to check environmental conditions or perform an action.

[0126] Exemplarily, the behavior tree includes a timer node and an indicator node. The timer node is used to customize the execution time of the evaluation task, and returns a null value when the task times out; the indicator node is used to calculate the evaluation indicator.

[0127] Step S605, starting from the root node of the behavior tree, execute an evaluation task and execute the execution logic of each child node in parallel until all nodes return success or failure information.

[0128] In some optional implementations, the evaluation service data is retrieved from a database or a file system, and the execution logic of each child node is executed in parallel starting from the root node according to the evaluation service data until all child nodes return success or failure information.

[0129] Step S606: According to the indicator calculation results of each child node of the behavior tree, the final evaluation result is summarized and returned to the backend service module.

[0130] Among them, the final evaluation result is returned to the back-end service module according to the callback instruction.

[0131] The present invention innovatively applies behavior trees to the evaluation system, reduces code intrusion into the scene operation service, and significantly improves the efficiency and accuracy of the evaluation. Using different types of nodes in the behavior tree to perform different operations is more flexible than the traditional hierarchical indicator structure.

[0132] In this embodiment, an evaluation system for an unmanned system is provided, which can be used for computer equipment, Figure 7 is a schematic diagram of the structure of an evaluation system for an unmanned system according to an embodiment of the present invention. Figure 7 As shown, the system includes: a scene controller, a database, a file system, a backend service module, and an evaluation service module; the backend service module is communicatively connected with the evaluation service module.

[0133] Among them, the scenario controller is used for simulation testing of the intelligent algorithms of unmanned systems and for generating simulation test data required for evaluation tasks.

[0134] The database is used to store the simulation test data required by the scenario controller to generate the evaluation task, and is used to store and provide the indicator tree and evaluation indicators to the evaluation service module.

[0135] The file system is used to store file data used by the scenario controller to generate evaluation tasks and to provide file data to the evaluation service module.

[0136] The backend service module is used to send evaluation requests to the evaluation service module.

[0137] The evaluation service module is used to construct an evaluation task according to the evaluation request, and to query the target indicator tree matching the evaluation task in multiple indicator trees in the database, and to create a behavior tree including a root node and multiple child nodes based on multiple evaluation indicators and logical relationships. It is also used to process the simulation test data using the multiple child nodes on the behavior tree to obtain the target evaluation result.

[0138] The evaluation service module is used to return the target evaluation results to the backend service module.

[0139] In this embodiment, an evaluation device for an unmanned system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0140] This embodiment provides an evaluation device for an unmanned system, such as Figure 8 As shown, including:

[0141] The task creation unit 801 is used to construct an evaluation task in response to an evaluation request; the evaluation request includes an indicator tree identifier.

[0142] The target indicator tree query unit 802 is used to query a target indicator tree matching the evaluation task from multiple indicator trees in the database according to the indicator tree identifier; the target indicator tree includes multiple evaluation indicators used to perform the evaluation task and the logical relationship between the multiple evaluation indicators.

[0143] The behavior tree creation unit 803 is used to create a behavior tree including a root node and multiple child nodes based on multiple evaluation indicators and logical relationships, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator.

[0144] The target evaluation result determination unit 804 is used to process the simulation test data using multiple child nodes on the behavior tree to obtain a target evaluation result.

[0145] In some optional implementations, the behavior tree creation unit 803 includes:

[0146] The node creation subunit is used to create a root node and multiple child nodes using multiple evaluation indicators.

[0147] The node connection subunit is used to connect the root node and multiple child nodes according to the logical relationship to obtain the initial behavior tree.

[0148] The node configuration subunit is used to configure execution logic for multiple subnodes according to the evaluation logic of multiple evaluation indicators to obtain a behavior tree.

[0149] In some optional implementations, the target evaluation result determination unit 804 includes:

[0150] The data processing subunit is used to execute the execution logic corresponding to the multiple sub-nodes on the behavior tree to process the simulation test data and obtain multiple execution results.

[0151] The result combination subunit is used to compose a target evaluation result from multiple execution results.

[0152] In some optional embodiments, the evaluation device of the unmanned system further comprises:

[0153] The sequential execution unit is used to add the evaluation task to the task execution queue according to the priority information; and control the multiple child nodes on the behavior tree corresponding to the evaluation task to process the simulation test data according to the task execution order in the task execution queue.

[0154] The further functional description of each of the above units is the same as that of the above corresponding embodiments and will not be repeated here.

[0155] The evaluation device of the unmanned system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0156] The embodiment of the present invention also provides a computer device having the above Figure 8 An evaluation device for an unmanned system is shown.

[0157] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0158] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0159] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0160] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0162] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0163] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0164] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0165] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An evaluation method for an unmanned system, characterized in that: The method comprises: In response to an evaluation request, constructing an evaluation task; the evaluation request includes an indicator tree identifier; According to the indicator tree identifier, a target indicator tree matching the evaluation task is searched in a database among a plurality of indicator trees; the target indicator tree includes a plurality of evaluation indicators used to perform the evaluation task and a logical relationship between the plurality of evaluation indicators; Based on the multiple evaluation indicators and the logical relationship, create a behavior tree including a root node and multiple child nodes, each evaluation indicator corresponds to a child node, the logical relationship corresponds to the relationship between the multiple child nodes, and the child node is configured with the execution logic of the corresponding evaluation indicator; Processing simulation test data using the multiple child nodes on the behavior tree to obtain a target evaluation result; The creating a behavior tree including a root node and a plurality of child nodes based on the plurality of evaluation indicators and the logical relationship comprises: Using the multiple evaluation indicators, create the root node and the multiple child nodes; According to the logical relationship, the root node and the plurality of child nodes are connected to obtain an initial behavior tree; The execution logic is configured for the multiple sub-nodes according to the evaluation logic of the multiple evaluation indicators to obtain the behavior tree.

2. The method according to claim 1, characterized in that The using multiple child nodes on the behavior tree to process the simulation test data to obtain a target evaluation result includes: Execute the execution logic corresponding to the multiple sub-nodes on the behavior tree to process the simulation test data to obtain multiple execution results; The target evaluation result is composed of the multiple execution results.

3. The method according to claim 1, characterized in that The multiple indicator trees and the multiple evaluation indicators are stored in a database in the form of a preset table, and different evaluation tasks call corresponding indicator trees.

4. The method according to claim 1, characterized in that The evaluation request includes priority information; and the processing of the simulation test data using the multiple sub-nodes on the behavior tree includes: According to the priority information, adding the evaluation task to the task execution queue; According to the task execution order in the task execution queue, the multiple sub-nodes on the behavior tree corresponding to the evaluation task are controlled to process the simulation test data.

5. The method according to claim 1, characterized in that The multiple sub-nodes include a timer node; the timer node is used to start timing when the simulation test data is processed using the multiple sub-nodes on the behavior tree until a preset time is reached, thereby terminating the evaluation task.

6. The method according to claim 1, characterized in that The evaluation request includes an evaluation mode; the evaluation mode includes a real-time evaluation mode, a post-evaluation mode and an emergency evaluation mode; The real-time evaluation mode is a mode in which the real-time simulation test data is processed by using the multiple sub-nodes on the behavior tree during the operation of the unmanned system; The post-evaluation mode is a mode in which the simulation test data generated during the operation of the unmanned system is processed using the multiple sub-nodes on the behavior tree after the operation of the unmanned system ends; The emergency assessment mode is a mode in which, after the operation of the unmanned system is completed, if the assessment logic changes, the execution logic of the sub-nodes related to the assessment indicators affected by the change in the assessment logic is updated, and the historical simulation test data is processed using the updated multiple sub-nodes, or the historical simulation test data is processed in response to an immediate assessment instruction.

7. An evaluation device for an unmanned system, characterized in that: The device comprises: A task creation unit, configured to construct an evaluation task in response to an evaluation request; the evaluation request includes an indicator tree identifier; A target indicator tree query unit is used to query a target indicator tree matching the evaluation task from a plurality of indicator trees in a database according to the indicator tree identifier; the target indicator tree includes a plurality of evaluation indicators used to perform the evaluation task and a logical relationship between the plurality of evaluation indicators; a behavior tree creation unit, configured to create a behavior tree including a root node and a plurality of child nodes based on the plurality of evaluation indicators and the logical relationship, wherein each evaluation indicator corresponds to a child node, the logical relationship corresponds to a relationship between the plurality of child nodes, and the child node is configured with execution logic corresponding to the evaluation indicator; a target evaluation result determination unit, configured to process the simulation test data using the multiple sub-nodes on the behavior tree to obtain a target evaluation result; The Behavior Tree Creation Unit includes: A node creation subunit is used to create a root node and multiple child nodes using multiple evaluation indicators; The node connection sub-unit is used to connect the root node and multiple child nodes according to the logical relationship to obtain the initial behavior tree; The node configuration subunit is used to configure execution logic for multiple subnodes according to the evaluation logic of multiple evaluation indicators to obtain a behavior tree.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the unmanned system evaluation method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the unmanned system evaluation method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the unmanned system evaluation method according to any one of claims 1 to 6.

Images