Intelligent level evaluation and verification system for unmanned aerial vehicle

By designing the drone intelligence level evaluation and verification system, using simulation technology and black box models to evaluate the intelligence level of the drone, the problem of missing drone intelligence evaluation standards is solved, and effective evaluation and grading of the intelligence level is achieved .

CN119067501AInactive Publication Date: 2024-12-03THE QUARTERMASTER RES INST OF THE GENERAL LOGISTICS DEPT OF THE CPLA
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411206138.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The lack of unified drone intelligence level evaluation standards and evaluation methods in China, and the inability to effectively evaluate the intelligence level of drones, resulting in the inability to verify its intelligence level.

Method used

A set of intelligent level evaluation and verification systems for drones are designed, including assessment and design subsystems, subject loading subsystems, deduction and simulation subsystems, evaluation and computing subsystems and system management subsystems, so as to realize the drone through test simulation. Grading evaluation of intelligence level, using black box model to send dynamic data, select testing options for human-computer interaction interface, evaluate question library construction rules, simulation engine performs simulation deduction, and evaluation calculation subsystem generates evaluation reports.

Benefits of technology

The evaluation and classification of the intelligence level of drones has been realized, platform support is provided for verification of the intelligence level of drones, and technical routes for intelligent level evaluation systems suitable for equipment in various fields have been explored.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067501A_ABST
    Figure CN119067501A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of unmanned intelligence, in particular to an intelligent level evaluation and verification system for an unmanned aerial vehicle. According to the unmanned aerial vehicle intelligent level evaluation and verification system, grading evaluation of the intelligent level of an unmanned aerial vehicle is achieved in a test simulation mode, and the system comprises an evaluation design subsystem, a subject loading subsystem, a deduction simulation subsystem, an evaluation calculation subsystem and a system management subsystem. According to the intelligent level evaluation system for the unmanned aerial vehicle, the intelligent degree of the unmanned aerial vehicle can be evaluated, platform support is provided for verifying the intelligent level of the unmanned aerial vehicle, and a technical route is explored for constructing the intelligent level evaluation system suitable for equipment in various fields.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of unmanned intelligent technology, and in particular to an unmanned aerial vehicle intelligence level assessment and verification system. Background Art

[0002] With the development of artificial intelligence technology, the importance of measuring the intelligence level of drones has become increasingly prominent. In recent years, my country has initially studied and formed equipment intelligence level classification standards, methods and processes, but its implementation lacks effective tools and means to support it. It is urgent to use typical intelligent equipment as the test object to build a set of equipment intelligence level classification evaluation and verification system.

[0003] As a typical intelligent equipment, drones have been increasingly widely used in daily life and wars. However, there are many types of drones in China with varying intelligence levels, and there is a lack of unified evaluation standards and methods. Therefore, it is urgent to build a drone intelligence level evaluation and verification system to explore the system architecture, functional elements, operating relationships and integration modes, and to achieve a graded evaluation of the drone intelligence level through semi-physical simulation testing.

[0004] Therefore, it is necessary to use drones as typical test objects, build a drone intelligence level assessment system, evaluate the intelligence level of drones, provide platform support for verifying the intelligence level of drones, and explore technical routes for building an intelligence level assessment system suitable for equipment in various fields. Summary of the invention

[0005] The purpose of the present invention is to construct a UAV intelligence level assessment system to assess the intelligence level of UAVs and provide a platform support for verifying the intelligence level of UAVs.

[0006] The technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a UAV intelligence level evaluation and verification system, which realizes a hierarchical evaluation of the UAV intelligence level by means of test simulation, and the system includes an evaluation design subsystem, a subject loading subsystem, a deduction simulation subsystem, an evaluation calculation subsystem and a system management subsystem;

[0008] Each subsystem is connected to the UAV device under test and evaluation through a physical interface and transmits data through a communication protocol. The dynamic model of the UAV under test is replaced by a black box model software program; the black box model sends the dynamic data of the UAV under test;

[0009] The assessment design subsystem includes a human-computer interaction interface, an assessment question library, and an assessment subject generator module; the assessment design subsystem has assessment scenario and subject generation functions, and can provide a human-computer interaction interface; it automatically generates preset subjects according to the content selected by the tested party, and the preset subjects include assessment scenarios and assessment questions;

[0010] The human-computer interaction interface provides test options for the evaluator, including the type of equipment to be tested, test scenario, test level and number of equipment; the evaluator selects the corresponding option according to the equipment to be tested, and the test option information of the human-computer interaction interface is transmitted to the test question library and the assessment subject generator, and the two modules configure the test environment scenario according to the selected option; each environment scenario is set with several difficulty assessment levels;

[0011] The evaluation question database provides question construction rules for the selected scenario according to the evaluation level of the drone to be evaluated, including the number range of targets and the density of obstacles. Different evaluation levels correspond to different question construction rules. The evaluation question database receives the scenario and evaluation level information sent by the human-computer interaction interface, and outputs the question construction rules for the corresponding scenario and level to the evaluation subject generator.

[0012] The assessment subject generator receives the assessment environment and assessment level data sent by the human-computer interface, and at the same time receives the information of the assessment question library, and configures the test subjects, including the assessment environment and assessment task information under the corresponding scenarios and corresponding assessment levels; the assessment subject generator module has the functions of loading the test environment, loading the test questions, and generating the assessment subjects; the assessment subject generator sends the test question information to the scenario generator and configurator in the simulation system to perform simulation assumption management and battlefield environment setting work;

[0013] The subject loading subsystem includes a subject loader, a simulation model library, and a scenario fusion generation and configurator module; the subject loading subsystem has a test scenario visualization function to construct simulation scenarios under different backgrounds;

[0014] The simulation model library is a database of simulation scenario models, providing basic models as the basis for scenario configuration generation during the evaluation simulation process;

[0015] The scene fusion generation and configurator module builds a 3D scene dynamic generation algorithm based on the test topic information generated by the assessment subject generator and calls the required model primitives or materials from the simulation model library to generate a 3D scene in real time;

[0016] The subject loader integrates the models and scenarios in the simulation model library and loads the configurator module data information to generate scenario information;

[0017] The simulation subsystem includes a simulation engine, which has a built-in time simulator and event simulator. The input of the simulation engine includes the simulation scene output by the subject loading subsystem and the output data of the drone black box. The time simulator and the event simulator judge the results of several events in the simulation process according to the logic of the event, and output the results to the integrated console of the system management subsystem. The simulation subsystem realizes the simulation function in the evaluation process.

[0018] The evaluation calculation subsystem includes an evaluation index calculation and evaluation report generation module and an index list and a scoring standard library;

[0019] The indicator list and scoring standard library integrate pre-defined scoring standards for different evaluation scenarios and different evaluation levels; this module passes the corresponding scoring standards to the evaluation indicator calculation and evaluation report generation module according to the level of the selected question;

[0020] The evaluation index calculation and evaluation report generation module calculates the score of the tested equipment according to the scoring criteria in the index list and the scoring criteria library and the information collected by the data collection module, through the evaluation algorithm and according to the received simulation data and the scoring criteria, generates an evaluation report, and gives the final evaluation result;

[0021] The system management subsystem includes a data conversion module, a data acquisition module and an integrated console module; the system management subsystem realizes the evaluation data conversion and acquisition functions, and at the same time realizes the overall integrated control function of the system;

[0022] The integrated console module realizes the overall integrated control function of the system and controls the start, pause, continue and end of the overall simulation process of the system;

[0023] The data acquisition module collects command information from the UAV ground station, the attitude, position, speed data of the UAV during the evaluation simulation, video stream data and flight log data, stores these data, and sends them to the evaluation index calculation and evaluation report generation module at the same time;

[0024] The data conversion module processes the data information sent by the drone board in a unified data format; the data conversion module interacts with the black box model, the drone sensor model and the data acquisition module, and passes the converted data that meets the requirements to the above modules for processing.

[0025] Preferably, the simulation scenes include mountains, cities, jungles, grasslands, oceans, straits, deserts, and polar regions.

[0026] Preferably, the black box model sends dynamic data of the tested drone including position, attitude, speed, acceleration and angular velocity.

[0027] Preferably, the UAV intelligence level assessment and verification system is connected to a number of different UAVs; the data conversion module sent by the UAV boards of the different units converts the data into a unified data format for processing according to the MAVLink protocol.

[0028] Preferably, the subject loading subsystem, indicator list and scoring standard library set up subject scenario data import and subject manual adjustment and modification modules.

[0029] Preferably, the difficulty assessment level is divided into six levels from L1 to L6, and different levels of scenes are distinguished by the presence, type, density or quantity of indicators. The scene indicators include weather, buildings, roads, bridges, vegetation, drones, unmanned vehicles, targets, water systems, topography, trees, interferences, and obstructions.

[0030] Preferably, the evaluation and calculation subsystem has a built-in mapping relationship system between the completion indicators of drones at different difficulty evaluation levels and the intelligence level to form evaluation criteria.

[0031] Furthermore, the evaluation calculation subsystem quantifies the performance of the drone's completion indicators and sets corresponding weights, obtains a total evaluation score through a series of weighted summations, and corresponds the evaluation score to the intelligence level.

[0032] Preferably, the drone issues external instructions via a control station or a remote controller.

[0033] In a second aspect, the present invention provides a use of the UAV intelligence level evaluation and verification system according to the first aspect, which specifically includes the following stages:

[0034] Evaluation preparation stage: The drone to be evaluated is connected to the system, and the number of equipment, number of evaluations, test environment and evaluation level are selected through the human-computer interaction interface;

[0035] Environment construction stage: The subject loading subsystem generates subjects and scenarios based on the number of equipment, number of evaluations, test environment, and evaluation level;

[0036] Evaluation and simulation stage: the simulation subsystem is simulated and evaluated;

[0037] Evaluation and assessment stage: The evaluation calculation subsystem generates an evaluation report based on the output data of the deduction and simulation subsystem.

[0038] Beneficial effects achieved by the present invention:

[0039] The present invention constructs a UAV intelligence level assessment system to assess the intelligence level of UAVs, provide a platform support for verifying the intelligence level of UAVs, and explore technical routes for building an intelligence level assessment system applicable to equipment in various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Functional composition diagram of the UAV intelligence level grading assessment system;

[0041] Figure 2 UAV intelligence level grading assessment system architecture diagram;

[0042] Figure 3 Evaluation and verification system operation flow chart;

[0043] Figure 4 Framework scene map generated by the evaluation design system

[0044] Figure 5 Overall scene map

[0045] Figure 6 Scene details in the scene map

[0046] Figure 7 Pan and rotate buildings in the map. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below based on the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more clear. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] Example 1

[0049] A UAV intelligence level evaluation and verification system realizes a hierarchical evaluation of the UAV intelligence level through test simulation. The system includes an evaluation design subsystem, a subject loading subsystem, a deduction simulation subsystem, an evaluation calculation subsystem and a system management subsystem.

[0050] Each subsystem is connected to the UAV device under test and evaluation through a physical interface and transmits data through a communication protocol. The dynamic model of the UAV under test is replaced by a black box model software program; the black box model sends the dynamic data of the UAV under test;

[0051] The assessment design subsystem includes a human-computer interaction interface, an assessment question library, and an assessment subject generator module; the assessment design subsystem has assessment scenario and subject generation functions, and can provide a human-computer interaction interface; it automatically generates preset subjects according to the content selected by the tested party, and the preset subjects include assessment scenarios and assessment questions;

[0052] The human-computer interaction interface provides test options for the evaluator, including the type of equipment to be tested, test scenario, test level and number of equipment; the evaluator selects the corresponding option according to the equipment to be tested, and the test option information of the human-computer interaction interface is transmitted to the test question library and the assessment subject generator, and the two modules configure the test environment scenario according to the selected option; each environment scenario is set with several difficulty assessment levels;

[0053] The evaluation question database provides question construction rules for the selected scenario according to the evaluation level of the drone to be evaluated, including the number range of targets and the density of obstacles. Different evaluation levels correspond to different question construction rules. The evaluation question database receives the scenario and evaluation level information sent by the human-computer interaction interface, and outputs the question construction rules for the corresponding scenario and level to the evaluation subject generator.

[0054] The assessment subject generator receives the assessment environment and assessment level data sent by the human-computer interface, and at the same time receives the information of the assessment question library, and configures the test subjects, including the assessment environment and assessment task information under the corresponding scenarios and corresponding assessment levels; the assessment subject generator module has the functions of loading the test environment, loading the test questions, and generating the assessment subjects; the assessment subject generator sends the test question information to the scenario generator and configurator in the simulation system to perform simulation assumption management and battlefield environment setting work;

[0055] The subject loading subsystem includes a subject loader, a simulation model library, and a scenario fusion generation and configurator module; the subject loading subsystem has a test scenario visualization function to construct simulation scenarios under different backgrounds;

[0056] The simulation model library is a database of simulation scenario models, providing basic models as the basis for scenario configuration generation during the evaluation simulation process;

[0057] The scene fusion generation and configurator module builds a 3D scene dynamic generation algorithm based on the test topic information generated by the assessment subject generator and calls the required model primitives or materials from the simulation model library to generate a 3D scene in real time;

[0058] The subject loader integrates the models and scenarios in the simulation model library and loads the configurator module data information to generate scenario information;

[0059] The simulation subsystem includes a simulation engine, which has a built-in time simulator and event simulator. The input of the simulation engine includes the simulation scene output by the subject loading subsystem and the output data of the drone black box. The time simulator and the event simulator judge the results of several events in the simulation process according to the logic of the event, and output the results to the integrated console of the system management subsystem. The simulation subsystem realizes the simulation function in the evaluation process.

[0060] The evaluation calculation subsystem includes an evaluation index calculation and evaluation report generation module and an index list and a scoring standard library;

[0061] The indicator list and scoring standard library integrate pre-defined scoring standards for different evaluation scenarios and different evaluation levels; this module passes the corresponding scoring standards to the evaluation indicator calculation and evaluation report generation module according to the level of the selected question;

[0062] The evaluation index calculation and evaluation report generation module calculates the score of the tested equipment according to the scoring criteria in the index list and the scoring criteria library and the information collected by the data collection module, through the evaluation algorithm and according to the received simulation data and the scoring criteria, generates an evaluation report, and gives the final evaluation result;

[0063] The system management subsystem includes a data conversion module, a data acquisition module and an integrated console module; the system management subsystem realizes the evaluation data conversion and acquisition functions, and at the same time realizes the overall integrated control function of the system;

[0064] The integrated console module realizes the overall integrated control function of the system and controls the start, pause, continue and end of the overall simulation process of the system;

[0065] The data acquisition module collects command information from the UAV ground station, the attitude, position, speed data of the UAV during the evaluation simulation, video stream data and flight log data, stores these data, and sends them to the evaluation index calculation and evaluation report generation module at the same time;

[0066] The data conversion module processes the data information sent by the drone board in a unified data format; the data conversion module interacts with the black box model, the drone sensor model and the data acquisition module, and passes the converted data that meets the requirements to the above modules for processing.

[0067] Specifically, taking the UAV test in urban scenes as an example, the specific design is as follows:

[0068] (1) Evaluation and Design Subsystem

[0069] The evaluation design subsystem generates an evaluation scenario framework map and evaluation subject task information based on the scenario, quantity, and level selected by the party to be evaluated, sends the evaluation scenario framework map to the subject loading system, and sends the evaluation subject task information to the UAV equipment to be evaluated.

[0070] The assessment design subsystem generates the assessment scenario framework map and assessment subject task information according to the subject generation criteria. First, the subjects of different levels and scenarios are broken down into different intelligent empowerment points according to the three intelligence level dimensions of autonomy, collaboration and learning. Then, each intelligent empowerment point is converted into corresponding scenario elements for implementation according to the selected scenario.

[0071] The grade characteristics are determined as follows:

[0072] Level 1 (machine-assisted, single-unit collaborative, pre-learning): Able to complete simple tasks within a known fixed task and environment, and improve equipment capabilities through offline learning

[0073] Level 2 (machine-assisted, homogeneous collaboration, pre-learning): Able to complete simple tasks within a known fixed task and environment, can collaborate with homogeneous equipment to form combat capabilities that exceed those of a single unit, and improve equipment capabilities through offline learning

[0074] Level 3 (machine-assisted, heterogeneous collaboration, pre-learning): Able to complete simple tasks within a known fixed mission and environment, can collaborate with heterogeneous equipment to form combat capabilities that exceed those of a single unit, and improve equipment capabilities through offline learning

[0075] L4 (human-machine collaboration, single-unit combat, pre-learning): Able to adapt to known changes in tasks or environments to complete complex tasks and improve equipment capabilities through offline learning

[0076] L5 (human-machine collaboration, homogeneous coordination, pre-learning): Able to adapt to known changes in tasks or environments to complete complex tasks, can collaborate with homogeneous equipment to form combat capabilities that exceed those of a single unit, and improve equipment capabilities through offline learning

[0077] Level 6 (human-machine collaboration, homogeneous coordination, autonomous learning): Able to adapt to known changes in tasks or environments to complete complex tasks, and can collaborate with homogeneous equipment to form combat capabilities that exceed those of a single unit, mainly improving equipment capabilities through self-learning

[0078] Evaluation content settings

[0079] The evaluation design system provides a human-computer interaction interface. The party to be evaluated selects options such as evaluation scene, evaluation level, and number of drones to be evaluated. The evaluation design system generates evaluation questions for the corresponding scene based on the options. In order to effectively complete the evaluation task, the evaluation environment configuration mainly completes the configuration of the evaluation environment parameters. This embodiment takes the urban scene as an example and summarizes the scene elements as shown in the following table:

[0080] Table 1 Scene description features

[0081]

[0082] The scene data generation module will construct different three-dimensional scene models based on multiple conditional factors such as scene characteristics and subject requirements to meet different testing needs.

[0083] The evaluation design system generates a basic scene framework file based on the scene, evaluation level and other options selected by the party being tested, including the basic scene framework and model size. The generated file format is XML. Taking the city scene as an example, the basic scene map generated by the evaluation design system is as follows: Figure 4 shown.

[0084] After the evaluators select the scene, quantity and level of the drones to be evaluated, the evaluation design subsystem will decompose the subject characteristics into different intelligent empowerment points from the three dimensions of learning, collaboration and autonomy according to the selected options and the content of the evaluation question bank. In order to effectively complete the evaluation task, the evaluation environment configuration mainly completes the configuration of the evaluation environment parameters, including the type of scene, range size, number and location of obstacles in the scene, and other information. Different scenes have different ground features.

[0085] (2) Subject loading subsystem

[0086] The subject loading subsystem includes scene loading, scene adjustment, and weather and lighting effects.

[0087] After the evaluation design subsystem generates the scene framework map, it sends it to the subject loading subsystem. The subject loading system calls the model in the model library to render the framework map and generate the scene map. The 3D scene dynamic generation algorithm module calls different model elements or materials to generate 3D scenes in real time. The main technical processes include: element decomposition, area segmentation, area filling and scene modification. The scene of the overall area is as follows: Figure 5 As shown, the scene details in the map are as follows Figure 6 shown.

[0088] Each building in the scene map has a corresponding number. The latitude, longitude and height information of the buildings in the map are saved in the corresponding xml files, and the building models in the map are fbx and jpg files. By performing operations such as translation and rotation on the building in the scene interface, the adjusted position information is saved again in the xml file. Figure 7 shown.

[0089] According to the relative motion of the earth and the sun, based on the weather type (sunny, cloudy) and simulation time set by the user, real-time solar-terrestrial illumination calculations are performed, and illumination tracking technology and shader technology are used to exert illumination effects on the scene to simulate the real scene environment.

[0090] (3) Simulation subsystem

[0091] The simulation engine of the simulation subsystem performs visual simulation, UAV modeling, UAV flight simulation, simulation data recording, and video streaming data. The simulation engine includes a visual simulation engine and a virtual simulation engine.

[0092] The visual simulation engine simulates and collects data for the test process according to the test scenario and test task settings, and feeds the collected data back to the intelligent processing device of the tested party. The virtual simulation engine is a professional simulation platform for the development and operation support of the virtual simulation evaluation system, which mainly includes the physics engine, rendering engine, network engine and artificial intelligence (AI) engine, providing core platform support for distributed three-dimensional virtual simulation. Among them, the physics engine realizes dynamics, kinematics and physical special effects simulation; the rendering engine optimizes the display of three-dimensional models to improve work efficiency; the network engine provides basic network connection services and supports multi-node distributed interconnected simulation in server / client mode; the AI ​​engine can make decisions based on the perception information of the intelligent body and the artificial intelligence model of the running system to generate intelligent entity behavior. The visual scene mainly includes environmental models and landform models. The geographical environment model is mainly based on typical areas, including the location and appearance of roads, bridges, buildings, etc. commonly used in key areas. The landform model mainly refers to various environmental models in key areas, mainly including streams, shoals, canals, wetlands, deserts, shoals, hills, depressions, ponds, swamps, Gobi, peaks and other types.

[0093] The UAV simulation module will build a high-precision three-dimensional model for the UAV model being tested, and import these high-precision models into the virtual scene during the flight test verification process to simulate the UAV flight process in a real environment.

[0094] The UAV flight scene simulation module receives and analyzes the UAV flight data sent by the black box model of the party to be evaluated in real time, and drives the UAV to execute the flight process. The data sent by the black box model mainly includes position, attitude, speed, acceleration, and angular velocity. During the UAV flight simulation process, the module will use the collision detection physics engine built into UE5 to detect in real time whether the UAV collides with other objects in the scene environment. These objects include static objects such as buildings, trees, and targets, as well as dynamic objects such as moving vehicles and aircraft. If a collision is detected, the visual software will apply an explosion effect to the colliding UAV, display the UAV crash screen, and stop receiving and processing the flight data of the UAV. The module will record the collision results and send them to the evaluation and calculation subsystem. During the flight of the UAV, the module will detect in real time whether the UAV is currently deviating from the test scene. If the UAV flies out of the boundary of the test scene, a warning message will pop up. The module will record the results of flying out of the boundary and send them to the evaluation and calculation subsystem. The UAV has the function of on-the-spot guidance during flight, simulating the situation of special situations, communication loss and crash of the UAV, and testing the autonomy, coordination and learning of the task replanning capability of the UAV cluster. During the flight, testers input the drone number through the human-computer interaction interface of the system management subsystem, specifying that the drone with that number has encountered a special situation and can no longer communicate, in order to evaluate the system's ability to detect and re-plan tasks of the drone cluster when encountering special situations.

[0095] During the UAV flight test, simulation-related data is recorded in real time and can be used for data playback and intelligent level evaluation. These data mainly include UAV flight data, UAV control instructions, etc. After the simulation is completed, the user can import the data and playback the entire simulation process, supporting operations such as play, pause, and speed adjustment.

[0096] (4) Evaluation and calculation subsystem

[0097] The evaluation calculation subsystem mainly includes the evaluation scheme setting and the evaluation result processing and display. The evaluation scheme setting mainly sets the evaluation index system and evaluation algorithm required in the UAV evaluation process. According to the current index setting, real-time data is collected during the evaluation process, and finally the evaluation data in the analysis and evaluation is formed. It mainly includes the index system classification management, index system design, index evaluation algorithm management, and evaluation task weight setting.

[0098] ① Classification management of indicator system

[0099] The classification management of the indicator system mainly completes the mapping or binding between the indicator system and the capabilities to be measured. A certain capability assessment has a corresponding indicator system to evaluate it.

[0100] To create a new indicator system classification, you need to configure the classification name, serial number, creator and creation time, notes and other related information.

[0101] Deleting a category will delete the category. If you need to keep the indicator system design under the category during deletion, the indicator system will be moved to the default category.

[0102] ②Indicator system design

[0103] The indicator system is provided by the tester, and the software provides management functions for the indicator system and the evaluation method, mainly including the establishment, modification, and deletion of the indicator system. The indicator system adopts a multi-level hierarchical design.

[0104] The system provides an open evaluation indicator management tool to add and modify evaluation indicators, evaluation content and evaluation rules to improve the openness, autonomy and dynamism of evaluation indicators.

[0105] ③Indicator evaluation algorithm management

[0106] The evaluation criteria are used to convert the overall performance value of the index system and the indicators in the index system into corresponding qualitative description information. The evaluation criteria design provides management of the criteria, and the criterion format is also saved in the index system as part of the index system. In this embodiment, the full score is 100, 0-60 is poor, 60-80 is good, and 80 or above is excellent.

[0107] ④Evaluation task weight setting

[0108] After the assessment is completed, the weight of the assessment task needs to be set. In order to reflect the impact of different assessment stages and steps on the entire subject operation process in the assessment, different assessment weights are set for different stages and steps.

[0109] Evaluation result display mainly refers to obtaining the evaluation plan of this evaluation according to the evaluation content, including the indicator system and evaluation algorithm, etc. The evaluation data processing part mainly realizes the real-time collection and processing of evaluation data by the drone evaluation client during the entire evaluation process. After the evaluation is completed, the evaluation data will be uploaded to the evaluation personnel management client for summary for subsequent analysis and evaluation. Analysis and evaluation display mainly completes the visualization of evaluation data, including graphics and tables.

[0110] (5) System Management Subsystem

[0111] The system management subsystem is mainly presented in the form of framework software, which is the hub connecting each subsystem and the human-computer interaction interface of the whole system, connecting the evaluation design subsystem, subject loading subsystem, visual simulation subsystem, and evaluation calculation subsystem. The framework software adopts an indexed structure, which calls each subsystem by sending instructions. The structure is robust and ensures the stability of the system. The callable functions include: user management, new project, question generation, scene generation, scene model editing, result evaluation, result display playback, etc.

[0112] ① The basic functions of user management include user registration, password modification, login verification, user query, user deletion, etc.

[0113] User registration function: After the user opens the framework software, clicks the relevant button on the login interface to complete the user registration, and then logs in using the set user name and password.

[0114] Password modification: Users can complete the password by clicking the relevant button on the software login interface.

[0115] Login verification: Users log in using their username and password, and the software automatically verifies their password and user permissions. Only those who pass the verification can successfully log in.

[0116] User query: After the user successfully logs in, the information of all currently registered users can be queried. User deletion: After the user successfully logs in, the specified user can be deleted.

[0117] The user login and user management interface is shown in the figure below.

[0118] Index call function. The most important function of the framework software is to call other subsystems, so the call options of each subsystem function are set on the interface, and the UI interface of each subsystem is displayed through the framework software.

Claims

1. A drone intelligence level assessment and verification system, characterized in that: The system includes an assessment design subsystem, a subject loading subsystem, a deduction and simulation subsystem, an evaluation calculation subsystem and a system management subsystem; Each subsystem is connected to the UAV device under test and evaluation through a physical interface and transmits data through a communication protocol. The dynamic model of the UAV under test is replaced by a black box model software program; the black box model sends the dynamic data of the UAV under test; The assessment design subsystem includes a human-computer interaction interface, an assessment question library, and an assessment subject generator module; the assessment design subsystem has assessment scenario and subject generation functions, and can provide a human-computer interaction interface; it automatically generates preset subjects according to the content selected by the tested party, and the preset subjects include assessment scenarios and assessment questions; The human-computer interaction interface provides test options for the evaluator, including the type of equipment to be tested, test scenario, test level and number of equipment; the evaluator selects the corresponding option according to the equipment to be tested, and the test option information of the human-computer interaction interface is transmitted to the test question library and the assessment subject generator, and the two modules configure the test environment scenario according to the selected option; each environment scenario is set with several difficulty assessment levels; The evaluation question database provides question construction rules for the selected scenario according to the evaluation level of the drone to be evaluated, including the number range of targets and the density of obstacles. Different evaluation levels correspond to different question construction rules. The evaluation question database receives the scenario and evaluation level information sent by the human-computer interaction interface, and outputs the question construction rules for the corresponding scenario and level to the evaluation subject generator. The assessment subject generator receives the assessment environment and assessment level data sent by the human-computer interface, and at the same time receives the information of the assessment question library, and configures the test subjects, including the assessment environment and assessment task information under the corresponding scenarios and corresponding assessment levels; The assessment subject generator sends the test question information to the scenario generator and configurator in the simulation system to manage simulation scenarios and set up the battlefield environment. The subject loading subsystem includes a subject loader, a simulation model library, and a scenario fusion generation and configurator module; the subject loading subsystem has a test scenario visualization function to construct simulation scenarios under different backgrounds; The simulation model library is a database of simulation scenario models, which serves as the basis for generating scenario configurations during the evaluation simulation process; The scene fusion generation and configurator module builds a 3D scene dynamic generation algorithm based on the test topic information generated by the assessment subject generator and calls the required model primitives or materials from the simulation model library to generate a 3D scene in real time; The subject loader integrates the models and scenarios in the simulation model library and loads the configurator module data information to generate scenario information; The simulation subsystem includes a simulation engine, which has a built-in time simulator and event simulator. The input of the simulation engine includes the simulation scene output by the subject loading subsystem and the output data of the drone black box. The time simulator and the event simulator judge the results of several events in the simulation process according to the logic of the event, and output the results to the integrated console of the system management subsystem. The simulation subsystem realizes the simulation function in the evaluation process. The evaluation calculation subsystem includes an evaluation index calculation and evaluation report generation module and an index list and a scoring standard library; The indicator list and scoring standard library integrate pre-defined scoring standards for different evaluation scenarios and different evaluation levels; This module passes the corresponding scoring criteria to the evaluation index calculation and evaluation report generation module according to the level of the selected question; The evaluation index calculation and evaluation report generation module calculates the score of the tested equipment according to the scoring criteria in the index list and the scoring criteria library and the information collected by the data collection module, through the evaluation algorithm and according to the received simulation data and the scoring criteria, generates an evaluation report, and gives the final evaluation result; The system management subsystem includes a data conversion module, a data acquisition module and an integrated console module; the system management subsystem realizes the evaluation data conversion and acquisition functions, and at the same time realizes the overall integrated control function of the system; The integrated console module realizes the overall integrated control function of the system and controls the start, pause, continue and end of the overall simulation process of the system; The data acquisition module collects command information from the UAV ground station, the attitude, position, speed data of the UAV during the evaluation simulation, video stream data and flight log data, stores these data, and sends them to the evaluation index calculation and evaluation report generation module at the same time; The data conversion module processes the data information sent by the drone board according to a unified data format; The data conversion module interacts with the black box model, the UAV sensor model and the data acquisition module, and passes the converted data that meets the requirements to the above modules for processing.

2. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The simulation scenes include mountains, cities, jungles, grasslands, oceans, straits, deserts, and polar regions.

3. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The black box model sends the dynamic data of the tested drone including position, attitude, speed, acceleration and angular velocity.

4. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The UAV intelligence level assessment and verification system is connected to a number of different UAVs; the data conversion module sent by the UAV boards of the different units converts the data into a unified data format for processing according to the MAVLink protocol.

5. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The subject loading subsystem, indicator list and scoring standard library set up subject scenario data import and subject manual adjustment and modification modules.

6. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The difficulty assessment level is divided into six levels from L1 to L6. Different levels of scenes are distinguished by the presence, type, density or quantity of indicators. The scene indicators include weather, buildings, roads, bridges, vegetation, drones, unmanned vehicles, targets, water systems, topography, trees, interferences, and obstructions.

7. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The evaluation and calculation subsystem has a built-in mapping relationship system between the completion indicators of drones at different difficulty evaluation levels and their intelligence levels to form evaluation criteria.

8. The UAV intelligence level evaluation and verification system according to claim 7 is characterized in that: The evaluation calculation subsystem quantifies the performance of the drone's completion indicators and sets corresponding weights, obtains a total evaluation score through a series of weighted summations, and corresponds the evaluation score to the intelligence level.

9. The UAV intelligence level evaluation and verification system according to claim 1 is characterized in that: The drone issues external commands via a control station or a remote controller.

10. A use of the UAV intelligence level evaluation and verification system according to any one of claims 1 to 9, characterized in that: The specific stages include: Evaluation preparation stage: The drone to be evaluated is connected to the system, and the number of equipment, number of evaluations, test environment and evaluation level are selected through the human-computer interaction interface; Environment construction stage: The subject loading subsystem generates subjects and scenarios based on the number of equipment, number of evaluations, test environment, and evaluation level; Evaluation and simulation stage: the simulation subsystem is simulated and evaluated; Evaluation and assessment stage: The evaluation calculation subsystem generates an evaluation report based on the output data of the deduction and simulation subsystem.

Citation Information

Patent Citations

  • Gas detection device for unmanned aerial vehicle

    CN113109509A

  • Intelligent unmanned cluster system area coverage relay communication application simulation and test system

    CN116522570A

  • Multi-domain unmanned system interoperation evaluation environment construction and application method

    CN117370425A

  • Air unmanned aerial vehicle group collaborative algorithm simulation verification system

    CN117539273A

  • Unmanned aerial vehicle system test method and device, electronic equipment and storage medium

    CN117775306A