A method for evaluating the degree of human intervention in unmanned systems

Abstract

The present invention relates to the field of unmanned system test evaluation, and specifically to a method for evaluating the degree of human intervention in an unmanned system. The present invention provides a method for evaluating the degree of human intervention in an unmanned system, which specifically includes the following steps: Step 1: preparing evaluation data; Step 2: determining the type of human intervention task; Step 3: evaluating the human intervention mode; Step 4: calculating the initiative of human intervention; Step 5: calculating the breadth of human intervention; Step 6: comprehensive calculation of the degree of human intervention. The present invention takes the degree of human intervention in an unmanned system as a "research object" and the constructed evaluation environment as a "simulation", and can quantify the degree of human intervention in an unmanned system with high timeliness, low cost and high confidence, and more accurately realize the autonomous evaluation of the unmanned system.

Description

Technical Field

[0001] The present invention relates to the field of unmanned system test evaluation, and in particular to an unmanned system manual intervention degree evaluation method. Background Art

[0002] Unmanned systems are usually designed as autonomous systems that can independently perform tasks, adapt to environmental changes and achieve predetermined goals. The development of autonomous system technology has enabled unmanned systems to achieve a higher degree of autonomous operation, reduce human intervention and improve the overall efficiency of the system.

[0003] The degree of human intervention refers to the degree to which personnel intervene in the equipment operation, decision-making and other task processes during the equipment's mission execution, indicating the degree to which the equipment relies on external operators to perform tasks. The higher the degree of human intervention, the higher the degree of dependence of the equipment on external operators, and the lower the level of autonomy of the equipment system.

[0004] At present, there are problems such as lack of evaluation methods and means for the human intervention degree of unmanned systems, and the inability to quantify. Therefore, it is necessary to develop an evaluation method for the human intervention degree of unmanned systems to accurately evaluate the human intervention degree of unmanned systems. Summary of the invention

[0005] The technical solution adopted by the present invention is as follows:

[0006] The present invention provides a method for evaluating the degree of human intervention in an unmanned system, which specifically comprises the following steps:

[0007] Step 1: Evaluation Data Preparation

[0008] By setting up typical mission scenarios for unmanned systems for test assessment, the interaction data between the unmanned system platform, ground station and operators are collected during the entire test process;

[0009] Step 2: Determine the type of manual intervention task

[0010] Evaluate the types of manual intervention tasks and divide the types of manual intervention content into functional intervention and safeguard intervention according to the nature of the manual intervention content;

[0011] The functional intervention is a human intervention whose content is directly related to the equipment completing the task; the guarantee intervention is a human intervention whose content is not directly related to the equipment completing the task, and only guarantees the successful completion of the task;

[0012] The task set R that the unmanned system can perform is obtained. The task set R contains Nr types of tasks that the unmanned system can perform, which can be expressed as:

[0013] R=[1,2,...,nr,...,Nr];

[0014] Step 3: Evaluate the human intervention model

[0015] For each task that requires human intervention, evaluate the human intervention model;

[0016] For the nrth task, the intervention mode coefficient S is set according to the task requirements; the intervention mode coefficient S is in the interval [0,1], and the higher the coefficient, the higher the degree of human intervention;

[0017] According to the three manual intervention modes of human control, human decision-making and human confirmation, S is divided into coefficients So, Sd and Sc of the corresponding modes;

[0018] So, Sd, and Sc are all in the interval [0,1] and So>Sd>Sc;

[0019] Step 4: Calculate the initiative of manual intervention

[0020] According to whether the system can detect abnormalities autonomously, set the manual intervention initiative coefficient, expressed as λ:

[0021] The higher the control initiative, the closer the coefficient λ is to 1 in the [0,1] interval;

[0022] If anomalies are discovered manually and controlled, the coefficient λ is the control initiative coefficient λa;

[0023] If the abnormality is detected autonomously, the coefficient λ is taken as the autonomous initiative coefficient λs;

[0024] And λa>λs;

[0025] Step 5: Calculate the extent of human intervention

[0026] The breadth of manual intervention includes the frequency, time, and subtask ratio of personnel intervention in the execution of tasks by equipment. The higher the frequency ratio, the greater the time ratio, and the higher the subtask ratio, the greater the breadth of manual intervention. The higher the manual intervention, the lower the autonomy of the equipment.

[0027] 501. Calculation of the proportion of manual intervention times: The manual intervention frequency refers to the total number of manual intervention times when the equipment performs a task, including the number of manual intervention operations and the number of manual intervention decisions; the proportion of manual intervention frequency is represented by fr, the manual intervention frequency is represented by fa, and the total number of unmanned system behaviors is represented by Sf. The calculation formula for fr is as follows:

[0028]

[0029] 502. Calculation of human intervention time ratio: Human intervention time refers to the duration of human intervention in the process of equipment performing tasks; human intervention time ratio refers to the ratio of human intervention operation time to the total task time when the equipment performs tasks; human intervention time ratio, human intervention duration, and total time of equipment performing tasks are represented by the letters tr, ta, and ts respectively. The calculation formula for tr is as follows:

[0030]

[0031] 503. Calculate the proportion of subtasks with manual intervention: The proportion of tasks with manual intervention is used as the measurement indicator; the proportion of subtasks with manual intervention, the number of subtasks with manual intervention, and the total number of subtasks are represented by letters wr, wa, and ws respectively; the calculation formula for wr is as follows:

[0032]

[0033] Step 6: Comprehensive calculation of human intervention degree

[0034] 601.Calculate the degree of human intervention for each task

[0035] The degree of manual intervention of each task is represented by Ia, the weight vector of the subtask is Wc, and the formula of manual intervention degree Ia is as follows:

[0036] Ia=S×λ×Wc(fr+tr+wr)

[0037] 602. Total human intervention degree of unmanned systems

[0038] In view of the different human intervention degrees of unmanned systems in different scenarios and for different objects, the human intervention degrees of various tasks are comprehensively calculated; the weight set corresponding to the task set R = [1, 2, ..., nr, ..., Nr] is W = [W1, W2, ..., Wnr, ..., WNr]; the human intervention degree of each task is represented by the set of Ia as I'; the formula for the total human intervention degree Is of the unmanned aerial vehicle system is as follows:

[0039] Is=WI'.

[0040] Preferably, in step 1, the collected interaction data includes text, voice, and action.

[0041] Preferably, in step 1, the test is performed in a simulation, real object or a combination of both.

[0042] Preferably, the tasks in the task set R include discovery, positioning, tracking, aiming, striking and evaluation.

[0043] Preferably, in step 2, functional intervention includes manual participation in target identification, task allocation, and path planning; and safeguard intervention includes abnormality detection.

[0044] Preferably, the intervention mode coefficient S and the manual intervention initiative degree coefficient λ are set by experts according to task requirements; So, Sd, Sc and λa, λs are set by experts separately according to task types.

[0045] Preferably, the number of subtasks of manual intervention includes the number of pictures identified by manual intervention and the number of routes planned by manual intervention tasks.

[0046] Preferably, in step five, the time, number of times, and subtask ratio are weighted together or 1-2 of them are selected for calculation; in step six, the degree of manual intervention when completing tasks in several typical scenarios are selected from the task set R as the total degree of manual intervention to simplify the calculation.

[0047] In a second aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0048] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0049] Beneficial effects achieved by the present invention:

[0050] The present invention takes the degree of human intervention in unmanned systems as the research object and uses the constructed evaluation environment as the simulation. It can quantify the degree of human intervention in unmanned systems with high timeliness, low cost and high confidence, and realize the autonomous evaluation of unmanned systems more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is the process of the unmanned system human intervention degree assessment method described in the present invention. DETAILED DESCRIPTION

[0052] Example 1

[0053] The present invention provides an implementation process of a method for evaluating the degree of human intervention in an unmanned system as follows:

[0054] Step 1: Evaluation Data Preparation

[0055] By setting up typical mission scenarios for unmanned systems, testing and assessment are carried out in simulation, physical or a combination of methods, and the interaction data between the unmanned system platform, ground station and operators during the entire test process is collected, including text, voice, action, etc.

[0056] Step 2: Determine the type of manual intervention task

[0057] Evaluate the type of manual intervention tasks. The types of manual intervention content can be divided into the following types according to the nature of the manual intervention content: functional intervention (referring to the content of human intervention that is directly related to the equipment's task completion, such as manual participation in target identification, task allocation, path planning, etc.);

[0058] Supportive intervention (the content of human intervention has no direct relationship with the equipment's mission completion, but can ensure the smooth completion of the mission, such as discovering anomalies, etc.);

[0059] Evaluate the type of human intervention tasks. The unmanned system can perform Nr types of tasks, and the task set is represented as

[0060] R = [1, 2, ..., nr, ..., Nr] may be of types such as discovery, positioning, tracking, aiming, attack and evaluation.

[0061] Step 3: Evaluate the human intervention model

[0062] For each task that requires human intervention, evaluate the human intervention mode. For the nrth task, the intervention mode coefficient is set by the expert according to the task requirements, etc. S represents the coefficient. For the three modes of human control, human decision-making, and human confirmation, the corresponding coefficients are So, Sd, and Sc respectively. The coefficient is in the range of [0,1]. The higher the coefficient, the higher the degree of human intervention.

[0063] Step 4: Calculate the initiative of manual intervention

[0064] For whether the system discovers abnormalities autonomously, a coefficient of the degree of initiative of manual intervention is given specifically, expressed as λ:

[0065] The higher the control initiative, the closer the coefficient λ is to 1 in the [0,1] interval.

[0066] 401. Artificially discover anomalies and control them, and control the initiative coefficient λa.

[0067] 402. Autonomously discover anomalies and request manual control (autonomous discovery ratio, false alarm probability, alarm coverage). The coefficient is represented by λs.

[0068] 501. Calculation of the proportion of manual intervention times: The manual intervention frequency refers to the total number of manual intervention times when the equipment performs a task, including the number of manual intervention operations and the number of manual intervention decisions; the proportion of manual intervention frequency is represented by fr, the manual intervention frequency is represented by fa, and the total number of unmanned system behaviors is represented by Sf. The calculation formula for fr is as follows:

[0069]

[0070] 502. Calculation of human intervention time ratio: Human intervention time refers to the duration of human intervention in the process of equipment performing tasks; human intervention time ratio refers to the ratio of human intervention operation time to the total task time when the equipment performs tasks; human intervention time ratio, human intervention duration, and total time of equipment performing tasks are represented by the letters tr, ta, and ts respectively. The calculation formula for tr is as follows:

[0071]

[0072] 503. Calculate the proportion of subtasks with manual intervention: The proportion of tasks with manual intervention is used as the measurement indicator; the proportion of subtasks with manual intervention, the number of subtasks with manual intervention, and the total number of subtasks are represented by letters wr, wa, and ws respectively; the calculation formula for wr is as follows:

[0073]

[0074] Step 6: Comprehensive calculation of human intervention degree

[0075] 601.Calculate the degree of human intervention for each task

[0076] The degree of manual intervention of each task is represented by Ia, the weight vector of the subtask is Wc, and the formula of manual intervention degree Ia is as follows:

[0077] Ia=S×λ×Wc(fr+tr+wr)

[0078] 602. Total human intervention degree of unmanned systems

[0079] In view of the different human intervention degrees of unmanned systems in different scenarios and for different objects, the human intervention degrees of various tasks are comprehensively calculated; the weight set corresponding to the task set R = [1, 2, ..., nr, ..., Nr] is W = [W1, W2, ..., Wnr, ..., WNr]; the human intervention degree of each task is represented by the set of Ia as I'; the formula for the total human intervention degree Is of the unmanned aerial vehicle system is as follows:

[0080] Is=WI'.

Claims

1. A method for evaluating the degree of human intervention in an unmanned system, characterized in that: The specific steps include: Step 1: Evaluation Data Preparation By setting up typical mission scenarios for unmanned systems for test assessment, the interaction data between the unmanned system platform, ground station and operators are collected during the entire test process; Step 2: Determine the type of manual intervention task Evaluate the types of manual intervention tasks and divide the types of manual intervention content into functional intervention and safeguard intervention according to the nature of the manual intervention content; The functional intervention is a human intervention whose content is directly related to the equipment completing the task; the guarantee intervention is a human intervention whose content is not directly related to the equipment completing the task, and only guarantees the successful completion of the task; The task set R that the unmanned system can perform is obtained. The task set R contains Nr types of tasks that the unmanned system can perform, which can be expressed as: R=[1,2,...,nr,...,Nr]; Step 3: Evaluate the human intervention model For each task that requires human intervention, evaluate the human intervention model; For the nrth task, the intervention mode coefficient S is set according to the task requirements; the intervention mode coefficient S is in the interval [0,1], and the higher the coefficient, the higher the degree of human intervention; According to the three manual intervention modes of human control, human decision-making and human confirmation, S is divided into coefficients So, Sd and Sc of the corresponding modes; So, Sd, and Sc are all in the interval [0,1] and So>Sd>Sc; Step 4: Calculate the initiative of manual intervention According to whether the system can detect abnormalities autonomously, set the manual intervention initiative coefficient, expressed as λ: The higher the control initiative, the closer the coefficient λ is to 1 in the [0,1] interval; If anomalies are discovered manually and controlled, the coefficient λ is the control initiative coefficient λa; If the abnormality is detected autonomously, the coefficient λ is taken as the autonomous initiative coefficient λs; And λa>λs; Step 5: Calculate the extent of human intervention The breadth of human intervention includes the frequency, time and subtask ratio of human intervention in the execution of tasks by equipment. The higher the frequency ratio, the greater the time ratio and the higher the subtask ratio, the greater the breadth of human intervention. The higher the degree of human intervention, the lower the autonomy of the equipment.

501. Calculation of the proportion of manual intervention times: The manual intervention frequency refers to the total number of manual intervention times when the equipment performs a task, including the number of manual intervention operations and the number of manual intervention decisions; the proportion of manual intervention frequency is represented by fr, the manual intervention frequency is represented by fa, and the total number of unmanned system behaviors is represented by Sf. The calculation formula for fr is as follows:

502. Calculate the proportion of manual intervention time: Manual intervention time refers to the length of time that equipment is manually intervened during the execution of a task; The manual intervention time ratio refers to the ratio of the manual intervention operation time to the total task time when the equipment is performing a task. The manual intervention time ratio, manual intervention duration, and total time for the equipment to perform a task are represented by the letters tr, ta, and ts respectively. The calculation formula for tr is as follows:

503. Calculate the proportion of subtasks with manual intervention: The proportion of tasks with manual intervention is used as the measurement indicator; the proportion of subtasks with manual intervention, the number of subtasks with manual intervention, and the total number of subtasks are represented by letters wr, wa, and ws respectively; the calculation formula for wr is as follows: Step 6: Comprehensive calculation of human intervention degree 601.Calculate the degree of human intervention for each task The degree of manual intervention of each task is represented by Ia, the weight vector of the subtask is Wc, and the formula of manual intervention degree Ia is as follows: Ia=S×λ×Wc(fr+tr+wr) 602. Total human intervention degree of unmanned systems In view of the different human intervention degrees of unmanned systems in different scenarios and for different objects, the human intervention degrees of various tasks are comprehensively calculated; the weight set corresponding to the task set R = [1, 2, ..., nr, ..., Nr] is W = [W1, W2, ..., Wnr, ..., WNr]; the human intervention degree of each task is represented by the set of Ia as I'; the formula for the total human intervention degree Is of the unmanned aerial vehicle system is as follows: Is=WI'.

2. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: In the step 1, the collected interaction data includes text, voice and action.

3. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: In the step 1, the test is performed in a simulation, a real object or a combination of the two.

4. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: The tasks in the task set R include discovery, positioning, tracking, aiming, striking and evaluation.

5. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: In the step 2, the functional intervention includes manual participation in target identification, task allocation and path planning; the safeguard intervention includes abnormality detection.

6. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: The intervention mode coefficient S and the manual intervention initiative coefficient λ are set by experts according to task requirements; the So, Sd, Sc and λa, λs are set by experts respectively and individually according to task types.

7. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: The number of subtasks of manual intervention includes the number of pictures identified by manual intervention and the number of routes planned by manual intervention tasks.

8. The method for evaluating the degree of human intervention for unmanned systems according to claim 1, characterized in that: In step five, the time, number of times, and subtask ratio are weighted together or 1-2 of them are selected for calculation; in step six, the degree of manual intervention when completing tasks in several typical scenarios is selected from the task set R as the total degree of manual intervention to simplify the calculation.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images