Intelligent grade evaluation method for unmanned equipment

Abstract

The invention discloses an intelligent level evaluation method for unmanned equipment. The method comprises the following steps: step 1, constructing intelligent level evaluation indexes of the unmanned equipment, wherein the intelligent level evaluation indexes comprise an autonomous intelligent level evaluation index system of the unmanned equipment and a collaborative intelligent level evaluation index system of the unmanned equipment; step 2, calculating autonomy and collaboration evaluation index values of the unmanned equipment; step 3, intelligent level comprehensive evaluation: comprehensively evaluating the autonomy and collaboration of the unmanned equipment by adopting an AHP technology, and establishing an autonomy and collaboration score grading model; and 4, according to the evaluation results of the autonomy and collaboration of the unmanned equipment, constructing a mapping relation table between the intelligent level and the autonomy and collaboration capabilities. The invention provides a feasible method for autonomy and collaboration grading quantitative evaluation of unmanned equipment by constructing an autonomy and collaboration grading model.

Description

Technical Field

[0001] The present invention relates to the field of equipment intelligence level assessment, and in particular to an intelligence level assessment method for unmanned equipment. Background Art

[0002] With the accelerated application of artificial intelligence technology in the military field, intelligent unmanned equipment has gradually become an important part of modern warfare. Intelligent equipment is represented by unmanned equipment, with autonomy as its typical feature, which can greatly reduce the consumption of human resources. By carrying a large number of intelligent algorithm models and systems such as perception, control, and decision-making, unmanned equipment can achieve multiple capabilities such as intelligent target detection and tracking, autonomous navigation, and unmanned collaboration. The world's major military powers all attach great importance to the application of intelligent unmanned equipment in the military field. In the future, intelligent unmanned equipment will profoundly affect the way of combat and subvert the rules of war.

[0003] However, intelligent technology has inherent vulnerabilities in models, algorithms, data, etc., which can easily lead to a sharp decline in the reliability and stability of unmanned equipment in complex combat environments, and the effectiveness of intelligent combat cannot be effectively exerted; at the same time, the current intelligence level of unmanned equipment is difficult to define, which makes it impossible for commanders to determine the application mode of the equipment based on its intelligence level. The above problems make it impossible for unmanned equipment to be used on a large scale in actual combat.

[0004] Before the actual installation and application, conducting comprehensive and multi-dimensional testing and verification of unmanned equipment can solve the above problems to a certain extent, so as to understand the autonomy and coordination capabilities of our unmanned equipment in advance, and know the capability boundaries of the equipment; to be familiar with the intelligence level of our unmanned equipment in advance, so as to be "targeted" on the battlefield. However, the current unmanned equipment testing, verification, and evaluation methods still have certain limitations and capability shortcomings, which hinder the pace of evaluating the intelligence level and capabilities of unmanned equipment. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method for intelligence level scoring and intelligence grade mapping for unmanned equipment. This method aims at the current problem of lack of means for testing and verifying the intelligence level of unmanned equipment, and proposes a method for intelligence level scoring and intelligence grade mapping for unmanned equipment.

[0006] The objective of the present invention is to achieve an intelligent level assessment method for unmanned equipment through the following technical solutions, comprising the following steps:

[0007] Step 1: Construct unmanned equipment intelligence level evaluation indicators: including autonomous unmanned equipment intelligence level evaluation indicator system and collaborative unmanned equipment intelligence level evaluation indicator system;

[0008] Step 2: Calculate the evaluation index values ​​of the autonomy and coordination of unmanned equipment;

[0009] Step 3: Comprehensive evaluation of intelligence level: Use AHP technology to comprehensively evaluate the autonomy and coordination of unmanned equipment, and establish a scoring and grading model for autonomy and coordination;

[0010] Step 4: Based on the evaluation results of the autonomy and collaboration of unmanned equipment, construct a mapping relationship table between the intelligence level and the autonomous capability and collaborative capability.

[0011] Preferably, in step 2, the autonomy evaluation indicators include autonomous perception capability, autonomous judgment capability, autonomous decision-making capability and autonomous execution capability; the collaboration evaluation indicators include collaborative perception capability, collaborative communication capability, collaborative decision-making capability and collaborative execution capability.

[0012] Preferably, the autonomous sensing capability includes: recognition accuracy and positioning accuracy;

[0013] Autonomous judgment capabilities include: target position judgment, target trajectory judgment, and friend-or-foe judgment;

[0014] Autonomous decision-making capabilities include: mission planning, obstacle avoidance path planning, and target path planning;

[0015] Autonomous execution capabilities include: hit rate, coverage rate and target destruction probability.

[0016] Preferably, the accuracy is used to quantify the recognition accuracy, and the calculation formula is:

[0017]

[0018] Among them, TP represents true positive examples, TN represents true negative examples, FP represents false positive examples, and FN represents false negative examples;

[0019] The quantification method used for positioning accuracy is the mean absolute error, and the calculation formula is:

[0020]

[0021] In the formula, Estimate i is the estimated target position, True i is the actual target position, and N is the number of sampling points.

[0022] Preferably, the target position determination is measured by the error between the estimated target position and the actual target position, which is calculated by the following formula:

[0023]

[0024] In the formula, (x est ,y est ,zest ) is the estimated target position, (x true ,y true ,z true ) is the actual target position;

[0025] The target trajectory judgment is quantitatively evaluated by the root mean square error, and the calculation formula is:

[0026]

[0027] In the formula, Estimate i is the estimated target position, True i is the actual target position, N is the number of sampling points;

[0028] IFF refers to the ability of unmanned equipment to accurately identify friendly and enemy targets in the surrounding environment, and is quantitatively evaluated using the following accuracy calculation formula:

[0029]

[0030] In the formula, Accuary represents the accuracy.

[0031] Preferably, the hit rate is calculated using the following formula:

[0032]

[0033] Where N hit is the number of targets hit, N all The total number of targets set;

[0034] The coverage rate is calculated using the following formula:

[0035]

[0036] In the formula, S covered is the area covered by the strike, S total The total area of ​​the region for which the target is established.

[0037] Preferably, the collaborative communication capability includes: communication timeliness, communication capacity, and node connectivity;

[0038] Collaborative decision-making capabilities include: task completion rate and dynamic decision-making capabilities;

[0039] Coordinated execution capabilities include: coordinated strike accuracy and coordinated destruction probability.

[0040] Preferably, the communication timeliness is calculated by the following formula:

[0041] Delay=T arrival -T send

[0042] Delay is the information from the sender T send To the receiving end T arrival the time required for the transfer;

[0043] The connectivity index is calculated using the following formula:

[0044]

[0045] Where N connected is the number of connected unmanned equipment nodes, N all is the number of all nodes.

[0046] Preferably, the coordinated strike accuracy is measured by the coordinated strike hit rate, as follows:

[0047]

[0048] Where N cohit is the number of targets hit by the coordinated strike, N coall The total number of established targets.

[0049] Preferably, in step 3, the index weights are determined by using the hierarchical analysis method, and the scores of the evaluation indicators in the first layer of the hierarchical system and the cumulative sum of the scores of the evaluation indicators in the first layer are calculated to calculate the autonomous scores of the unmanned equipment. A and synergy score B , as shown below:

[0050]

[0051] In the formula, A i is the score of the ith autonomy evaluation indicator, w i is the weight of the ith autonomy evaluation index, p is the number of autonomy evaluation indicators, the value range of i is [1, p], w i The value range of B is [0, 1]; j is the score of the j-th synergy evaluation index, w j is the weight of the jth synergy evaluation index, q is the number of synergy evaluation indexes, and the value range of j is [1, q], w j The value range is [0, 1].

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] The present invention provides an intelligent level assessment method for unmanned equipment. By conducting research on the intelligent level assessment technology of unmanned equipment, an unmanned equipment intelligent level assessment index system is established, and the assessment indicators of two dimensions, autonomy and collaboration, are integrated to finally obtain the unmanned equipment intelligent level assessment result.

[0054] The present invention constructs evaluation indicators of two dimensions, autonomy and coordination, and combines the core capabilities of unmanned equipment decomposed by the intelligent combat OODA loop to form an intelligent level evaluation index system for unmanned equipment, laying a foundation for the quantitative evaluation of the intelligence level of unmanned equipment. The present invention provides a feasible method for the hierarchical quantitative evaluation of the autonomy and coordination of unmanned equipment by constructing a scoring and grading model for autonomy and coordination; by establishing a mapping relationship between the intelligent level mapping and the autonomous capability and the collaborative capability, the intelligent level is divided into 9 levels according to autonomy and coordination. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 The present invention is a flow chart of an intelligent level assessment method for unmanned equipment. DETAILED DESCRIPTION

[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0057] like Figure 1 As shown, the technical solution of the present invention provides an intelligent level assessment method for unmanned equipment, comprising the following steps:

[0058] Step 1: Construct an evaluation index system for the intelligent level of unmanned equipment: Construct an evaluation index system for the intelligent level of unmanned equipment that includes two dimensions: autonomy and collaboration.

[0059] Step 2: Calculate the autonomy and coordination index values ​​of unmanned equipment: Calculate the values ​​of each index according to the index calculation formula or expert scores.

[0060] Step 3: Comprehensive evaluation of intelligence level: Use AHP technology to comprehensively evaluate the autonomy and collaboration of unmanned equipment, and establish a scoring and grading model for autonomy and collaboration.

[0061] Step 4: Construct an intelligence level scoring and intelligence grade mapping model for unmanned equipment: Based on the evaluation results of the autonomy and collaboration of unmanned equipment, construct a mapping relationship table between the intelligence grade and the autonomous capability and collaborative capability, as shown in Table 1.

[0062] Table 1 Unmanned equipment intelligence level evaluation index system

[0063]

[0064]

[0065] In one embodiment of the present invention, in step 2, the construction and calculation method of the autonomy evaluation index is as follows:

[0066] 1) Autonomous perception capability

[0067] a. Recognition accuracy

[0068] Recognition accuracy refers to the ratio of the number of targets correctly predicted by the model to the total number of targets when performing tasks such as image classification, detection or segmentation. The present invention uses accuracy to quantify the measurement.

[0069] Accuracy: refers to the ratio of the number of samples correctly classified by the model to the total number of samples. The calculation formula is:

[0070]

[0071] Among them, TP represents true positive examples, TN represents true negative examples, FP represents false positive examples, and FN represents false negative examples.

[0072] b. Positioning accuracy

[0073] Positioning accuracy refers to the accuracy of the unmanned equipment in positioning itself, that is, the difference between the system's estimated position and its actual position. The quantitative method used in this paper is the mean absolute error (MAE), and the calculation formula is:

[0074]

[0075] In the formula, Estimate i is the estimated target position, True i is the actual target position, and N is the number of sampling points.

[0076] 2) Independent judgment ability

[0077] a. Target location determination

[0078] Target position judgment refers to the ability of unmanned equipment to determine the target position based on sensor data. It is measured by the error between the estimated target position and the actual target position. The calculation formula is:

[0079]

[0080] In the formula, (x est ,y est ,z est ) is the estimated target position, (x true ,y true ,z true ) is the actual target position.

[0081] b. Target trajectory judgment

[0082] Target trajectory judgment refers to the ability of unmanned equipment to accurately judge and analyze the target trajectory, which can be quantitatively evaluated by the root mean square error. Root mean square error (RMSE): Calculate the root mean square error between the target trajectory estimated by the unmanned equipment and the actual target trajectory. The calculation formula is:

[0083]

[0084] In the formula, Estimate i is the estimated target position, True i is the actual target position, and N is the number of sampling points.

[0085] c. Identification of friend or foe

[0086] IFF refers to the ability of unmanned equipment to accurately identify friendly and enemy targets in the surrounding environment, which can be quantitatively evaluated using the following accuracy calculation formula:

[0087]

[0088] In the formula, Accuary represents the accuracy.

[0089] (3) Autonomous decision-making ability

[0090] a. Mission Planning

[0091] Mission planning refers to the ability of unmanned equipment to plan and execute when performing tasks. Quantitative evaluation can be based on the accuracy of mission planning. The accuracy of mission planning reflects the degree of conformity between the mission planned by the unmanned equipment and the actual mission performed. The present invention uses expert scoring to determine the accuracy.

[0092] b. Obstacle avoidance path planning

[0093] Obstacle avoidance path planning refers to the ability of unmanned equipment to plan a safe and feasible path based on the obstacles in the environment when performing a task. It can be measured by the length of the obstacle avoidance path.

[0094] c. Target path planning

[0095] Target path planning refers to the ability of unmanned equipment to plan the optimal path to achieve the mission objectives based on the mission requirements and target location. The present invention is measured based on the target path length.

[0096] (4) Autonomous execution capability

[0097] a. Hit Rate: The ratio of successfully hitting the target when executing a strike mission. The calculation formula is:

[0098]

[0099] Where N hit is the number of targets hit, N all The total number of targets set;

[0100] b. Convergence Rate: The ability to cover the target area when performing a strike mission. The calculation formula is:

[0101]

[0102] In the formula, S covered is the area covered by the strike, S total The total area of ​​the region for which the target is established.

[0103] c. Target damage probability

[0104] The target damage probability refers to the probability of unmanned equipment successfully destroying the target when performing a strike mission, that is, the probability of causing substantial damage to the target. Generally speaking, it is calculated by the product of the hit probability and the probability of hitting the vital part after hitting.

[0105] In one embodiment of the present invention, in step 2, the construction and calculation method of the synergy evaluation index is as follows:

[0106] 1) Collaborative perception capability

[0107] Collaborative perception capability refers to the ability of unmanned equipment to collaboratively detect and identify the surrounding environment using typical sensors such as optics, radar, and sonar. It is mainly measured by detection range and recognition accuracy.

[0108] 2) Collaborative communication capabilities

[0109] a. Communication timeliness

[0110] Communication timeliness refers to the speed at which unmanned equipment transmits information during communication, and is mainly quantitatively evaluated through transmission delay. The calculation formula is:

[0111] Delay=T arrival -T send

[0112] Delay is the information from the sender T send To the receiving end T arrival The time required for the transfer.

[0113] b. Communication capacity

[0114] Communication capacity refers to the amount of data or information that unmanned equipment can transmit and process per unit time, and is quantitatively evaluated by bandwidth: the maximum amount of data that a communication system can transmit per unit time.

[0115] c. Node connectivity

[0116] Node connectivity refers to the ability to establish connections and transfer information between nodes in unmanned equipment, which can be quantitatively evaluated through the following aspects:

[0117] Connectivity index: The degree or density of connections between unmanned equipment nodes, which can be calculated using the following formula:

[0118]

[0119] Where N connected is the number of connected unmanned equipment nodes, N all is the number of all nodes.

[0120] The connectivity index ranges from 0 to 1, and the closer it is to 1, the higher the degree of connectivity between unmanned equipment.

[0121] 3) Collaborative decision-making capabilities

[0122] a.Task completion rate

[0123] The ability of unmanned equipment to work together and complete tasks, including command and control, task allocation, resource scheduling, etc., can be measured by task completion rate in combination with expert prior knowledge.

[0124] b. Dynamic decision-making ability

[0125] Dynamic decision-making capability refers to the ability of each node in the unmanned equipment cluster to make fast and flexible decisions based on real-time conditions. The commonly used evaluation method is the expert scoring method, which can be developed from the perspectives of decision response time, decision accuracy, and decision flexibility.

[0126] 4) Collaborative execution capabilities

[0127] a. Coordinated strike accuracy

[0128] Coordinated strike accuracy refers to the accuracy of multiple unmanned equipment working together to strike the target. Similarly, the coordinated strike hit rate can be used to measure it:

[0129]

[0130] Where N cohit is the number of targets hit by the coordinated strike, N coall The total number of established targets.

[0131] b. Probability of coordinated damage

[0132] The probability of coordinated destruction refers to the probability of multiple unmanned equipment working together to successfully destroy the target. Similarly, it can be calculated by multiplying the probability of hitting the target and the probability of hitting the vital part after hitting.

[0133] In one embodiment of the present invention, in step 3, the comprehensive evaluation method of the intelligence level is as follows:

[0134] The weight w of the i-th evaluation index of autonomy in the hierarchical system is calculated by quantification and weighting method i and score A i , will score A i Normalize; where p is the number of autonomous evaluation indicators, i ranges from [1, p], w i The value range is [0,1]. The weight w of the jth evaluation index of synergy j and score B j , the score will be B j Normalize; where q is the number of synergy evaluation indicators, j ranges from [1, q], and w j The value range is [0,1].

[0135] This embodiment uses the hierarchical analysis method to determine the indicator weights, calculate the scores of each evaluation indicator in the first layer of the hierarchical system and the cumulative sum of the scores of the first layer evaluation indicators, and calculate the unmanned equipment autonomy score Score A and synergy score B .

[0136]

[0137] A scoring and grading model for autonomy and collaboration is established as shown in Table 2, and the autonomy and collaboration levels of unmanned equipment are queried from Table 2.

[0138] Table 2 Grading model for autonomy and collaboration

[0139]

[0140] In one embodiment of the present invention, a method for constructing an intelligence level scoring and intelligence grade mapping model for unmanned equipment is as follows:

[0141] According to the evaluation results of the autonomy and coordination of unmanned equipment, a mapping relationship table between the intelligence level and the autonomy and coordination capabilities shown in Table 3 is constructed to assess the intelligence level of the unmanned system.

[0142] Table 3 Intelligent level mapping model

[0143]

[0144] The above are preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An intelligent level assessment method for unmanned equipment, characterized by: The evaluation method comprises the following steps: Step 1: Construct unmanned equipment intelligence level evaluation indicators: including autonomous unmanned equipment intelligence level evaluation indicator system and collaborative unmanned equipment intelligence level evaluation indicator system; Step 2: Calculate the evaluation index values ​​of the autonomy and coordination of unmanned equipment; Step 3: Comprehensive evaluation of intelligence level: Use AHP technology to comprehensively evaluate the autonomy and coordination of unmanned equipment, and establish a scoring and grading model for autonomy and coordination; Step 4: Based on the evaluation results of the autonomy and collaboration of unmanned equipment, construct a mapping relationship table between the intelligence level and the autonomous capability and collaborative capability.

2. The intelligent level assessment method for unmanned equipment according to claim 1, characterized in that: In step 2, the autonomy evaluation indicators include autonomous perception capability, autonomous judgment capability, autonomous decision-making capability and autonomous execution capability; the collaboration evaluation indicators include collaborative perception capability, collaborative communication capability, collaborative decision-making capability and collaborative execution capability.

3. The intelligent level assessment method for unmanned equipment according to claim 2, characterized in that: Autonomous perception capabilities include: recognition accuracy and positioning accuracy; Autonomous judgment capabilities include: target position judgment, target trajectory judgment, and friend-or-foe judgment; Autonomous decision-making capabilities include: mission planning, obstacle avoidance path planning, and target path planning; Autonomous execution capabilities include: hit rate, coverage rate and target destruction probability.

4. The intelligent level assessment method for unmanned equipment according to claim 3, characterized in that: The accuracy is used to quantify the recognition accuracy, and the calculation formula is: Among them, TP represents true positive examples, TN represents true negative examples, FP represents false positive examples, and FN represents false negative examples; The quantification method used for positioning accuracy is the mean absolute error, and the calculation formula is: In the formula, Estimate i is the estimated target position, True i is the actual target position, and N is the number of sampling points.

5. The intelligent level assessment method for unmanned equipment according to claim 4, characterized in that: The target position judgment is measured by the error between the estimated target position and the actual target position, which is calculated by the following formula: In the formula, (x est ,y est ,z est ) is the estimated target position, (x true ,y true ,z true ) is the actual target position; The target trajectory judgment is quantitatively evaluated by the root mean square error, and the calculation formula is: In the formula, Estimate i is the estimated target position, True i is the actual target position, N is the number of sampling points; IFF refers to the ability of unmanned equipment to accurately identify friendly and enemy targets in the surrounding environment, and is quantitatively evaluated using the following accuracy calculation formula: In the formula, Accuary represents the accuracy.

6. The intelligent level assessment method for unmanned equipment according to claim 5, characterized in that: The hit rate is calculated using the following formula: Where N hit is the number of targets hit, N all The total number of targets set; The coverage rate is calculated using the following formula: In the formula, S corered is the area covered by the strike, S total The total area of ​​the region for which the target is established.

7. The intelligent level assessment method for unmanned equipment according to claim 2, characterized in that: The collaborative communication capabilities include: communication timeliness, communication capacity, and node connectivity; Collaborative decision-making capabilities include: task completion rate and dynamic decision-making capabilities; Coordinated execution capabilities include: coordinated strike accuracy and coordinated destruction probability.

8. The intelligent level assessment method for unmanned equipment according to claim 7, characterized in that: The communication timeliness is calculated by the following formula: Delay=T arrival -T send Delay information is sent from the sender T send To the receiving end T arrival the time required for the transfer; The connectivity index is calculated using the following formula: Where N connected is the number of connected unmanned equipment nodes, N all is the number of all nodes.

9. The intelligent level assessment method for unmanned equipment according to claim 8, characterized in that: The coordinated strike accuracy is measured by the coordinated strike hit rate, which is as follows: Where N cohit is the number of targets hit by the coordinated strike, N coall The total number of established targets.

10. The intelligent level assessment method for unmanned equipment according to claim 9, characterized in that: In step 3, the analytic hierarchy process is used to determine the indicator weights, calculate the scores of the first-level evaluation indicators of the hierarchical system and the cumulative sum of the first-level evaluation indicator scores, and calculate the unmanned equipment autonomy score Score A and synergy score B , as shown below: In the formula, A i is the score of the ith autonomy evaluation indicator, w i is the weight of the ith autonomy evaluation index, p is the number of autonomy evaluation indicators, the value range of i is [1, p], w i The value range of B is [0, 1]; j is the score of the j-th synergy evaluation index, w j is the weight of the jth synergy evaluation index, q is the number of synergy evaluation indexes, and the value range of j is [1, q], w j The value range is [0, 1].

Images