Unmanned equipment test scene construction and capability evaluation positive design method

Abstract

This invention belongs to the field of equipment performance testing and evaluation technology. It discloses a forward design method for constructing test scenarios and evaluating the capabilities of unmanned equipment, comprising two main stages: environmental scenario construction and capability evaluation. The specific steps are as follows: Step 1: Taking the rapid clearing of targets in underground space as the actual task orientation, and based on the typical environmental characteristics of underground space—darkness, signal rejection, and complex terrain—a typical task scenario for target identification and positioning in underground space is demonstrated. Step 2: The typical task scenario is refined into key actions: target detection, area search and positioning, obstacle crossing, and target confirmation. By breaking down these key actions, the core difficulties of unmanned equipment in performing the task are identified. This invention forms a forward design method for constructing environmental scenarios and evaluating the capabilities of unmanned equipment for underground space target detection and identification throughout the entire process from actual task requirements to capability assessment and evaluation, supporting the overall planning and design of performance tests for unmanned equipment in underground space target identification and positioning.

Description

Technical Field

[0001] This invention belongs to the field of equipment performance testing and evaluation technology, and in particular relates to a forward design method for constructing test scenarios and evaluating the capabilities of unmanned equipment. Background Technology

[0002] Currently, unmanned and intelligent equipment, represented by unmanned vehicles and drones, is playing an increasingly important role in disaster relief, counter-terrorism, and other missions. Underground spaces are vital lifelines or key passageways, often used by dangerous elements as strongholds to hinder search and capture operations. Due to the darkness and confinement of underground spaces, large equipment generally cannot enter, making unmanned equipment even more valuable in such environments, especially in the identification and location of dangerous targets. To enable unmanned equipment to solve the challenges of target detection and identification in dimly lit underground environments and to further drive the development of related cutting-edge technologies, it is urgent to conduct technical evaluations under relevant mission conditions and construct underground target detection and identification capability verification mission scenarios. These scenarios serve as crucial inputs for the planning and design of unmanned equipment performance tests. Developing assessment and testing procedures and capability evaluation systems is essential for accurately evaluating the comprehensive capabilities of unmanned equipment in performing underground target detection and identification tasks, providing feedback to support performance iteration and functional optimization design of unmanned equipment. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a forward design method for test scenario design and capability evaluation of unmanned equipment for underground space target detection and identification.

[0004] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows: A forward design method for constructing test scenarios and evaluating the capabilities of unmanned equipment includes two main stages: environmental scenario construction and capability evaluation. The specific steps are as follows: Step 1: Taking the rapid removal of targets in underground space as the actual task orientation, and based on the typical environmental characteristics of underground space such as dim lighting, signal rejection, and complex terrain, demonstrate the typical task scenarios for target identification and positioning in underground space. Step 2: Break down the typical task scenario into key actions such as target detection, area search and localization, obstacle crossing, and target confirmation. By breaking down the key actions, extract the core difficulties of unmanned equipment in performing the task. Step 3: Based on the core difficulties and the actual needs of underground space target detection and identification, propose key capabilities for unmanned equipment reconnaissance and detection in underground space, focus on the micro-task scenario of unmanned equipment searching and locating targets in underground space, summarize the evaluation and assessment points of obstacle crossing effect, number of targets discovered and identified, and recognition accuracy, and decompose each assessment point into a corresponding assessment process. Step 4: Based on the analysis results of the assessment indicators for relevant research topics, construct an assessment indicator system and establish an overall indicator evaluation system according to the focus of the indicator settings. Step 5: Collect data for each indicator based on the assessment process, process and analyze the data through the overall indicator evaluation system, obtain the overall evaluation result of underground space target detection and identification, complete the assessment of the unmanned equipment's ability to perform underground space target detection and identification tasks, and provide feedback support for the demonstration of unmanned equipment mission requirements.

[0005] Furthermore, the core challenges of unmanned equipment performing tasks in underground space as described in step two include the difficulty of real-time path planning and the difficulty of identifying and locating small, non-moving targets.

[0006] Furthermore, the construction of the micro-task scenarios adopts a multi-layered, integrated experimental scenario design method encompassing task requirements, key actions, functional scenarios, logical scenarios, and specific scenarios. The specific process is as follows: S41. Determine the task requirements for target detection, identification, and positioning in underground space, and clarify the capability assessment indicators for target detection and identification completion and target positioning accuracy. S42. Define the key actions for unmanned platforms to penetrate underground spaces to search for dangerous targets and complete target detection, identification and location. S43. Construct functional scenarios for unmanned equipment to travel along underground passages, avoid / overcome obstacles, and detect, identify, and locate targets within the passages; S44. Set the site environment and passage space parameters for underground darkness denial, determine the test objects of unmanned equipment and dangerous target identification objects with different locations and different characteristic attributes, and form a logical scenario; S45. Specify the exact dimensions of the underground passage site, the number of test subjects, the types and number of objects to be identified, and the types and number of obstacles. Develop specific test and assessment procedures for the unmanned equipment being tested to enter the mission area, handle obstacles, and search for and locate targets, thus forming a specific test scenario.

[0007] Furthermore, the construction of the specific test scenario also includes the construction of the equipment test environment. The construction of the equipment test environment includes the requirements for test items, the requirements for accompanying test items, and the setting of specific task environment. The requirements for test items include the determination of the type, quantity, and operation method of the test equipment. The requirements for accompanying test items include the determination of the type of equipment, the quantity of test targets, and the motion attributes of the test targets. The specific task environment includes the division of air area and ground area.

[0008] Furthermore, the capability evaluation described in step five adopts a percentage-based quantitative evaluation method, based on the overall success rate assessment of unmanned equipment in searching for and locating targets. The specific evaluation steps are as follows: S51. Count the total number N of target image photos automatically acquired or locked by unmanned equipment, and the total number n of valid image photos among them. The valid image photos are those where personnel / dangerous materials / explosive targets occupy no more than 1 / 3 of the image area and are marked with a rectangular frame + crosshair. Multiple valid photos of the same target are counted as 1. S52. Calculate the recognition accuracy X: X = n / N; S53. Determine the autonomous driving capability coefficient Y. In autonomous mode, Y=1.0, and in remote control mode, Y=0.9. In autonomous mode, all tested unmanned equipment must be operated autonomously. S54, Task completion time evaluation η: η=1-T / Tmax, where Tmax is the maximum expected time to complete the task and T is the actual task completion time; S55. Calculate the final score SUM: SUM = X × Y × η × 100 - 10K, where K is the number of times human intervention was performed; S56. If the actual task completion time T reaches Tmax, the test ends, and only the task completion result SUM before the end of the test is calculated. S57. Based on the final score SUM, classify the capability evaluation level and complete the capability evaluation of unmanned equipment.

[0009] Furthermore, the ability evaluation level is divided into 5 levels according to the score range: 80-100 points is good, 60-80 points is relatively good, 40-60 points is average, 20-40 points is poor, and 0-20 points is very poor.

[0010] Furthermore, the unmanned equipment includes one or more of unmanned vehicles, drones, and robots; the identified target includes personnel targets and simulated hazardous targets; and the obstacle includes one or more of barricades and barriers.

[0011] The present invention has the following advantages: (1) Based on the actual task requirements and the entire process of capability assessment and evaluation, combined with task scenario research, topic research, expert consultation and other channels, a positive design method for constructing and evaluating the environment scenario of unmanned equipment underground space target detection and recognition is formed to support the overall planning and design of the test of the effectiveness of unmanned equipment underground space target recognition and positioning.

[0012] (2) Based on the multi-layer mapping through-type equipment test scenario design method of “task requirements-key actions-functional scenarios-logical scenarios-specific scenarios”, a test scenario for target detection and identification of unmanned platforms in underground space is constructed. The establishment of this test scenario can provide reference and support for the test planning, design and specific implementation of single equipment and system-level equipment in relevant environmental scenarios.

[0013] (3) Set up capability assessment indicators based on key assessment points. Through aggregation method, the final evaluation results of the unmanned equipment's ability to perform spatial target reconnaissance, identification and positioning tasks in underground passages are formed, and the relevant data support the equipment's functional optimization design and performance iteration verification. Attached Figure Description

[0014] Figure 1 This invention presents a forward design approach for constructing and evaluating the environmental scenarios for target detection and identification in underground space for unmanned equipment. Figure 2 Design methodology and process for multi-layered, continuous scene layout; Figure 3 A schematic diagram of a test scenario for target detection and identification of unmanned equipment in underground space. Detailed Implementation

[0015] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0016] To address the lack of effective test scenario design and capability assessment methods for unmanned equipment performing target detection and identification in underground spaces, this embodiment provides a forward design method for test scenario design and capability evaluation of unmanned equipment performing target detection and identification in underground spaces. Based on relevant mission requirements and key actions, a test scenario for mission execution is constructed, forming a corresponding test scenario design method. Based on the setting of key assessment indicators and related weights, a method for evaluating the overall capability of unmanned equipment in underground space target detection and identification tasks is constructed.

[0017] like Figure 1 As shown, this embodiment is based on the actual mission requirement of rapidly clearing targets in an underground space. Starting from the dark and denied environment underground, it demonstrates the typical scenario of target identification and positioning in underground space, and refines it into key actions such as target detection, search and positioning, and obstacle crossing. It summarizes the difficulties in performing the mission, such as the difficulty of real-time path planning for unmanned equipment in underground space and the difficulty of identifying and locating small non-moving targets. Then, it proposes key capabilities for underground space reconnaissance and detection, and finally focuses on the micro-task scenario of target search and positioning in underground space for unmanned equipment (for details on the specific equipment, number of targets and attribute settings, please refer to the experimental scenario design method for target detection and identification of unmanned equipment in underground space). It summarizes the evaluation and assessment points such as obstacle crossing effect, number of targets found and identified, and recognition accuracy, and decomposes them into corresponding assessment processes.

[0018] Based on the analysis of assessment indicators for related fields and the experience of relevant experts, an assessment indicator system is constructed. According to the focus of the indicator settings, an overall indicator evaluation system is established. Based on the data obtained from each indicator, an overall evaluation result for underground space target detection and identification is derived, thereby providing an assessment of the overall capability of unmanned equipment to perform this task, and providing feedback and support for the demonstration of unmanned equipment mission requirements. In summary, based on a full-process approach encompassing mission requirements, typical scenarios, key actions, mission execution difficulties, key capabilities and urgently needed technologies, test mission scenarios, assessment indicators, assessment schemes, and evaluation methods, a positive design method for constructing environmental scenarios and evaluating the capabilities of unmanned equipment for underground space target detection and identification is formed.

[0019] Based on the multi-layered, integrated test scenario design method of "task requirements - key actions - functional scenarios - logical scenarios - specific scenarios" (see...) Figure 2 As shown in the figure, a test scenario for unmanned equipment to detect and identify targets in underground space is constructed, and its design process is as follows: 1) Mission Requirements: The mission focuses on target detection, identification, and localization in underground spaces. It assesses the unmanned platform's ability to autonomously detect, identify, and locate targets in dark, restricted, and enclosed spaces, and to apprehend dangerous targets. Assessment indicators include target detection and identification completion rate and target localization accuracy.

[0020] 2) Key Operations: Against the backdrop of our side deploying unmanned platforms to infiltrate underground spaces to search for terrorists, identify threatening targets, and effectively support subsequent support forces, the unmanned platforms, equipped with autonomous reconnaissance, identification, and positioning capabilities, completed the detection, identification, and effective positioning of dangerous targets during the mission.

[0021] 3) Functional scenarios, i.e. mission scenarios: When unmanned vehicles or drones travel along underground passages, they avoid or traverse obstacles, and detect, identify, and locate targets such as people and dangerous objects in the passages during the maneuver.

[0022] 4) Logical scenario, i.e., site environment and target setting: underground dark denial environment, length, width and height of the passage space; assessment object: unmanned vehicle, drone or robot with detection, identification and positioning functions; identification object: dangerous targets with characteristic attributes distributed in different positions in the passage space.

[0023] 5) Specific test scenarios and testing procedures: including the venue environment: corridor length, width, and height (X... Y Z)m; Assessment objects: 1 unmanned vehicle, 1 drone, or 1 robot; Identification objects: 2 personnel targets distributed in different locations (hidden behind cover, exhibiting certain task actions), 3 simulated hazards, and 4 sets of obstacles (including barricades, barriers, etc., specific locations and sizes depending on the situation). Specifically: The tested unmanned equipment autonomously or remotely enters the task area, travels along the target underground passage, avoids or traverses the 4 sets of obstacles within the task area, and sequentially searches for and locates the deployed detection and identification objects within the task area. The results are evaluated based on the identified and annotated images or the automatically stored video of the unmanned equipment's real-time reconnaissance and detection, and the task completion time is recorded.

[0024] See details Figure 3 As shown.

[0025] The equipment testing environment construction includes requirements for test items (including determining the type of test equipment, the number of test equipment, and the operation method of the test equipment), requirements for accompanying test items (including determining the type of equipment, the number of test targets, and the motion attributes of the test targets), and specific mission environment (including air area, ground area, etc.).

[0026] (3) Evaluation method for target detection and identification capability of unmanned equipment in underground space Based on the above test scenarios and assessment procedures, the overall success rate of unmanned equipment in searching for and locating targets will be evaluated (the specific number of unmanned equipment and targets deployed will depend on the specific test conditions). A score of 100 points will be used.

[0027] 1) Total number of images and photos N: The total number of target images and photos automatically acquired or locked by unmanned equipment.

[0028] 2) Total number of valid image photos n: The number of valid photos among those automatically acquired or locked by unmanned equipment, i.e., the number of personnel targets / hazardous material targets / explosive targets that are effectively identified. A valid photo is one in which the personnel target / hazardous material target / explosive target occupies no more than 1 / 3 of the image area and is marked with a rectangular frame and a crosshair. If two or more valid photos are actually of the same target, they are counted as only one target photo.

[0029] 3) Recognition accuracy X: X = n / N.

[0030] 4) Autonomous driving capability Y: Autonomous mode Y=1.0 (all unmanned equipment tested must be in autonomous mode), remote control mode Y=0.9.

[0031] 5) Task completion time evaluation η: η = 1 - T / T max T max T represents the maximum expected time to complete the task, and T represents the actual time to complete the task.

[0032] 6) Final score SUM: SUM = X × Y × η × 100 - 10K. Where K is the number of times human intervention was performed.

[0033] 7) Set T max The maximum expected time to complete the task is T. If the actual task completion time T reaches T... max If the test ends, only the task completion result SUM before the test ends will be calculated.

[0034] The corresponding grade evaluation will be given based on the assessment results. The specific grade evaluation is shown in Table 1.

[0035] Table 1. Competency Assessment Levels and Value Ranges Assessment of competency levels The range / score of the evaluation result G 1: Good 80-100 2: Better 60-80 3: General 40-60 4: Poor 20-40 5: Poor 0-20 Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art will be able to make various modifications and improvements without departing from the principles of the present invention, and these modifications and improvements should also be considered to fall within the scope of protection of the present invention.

Claims

1. A forward design method for constructing test scenarios and evaluating the capabilities of unmanned equipment, characterized in that, It includes two main stages: environmental scenario construction and capability assessment. The specific steps are as follows: Step 1: Taking the rapid removal of targets in underground space as the actual task orientation, and based on the typical environmental characteristics of underground space such as dim lighting, signal rejection, and complex terrain, demonstrate the typical task scenarios for target identification and positioning in underground space. Step 2: Break down the typical task scenario into key actions such as target detection, area search and localization, obstacle crossing, and target confirmation. By breaking down the key actions, extract the core difficulties of unmanned equipment in performing the task. Step 3: Based on the core difficulties and the actual needs of underground space target detection and identification, propose key capabilities for unmanned equipment reconnaissance and detection in underground space, focus on the micro-task scenario of unmanned equipment searching and locating targets in underground space, summarize the evaluation and assessment points of obstacle crossing effect, number of targets discovered and identified, and recognition accuracy, and decompose each assessment point into a corresponding assessment process. Step 4: Based on the analysis results of the assessment indicators for relevant research topics, construct an assessment indicator system and establish an overall indicator evaluation system according to the focus of the indicator settings. Step 5: Collect data for each indicator based on the assessment process, process and analyze the data through the overall indicator evaluation system, obtain the overall evaluation result of underground space target detection and identification, complete the assessment of the unmanned equipment's ability to perform underground space target detection and identification tasks, and provide feedback support for the demonstration of unmanned equipment mission requirements.

2. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 1, characterized in that, The core challenges of unmanned equipment performing tasks in underground space as described in step two include the difficulty of real-time path planning and the difficulty of identifying and locating small, non-moving targets.

3. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 1, characterized in that, The micro-task scenarios are constructed using a multi-layered, integrated experimental scenario design method, which includes task requirements, key actions, functional scenarios, logical scenarios, and specific scenarios. The specific process is as follows: S41. Determine the task requirements for target detection, identification, and positioning in underground space, and clarify the capability assessment indicators for target detection and identification completion and target positioning accuracy. S42. Define the key actions for unmanned platforms to penetrate underground spaces to search for dangerous targets and complete target detection, identification and location. S43. Construct functional scenarios for unmanned equipment to travel along underground passages, avoid / overcome obstacles, and detect, identify, and locate targets within the passages; S44. Set the site environment and passage space parameters for underground darkness denial, determine the test objects of unmanned equipment and dangerous target identification objects with different locations and different characteristic attributes, and form a logical scenario; S45. Specify the exact dimensions of the underground passage site, the number of test subjects, the types and number of objects to be identified, and the types and number of obstacles. Develop specific test and assessment procedures for the unmanned equipment being tested to enter the mission area, handle obstacles, and search for and locate targets, thus forming a specific test scenario.

4. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 3, characterized in that, The construction of the specific test scenario also includes the construction of the equipment test environment, which includes requirements for test items, requirements for accompanying test items, and specific task environment settings. The requirements for test items include the determination of the type, quantity, and operation method of the test equipment. The requirements for accompanying test items include the determination of the type of equipment, the quantity of test targets, and the motion attributes of the test targets. The specific task environment includes the division of air and ground areas.

5. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 1, characterized in that, The capability evaluation described in step five adopts a percentage-based quantitative evaluation method, based on the overall success rate assessment results of unmanned equipment searching for and locating targets. The specific evaluation steps are as follows: S51. Count the total number N of target image photos automatically acquired or locked by unmanned equipment, and the total number n of valid image photos among them. The valid image photos are those where personnel / dangerous materials / explosive targets occupy no more than 1 / 3 of the image area and are marked with a rectangular frame + crosshair. Multiple valid photos of the same target are counted as 1. S52. Calculate the recognition accuracy X: X = n / N; S53. Determine the autonomous driving capability coefficient Y. In autonomous mode, Y=1.0, and in remote control mode, Y=0.

9. In autonomous mode, all tested unmanned equipment must be operated autonomously. S54. Task completion time evaluation score η: η = 1 - T / T max T max T represents the maximum expected time to complete the task, and T represents the actual time to complete the task. S55. Calculate the final score SUM: SUM = X × Y × η × 100 - 10K, where K is the number of times human intervention was performed; S56. If the actual task completion time T reaches T max If the result is not found, the test ends, and only the task completion result SUM before the end of the test is calculated; S57. Based on the final score SUM, classify the capability evaluation level and complete the capability evaluation of unmanned equipment.

6. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 5, characterized in that, The ability evaluation level is divided into 5 levels according to the score range: 80-100 points is good, 60-80 points is relatively good, 40-60 points is average, 20-40 points is poor, and 0-20 points is very poor.

7. The forward design method for constructing test scenarios and evaluating capabilities of unmanned equipment according to claim 1, characterized in that, The unmanned equipment includes one or more of unmanned vehicles, drones, and robots; the identified targets include personnel targets and simulated hazardous targets; and the obstacles include one or more of barricades and barriers.

Images