A multi-domain unmanned system interoperability evaluation environment construction and utilization method

Abstract

The application provides a multi-domain unmanned system interoperability evaluation environment construction and application method, which combines the characteristics of multi-domain unmanned systems, constructs an evaluation environment around the interoperability test evaluation requirements, and accesses test resources in the form of installation (L), simulation (V) and simulation (C), as well as multi-domain unmanned system test resources. During the interoperability evaluation test, static compliance testing and dynamic simulation testing are included. The static testing can collect multi-domain unmanned system interoperability evaluation index data through manual and automatic collection. The dynamic simulation test supports information and instruction transmission testing between unmanned system test resources for typical application scenarios. Finally, the test data is evaluated from four dimensions of procedures (P), applications (A), infrastructure (I) and data (D), and the multi-domain unmanned system interoperability level is obtained, which provides guidance for unmanned system interoperability level evaluation and capability improvement.

Description

TECHNICAL FIELD

[0001] The application relates to a multi-domain unmanned system interoperability evaluation environment construction and application method. BACKGROUND

[0002] With the development of unmanned equipment, more and more unmanned equipment is put into various fields, such as material delivery, disaster relief, city security, resource exploration and power inspection in the civil field, and front-line reconnaissance and precision strike in the military field. As a military force, unmanned equipment has the characteristics of low cost, zero casualties, fearlessness of harsh conditions and high response accuracy, and will become the development trend of future war. Compared with the use of single-domain unmanned equipment, the cross-domain collaborative use of multi-domain unmanned equipment can effectively make up for the shortcomings of single-domain unmanned equipment, further exert the effectiveness of unmanned equipment and improve the systematized use level of unmanned equipment. Under this background, unmanned system interoperability technology emerges as the times require. Unmanned system interoperability refers to the provision and acceptance of services (including data, information, tasks, instructions and functions) between two or more unmanned systems or elements, so as to realize collaborative operation. Unmanned system interoperability can fully improve the use efficiency of unmanned equipment and exert the potential effectiveness of unmanned equipment, and is the development focus of future unmanned equipment.

[0003] At present, domestic and foreign countries are strengthening the construction of unmanned equipment interoperability. Taking the United States as an example, the US Department of Defense has released a new 30-year unmanned system plan, namely, the "2017-2042 Unmanned System Integrated Roadmap", which emphasizes the demand for interoperability between systems and analyzes the technical challenges faced by unmanned system interoperability, including the need to improve testing, evaluation, verification and certification of interoperability standards, and the need for more collaboration between military services. In terms of unmanned system interoperability level evaluation, Wu Lizhen of the National University of Defense Technology proposed an interoperability level capability model in combination with the "NATO Unmanned Aircraft Control System (UCS) Interface Standard" (STANAG 4586), and proposed a cross-domain heterogeneous unmanned system interoperability level evaluation method according to the dimensions of isomorphism / heterogeneity, via ground station / without ground station, and payload control / collaborative use. However, as a whole, the research on multi-domain unmanned system interoperability standard testing and evaluation environment in China is still in its infancy.

[0004] In order to improve the interoperability level of unmanned equipment, it is necessary to strengthen the interoperability testing and certification capability of unmanned equipment. By establishing a scientific and normative multi-domain unmanned equipment interoperability evaluation environment, carrying out multi-domain unmanned equipment interoperability level evaluation based on the environment, the interoperability level of cross-domain unmanned equipment can be effectively determined, which is of great significance to standardize the application of unmanned system interoperability and improve the level of unmanned system interoperability. SUMMARY

[0005] The technical problem solved by the present application is to provide a multi-domain unmanned system interoperability evaluation environment construction and application method to improve the interoperability test capability of unmanned equipment and standardize the multi-domain unmanned system interoperability application.

[0006] The method of the present application comprises the following steps:

[0007] Step 1, clearly define the multi-domain unmanned system interoperability test evaluation task requirements (such as the need to evaluate the interoperability level of unmanned system control station and other information systems, the interoperability level of unmanned system control station and other unmanned platforms), determine the test unmanned equipment and test resources (determine the evaluation object according to the evaluation task requirements, determine which information systems, unmanned system control stations, and air, ground, underwater multi-domain unmanned platform resources are needed to participate in the test);

[0008] Step 2, construct a multi-domain unmanned system interoperability test evaluation environment test system, including 4 subsystems of evaluation task management and control, evaluation index construction, data collection management, and evaluation result generation;

[0009] Step 3, construct a multi-domain unmanned system interoperability test evaluation environment test resource area, including LVC form (real equipment (L), simulation (V), simulation (C)) test resources and multi-domain unmanned system test resources, and unified access to the test environment;

[0010] Step 4, based on the distributed service bus and the test support network, carry out interoperability evaluation, including static conformity test and dynamic simulation test, and collect the procedure P, application A, infrastructure I, and data D attribute evaluation data required for multi-domain unmanned system interoperability level evaluation;

[0011] Step 5, based on the test system, analyze the collected data, construct a multi-level evaluation index system from the procedure P, application A, infrastructure I, and data D four dimensions, and comprehensively obtain the multi-domain unmanned system interoperability level evaluation result.

[0012] Step 2 comprises:

[0013] Step 2-1, design the evaluation task management and control subsystem, including an evaluation task planning module, a test environment design module, and a task state monitoring module;

[0014] Step 2-2, design the evaluation index construction subsystem, including an index system template management module, an evaluation index establishment module, and an index weight distribution module;

[0015] Step 2-3, design the data collection management subsystem, including a manual data collection module, an automatic data collection module, and a data management module;

[0016] Step 2-4, design evaluation result generation subsystem, including data compliance verification module, index calculation comprehensive module, evaluation result display module.

[0017] In step 2-1, the evaluation task planning module has evaluation task establishment, evaluation task decomposition, and evaluation task management functions, and can edit test purposes, test subjects, and test content, and perform subject decomposition, while supporting deletion and viewing of existing test tasks:

[0018] The test environment design module supports selection based on evaluation tasks, and can select from existing test resource information in the database or newly registered test resources, and configure the connection relationship (such as control relationship, information interaction relationship) between test resources to guide the construction of test environment;

[0019] The task state monitoring module can display and monitor the development state, execution progress, and completion of each evaluation task or subtask.

[0020] In step 2-2, after the evaluation task is established, the corresponding evaluation index needs to be designed. The index system template management module is mainly provided to meet the reuse function of the evaluation task, and provides functions of creating, opening, modifying, and deleting index system templates;

[0021] The evaluation index establishment module provides a graphic operation interface, supports attribute supplementation of the index system template based on the evaluation task, and establishes a multi-layer tree-shaped evaluation index system in a visual manner, mainly including the establishment and decomposition of procedure (P), application (A), infrastructure (I), and data (D) attribute indexes;

[0022] The index weight allocation module provides index weight calculation functions, and realizes weight coefficient calculation of each index leaf node and subnode through subjective and objective weighting methods. The subjective and objective weighting methods include the analytic hierarchy process and expert weighting method.

[0023] In step 2-3, after the evaluation index design is completed, the test can be carried out, and the main way is to manually operate the information system, control station, and unmanned platform to send information to each other, record the data transmission situation, and support test data collection. Among them, the manual data collection module is based on the evaluation index system, and the evaluation index template is preset based on the procedure (P), application (A), infrastructure (I), and data (D) attributes. Combined with the data transmission situation, the data is collected by manual judgment and selection / entry;

[0024] The automatic data collection module automatically acquires the required evaluation data through packet capture during data transmission;

[0025] The data management module supports filtering and storage management of data obtained by manual and automatic methods.

[0026] In steps 2-4, after data collection is completed, the data is transmitted to the evaluation result generation subsystem to carry out evaluation, and the data compliance verification module is used to verify the compliance of the load work product data, load control and response data, platform control and response data transmitted between the test objects in the test process, calculate the compliance degree with the relevant standards and specifications, and give the data D attribute evaluation result;

[0027] The index calculation and synthesis module supports the synthesis of manually entered data by the user and calculated data based on compliance verification, and obtains the evaluation results of the attributes of the procedures P, applications A, infrastructures I, data D and the overall interoperability level of unmanned systems.

[0028] The evaluation result display module visually displays the evaluation results in the form of column chart, pie chart and line chart.

[0029] Step 3 includes:

[0030] Step 3-1, access LVC test resources based on the test local area network, wherein the LVC test resources include information system implementation and other equipment platform simulator resources.

[0031] Step 3-2, access multi-domain unmanned system test resources based on the test local area network, wherein the multi-domain unmanned system test resources include unmanned aerial vehicles and unmanned vehicles.

[0032] Step 4 includes:

[0033] Step 4-1, establish evaluation tasks and corresponding evaluation subtasks by using the evaluation task control subsystem: use the evaluation task planning module to select the evaluation task type, such as evaluating the interoperability level of the unmanned system control station and other information systems, and the interoperability level of the unmanned system control station and other unmanned platforms, and then automatically generate evaluation subtasks according to the evaluation tasks. The subtasks are used to obtain and evaluate the specific attributes of the test objects. Here, four subtasks of procedures (P), applications (A), infrastructures (I) and data (D) are taken as examples for illustration.

[0034] Step 4-2, use the evaluation index construction subsystem to construct a multi-level evaluation index system from the four dimensions of procedures, applications, infrastructures and data according to the evaluation tasks and evaluation subtasks, and generate an evaluation data collection template. The collection requirements can be determined according to different specific evaluation models, and only one example is given in the present application for reference.

[0035] The procedure attribute refers to whether the integration and collaborative use of unmanned systems comply with relevant standards, policies, regulations, use rules, safety strategies and operation procedures. The collection requirement example is as follows:

[0036] Whether to follow the industry-wide uniform action coordination procedure;

[0037] Whether to follow the general control station-based unmanned platform action coordination procedure;

[0038] Whether to follow the homogeneous unmanned platform action coordination procedure;

[0039] Whether to follow the special control station-based unmanned platform action coordination procedure;

[0040] Whether to follow the industry-wide uniform task coordination procedure;

[0041] Whether to follow the general control station-based unmanned platform task coordination procedure;

[0042] Whether to follow the same task coordination procedure;

[0043] Whether to follow the special control station-based unmanned platform task coordination procedure;

[0044] Whether to follow the information sharing procedure between heterogeneous unmanned platforms;

[0045] Whether to follow the general control station information sharing procedure;

[0046] Whether to follow the special control station information sharing procedure;

[0047] Application attributes refer to the generalization degree of application software related to unmanned platform control software and task control software, such as special software / single-class general software / single-domain general software / multi-domain general software, and the collection demand categories are as follows:

[0048] Whether the platform uses an action coordination software that meets the industry standard;

[0049] Whether the control station uses a general action coordination software, and the platform uses a self-defined action control software;

[0050] Whether the platform uses an action coordination software that meets the professional field standard;

[0051] Whether the control station uses an action coordination software that meets the professional field standard, and the platform uses a self-defined action control software;

[0052] Whether the platform uses a task coordination software that meets the industry standard;

[0053] Whether the control station uses a general task coordination software, and the platform uses a self-defined task coordination software;

[0054] Whether the platform uses a task coordination software that meets the professional field standard;

[0055] Whether the control station uses a task coordination software that meets the professional field standard, and the platform uses a self-defined task coordination software;

[0056] Whether the control station adopts general TT&C software and the platform adopts general information exchange software;

[0057] Whether the control station adopts general TT&C software and the platform adopts self-defined information exchange software;

[0058] Whether the control station adopts special TT&C software and the platform adopts self-defined information exchange software;

[0059] Infrastructure attributes refer to the capability level of communication, computing and storage, security and privacy, and terminal equipment and facilities supporting the interoperability of unmanned systems, and the collection demand categories are as follows:

[0060] Whether to adopt general unmanned platform action coordination link (ad hoc network);

[0061] Whether to adopt general control station-unmanned platform action coordination link (point-to-multipoint);

[0062] Whether to adopt special unmanned platform action coordination link (ad hoc network);

[0063] Whether to adopt special control station-unmanned platform action coordination link (point-to-multipoint);

[0064] Whether to adopt general unmanned platform task coordination link (ad hoc network);

[0065] Whether to adopt general control station-unmanned platform task coordination link (point-to-multipoint);

[0066] Whether to adopt special unmanned platform task coordination link (ad hoc network);

[0067] Whether to adopt special control station-unmanned platform task coordination link (point-to-multipoint);

[0068] Whether to adopt general information transmission and exchange infrastructure (ad hoc network);

[0069] Whether to adopt general information transmission and exchange infrastructure (point-to-multipoint);

[0070] Whether to adopt special information transmission and exchange infrastructure (point-to-point);

[0071] Data attributes refer to the standardization level of the content and format of the business and monitoring data processed and exchanged by the unmanned system, and the collection demand categories are as follows:

[0072] Whether the unmanned platform adopts the unmanned system action coordination platform control instruction format and content conforming to the industry standard;

[0073] Whether the control station adopts the unmanned system action coordination platform control instruction format and content conforming to the industry standard;

[0074] whether the unmanned platform adopts the unmanned system action coordination platform control instruction format and content conforming to the professional field standard;

[0075] whether the control station adopts the unmanned system action coordination platform control instruction format and content conforming to the professional field standard;

[0076] whether the unmanned platform adopts the task coordination instruction format and content conforming to the industry standard;

[0077] whether the control station adopts the unmanned system task coordination platform control instruction format and content conforming to the industry standard;

[0078] whether the unmanned platform adopts the task coordination instruction format and content conforming to the professional field standard;

[0079] whether the control station adopts the unmanned system task coordination platform control instruction format and content conforming to the professional field standard;

[0080] whether the data format and content (payload state information, monitoring instruction) conform to the industry standard;

[0081] whether the data format and content (payload detection information, task data) conform to the professional field standard;

[0082] whether the data format and content (payload detection information, task data) are self-defined;

[0083] Step 4-3, static compliance test is carried out, and the four types of attributes of the regulations P, the application A, the infrastructure I and the data D are statically verified, the corresponding attribute values are selected by checking the data files of the unmanned system and the questionnaire, and the results are input into the data collection and management subsystem;

[0084] Step 4-4, dynamic simulation test is carried out, the test resources and the test resources are sent according to the assumption, including payload work product data, payload control instruction, payload response data, platform control instruction, platform response data, whether the unmanned system can receive data and respond as expected, mainly through artificial judgment, for example, the unmanned system control station sends control instruction to another unmanned platform, artificial checks whether the unmanned platform responds, and whether the unmanned system control station receives the expected response data, according to the receiving and responding situation, the results that cannot be obtained by static compliance test and the results that do not conform to static compliance test are further corrected and improved.

[0085] Step 5 includes:

[0086] Step 5-1, analyzing the attribute test results of the procedure P, application A, infrastructure I and data D attribute data, wherein the procedure P, application A and infrastructure I are determined by artificial comprehensive analysis, and the data D attribute is calculated by a data compliance verification module, and the message verification process comprises:

[0087] Step 5-1-1, creating a message verification criterion model:

[0088] Adding a verification criterion model: in the verification tool, the standard message format is imported, the message format is visually processed, and the standard message structure and attribute value are generated.

[0089] Editing the standard verification rule of the message: according to the requirement, the elements of the message to be verified are edited, including whether it is a mandatory node, the maximum number of node occurrences and attribute value verification.

[0090] Step 5-1-2, executing the message verification task:

[0091] Establishing a message verification task: establishing a message verification task, and importing the standard verification criterion edited in step 5-1-1.

[0092] Executing verification: reading the message to be verified and matching the standard verification criterion for verification.

[0093] Step 5-1-3, analyzing the message verification result:

[0094] The message sent by the unmanned system is automatically checked and analyzed with the standard message verification result, and the evaluation level of the data D attribute is matched.

[0095] Step 5-2, analyzing the procedure P, application A, infrastructure I and data D attribute evaluation level, and determining the highest level that each attribute can reach according to the test satisfaction degree.

[0096] Step 5-3, selecting the lowest one of the evaluation levels of the procedure P, application A, infrastructure I and data D, that is, the interoperability level that can be reached between the multi-domain unmanned systems.

[0097] The application also provides a storage medium storing a computer program or instructions, which, when executed, implement the multi-domain unmanned system interoperability evaluation environment construction and utilization method.

[0098] The beneficial effects of the present application are that: (1) the present application provides an evaluation environment construction method based on LVC simulation to solve the problem of joint use of multiple subject resources and test resources for multi-domain unmanned system interoperability level evaluation; the subject and test resources support multiple forms of access of real equipment, simulators and digital models, and solve the problems of high cost and difficult resource coordination caused by pure real equipment resource testing; (2) the present application integrates the evaluation model and evaluation criteria into the evaluation software to facilitate operators to carry out evaluation tasks according to the process, improve the standardization of the evaluation process and the efficiency of the evaluation; (3) the present application provides a data (D) attribute automatic evaluation function based on message verification to solve the problem that the evaluation is difficult to be accurate due to the involvement of numerous formats of information, instruction data in multi-domain unmanned system interoperability level evaluation; the machine automatic reading function of data attribute is realized through the preset judgment algorithm for various complex data such as payload task information, payload control instruction, platform task information and platform control instruction, and the reliability and accuracy of the evaluation result are improved. BRIEF DESCRIPTION OF DRAWINGS

[0099] The above and / or other aspects of the present application will become apparent by referring to the following description in conjunction with the accompanying drawings, in which:

[0100] Figure 1a is a schematic diagram of the evaluation task management and control subsystem in the present application.

[0101] Figure 1b is a schematic diagram of the relationship of the evaluation task management and control subsystem in the present application.

[0102] Figure 2a is a schematic diagram of the evaluation index construction subsystem in the present application.

[0103] Figure 2b is a schematic diagram of the information relationship of the evaluation index construction subsystem in the present application.

[0104] Figure 3a is a schematic diagram of the data acquisition management subsystem in the present application.

[0105] Figure 3b is a schematic diagram of the information relationship of the data acquisition management subsystem in the present application.

[0106] Figure 4a is a schematic diagram of the evaluation result generation subsystem in the present application.

[0107] Figure 4b is a schematic diagram of the information relationship of the evaluation result generation subsystem in the present application.

[0108] Figure 5 is a schematic diagram of the multi-domain unmanned system interoperability evaluation environment finally formed by the present application.

[0109] Figure 6 is a typical cross-domain unmanned system interoperability evaluation case test environment design schematic diagram in the application. DETAILED DESCRIPTION

[0110] The application provides a multi-domain unmanned system interoperability evaluation environment construction and use method, comprising:

[0111] Step 1, clearly defining the multi-domain unmanned system interoperability test evaluation task requirements, determining the test unmanned equipment and test resources;

[0112] Step 2, constructing a multi-domain unmanned system interoperability test evaluation environment test system, including four subsystems of evaluation task management and control, evaluation index construction, data acquisition management and evaluation result generation;

[0113] (1) The evaluation task management and control subsystem is composed as shown in Figure 1a , including an evaluation task planning module, a test environment design module and a task state monitoring module. The evaluation task planning module, as the initiator of the entire evaluation task, includes evaluation task establishment, evaluation task decomposition and evaluation task management functions; the test environment design module supports selecting existing test resources or adding new test resources according to the evaluation task, and configuring connection relationships to guide the construction of the test environment; the task state monitoring includes the display monitoring of the development state, execution progress and completion of each evaluation task / subtask. The information relationship of the evaluation task management and control subsystem is as shown in Figure 1b , the test resource requirement generated by the evaluation task planning module is provided to the test environment design module to guide the test environment design; the evaluation task planning module provides an evaluation task list to the task state monitoring to support task process monitoring.

[0114] (2) The evaluation index construction subsystem is composed as shown in Figure 2a , including an index system template management module, an evaluation index establishment module and an index weight allocation module. The index system template management module mainly exists to meet the reuse function of the evaluation task, and provides functions of newly creating, opening, modifying and deleting the index system template; the evaluation index establishment module provides a graphic operation interface, supports attribute supplementing of the index system template according to the evaluation task, and establishes a multi-layer tree-shaped evaluation index system in a visual manner, including the establishment and decomposition of the attribute indexes of procedures (P), applications (A), infrastructure (I) and data (D); the index weight allocation module provides an index weight calculation function, and realizes the weight coefficient calculation of each index leaf node and subnode through subjective and objective weighting methods. The subjective and objective weighting methods are analytic hierarchy process and expert weighting method. The information relationship of the evaluation index construction subsystem is as shown in Figure 2bAs shown, the index system construction module generates an index system according to the evaluation task input and provides the index system template management module for storage and modification; the index system construction module provides the index weight distribution module with the relationship information between indexes for subsequent index weight modification.

[0115] (3) The data acquisition management subsystem is composed of Figure 3a As shown, the data acquisition management subsystem is composed of manual data acquisition, automatic data acquisition, and data management module. The manual data acquisition is based on the evaluation index template preset according to the evaluation index system, the procedure (P), application (A), infrastructure (I), and data (D) attributes, and is collected by manual selection / entry of the user; the automatic data acquisition supports networked acquisition in the cross-domain unmanned system interoperability evaluation verification environment for the data (D) index, and automatically acquires the required evaluation data by packet capture during the data transmission between the test unmanned systems; the data management supports filtering and storage management of the data obtained by manual and automatic methods. The information relationship of the data acquisition management subsystem is as shown in Figure 3b As shown, the automatic data acquisition module and the manual data acquisition module provide data supplement requirements to each other to enrich the data sources, and at the same time, provide the collected data to the data management module for data storage management.

[0116] (4) The evaluation result generation subsystem is composed of Figure 4a As shown, the evaluation result generation subsystem is composed of data compliance verification, index calculation and synthesis, and evaluation result display module. The data compliance verification supports compliance verification of the payload work product data, payload control and response data, and platform control and response data transmitted between the test unmanned systems during the test, calculates the compliance degree with the relevant standards and specifications, and gives the data (D) attribute evaluation result; the index calculation and synthesis supports the synthesis of the manually entered data and the calculated data obtained based on the compliance verification to obtain the PAID attribute evaluation result and the unmanned overall interoperability level evaluation result; the evaluation result display visually displays the evaluation result in the form of column chart, pie chart, and line chart. The information relationship of the evaluation result generation subsystem is as shown in Figure 4b As shown, the data compliance verification module generates the data (D) attribute evaluation result and provides it to the index calculation and synthesis module, the index calculation and synthesis module fuses the manually collected other attribute evaluation to generate a comprehensive evaluation result and provides it to the evaluation result display module to support subsequent result visualization.

[0117] Step 3, construct a multi-domain unmanned system interoperability test evaluation environment test resource area, including LVC form test resources and multi-domain unmanned system test resources, and uniformly access the test environment, the specific steps are as follows:

[0118] (1) Access to test resources. Test resources are mainly used to cooperate with the development of multi-domain unmanned system interoperability level evaluation equipment or information system, such as data processing information system:

[0119] For entity test resources, design related proxy gateway and access test environment, complete protocol conversion between entity resources and test system in test control area through proxy gateway, help entity test resources complete dynamic access and receive simulation control action;

[0120] For model test resources, encapsulate the model into an installable software package and a container image. The image package contains the basic environment and dependent environment necessary for model running. Deploy the model to the server through the image package to complete the access of test resources.

[0121] (2) Access to test resources. The test resources are LVC multi-domain unmanned system test objects. The access methods of test multi-domain unmanned system entity resources and model resources are the same as steps 3 and (1).

[0122] Step 4, form a multi-domain unmanned system interoperability evaluation environment as shown in Figure 5 The following describes the application process of the interoperability evaluation environment. Based on the distributed service bus and test support network, carry out interoperability evaluation, including static compliance test and dynamic simulation test. Collect the required procedure (P), application (A), infrastructure (I), and data (D) attribute evaluation data for multi-domain unmanned system interoperability level evaluation. The specific steps are as follows:

[0123] (1) Use the evaluation task control subsystem to establish evaluation tasks and corresponding evaluation subtasks, and use the evaluation index construction subsystem to construct a multi-level evaluation index system from the four dimensions of procedure (P), application (A), infrastructure (I), and data (D), and generate evaluation data collection templates;

[0124] (2) Carry out static compliance test, and verify the four types of attributes of procedure (P), application (A), infrastructure (I), and data (D). Record the results in the data collection management subsystem. The attribute data collection requirements are shown in Table 1 as follows:

[0125] Table 1

[0126]

[0127]

[0128] (3) Carry out dynamic simulation test. The test resources and test resources send data according to the scenario, including load work product data, load control and response data, and platform control and response data.

[0129] Payload working product data is sent by the test unmanned system to the interoperability level evaluation site through the control station / operation terminal, such as image data, synthetic aperture radar GMTI data;

[0130] Payload control and response data is sent by the test unmanned system A to the test unmanned system B, and the test unmanned system B makes a payload control response to the test unmanned system A;

[0131] Platform control and response data is sent by the test unmanned system A to the test unmanned system B, and the test unmanned system B makes a platform control response to the test unmanned system A.

[0132] (4) The data acquisition management subsystem acquires each attribute evaluation data, wherein the procedure (P), application (A), and infrastructure (I) attribute values are determined by manual input by the user to determine the procedure (P), application (A), and infrastructure (I) attribute evaluation data between the evaluated unmanned systems, and are stored in the data acquisition management subsystem;

[0133] The data (D) attribute values are collected and stored through network collection means, such as packet capture tools, for payload working product data, payload control and response data, and platform control and response data, and are stored in the data acquisition management subsystem.

[0134] Step 5, based on the test system, the collected data is analyzed, a multi-level evaluation index system is constructed from the four dimensions of procedure (P), application (A), infrastructure (I), and data (D), and a multi-domain unmanned system interoperability level evaluation result is obtained, and the specific steps are as follows;

[0135] (1) The procedure (P), application (A), infrastructure (I), and data (D) attribute data are analyzed to obtain each attribute test result; wherein the procedure (P), application (A), and infrastructure (I) are determined by artificial comprehensive analysis, and the data (D) attribute can be calculated by the data compliance verification function of the test evaluation result generation subsystem, such as the main process of message verification:

[0136] 1) New message verification rule model

[0137] a) Add verification rule model

[0138] In the verification tool, the standard message format is imported, and the software automatically visualizes the message format to generate a standard message structure and attribute value.

[0139] b) Edit message verification rules

[0140] According to the requirement, the elements of the message to be verified are edited, including whether it is a mandatory node, the maximum number of node occurrences, and attribute value verification (including fixed value, enumeration value, regular expression verification, interval verification).

[0141] 2) Perform message verification task

[0142] a) Establish message verification task

[0143] Establish a message verification task, and import the standard verification criteria edited in step 1).

[0144] b) Perform verification

[0145] Read the message to be verified and match it with the standard verification criteria for verification.

[0146] 3) Analyze message verification results

[0147] Automatically check and analyze the message sent by the unmanned system and the standard message verification results, and match the evaluation level of the data D attribute listed in Table 1;

[0148] (2) Analyze the procedure (P), application (A), infrastructure (I), and data (D) attribute evaluation levels, and determine the highest level that each attribute can reach according to the test satisfaction level. The evaluation level reference table is shown in Table 2:

[0149] Table 2

[0150]

[0151]

[0152]

[0153] (3) Select the lowest one of the procedure (P), application (A), infrastructure (I), and data (D) four attributes, which is the interoperability level that the multi-domain unmanned system can reach. For example, the procedure (P), application (A), infrastructure (I), and data (D) four attribute evaluation levels are L07, L08, L05, and L06, respectively, and the interoperability level that the multi-domain unmanned system can reach is L05.

[0154] The application will be further described in detail in conjunction with the following examples:

[0155] Example 1

[0156] Taking the evaluation of the interoperability level of a certain unmanned platform control terminal and multi-domain unmanned platform as an example.

[0157] (1) According to step 1, the task requirements of this multi-domain unmanned system interoperability test evaluation are determined as follows: evaluating the interoperability level of a certain unmanned platform control terminal and multi-domain unmanned platform; the test equipment includes the unmanned platform control terminal, unmanned aerial vehicle A, unmanned aerial vehicle B, unmanned vehicle A, and unmanned vehicle B; the test resources include information system resources and platform equipment simulators.

[0158] (2) According to step 2, the multi-domain unmanned system interoperability test evaluation environment test system is constructed, and four subsystems of evaluation task management and control, evaluation index construction, data acquisition management, and evaluation result generation are deployed.

[0159] (3) According to step 3, the multi-domain unmanned system interoperability test evaluation environment test resource area is constructed, and the test equipment and test resources are connected. The test equipment includes the unmanned platform control terminal, unmanned aerial vehicle A, unmanned aerial vehicle B, unmanned vehicle A, and unmanned vehicle B; the test resources include information system resources and platform equipment simulators. A typical cross-domain unmanned system interoperability evaluation case test environment is formed as shown in FIG. 1. Figure 6

[0160] (4) According to step 4(1), the evaluation task management and control subsystem is used to establish the evaluation task of the interoperability level of the certain unmanned platform control terminal and the multi-domain unmanned platform, which includes four evaluation sub-tasks of the interoperability level of the unmanned platform control terminal and unmanned aerial vehicle A, the interoperability level of the unmanned platform control terminal and unmanned aerial vehicle B, the interoperability level of the unmanned platform control terminal and unmanned vehicle A, and the interoperability level of the unmanned platform control terminal and unmanned vehicle A.

[0161] (5) According to step 4(1), the evaluation index construction subsystem is used to construct a multi-level evaluation index system from four dimensions of procedures (P), applications (A), infrastructure (I), and data (D), and generate an evaluation data acquisition template. The acquisition template used this time is shown in Table 1.

[0162] (6) According to step 4(2), the four types of properties of procedures (P), applications (A), infrastructure (I), and data (D) are statically verified, and the results of each data acquisition are determined by artificial judgment. It is assumed that the results of static verification are as follows:

[0163] ① Unmanned platform control terminal and unmanned aerial vehicle A interoperability property acquisition:

[0164] Procedures (P) properties: follow the isomorphic unmanned platform action coordination procedures;

[0165] Applications (A) properties: the platform uses action coordination software that meets the professional field standards;

[0166] Infrastructure (I) properties: use dedicated unmanned platform action coordination links (ad hoc networks);​

[0167] Data (D) attribute: unable to collect through static verification;

[0168] ②Unmanned platform control terminal and unmanned vehicle B interoperation attribute collection:

[0169] Procedures (P) attribute: follow the special control station-based unmanned platform action coordination procedures;

[0170] Application (A) attribute: control station uses action coordination software that meets professional standards, and platform uses self-defined action control software;

[0171] Infrastructure (I) attribute: use special control station-unmanned platform action coordination link (point-to-multipoint);

[0172] Data (D) attribute: unable to collect through static verification;

[0173] ③Unmanned platform control terminal and unmanned vehicle A interoperation attribute collection:

[0174] Procedures (P) attribute: follow the industry-wide unified task coordination procedures;

[0175] Application (A) attribute: platform uses task coordination software that meets industry standards;

[0176] Infrastructure (I) attribute: use general unmanned platform task coordination link (ad hoc network);

[0177] Data (D) attribute: unable to collect through static verification;

[0178] ④Unmanned platform control terminal and unmanned vehicle B interoperation attribute collection:

[0179] Procedures (P) attribute: follow the special control station-based unmanned platform action coordination procedures;

[0180] Application (A) attribute: control station uses action coordination software that meets professional standards, and platform uses self-defined action control software;

[0181] Infrastructure (I) attribute: use special control station-unmanned platform action coordination link (point-to-multipoint);

[0182] Data (D) attribute: unable to collect through static verification;

[0183] (7) According to step 4(3), step 4(4), dynamic simulation test is carried out, and data is sent between the test resources and the test resources according to the assumption. The unmanned platform control terminal sends the load control data and the platform control data to the unmanned aerial vehicle A, the unmanned aerial vehicle B, the unmanned vehicle A and the unmanned vehicle B respectively. The unmanned aerial vehicle A, the unmanned aerial vehicle B, the unmanned vehicle A and the unmanned vehicle B send the load work product data, the load response data and the platform response data to the unmanned platform control terminal respectively, and store the data in the data acquisition management subsystem.

[0184] (8) According to step 5(1), by automatically checking the collected data and the standard format data, the dynamic verification collection result is obtained as

[0185] ① Unmanned platform control terminal and unmanned aerial vehicle A interoperation attribute collection:

[0186] Data (D) attribute: The unmanned aerial vehicle A adopts the unmanned system action collaborative platform control instruction format and content conforming to the professional field standard.

[0187] ② Unmanned platform control terminal and unmanned aerial vehicle B interoperation attribute collection:

[0188] The control station adopts the unmanned system action collaborative platform control instruction format and content conforming to the professional field standard.

[0189] ③ Unmanned platform control terminal and unmanned vehicle A interoperation attribute collection:

[0190] The unmanned vehicle A adopts the task collaborative instruction format and content conforming to the industry standard.

[0191] ④ Unmanned platform control terminal and unmanned vehicle B interoperation attribute collection:

[0192] The control station adopts the unmanned system task collaborative platform control instruction format and content conforming to the industry standard.

[0193] (9) According to step 5(2), step 5(3), the evaluation levels of the regulations (P), the application (A), the infrastructure (I) and the data (D) attributes are comprehensively analyzed, and the lowest one of the four attributes is selected as the interoperation level that can be reached between the multi-domain unmanned systems. Referring to the evaluation level reference table given in table 2, it can be obtained that:

[0194] ① The interoperation level of the unmanned platform control terminal and the unmanned aerial vehicle A is L09;

[0195] ② The interoperation level of the unmanned platform control terminal and the unmanned aerial vehicle B is L08;

[0196] ③ The interoperation level of the unmanned platform control terminal and the unmanned vehicle A is L07;

[0197] (4) The unmanned platform control terminal and the unmanned vehicle B have an L06 level of interoperation.

[0198] Finally, based on the multi-domain unmanned system interoperation evaluation environment, the unmanned platform control terminal and the multi-domain unmanned platform interoperation level evaluation task is completed.

[0199] The application provides a multi-domain unmanned system interoperation evaluation environment construction and use method, and there are many methods and approaches to realize the technical scheme, and the above description is only a preferred embodiment of the application, and it should be pointed out that, for ordinary technical personnel in the technical field, some improvements and refinements can be made without departing from the principle of the application, and these improvements and refinements should also be regarded as the protection scope of the application. The components not explicitly described in the embodiment can be realized by using the existing technology.

Claims

1. A method for constructing and using a multi-domain unmanned system interoperability assessment environment, characterized in that, Comprising the following steps: Step 1, clearly define the multi-domain unmanned system interoperability test evaluation task requirements, determine the test unmanned equipment and test resources; Step 2, build a multi-domain unmanned system interoperability test evaluation environment test system, including evaluation task management, evaluation index construction, data collection management, evaluation result generation 4 subsystems; Step 3, build a multi-domain unmanned system interoperability test evaluation environment test resource area, including LVC form, i.e. real equipment L, simulation V, simulation C test resources and multi-domain unmanned system test resources, unified access to the test environment; Step 4, based on the distributed service bus and the test support network to carry out interoperability evaluation, including static compliance test and dynamic simulation test two processes, collecting the procedure P, application A, infrastructure I, data D attribute evaluation data required for multi-domain unmanned system interoperability level evaluation; Step 5, based on the test system to analyze the collected data, from the procedure P, application A, infrastructure I, data D 4 dimensions to build a multi-level evaluation index system, and comprehensively obtain the multi-domain unmanned system interoperability level evaluation result; Step 5 includes: Step 5-1, analyze the procedure P, application A, infrastructure I, data D attribute data to obtain the test results of each attribute; wherein the procedure P, application A, infrastructure I are determined by comprehensive analysis, and the data D attribute is calculated by the test data compliance verification module, and the message verification process includes: Step 5-1-1, new message verification criterion model: Add verification criterion model: in the verification tool, import the standard message format, visually process the message format, generate the standard message structure and attribute value; Edit the standard verification rule of the message: according to the demand, edit the elements of the message that need to be verified, including whether it is a mandatory node, the maximum number of node occurrences and attribute value verification; Step 5-1-2, execute the message verification task: Establish message verification task: establish message verification task, import the standard verification criterion edited in step 5-1-1; Execute verification: read the message to be verified and match the standard verification criterion for verification; Step 5-1-3, analyze the message verification result: The message sent by the unmanned system and the standard message verification result are automatically checked and analyzed, and the evaluation level to which the data D attribute belongs is matched; Step 5-2, analyze the procedure P, application A, infrastructure I, data D attribute evaluation level, and determine the highest level that each attribute can reach according to the test satisfaction degree; Step 5-3, select the lowest one of the procedure P, application A, infrastructure I, data D four attributes, that is, the interoperability level that the multi-domain unmanned system can reach.

2. The method of claim 1, wherein, Step 2 includes: Step 2-1, design the evaluation task management subsystem, including the evaluation task planning module, the test environment design module, and the task state monitoring module; Step 2-2, design the evaluation index construction subsystem, including the index system template management module, the evaluation index establishment module, and the index weight distribution module; Step 2-3, design the data collection management subsystem, including the manual data collection module, the automatic data collection module, and the data management module; Step 2-4, design evaluation result generation subsystem, including data compliance verification module, index calculation comprehensive module, evaluation result display module.

3. The method of claim 2, wherein, In step 2-1, the evaluation task planning module has evaluation task establishment, evaluation task decomposition, and evaluation task management functions, and can edit test purposes, test subjects, and test content, and can also delete and view existing test tasks: The test environment design module supports selection based on evaluation tasks, and can select from existing test resource information in the database or add new test resources, and configure the connection relationship between test resources to guide the construction of the test environment; The task state monitoring module can display and monitor the development state, execution progress, and completion of each evaluation task or subtask.

4. The method of claim 3, wherein, In step 2-2, after the evaluation task is established, the corresponding evaluation index needs to be designed; the index system template management module provides functions of creating, opening, modifying, and deleting index system templates; The evaluation index establishment module provides a graphic operation interface, supports attribute supplementation based on the evaluation task to the index system template, and establishes a multi-layer tree-shaped evaluation index system in a visual manner, including the establishment and decomposition of attribute indexes; The index weight allocation module provides index weight calculation functions, and realizes the weight coefficient calculation of each index leaf node and subnode through subjective and objective weighting methods.

5. The method of claim 4, wherein, In step 2-3, after completing the evaluation index design, the information system, control station, and unmanned platform send information to each other, record the data transmission situation, and support test data collection, wherein the manual data collection module collects data according to the evaluation index system based on the attribute preset evaluation index template combined with the data transmission situation; The automatic data collection module automatically obtains the required evaluation data through packet capture during data transmission; The data management module supports filtering and storage management of data obtained by manual and automatic methods.

6. The method of claim 5, wherein, In step 2-4, after data collection is completed, the data is transmitted to the evaluation result generation subsystem for evaluation, the data compliance verification module verifies the compliance of the load work product data, load control and response data, platform control and response data transmitted between the test subjects and unmanned systems during the test, calculates the compliance degree with the relevant standards and specifications, and gives the data D attribute evaluation result; The index calculation comprehensive module supports the comprehensive calculation of user-manual-entered data and calculation data obtained based on compliance verification to obtain the evaluation results of the attributes of procedures P, applications A, infrastructure I, data D, and the overall interoperability level of unmanned systems; The evaluation result display module displays the evaluation results in the form of column chart, pie chart, and line chart.

7. The method of claim 6, wherein, Step 3 includes: Step 3-1, access LVC test resources based on the test local area network; Step 3-2, access multi-domain unmanned system test resources based on the test local area network, including unmanned aerial vehicles and unmanned vehicles.

8. The method of claim 7, wherein, Step 4 includes: Step 4-1, establishing an evaluation task and corresponding evaluation subtasks by using the evaluation task management system: using the evaluation task planning module, selecting the evaluation task type, and then automatically generating evaluation subtasks according to the evaluation task, the subtasks being to obtain and evaluate the specific attributes of the subject; Step 4-2, using the evaluation index construction system, constructing a multi-level evaluation index system from the aspects of regulations, applications, infrastructures, and data according to the evaluation task and the evaluation subtasks, and generating an evaluation data collection template; The regulation attribute refers to whether the unmanned system integration and collaborative operation complies with relevant standards, policies, regulations, operation rules, safety strategies, and operation regulations; The application attribute refers to the generalization degree of application software such as unmanned platform control software and task control software; The infrastructure attribute refers to the capability level of communication, computing and storage, security and privacy, and terminal equipment facilities supporting the interoperability of unmanned systems; The data attribute refers to the standardization level of the content and format of the business and monitoring data processed and exchanged by the unmanned system; Step 4-3, carrying out static compliance testing, verifying the regulations P, applications A, infrastructures I, and data D four attributes statically, selecting the corresponding attribute values by checking the data files of the unmanned system and through questionnaire survey, and inputting the results into the data collection management system; Step 4-4, carrying out dynamic simulation tests, the subject resources and the test resources sending data according to the scenario, including load work product data, load control instructions, load response data, platform control instructions, and platform response data, verifying whether the unmanned systems can receive data and respond as expected, further correcting and improving the results that cannot be obtained by static compliance testing and the results that are inconsistent with static compliance testing.

9. A storage medium, characterized by The computer program or instructions stored therein, when executed, implement the method of any one of claims 1 to 8.

Images