Matrix-based evaluation scene automatic generation system

Abstract

The invention discloses an evaluation scene automatic generation system based on a matrix, and the system comprises a test factor extraction module which is used for extracting elements which affect an evaluation result from different types of scene components, and enabling the elements to serve as test factors; the test factor system construction module is used for selecting required test factors to form a test factor system; the test factor level design module is used for setting the factor level value of each test factor in the test factor system, and sampling in a sample space formed by combining each test factor and the corresponding factor level value to obtain a test sample; when the number of test factors is larger than or equal to 6, an improved Latin hypercube test design method based on algorithm optimization is provided for sampling, and the test efficiency is improved; and the test parameter distribution module is used for mapping test factors in the test samples and element attributes in the evaluation scene, and distributing factor level values to the evaluation scene to obtain a complete evaluation scene for simulation deduction.

Description

Technical Field

[0001] The present invention relates to the field of intelligent simulation, and in particular to a matrix-based automatic generation system for evaluation scenarios. Background Art

[0002] In the generation test of intelligent task simulation scenarios, the generated test scenarios are used to input perception and decision-making algorithms, and the perception and decision-making algorithms generate test results to complete the testing of the perception and decision-making algorithms. In the existing evaluation scenario generation system, it is usually difficult to fully cover the evaluation of low-probability and multi-coupled events, which leads to the weakness of perception and decision-making algorithms when facing actual mission scenarios. In order to solve this problem, it is necessary to improve the full coverage of the test scenario library and increase the diversity and complexity of the generated scenarios to cover complex scenarios under conditions such as different ships and mission strategies. However, a high degree of scenario coverage and complexity will lead to a surge in the number of test scenarios, thereby reducing test efficiency. Summary of the invention

[0003] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art and provide a matrix-based automatic generation system for evaluation scenarios, which improves the test efficiency in the case of a surge in the number of test scenarios under a high degree of scenario coverage and complexity.

[0004] The technical solution of the present invention is to provide a matrix-based automatic generation system for evaluation scenarios, including:

[0005] The test factor extraction module is used to extract the factors that affect the evaluation results from the components of different types of scenarios as test factors;

[0006] The experimental factor system building module is used to select the required experimental factors and form an experimental factor system;

[0007] The experimental factor level design module is used to set the factor level value of each experimental factor in the experimental factor system; sampling is performed from the sample space formed by the combination of each experimental factor and the corresponding factor level value to obtain the experimental sample; wherein the sampling method includes an improved Latin hypercube experimental design method based on algorithm optimization, which removes redundant information from the sample space formed by the combination of each experimental factor and the corresponding factor level value, performs information sampling and information dimension reduction, and finds the optimal solution for the spatial distribution of sample points to obtain the optimized experimental sample;

[0008] The test parameter distribution module maps the test factors in the test samples and the element attributes in the evaluation scenario, distributes the factor level values ​​to the evaluation scenario, and obtains a complete evaluation scenario for simulation deduction.

[0009] Furthermore, when the number of experimental factors is ≥ 6, an improved Latin hypercube experimental design method based on algorithm optimization is used; the improved Latin hypercube experimental design method based on algorithm optimization specifically includes:

[0010] The experimental factors are grouped, and the number of experimental factors in each group is ≤ 6. Each group of experimental factors performs the following steps: an extended translation Latin hypercube design of the experimental factors of the group is generated, and an A×A Sudoku grid is generated according to the number A of the experimental factors of the group, and samples are taken from the value range of the factor level of each experimental factor of the group, and the sampled values ​​are filled into the Sudoku grid, and the Sudoku grid is sorted according to the value range of the experimental factors;

[0011] Merge the sorted Sudoku grids of each group to obtain a complete extended translation Latin hypercube test scenario matrix based on Sudoku grouping;

[0012] Eliminate redundant noise information in the extended translation Latin hypercube test scene matrix based on Sudoku grouping; perform information sampling and information dimension reduction on the test scene matrix after eliminating redundant information, continue to optimize the sample space, find the optimal solution for the spatial distribution of sample points, and obtain the optimized test samples.

[0013] Furthermore, the number of experimental factors is 50-100.

[0014] Furthermore, the variance analysis method is used to eliminate redundant noise information; one or more of the sensitive information sampling method, importance sampling and weighted method or mixed sampling method is used for information sampling; PCA principal component analysis method and / or factor analysis method is used for information dimension reduction; and the population optimization algorithm is used to find the optimal solution for the spatial distribution of sample points.

[0015] Furthermore, the sampling methods also include random experiment design method, boundary experiment design method, and Latin hypercube experiment design method; the user can select any one or more experiment design methods to sample the sample space according to needs.

[0016] Furthermore, in the experimental factor system construction module, the experimental factor system construction includes two modes: automatic generation and manual entry, which are operated through the human-computer interaction interface; automatic generation mode: the experimental factors extracted by the experimental factor extraction module are classified according to the types of components of different scenarios to form an experimental factor system; manual entry mode: import pre-established evaluation indicators, select experimental factors associated with the indicator items, and form an experimental factor system; in both modes, each experimental factor can be edited and deleted.

[0017] Furthermore, in the experimental factor level design module, the factor level setting includes two modes: automatic generation and manual entry, which are operated through the human-computer interaction interface; automatic generation mode: first select the factor level type, and automatically generate the corresponding factor level values ​​according to different factor level types; manual entry mode: for each experimental factor, enter the factor level values ​​one by one; in both modes, the number of factor levels can be increased and deleted.

[0018] Furthermore, in the experimental factor level design module, the factor level types include two-terminal type, discrete type and continuous type; for the two-terminal type experimental factors, the value-taking method is to set the minimum and maximum values ​​of the factors, and use the minimum and maximum values ​​as the values ​​of the factor levels; for the discrete type experimental factors, the value-taking method is to set the discrete value of the factors, and use each discrete value as the value of the factor level; for the continuous type experimental factors, the value-taking method is to set the value interval of the factors, discretize them according to uniform intervals or variable intervals within the value interval, and use the discretized values ​​as the values ​​of the factor levels.

[0019] Furthermore, the types of scene constituent elements include red side elements, blue side elements, and environmental elements; the red side elements include the type, location, and quantity of the red side's equipment; the blue side elements include the type, location, and quantity of the blue side's equipment; and the environmental elements include the geographical environment and weather environment.

[0020] Furthermore, the automatic generation system of evaluation scenarios is divided into resource layer, resource access layer, functional logic layer and human-computer interaction layer; the experimental factor extraction module, experimental factor system construction module, experimental factor level design module and experimental parameter distribution module all belong to the functional logic layer to realize specific functional logic; the human-computer interaction layer provides an operation interface for users and transmits user operation results to the functional logic layer; when resource operations are needed, the functional logic layer reads, adds, deletes or modifies resource files stored in the resource layer through the access interface of the resource access layer.

[0021] The advantages of the present invention compared with the prior art are:

[0022] (1) Traditional intelligent algorithm performance evaluation decouples the system scenario of intelligent algorithm application into a single fixed scenario for simulation. This single scenario replacement method cannot fully cover the system elements and does not conform to the actual scenario, resulting in a large deviation in the performance evaluation results. The present invention constructs diversified test evaluation elements for multi-task scenarios and determines a matrix evaluation test design scheme for conducting test evaluations of different test factors and levels related to the overall performance of intelligent algorithms.

[0023] (2) Traditional simulation evaluation tests intelligent algorithms mainly by repeating multiple rounds of simulation deductions in fixed scenarios, which cannot refine the definition and simulate the various uncertainties faced in real scenarios, and the situations where uncertainties interact with each other. The matrix evaluation scenario automatic generation method proposed in the present invention adopts the improved ETPLHD method. Compared with the ETPLHD method, on the one hand, the number of sample points is smaller and the calculation time is shorter, which significantly reduces the time required for simulation experiments; on the other hand, the sample space distribution is more uniform, closer to the optimal Latin hypercube design, making the simulation test results more representative; the spatial distribution uniformity is used to reduce the correlation between factors, effectively solving the confounding problem caused by factor grouping. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the system composition of the present invention.

[0025] Figure 2 The present invention is a logical schematic diagram of the main steps of the improved Latin hypercube experimental design method based on algorithm optimization.

[0026] Figure 3 This is an example diagram of 20 experimental factors of the present invention. DETAILED DESCRIPTION

[0027] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0028] The automatic generation system of evaluation scenarios provided in this embodiment adopts a hierarchical construction concept, such as Figure 1 As shown, the system is divided into four parts: human-computer interaction layer, functional logic layer, resource access layer and resource layer. The functional logic layer is the core layer that reflects the innovation of this system.

[0029] The human-computer interaction layer provides users with a trial operation interface, through which users can perform file import and export operations, file editing operations, factor selection operations, and factor setting operations.

[0030] The functional logic layer is responsible for implementing all functional logics of the experimental design, including the experimental factor extraction module, the experimental factor system construction module, the experimental factor level design module, and the experimental parameter distribution module. The operations performed by the user through the human-computer interaction layer are implemented by the functional logic layer. In the case of resource access, the functional logic layer will obtain the relevant resources of the experimental design through the resource access layer. The specific functions of each module are:

[0031] The test factor extraction module is used to extract the factors that affect the evaluation results from the components of different types of scenarios as test factors;

[0032] The experimental factor system building module is used to select the required experimental factors and form an experimental factor system;

[0033] The experimental factor level design module is used to set the factor level value of each experimental factor in the experimental factor system; and then obtain the experimental sample after sampling from the sample space formed by the combination of each experimental factor and the corresponding factor level value;

[0034] The test parameter distribution module maps the test factors in the test samples to the corresponding element attributes in the evaluation scenario, distributes the factor level values ​​to the evaluation scenario, and obtains a complete evaluation scenario for simulation deduction.

[0035] The resource access layer provides resource access interfaces for the functional logic layer, including experimental design method access interfaces and database access interfaces. Through the resource access interfaces, the functional logic layer can obtain, add, delete or update resources in the resource layer.

[0036] The resource layer includes various resources related to experimental design, mainly including various files such as scenario samples, factor level samples, experimental design method library and platform resource library. Among them, the experimental design method library includes random design method, boundary design method, Latin hypercube design method and improved Latin hypercube method based on algorithm optimization. When connected to the platform resource library, the resource access layer can obtain relevant data from the platform resource library through the database access interface to carry out experimental design.

[0037] (1) Experimental factor extraction module

[0038] In order to solve the problems of scenario diversity and scenario library coverage, ontology is used to structurally describe the components of different types of scenarios to extract experimental factors.

[0039] The types of scene constituent elements mainly include red team elements, blue team elements, and environmental elements; among them, red team elements include the red team’s equipment type, deployment location, and quantity; blue team elements include the blue team’s equipment type, deployment location, and quantity; environmental elements include geographical environment and weather environment.

[0040] Specifically, in the aircraft system test mission, the equipment types mainly include various test entities, such as measurement and control equipment, support equipment; various scenario mission entities, such as task formation command nodes, aircraft systems, mission units, support equipment, damage targets, early warning and detection, interference, etc.

[0041] The function of the test factor extraction module is to extract the factors that affect the evaluation results from the components of different types of scenarios as test factors, such as the location of equipment deployment, the number of aircraft carried in the aircraft system, etc.

[0042] (2) Experimental factor system construction module

[0043] In order to solve the complexity problem of multi-coupling scenarios, a human-computer interaction interface is provided, and experimental factors are screened and combined to form an experimental factor system in combination with experimental tasks.

[0044] The construction of the experimental factor system includes two modes: automatic generation and manual entry, and is operated through a human-computer interaction interface. During automatic generation, the experimental factors extracted by the experimental factor extraction module are classified according to the types of elements of different scenarios (red side, blue side, environment, etc.) to form an experimental factor system, and the experimental factor system is displayed in a human-computer interaction interface. During manual entry, the intelligent algorithm evaluation index system can be loaded, and according to the experimental task, the experimental evaluation index combination is selected based on prior experience knowledge, and the associated experimental factors are selected for the index items to form an experimental factor system.

[0045] In both modes, the types and data of experimental factors can be defined, and factors can be edited, deleted, and other operations can be supported; the designed experimental factor set can be exported and saved.

[0046] (3) Experimental factor level design module

[0047] In order to solve the problems of data sampling method selection and test efficiency, a certain number of factor levels of the test factors in the test factor system are reasonably designed by manual setting or automatic generation. After all the test factor levels are discretized, a sample space is formed, and a variety of test design methods are loaded for test design. According to the required number of test samples, the sample space is sampled reasonably.

[0048] For factor level settings, two modes are provided: automatic generation and manual entry, which are operated through the human-computer interaction interface. In automatic generation, first select the factor level type, and automatically generate the corresponding factor level values ​​according to different factor level types; in manual entry, for each experimental factor, enter the factor level values ​​one by one. In both modes, the number of factor levels can be added or deleted.

[0049] Specifically, the factor level types include two-terminal, discrete and continuous types; for two-terminal experimental factors, the value-taking method is to set the minimum and maximum values ​​of the factor, and use the minimum and maximum values ​​as the values ​​of the factor level; for discrete experimental factors, the value-taking method is to set the discrete value of the factor, and use each discrete value as the value of the factor level; for continuous experimental factors, the value-taking method is to set the value interval of the factor, discretize it according to uniform intervals or variable intervals within the value interval, and use the discretized value as the value of the factor level.

[0050] For the experimental design, the sampling of the sample space is realized by calling a variety of experimental design methods, and the experimental design methods at least include a randomized experimental design method, a boundary experimental design method, a Latin hypercube design method, and an improved Latin hypercube experimental design method based on algorithm optimization. Among them, the randomized experimental design method and the boundary experimental design method adopt the existing technical methods, which will not be repeated in this embodiment. The Latin hypercube design method is also a prior art method. In order to better understand the improved Latin hypercube experimental design method based on algorithm optimization proposed by the present invention, the Latin hypercube design method is briefly described here.

[0051] The design idea of ​​the Latin hypercube design method is to transform a v factors, each with n p An experimental design with n levels is denoted as p ×n v matrix Each of these lines is a set of test points. The Latin hypercube test design randomly selects n in the test space. p There are n experimental points, and each row and column of the design matrix has exactly one level. This method requires a total of n p In one test space, there are Different Latin hypercube designs.

[0052] Using Latin hypercube design can ensure that samples are evenly distributed in each dimension, which means that it can better cover the parameter space. However, there can only be one sample in each row and column of any two-dimensional subspace, and the test space filling uniformity is poor. In addition, for multi-sample experiments, there is the problem of multiple factors changing at the same time. From the perspective of result analysis, it may be difficult to distinguish their respective effects. This phenomenon is called confounding. Therefore, orthogonality is another optimization goal of Latin hypercube design. The lower the pairwise correlation between factors, the better the orthogonality.

[0053] In order to optimize the orthogonality and uniformity of the Latin hypercube design, the Translational Propagation Latin Hypercube Design (TPLHD) was developed to optimize the space filling uniformity of the large number of sample experimental designs. However, there is still a problem that the number of experiments is equal to the number of levels, and n can only be generated according to the number of factor levels. p This results in that when the number of factors is large, the test points are very sparse in the entire sample space. To address this problem, the Expand Translational Propagation Latin Hypercube Design (ETPLHD) was developed based on the TPLHD method.

[0054] The application scenarios of the present invention face the problem of diversity and complexity of intelligent task simulation scenarios. The number of experimental factors is much larger than that of general scenarios. The TPLHD method performs poorly when the factor dimension (i.e., the number of experimental factors) is higher than 6, because the number of levels needs to be greatly expanded, and the subsequent level deletion may affect the uniform distribution of the sample space and increase the time cost. The ETPLHD method is based on TPLHD and is not suitable for experimental design problems with dimensions higher than 6.

[0055] Therefore, the present invention proposes an improved Latin hypercube experimental design method based on algorithm optimization, which is suitable for experimental design with the number of experimental factors greater than 6, especially for experimental factors with the number of 50 to 100. From the sample space formed by the combination of each experimental factor and the corresponding factor level value, redundant information is removed, information sampling and information dimension reduction are performed, and the optimal solution for the spatial distribution of sample points is found to obtain a highly uniform and highly correlated experimental sample. The specific steps are as follows:

[0056] 1) The experimental factors are divided into groups, and the number of experimental factors in each group is ≤ 6.

[0057] 2) Perform the following steps for each group of experimental factors:

[0058] 2.1) Taking the experimental factors and corresponding factor levels of the group as input, an extended translation Latin hypercube design is generated according to the number of experimental factors to obtain the Latin hypercube structure of the group of factors.

[0059] 2.2) Take the square of the number of test factors in this group and generate a Sudoku grid of this square number.

[0060] 2.3) Sampling is performed from the factor level value range of each experimental factor in the group, the sampled values ​​are filled into the Sudoku grid, and the Sudoku grid is sorted according to the experimental factor value range, that is, the results of each group in each row and column are serially combined and rearranged in ascending order according to the factor value range.

[0061] 3) Merge the sorted Sudoku grids in each group to obtain a complete extended translation Latin hypercube test scenario matrix based on Sudoku grouping. Specifically, arrange the sampling results corresponding to the selected grid rows / columns in parallel to obtain multiple groups of Sudoku grid value strings after removing the values ​​greater than the total number of factors. Each group of value strings corresponds to a factor number combination to generate a complete extended translation Latin hypercube test scenario matrix based on Sudoku grouping.

[0062] 4) Optimize the extended translation Latin hypercube test scenario matrix based on Sudoku grouping, see Figure 2 .

[0063] 4.1) Using variance analysis, the redundant noise information in the extended translation Latin hypercube test scene matrix based on Sudoku grouping is eliminated to obtain the test scene matrix after eliminating redundancy.

[0064] 4.2) After removing redundant information, the scenario matrix is ​​sampled and reduced in dimension to obtain the experimental scenario matrix after information sampling and dimension reduction, so as to preliminarily complete the design of mixed scenarios with multi-dimensional factors and complex factor level distribution, and preliminarily achieve the spatial dimension reduction of the scenario to improve the temporal executability of the scenario. Specifically, the sensitivity information sampling method, importance sampling and weighting method, mixed sampling method, etc. can be used to complete the information sampling; and the PCA principal component analysis method and factor analysis method can be used to complete the information dimension reduction.

[0065] 4.3) After the information sampling and dimensionality reduction of the test scene matrix, further optimize the sample space to find the optimal solution for the spatial distribution of sample points and obtain the optimized test samples.

[0066] Specifically, the population optimization algorithm is used to find the optimal solution for the spatial distribution of sample points, which mainly includes the following steps:

[0067] ① Encode chromosomes at the factor level, initialize the population pop, and set algorithm parameters

[0068] Input: When constituting a chromosome, it is stipulated that only one chromosome segment gene bit corresponding to each factor is 1, and the rest are 0. i They are linked together to form chromosomes.

[0069] Output: Each Li segment chromosome corresponds to x i Factors, for example, x i The corresponding chromosome segment has 5 expressions and

[0070] ② Use microecological technology to measure the similarity of individuals in the population.

[0071] Input: Calculate the shared relationship share_d(i,j) between pop(i) and pop(j), where d(i,j) is the Euclidean distance between individuals and r is the small ecological radius.

[0072]

[0073] Output: Make the similarity of individuals searched by the algorithm in the population as small as possible, and the projection characteristics as large as possible. The fitness function of individual i in the population is selected as:

[0074] objvalue(i)=-f cd (i)d min (i)

[0075] ③Iteratively update the population to find the optimal solution and the corresponding optimal test factor combination

[0076] Input: Update the population through adaptive strategy crossover operation, iteratively calculate the fitness function value, and the individual with the largest fitness value in the population is the optimal solution.

[0077] Output: Store the optimal solution and its fitness value, sort the fitness values ​​in descending order, select a specified number of non-repetitive optimal test factor combination matrices, prune the scene through the population optimization algorithm, and obtain the optimized test samples.

[0078] Taking 20 experimental factors as an example, the generated samples are as follows: Figure 3 shown.

[0079] It is to be understood that the present invention is described by way of embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and embodiments that can fall within the scope of the claims of this application all fall within the scope protected by the present invention.

[0080] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A matrix-based automatic evaluation scenario generation system, characterized in that: include: The test factor extraction module is used to extract the factors that affect the evaluation results from the components of different types of scenarios as test factors; The experimental factor system building module is used to select the required experimental factors and form an experimental factor system; The experimental factor level design module is used to set the factor level value of each experimental factor in the experimental factor system; sampling is performed from the sample space formed by the combination of each experimental factor and the corresponding factor level value to obtain the experimental sample; wherein the sampling method includes an improved Latin hypercube experimental design method based on algorithm optimization, which removes redundant information from the sample space formed by the combination of each experimental factor and the corresponding factor level value, performs information sampling and information dimension reduction, and finds the optimal solution for the spatial distribution of sample points to obtain the optimized experimental sample; The test parameter distribution module maps the test factors in the test samples and the element attributes in the evaluation scenario, distributes the factor level values ​​to the evaluation scenario, and obtains a complete evaluation scenario for simulation deduction.

2. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: When the number of experimental factors is ≥ 6, an improved Latin hypercube experimental design method based on algorithm optimization is used; the improved Latin hypercube experimental design method based on algorithm optimization specifically includes: The experimental factors are grouped, and the number of experimental factors in each group is ≤ 6. Each group of experimental factors performs the following steps: an extended translation Latin hypercube design of the experimental factors of the group is generated, and an A×A Sudoku grid is generated according to the number A of the experimental factors of the group, and samples are taken from the value range of the factor level of each experimental factor of the group, and the sampled values ​​are filled into the Sudoku grid, and the Sudoku grid is sorted according to the value range of the experimental factors; Merge the sorted Sudoku grids of each group to obtain a complete extended translation Latin hypercube test scenario matrix based on Sudoku grouping; Eliminate redundant noise information in the extended translation Latin hypercube test scene matrix based on Sudoku grouping; perform information sampling and information dimension reduction on the test scene matrix after eliminating redundant information, continue to optimize the sample space, find the optimal solution for the spatial distribution of sample points, and obtain the optimized test samples.

3. The matrix-based automatic evaluation scenario generation system according to claim 2 is characterized in that: The number of experimental factors ranges from 50 to 100.

4. The matrix-based automatic evaluation scenario generation system according to claim 2, characterized in that: Use variance analysis to eliminate redundant noise information; use one or more of the sensitive information sampling method, importance sampling and weighted method, or mixed sampling method to sample information; The PCA principal component analysis method and / or factor analysis method are used to reduce the dimension of information; the population optimization algorithm is used to find the optimal solution for the spatial distribution of sample points.

5. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: Sampling methods also include random experiment design method, boundary experiment design method, and Latin hypercube experiment design method; users can select any one or more experiment design methods to sample the sample space according to their needs.

6. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: In the test factor system construction module, the test factor system construction includes two modes: automatic generation and manual entry, which are operated through the human-computer interaction interface; automatic generation mode: the test factors extracted by the test factor extraction module are classified according to the types of elements of different scenarios to form a test factor system; Manual entry mode: import pre-established evaluation indicators, select experimental factors associated with the indicator items, and form an experimental factor system; in both modes, each experimental factor can be edited and deleted.

7. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: In the experimental factor level design module, the factor level setting includes two modes: automatic generation and manual entry, which are operated through the human-computer interaction interface; automatic generation mode: first select the factor level type, and automatically generate the corresponding factor level values ​​according to different factor level types; manual entry mode: for each experimental factor, enter the factor level values ​​one by one; in both modes, the number of factor levels can be increased and deleted.

8. The matrix-based automatic evaluation scenario generation system according to claim 7, characterized in that: In the experimental factor level design module, the factor level types include two-terminal type, discrete type and continuous type; for the two-terminal type experimental factors, the value-taking method is to set the minimum and maximum values ​​of the factors, and use the minimum and maximum values ​​as the values ​​of the factor levels; for the discrete type experimental factors, the value-taking method is to set the discrete value of the factors, and use each discrete value as the value of the factor level; for the continuous type experimental factors, the value-taking method is to set the value interval of the factors, discretize them according to uniform intervals or variable intervals within the value interval, and use the discretized values ​​as the values ​​of the factor levels.

9. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: The types of scene constituent elements include red side elements, blue side elements, and environmental elements; red side elements include the type, location, and quantity of the red side’s equipment; blue side elements include the type, location, and quantity of the blue side’s equipment; environmental elements include the geographical environment and weather environment.

10. The matrix-based automatic evaluation scenario generation system according to claim 1, characterized in that: The automatic generation system of evaluation scenarios is divided into resource layer, resource access layer, functional logic layer and human-computer interaction layer; the experimental factor extraction module, experimental factor system construction module, experimental factor level design module and experimental parameter distribution module all belong to the functional logic layer to realize specific functional logic; the human-computer interaction layer provides an operation interface for users and transmits user operation results to the functional logic layer; when resource operations are needed, the functional logic layer reads, adds, deletes or modifies resource files stored in the resource layer through the access interface of the resource access layer.

Images