A satellite simulation system performance evaluation method and system

Abstract

The present application provides a method and system for evaluating the performance of a satellite simulation system, which solves the technical problem of the complex indicator configuration process when the existing performance evaluation method performs performance evaluation on different evaluation objects. It includes the following steps: indicator configuration: configuring the evaluation indicators of each level, the affiliation of each indicator item, the indicator quantification method and the indicator normalization method of the lowest level indicator item according to the evaluation task; performance evaluation: assigning indicator weights to the evaluation indicators of each level, reading the test data of the satellite simulation system at that time, and generating performance evaluation values. The present application can be widely used in the field of spacecraft simulation evaluation technology.

Description

Technical Field

[0001] The present application relates to spacecraft performance evaluation, and more specifically, to a method and system for evaluating the performance of a satellite simulation system. Background Art

[0002] With the rapid development of computer simulation technology and satellite systems, satellite simulation systems and performance evaluation have gradually become an indispensable part of the design of new satellites and the process of mission decision-making. The efficiency and reusability of performance evaluation schemes and evaluation system development directly affect the progress of decision-making. Existing performance evaluation systems and evaluation index systems are often bound to specific model requirements and simulation platform interfaces, resulting in poor versatility and reusability of performance evaluation models, and repeated modeling and development are required for different model tasks.

[0003] Due to different simulation systems, different simulation task scenarios, and different objects to be evaluated, the indicator configuration required for performance evaluation is different. The indicator configuration includes the evaluation indicator system, the quantitative model of the underlying indicator items, and the normalization algorithm used. At present, when building the performance evaluation system, the indicator configuration of different evaluation objects is realized by modifying the software code, which is complex, inefficient, and requires a lot of manpower. Summary of the invention

[0004] To solve the above problems, the technical solution adopted in this application is: to provide a method for evaluating the effectiveness of a satellite simulation system, comprising the following steps:

[0005] Indicator configuration: configure the evaluation indicators at each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicator according to the evaluation task;

[0006] Performance evaluation: Assign indicator weights to each level of evaluation indicators, read the current satellite simulation system test data, and generate performance evaluation values.

[0007] Preferably, configuring the indicator normalization method refers to obtaining configuration parameters of the normalization calculation formula of the indicator items of the evaluation task, and instantiating the normalization formula according to the normalization formula and constant parameters.

[0008] Preferably, the indicator quantification method includes an indicator value quantification formula and required parameters, and the required parameters are expressed as formal parameters. When performing an evaluation task, specific actual parameters are configured for the indicator quantification model.

[0009] Preferably, the indicator configuration is implemented through an indicator system, which includes indicator items, a hierarchy of indicator items, and weights of indicators at each level;

[0010] The indicator items include the indicator item name, the position relationship of the indicator items, the indicator item calculation data source, the indicator quantification formula of the bottom-level indicator items and the indicator item normalization model.

[0011] Preferably, the indicator configuration specifically includes the following steps:

[0012] Parse the configuration data of the evaluation indicator system and store the configuration parameters of the indicator items in the database table;

[0013] According to the corresponding field data, the hierarchy and affiliation of the indicator system, the quantitative formula and normalization formula of the indicator value of the bottom-level indicator items are constructed;

[0014] At the same time, the indicator quantification formula is bound to the address of the database where the satellite simulation system test data source is located and the information in the corresponding database table.

[0015] Preferably, in the performance evaluation, indicator weights are assigned to evaluation indicator items at each level, by generating a judgment matrix of indicator items at each level with respect to the subordinate indicator items, filling in the judgment matrix according to the 9-level scaling method, and calculating the weight values ​​of indicator items at each level.

[0016] Preferably, in the performance evaluation, generating the performance evaluation value means generating the normalized utility value of the lowest-level indicator item based on the test data of the satellite simulation system, combining the indicator weights of each level and using the weighted sum method to obtain the comprehensive performance evaluation value of the satellite simulation task at that time.

[0017] The present invention also provides a satellite simulation system performance evaluation system, comprising:

[0018] Indicator configuration module: configures the evaluation indicators at each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicators according to the evaluation task;

[0019] Performance evaluation module: assigns indicator weights to evaluation indicators at each level, reads the test data of the satellite simulation system, and generates performance evaluation values.

[0020] The beneficial effects of the present invention are as follows: according to the performance evaluation requirements of the evaluation task, the evaluation index items at each level, the affiliation of each index item, the index quantification method and the index normalization method of the bottom-level index item can be quickly configured; different satellite simulation systems can be connected according to the way of configuring the evaluation object data in the index item; and according to the weights of the evaluation indicators at each level, the evaluation effect of the satellite simulation system can be obtained more quickly; the processing is simple, and the efficiency of index system construction and performance evaluation is improved, which can provide a decision-making basis for the optimization of satellite simulation solutions including spacecraft design, spacecraft mission decision-making, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 A schematic diagram of a configuration flow of an efficiency evaluation index system for a satellite simulation system provided in one embodiment of the present application;

[0023] Figure 2 A schematic diagram of the connection relationship of the indicator system configuration of the indicator items provided in one embodiment of the present application;

[0024] Figure 3 A performance evaluation index system obtained by configuring according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] See also Figure 1 , is a schematic diagram of a configuration flow of an efficiency evaluation index system of a satellite simulation system provided by an embodiment of the present application. For ease of explanation, only the part related to the present embodiment is shown, which is described in detail as follows:

[0027] The present application provides a satellite simulation system performance evaluation method, which is used to directly serve the performance evaluation of different spacecraft simulation systems. It includes the following steps:

[0028] S101, indicator configuration: configure the evaluation indicators of each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicator according to the evaluation task, that is, complete the initialization of the performance evaluation;

[0029] Specifically, the configuration index item normalization method refers to obtaining the configuration parameters of the index item normalization calculation formula, instantiating the normalization formula according to the normalization formula and constant parameters, and dimensionlessly processing the attribute values ​​of the index items in different units so that different index items can be evaluated under the same standard.

[0030] Among them, the indicator quantification method includes the indicator value quantification formula and the required parameters. The required parameters are expressed as formal parameters. When performing an evaluation on a system, specific actual parameters need to be configured for the indicator quantification model.

[0031] In one embodiment, the indicator configuration is implemented through an indicator system, see Figure 2 The indicator system includes indicator items, indicator item hierarchy, and indicator weights at each level; among them, indicator items include indicator item names, indicator item position relationships, indicator item calculation data sources, indicator quantification formulas of the lowest level indicator items, and indicator item normalization models.

[0032] Abstraction is performed based on the evaluation task and object-oriented thinking, and the abstraction is into an indicator template object. The indicator template can be instantiated into the required indicator item by configuring different parameters.

[0033] The indicator item template is designed as an 11-tuple:

[0034] {id,name,type,level,parent_id,is_parent,parameter_f,formula,normalize,normalize_d,parameter_n}

[0035] The indicator item template definition is shown in the following table:

[0036]

[0037]

[0038] The indicator item template has the following characteristics:

[0039] (1) By modifying the level and parent_id of an indicator and linking it to other parent indicators, the position of the indicator item can be flexibly adjusted, and the sub-indicator set of the indicator will change accordingly;

[0040] (2) For the parameter_f field, its subtable contains three attributes: data table, data field, and identifier. The data table indicates the name of the database data table where the data is stored, and the data field indicates the field name where the parameter is stored in the data table. English letters are used as identifiers. In addition, to simplify the data parameter selection operation, a data source containing parameters such as database address, user name, and password can be configured in advance, and the data source name is used as an index to read the database data pointed to;

[0041] (3) For the normalization algorithm, its function is to make the attribute values ​​of indicator items of different units dimensionless so that different indicator items can be evaluated under the same standard;

[0042] (3.1) Different types of normalization algorithms are solidified using code, and the parameters required by the normalization algorithm are encapsulated as parameter interfaces. When a normalization algorithm is selected, the parameters required by the normalization algorithm are assigned values ​​according to the value of the parameter_n field;

[0043] (3.2) When adding the lowest-level indicator item, the configuration parameters on the visualization page are parsed and the normalization algorithm is initialized, and the normalization algorithm callback function is registered to normalize. When evaluating, the normalization of the indicator becomes the process of calling the normalization algorithm callback function, thus realizing the configurability of the normalization algorithm.

[0044] Based on the indicator system, the specific implementation process of indicator configuration is as follows:

[0045] S1011, parsing the configuration data of the evaluation index system, and storing the configuration parameters of the index items in a database table, where the fields in the database table are consistent with the fields in the index item template;

[0046] S1012. According to the three field data {level, parent_id}, construct the level and affiliation relationship of the indicator system - parent_id;

[0047] S1013. According to the three fields {is_parent, parameter_f, formula}, find the bottom-level indicator, construct the indicator value quantification formula - formula of the bottom-level indicator item, and bind the indicator value quantification formula with the test data - parameter_f generated by the satellite simulation system, that is, bind the address of the database where the satellite simulation system test data source is located and the information in the corresponding database table, and complete the binding of the calculation data source;

[0048] S1014. According to the three fields {normalize, normalize_d, parameter_n}, construct a normalization algorithm formula of the indicator item - normalize_d, wherein the constant in the normalization algorithm formula is specified by parameter_n;

[0049] At this point, the indicator system is configured and the performance evaluation initialization is completed.

[0050] S102, performance evaluation: assign indicator weights to evaluation indicators at each level, and after the simulation test is completed, read the satellite simulation system test data of that time to generate the performance evaluation value. The indicator weights at each level are obtained using the hierarchical analysis method.

[0051] In one embodiment, the weight of indicators at each level is obtained by generating a judgment matrix of indicator items at each level with respect to the indicator items belonging to each level. Experts or evaluators fill in the judgment matrix according to the 9-level scaling method and calculate the weight values ​​of indicator items at each level.

[0052] In one embodiment, the comprehensive performance evaluation is implemented by generating normalized utility values ​​of the lowest-level indicators based on the recorded simulation test data, combining the weights of the indicators at each level and using the weighted sum method to obtain the comprehensive performance evaluation value of the satellite simulation mission at that time.

[0053] The present invention can quickly construct an evaluation index system according to the performance evaluation requirements, and can access different satellite simulation systems according to the way of configuring the evaluation object data in the index items, thereby improving the efficiency of index system construction and performance evaluation, and obtaining the evaluation effect of the satellite simulation system more quickly, and can provide a decision basis for the optimization of satellite simulation solutions including spacecraft design, spacecraft mission decision-making, etc. The code of the index weight distribution and the hierarchical conduction relationship can be solidified, and combined with the index system, the modular design and development of the performance evaluation system can be realized.

[0054] A second aspect of the present application provides a satellite simulation system performance evaluation system, including an indicator configuration module and a performance evaluation module.

[0055] Indicator configuration module: configures the evaluation indicators at each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicators according to the evaluation task;

[0056] Performance evaluation module: assigns indicator weights to evaluation indicators at each level, reads the test data of the satellite simulation system, and generates performance evaluation values.

[0057] It should be noted that the performance evaluation system of a satellite simulation system in this embodiment is an embodiment of the system corresponding to the performance evaluation method of the above-mentioned satellite simulation system. Therefore, the specific implementation of the software methods in each module of the performance evaluation system of the satellite simulation system will not be described in detail here.

[0058] Specific implementation example: Conducting performance evaluation for a remote sensing satellite simulation system reconnaissance mission.

[0059] See also Figure 3 , the specific implementation process of indicator configuration is as follows:

[0060] Step 1: Add the first-layer indicators as shown in the following table, parse its configuration parameters, and get the indicator name as reconnaissance mission execution capability, the indicator type as benefit-based indicator, the indicator level as the first layer, and contain sub-indicators. Since it contains sub-indicators, the fields related to the quantization and normalization algorithms of the indicator items are not displayed;

[0061] Field Name Field data name Reconnaissance mission execution capability type 1 level 1 parent_id / is_parent 0

[0062] Step 2: Add the second-level indicators shown in the following table, parse their visualization configuration parameters, and get the indicator name as imaging time satisfaction rate, indicator type as benefit-based indicator, indicator level as the second level, and contain sub-indicators. Because of the sub-indicators, the indicator items related to quantization and normalization algorithms are not displayed on the front end of the page;

[0063]

[0064] Step 3: Add and analyze target recognition rate steps similar to step 2;

[0065] Step 4: Add the third-level indicators shown in the following table, parse their visualization configuration parameters, and the indicator name obtained by parsing is task response time, the indicator type is cost-based indicator, the indicator level is the third level, and it does not contain sub-indicators.

[0066]

[0067] Since there are no sub-indicators, the following parameters related to the indicator quantification and normalization algorithm are further read as follows:

[0068] 1) Parse "{task start time - a, task end time - b}, @b - @a". The quantification of the task response time indicator requires two parameters: task start time and task end time, and the quantification formula is "task end time - task start time";

[0069] 2) Further analysis of the "Min-Max normalization algorithm, {1.19, 96.8}", according to the configuration, obtain the Min-Max normalization algorithm formula According to {1.19, 96.8}, we know that x min =1.19 and x max =96.8, that is, for a certain target, the maximum response time of the satellite reconnaissance mission is 96.8 (hours), and the minimum response time is 1.19 (hours);

[0070] 3) This has achieved The configuration of this indicator item is completed.

[0071] Step 5: Refer to the task response time indicator item, analyze the information transmission delay, spatial resolution, coverage probability, detection probability and other indicators, and complete the configuration of the indicator system.

[0072] The weights of indicators at each level are obtained by combining the analytic hierarchy process. After completing the simulation task, the normalized utility value of the third-level indicator items is calculated according to the recorded simulation test data. Then, the utility value of the reconnaissance task execution capability is obtained by combining the weights of indicators at each level.

[0073] The above configuration of the performance evaluation index system for a remote sensing satellite simulation system to perform target reconnaissance tasks is completed. After the satellite simulation system completes the simulation task, the simulation test data in the database can be read to calculate the normalized value of the third-level indicator items. Combined with the weights of indicators at each level, the performance value of the task execution capability indicator can be obtained.

[0074] The present invention provides a method and system for evaluating the effectiveness of a satellite simulation system:

[0075] 1) Indicator configuration is achieved through the indicator system, which can configure evaluation indicators under different task requirements;

[0076] 2) By binding the index value quantification formula with the address of the database where the test data source generated by the satellite simulation system is located and the information in the corresponding database, the calculation data source is bound, and the index system can be connected to different satellite simulation systems;

[0077] 3) Through the visual indicator configuration method of the indicator system, there is no need to repeatedly develop indicator modeling code, and the construction of the evaluation indicator system can be completed with reliable quality and reusability.

[0078] 4) This performance evaluation index system configuration method is not limited to satellite simulation systems, but is also applicable to the test performance evaluation of similar aerospace / aircraft equipment.

[0079] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0080] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for evaluating the effectiveness of a satellite simulation system, characterized in that: The following steps are involved: Indicator configuration: configure the evaluation indicators at each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicator according to the evaluation task; Performance evaluation: assign indicator weights to each level of evaluation indicators, read the current satellite simulation system test data, and generate performance evaluation values; Configuring the indicator normalization method refers to obtaining configuration parameters of the normalization calculation formula of the indicator item of the evaluation task, and instantiating the normalization formula according to the normalization formula and the constant parameters; In the performance evaluation, generating the performance evaluation value means generating the normalized utility value of the lowest level indicator item according to the current satellite simulation system test data and the instantiated normalization formula, combining the indicator weights of each level and using the weighted sum method to obtain the comprehensive performance evaluation value of the current satellite simulation task; The indicator configuration specifically includes the following steps: Parse the configuration data of the evaluation indicator system and store the configuration parameters of the indicator items in the database table; According to the corresponding field data, the hierarchy and affiliation of the indicator system, the quantitative formula and normalization formula of the indicator value of the bottom-level indicator items are constructed; At the same time, the indicator value is quantified into a formula, and is bound to the database address where the satellite simulation system test data source is located and the information in the corresponding database table.

2. The method for evaluating the effectiveness of a satellite simulation system as claimed in claim 1, wherein: The indicator quantification method includes an indicator value quantification formula and required parameters. The required parameters are expressed as formal parameters. When performing an evaluation task, specific actual parameters are configured for the indicator quantification model.

3. The method for evaluating the effectiveness of a satellite simulation system as claimed in claim 1, wherein: The indicator configuration is implemented through an indicator system, which includes indicator items, an indicator item hierarchy, and indicator weights at each level; The indicator items include indicator item names, positional relationships of indicator items, indicator item calculation data sources, indicator quantification formulas of bottom-level indicator items, and indicator item normalization models.

4. The method for evaluating the effectiveness of a satellite simulation system according to any one of claims 1 to 3, characterized in that: In the performance evaluation, the indicator weights are assigned to the evaluation indicator items at each level, by generating a judgment matrix of the indicator items at each level with respect to the subordinate indicator items, filling in the judgment matrix according to the 9-level scaling method, and calculating the weight values ​​of the indicator items at each level.

5. A satellite simulation system performance evaluation system, characterized in that: include: Indicator configuration module: configures the evaluation indicators at each level, the affiliation of each indicator, the indicator quantification method and indicator normalization method of the lowest level indicators according to the evaluation task; Performance evaluation module: assigns indicator weights to each level of evaluation indicators, reads the current satellite simulation system test data, and generates performance evaluation values; Configuring the indicator normalization method refers to obtaining configuration parameters of the normalization calculation formula of the indicator item of the evaluation task, and instantiating the normalization formula according to the normalization formula and the constant parameters; In the performance evaluation, generating the performance evaluation value means generating the normalized utility value of the lowest level indicator item according to the current satellite simulation system test data and the instantiated normalization formula, combining the indicator weights of each level and using the weighted sum method to obtain the comprehensive performance evaluation value of the current satellite simulation task; The indicator configuration module is specifically used for: Parse the configuration data of the evaluation indicator system and store the configuration parameters of the indicator items in the database table; According to the corresponding field data, the hierarchy and affiliation of the indicator system, the quantitative formula and normalization formula of the indicator value of the bottom-level indicator items are constructed; At the same time, the indicator value is quantified into a formula, and is bound to the database address where the satellite simulation system test data source is located and the information in the corresponding database table.

Images