A method, system, device and medium for performance evaluation of a flight test mission

Abstract

The application relates to the technical field of flight test mission evaluation, in particular to a flight test mission efficiency evaluation method, system, device and medium, which comprises the following steps: constructing a flight test concept meta-model library; generating a computable task model based on the requirements of a target flight test mission; constructing a multi-level index system based on the computable task model; establishing an index mapping relationship table of performance indexes and test data sources; obtaining test data, extracting specific parameters of each performance index from the test data based on the index mapping relationship table, performing standardized assignment, data feature analysis and weight distribution, and obtaining a weight coefficient system of the multi-level index system; and based on the standardized assignment result of the performance index and the weight coefficient system, calculating the evaluation value of the efficiency index through index aggregation to obtain the efficiency evaluation result of the flight test mission. The application can improve the comprehensiveness, accuracy and adaptability of flight test efficiency evaluation.

Description

Technical Field

[0001] This application relates to the field of flight test mission evaluation technology, specifically to a method, system, equipment, and medium for evaluating the effectiveness of flight test missions. Background Technology

[0002] The purpose of flight test mission effectiveness evaluation is to scientifically measure mission performance, optimize system design, and improve overall mission execution capabilities. As the complexity of flight tests continues to increase, the evaluation process needs to handle heterogeneous data from multiple sources, including simulations, experiments, and sensors, and involves multi-dimensional indicators and dynamic mission scenarios, placing higher demands on the adaptability, accuracy, and efficiency of evaluation methods.

[0003] In existing technologies, flight test performance evaluation typically employs static methods based on predefined indicator systems. Evaluation models are constructed using expert experience or historical data, and a fixed algorithm library is used for indicator calculation and result aggregation. For example, some systems use traditional methods such as the analytic hierarchy process (AHP), fuzzy comprehensive evaluation, or weighted average method, manually setting indicator weights and evaluation rules to analyze test data and derive performance conclusions. These methods can provide some evaluation support in scenarios with relatively fixed indicator structures and stable data sources.

[0004] However, the flight test effectiveness evaluation methods have significant shortcomings: the evaluation models lack flexibility and dynamic adaptability, making it difficult to quickly respond to the specific needs of different flight test missions, resulting in a rigid evaluation process with poor reusability; the construction of the indicator system relies heavily on manual intervention, failing to automatically match and generate relevant indicators based on mission characteristics, affecting the comprehensiveness and objectivity of the evaluation; furthermore, the mapping relationship between data and indicators is unclear, and the standardization process is insufficient, making the evaluation results susceptible to the influence of inconsistent data quality and format; weight allocation is mostly based on subjective experience, lacking a data-driven dynamic adjustment mechanism, which reduces the scientific rigor and reliability of the evaluation. Summary of the Invention

[0005] To address the technical problems of existing flight test performance evaluation methods, such as rigid models, reliance on manual construction of indicator systems, unclear data-indicator mapping relationships, and strong subjectivity in weight allocation, this application provides a flight test mission performance evaluation method, system, equipment, and medium. Through dynamic mission modeling, intelligent generation of indicator systems, standardized data processing, and dynamic weight optimization, the comprehensiveness, accuracy, and adaptability of flight test performance evaluation are improved.

[0006] In a first aspect, this application provides a method for evaluating the effectiveness of a flight test mission, comprising the following steps:

[0007] S1. Construct a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements used to describe flight test missions. The meta-model elements include test object meta-models and test process meta-models.

[0008] S2. Instantiate the meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test mission, generate the target flight test mission flowchart, and convert the target flight test mission flowchart into a computable mission model. The computable mission model is a machine-readable structured data representation that contains a complete description of the test object, the test process, and their relationships.

[0009] S3. Based on the computable task model, candidate indicators are matched in a predefined indicator algorithm library, and a multi-level indicator system is constructed based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is various performance indicators.

[0010] S4. Match the corresponding test data source for each performance indicator and establish a mapping relationship table between the performance indicators and the test data source;

[0011] S5. Obtain the test data of the target flight test mission, extract the specific parameters of each performance index from the test data based on the index mapping relationship table, and standardize and assign values ​​to each performance index to obtain the standardized assignment results of each performance index.

[0012] S6. Based on the standardized assignment results, perform data feature analysis and weight allocation to obtain the weight coefficient system of the multi-level indicator system;

[0013] S7. Based on the standardized performance index assignment results and weighting coefficient system, the evaluation value of the effectiveness index is obtained through index aggregation calculation, which serves as the effectiveness evaluation result of the flight test mission.

[0014] It should be further noted that in step S1, the meta-model of the test object includes the attribute definitions of the detection equipment, processing equipment, and execution equipment involved in the flight test;

[0015] The experimental process meta-model includes the process definitions for target detection, signal processing, and mission execution involved in the flight test;

[0016] Meta-model elements are formally described using the Unified Modeling Language, including metadata such as name, type, input parameters, output parameters, and constraints.

[0017] It should be further noted that step S2 includes:

[0018] S201. Instantiate meta-model elements by dragging and dropping them from the flight test concept meta-model library using a graphical modeling tool, then configure the attribute parameters of each instance, and define the logical relationships between instances through connecting lines to generate a visual target flight test mission flowchart.

[0019] S202. Analyze the semantic information of each graphic element in the target flight test mission flowchart, then map the graphic elements to computing nodes, map the connection relationships to data streams, and generate a computable mission model.

[0020] It should be further noted that step S3 includes:

[0021] S301. In the computable task model, the test object type and test process type are parsed, and then semantic matching is performed in the indicator algorithm library using the test object type and test process type as keywords to obtain relevant candidate indicators;

[0022] S302. Based on the hierarchical relationship of candidate indicators in the computable task model, a topological sorting algorithm is used to construct the dependency relationship between indicators, forming a multi-level indicator system with a tree structure.

[0023] It should be further noted that step S4 includes:

[0024] S401. Analyze the data requirements for each performance metric, including parameter type, data format, and sampling frequency;

[0025] S402. Locate the data source that meets the data requirements in the flight test data source list. The flight test data source list is a list of test data types that can be collected in the flight test mission, including flight status data, radar detection data, and action execution records.

[0026] S403. Establish the mapping relationship between each performance indicator and its corresponding data source, forming an indicator mapping relationship table.

[0027] It should be further noted that step S5 includes:

[0028] S501. Obtain the current test dataset for the target flight test mission;

[0029] S502. Based on the index mapping relationship table, locate and extract the specific parameters corresponding to each performance index from the current test dataset;

[0030] S503. Determine the raw values ​​for each performance metric, where:

[0031] When a performance metric directly corresponds to a specific parameter, the corresponding specific parameter is directly used as the original value of the performance metric.

[0032] When a performance metric needs to be calculated based on specific parameters, the corresponding specific parameters are used to perform the calculation to obtain the original value of the performance metric.

[0033] S504. Standardize the raw values ​​of each performance metric to obtain the corresponding standardized values, where:

[0034] For performance indicators where a larger original value indicates better performance, the standardization formula is:

[0035] ;

[0036] For performance indicators where smaller original values ​​indicate better performance, the standardization formula is:

[0037] ;

[0038] in, This represents the standardized value assigned to the current performance metric.

[0039] Represents the raw value of the current performance metric;

[0040] This indicates the preset upper limit of the original value of the current performance indicator;

[0041] This indicates the lower limit of the original value of the current performance indicator.

[0042] It should be further noted that step S6 includes:

[0043] S601. For each upper-level indicator with lower-level indicators, perform feature analysis on the standardized assignment results of all its lower-level indicators to obtain data features, including the correlation coefficient between each lower-level indicator and the coefficient of variation of each lower-level indicator.

[0044] S602. Based on data characteristics, select the weight calculation method for the lower-level indicators for each upper-level indicator through a predefined decision rule base. The decision rule base contains rules for selecting weight calculation methods based on data characteristics.

[0045] S603. Using the selected weight calculation method, calculate the weight value of each lower-level indicator under each upper-level indicator to form a complete weight coefficient system. The sum of the weight values ​​of all lower-level indicators under a given upper-level indicator is 1.

[0046] It should be further noted that the weight calculation methods include the analytic hierarchy process (AHP), fuzzy evaluation method, weighted average method, SEA performance evaluation method, and ADC performance evaluation method, among which:

[0047] The Analytic Hierarchy Process (AHP) constructs pairwise comparison judgment matrices between each lower-level indicator, calculates the eigenvectors of each pairwise judgment matrix as weight vectors, and performs consistency checks on the judgment matrices to finally obtain the weight allocation of each lower-level indicator relative to its upper-level indicator.

[0048] The fuzzy evaluation method constructs the membership function and fuzzy relation matrix of each lower-level indicator, performs fuzzy synthesis operation, and then normalizes the result vector to obtain the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0049] The weighted average method calculates the relative frequency or importance ratio of each indicator by statistically analyzing the historical data of each lower-level indicator, and obtains the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0050] The SEA system performance evaluation method calculates the intersection measure of the system trajectory and mission trajectory in the common performance energy dimension space, analyzes the contribution of each lower-level indicator to the intersection, and obtains the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0051] ADC performance evaluation method: By analyzing the system's performance in three dimensions—availability, reliability, and capability—the contribution ratio of each lower-level indicator to the overall system performance is calculated, and the weight coefficient of each lower-level indicator relative to its upper-level indicator is obtained.

[0052] It should be further noted that step S7 includes:

[0053] Based on the standardized performance index assignment results and weight coefficient system of each performance index, the evaluation value of the upper-level index is calculated layer by layer until the evaluation value of the performance index is obtained.

[0054] The evaluation value of each upper-level indicator is calculated using the following formula:

[0055]

[0056] in, This indicates the current evaluation value of the upper-level indicator;

[0057] This indicates the first [level] under this higher-level indicator. The weighting coefficients of each lower-level indicator;

[0058] This indicates the first [level] under this higher-level indicator. The evaluation value of each lower-level indicator; when the lower-level indicator is a performance indicator. For the first Standardized assignment results of each lower-level indicator;

[0059] This indicates the total number of lower-level indicators under this upper-level indicator.

[0060] It should be further noted that step S8 is also included: generating visualization charts and evaluation reports based on the evaluation results, and then displaying them visually.

[0061] It should be further noted that the visualization charts include at least one of the following:

[0062] The hierarchical structure diagram of the indicator system is generated based on the topological relationships of the multi-level indicator system;

[0063] Capability assessment radar chart, generated based on the assessment values ​​and weighting coefficients of each indicator in a multi-level indicator system;

[0064] The pie chart showing the weight distribution of each higher-level indicator is generated based on the weight coefficients of each level of indicator in the multi-level indicator system, and shows the weight allocation ratio of each lower-level indicator under the higher-level indicator.

[0065] Indicator mapping relationship table;

[0066] The evaluation report includes at least one of the following:

[0067] Effectiveness evaluation results of the flight test mission;

[0068] Definitions of each indicator;

[0069] Evaluation values ​​and weighting coefficients for each indicator;

[0070] The specific parameters corresponding to each performance index in the test data.

[0071] Secondly, this application provides a performance evaluation system for flight test missions, used to implement the above-mentioned performance evaluation method, including:

[0072] The meta-model library construction module is used to build a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements for describing flight test missions. The meta-model elements include test object meta-models and test process meta-models.

[0073] The computable task model generation module is used to instantiate meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test task, generate a target flight test task flowchart, and convert the target flight test task flowchart into a computable task model.

[0074] The multi-level indicator system construction module is used to match candidate indicators in a predefined indicator algorithm library based on a computable task model, and to construct a multi-level indicator system based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is the various performance indicators.

[0075] The indicator mapping relationship table construction module is used to match the corresponding experimental data source for each performance indicator and establish an indicator mapping relationship table between the performance indicator and the experimental data source.

[0076] The standardization assignment module is used to acquire the test data of the target flight test mission, extract specific parameters related to each performance index from the test data based on the index mapping relationship table, and perform standardization assignment on each performance index to obtain the standardization assignment result of each performance index.

[0077] The weight coefficient system construction module is used to perform data feature analysis and weight allocation based on the standardized assignment results, and obtain the weight coefficient system of the multi-level indicator system.

[0078] The performance evaluation result generation module is used to calculate the performance index evaluation value based on the standardized performance index assignment results and weight coefficient system, and to serve as the performance evaluation result of the flight test mission.

[0079] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described performance evaluation method.

[0080] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described performance evaluation method.

[0081] As can be seen from the above technical solutions, this application has the following advantages:

[0082] 1. This application constructs a flight test concept meta-model library and instantiates it to generate a computable task model, realizing dynamic modeling of the task process and machine-readable structured representation. This avoids the repetitive design and adjustment work caused by the fixed model in traditional methods, ensuring that the evaluation process can flexibly adapt to different task scenarios, and enabling the evaluation system to quickly adapt to the specific needs of different flight test tasks.

[0083] 2. This application automatically matches candidate indicators based on a computable task model and constructs a multi-level indicator system. It utilizes a predefined indicator algorithm library for semantic matching, automatically identifies task-related candidate indicators, and constructs dependencies between indicators using a topological sorting algorithm, forming a hierarchical structure. This automation reduces potential subjective biases that may occur during manual indicator selection, ensuring the indicator system comprehensively covers evaluation elements and maintains clear hierarchical relationships, thereby improving the accuracy and reliability of the evaluation results.

[0084] 3. This application establishes a mapping table between performance indicators and test data sources, standardizes and assigns values ​​to test data, analyzes the data requirements of each performance indicator, then searches for data sources that meet the requirements in the flight test data source list, establishes a clear mapping relationship, and then standardizes the raw test data. This achieves an effective association between data and indicators, unifies the dimensions and numerical ranges of data from different sources, eliminates the interference of data heterogeneity on the evaluation process, and provides standardized and unified input data for subsequent calculations.

[0085] 4. This application combines data feature analysis with weight allocation to perform feature analysis on the standardized assignment results of all lower-level indicators under each upper-level indicator, obtain statistical features such as correlation coefficient and coefficient of variation among indicators, and automatically select the most suitable weight calculation method through a predefined decision rule library based on these data features, thereby realizing the dynamic optimization calculation of indicator weights. This overcomes the limitations of traditional methods that rely too much on subjective judgment of experts, and makes weight allocation more in line with the objective laws of data.

[0086] 5. This application calculates the final performance evaluation value by aggregating indicators layer by layer, obtains a comprehensive score of the top-level performance indicators, and forms a complete evaluation output system. This enables the evaluation results to intuitively reflect the performance of each stage of the flight test mission and provide decision-makers with detailed analytical basis. Attached Figure Description

[0087] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1 This is a flowchart of a flight test mission performance evaluation method in one embodiment of this application.

[0089] Figure 2 This is a schematic block diagram of a flight test mission performance evaluation system in one embodiment of this application.

[0090] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation

[0091] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0092] The performance evaluation method involved in this application will be described in detail below. Specific details such as particular system structures and technologies are presented for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0093] In the performance evaluation methods involved in this application, the term "comprising" indicates the presence of the described feature, whole, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0094] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0095] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0096] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0097] The performance evaluation method provided in this application embodiment is executed by computer equipment, and correspondingly, the performance evaluation system for flight test missions runs in computer equipment.

[0098] Figure 1This is a flowchart of a performance evaluation method for a flight test mission according to an embodiment of this application. Figure 1 The implementing entity can be a performance evaluation system. Depending on different needs, the order of the steps in this flowchart can be changed, and some can be omitted.

[0099] like Figure 1 As shown, the effectiveness evaluation method for this flight test mission includes:

[0100] Step S1: Construct a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements used to describe flight test missions. The meta-model elements include test object meta-models and test process meta-models.

[0101] By constructing a flight test concept meta-model library that includes meta-models of test objects and meta-models of test processes, a unified and standardized description of all elements of flight test missions is achieved. This enables different types of test missions to be modeled based on a unified semantic standard, enhancing the system's adaptability to diverse flight test missions and improving model reuse efficiency.

[0102] In some specific embodiments, the test object meta-model includes the attribute definitions of the detection equipment, processing equipment, and execution equipment involved in the flight test;

[0103] The experimental process meta-model includes the process definitions for target detection, signal processing, and mission execution involved in the flight test;

[0104] Meta-model elements are formally described using the Unified Modeling Language, including metadata such as name, type, input parameters, output parameters, and constraints.

[0105] By specifically defining the meta-model of the test object, which includes various equipment attributes, and the meta-model of the test process, which includes key process definitions, fine-grained characterization of flight test mission elements is achieved. This enables the model to accurately describe the equipment behavior and business processes in the actual system, thereby improving the model's expressive power and practicality.

[0106] Step S2: Instantiate the meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test mission, generate the target flight test mission flowchart, and convert the target flight test mission flowchart into a computable mission model. The computable mission model is a machine-readable structured data representation that contains a complete description of the test object, the test process, and their relationships.

[0107] By instantiating meta-model elements based on the requirements of the target flight test mission to generate a mission flowchart and convert it into a computable mission model, the automatic conversion from business requirements to a computational model is realized. This enables complex test processes to be expressed in a machine-readable structured form, significantly improving the accuracy of mission modeling and execution efficiency.

[0108] In some specific embodiments, step S2 includes:

[0109] S201. Instantiate meta-model elements by dragging and dropping them from the flight test concept meta-model library using a graphical modeling tool, then configure the attribute parameters of each instance, and define the logical relationships between instances through connecting lines to generate a visual target flight test mission flowchart.

[0110] S202. Analyze the semantic information of each graphic element in the target flight test mission flowchart, then map the graphic elements to computing nodes, map the connection relationships to data streams, and generate a computable mission model.

[0111] By using graphical modeling tools to visualize and instantiate meta-model elements and automatically convert them into computable task models, the technical threshold for task modeling is significantly reduced, while ensuring the structural standardization and computational feasibility of the generated models.

[0112] Step S3: Based on the computable task model, candidate indicators are matched in the predefined indicator algorithm library, and a multi-level indicator system is constructed based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is the various performance indicators.

[0113] By analyzing the types of experimental elements in a computable task model and matching candidate indicators to construct a multi-level indicator system, an intelligent association between evaluation indicators and task characteristics is achieved. This ensures that the indicator system comprehensively covers key elements of the task while maintaining a clear hierarchical structure, effectively improving the pertinence and systematic nature of the evaluation.

[0114] In some specific embodiments, step S3 includes:

[0115] S301. In the computable task model, the test object type and test process type are parsed, and then semantic matching is performed in the indicator algorithm library using the test object type and test process type as keywords to obtain relevant candidate indicators;

[0116] S302. Based on the hierarchical relationship of candidate indicators in the computable task model, a topological sorting algorithm is used to construct the dependency relationship between indicators, forming a multi-level indicator system with a tree structure.

[0117] By parsing task element types and performing semantic matching in a computable model to obtain candidate indicators, and then constructing a hierarchical indicator system through topological sorting, a deep fit between the indicator system and task characteristics is achieved, ensuring that the evaluation indicators are both comprehensive and well-structured, thereby improving the accuracy and interpretability of the evaluation.

[0118] Step S4: Match each performance indicator with a corresponding test data source and establish a mapping table between the performance indicators and the test data sources.

[0119] By analyzing the performance indicator data requirements and establishing a mapping table with the experimental data sources, a precise connection between the evaluation indicators and the data sources was achieved, ensuring that each performance indicator could obtain data support that meets the specifications, and providing reliable data assurance for subsequent evaluation calculations.

[0120] In some specific embodiments, step S4 includes:

[0121] S401. Analyze the data requirements for each performance metric, including parameter type, data format, and sampling frequency;

[0122] S402. Locate the data source that meets the data requirements in the flight test data source list. The flight test data source list is a list of test data types that can be collected in the flight test mission, including flight status data, radar detection data, and action execution records.

[0123] S403. Establish the mapping relationship between each performance indicator and its corresponding data source, forming an indicator mapping relationship table.

[0124] By analyzing the data requirement characteristics of performance metrics in detail and accurately matching them in the data source list, a multi-dimensional relationship between metrics and data sources was established, ensuring that each metric can obtain data support that meets its specific requirements, thereby improving the accuracy and completeness of data acquisition.

[0125] Step S5: Obtain the test data of the target flight test mission for the current test, extract specific parameters related to each performance index from the test data based on the index mapping relationship table, and standardize and assign values ​​to each performance index to obtain the standardized assignment results of each performance index.

[0126] By acquiring the data from the current experiment and extracting parameters based on the mapping relationship for standardized assignment, the original experimental data was standardized, eliminating the impact of differences in multi-source data on the evaluation results and ensuring the comparability and fairness of the evaluation results between different experimental tasks.

[0127] In some specific embodiments, step S5 includes:

[0128] S501. Obtain the current test dataset for the target flight test mission;

[0129] S502. Based on the index mapping relationship table, locate and extract the specific parameters corresponding to each performance index from the current test dataset;

[0130] S503. Determine the raw values ​​for each performance metric, where:

[0131] When a performance metric directly corresponds to a specific parameter, the corresponding specific parameter is directly used as the original value of the performance metric.

[0132] When a performance metric needs to be calculated based on specific parameters, the corresponding specific parameters are used to perform the calculation to obtain the original value of the performance metric.

[0133] S504. Standardize the raw values ​​of each performance metric to obtain the corresponding standardized values, where:

[0134] For performance indicators where a larger original value indicates better performance, the standardization formula is:

[0135] ;

[0136] For performance indicators where smaller original values ​​indicate better performance, the standardization formula is:

[0137] ;

[0138] in, This represents the standardized value assigned to the current performance metric.

[0139] Represents the raw value of the current performance metric;

[0140] This indicates the preset upper limit of the original value of the current performance indicator;

[0141] This indicates the lower limit of the original value of the current performance indicator.

[0142] By accurately locating and extracting index parameters from the experimental dataset and completing the calculation and standardization of raw values, the effective transformation of massive experimental data into standardized evaluation values ​​was achieved, ensuring the quality and consistency of the input data for evaluation calculation and preparing high-quality input data for subsequent weight allocation and aggregation calculation.

[0143] Step S6: Based on the standardized assignment results, perform data feature analysis and weight allocation to obtain the weight coefficient system of the multi-level indicator system.

[0144] By performing feature analysis on standardized data and dynamically selecting weight calculation methods based on decision rules, the weight allocation has been transformed from subjective experience to data-driven, making the weight setting more in line with the actual data characteristics and significantly improving the objectivity and scientific nature of the evaluation results.

[0145] In some specific embodiments, step S6 includes:

[0146] S601. For each upper-level indicator with lower-level indicators, perform feature analysis on the standardized assignment results of all its lower-level indicators to obtain data features, including the correlation coefficient between each lower-level indicator and the coefficient of variation of each lower-level indicator.

[0147] S602. Based on data characteristics, select the weight calculation method for the lower-level indicators for each upper-level indicator through a predefined decision rule base. The decision rule base contains rules for selecting weight calculation methods based on data characteristics.

[0148] S603. Using the selected weight calculation method, calculate the weight value of each lower-level indicator under each upper-level indicator to form a complete weight coefficient system. The sum of the weight values ​​of all lower-level indicators under a given upper-level indicator is 1.

[0149] By analyzing the statistical characteristics of lower-level indicator data and intelligently selecting weight algorithms based on rule bases, adaptive optimization of weight calculation methods is achieved, enabling weight allocation to be dynamically adjusted according to data characteristics, thereby improving the evaluation system's adaptability to different data patterns.

[0150] In some specific embodiments, the weight calculation methods include the analytic hierarchy process (AHP), fuzzy evaluation method, weighted average method, SEA performance evaluation method, and ADC performance evaluation method, wherein:

[0151] The Analytic Hierarchy Process (AHP) constructs pairwise comparison judgment matrices between each lower-level indicator, calculates the eigenvectors of each pairwise judgment matrix as weight vectors, and performs consistency checks on the judgment matrices to finally obtain the weight allocation of each lower-level indicator relative to its upper-level indicator.

[0152] The fuzzy evaluation method constructs the membership function and fuzzy relation matrix of each lower-level indicator, performs fuzzy synthesis operation, and then normalizes the result vector to obtain the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0153] The weighted average method calculates the relative frequency or importance ratio of each indicator by statistically analyzing the historical data of each lower-level indicator, and obtains the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0154] The SEA system performance evaluation method calculates the intersection measure of the system trajectory and mission trajectory in the common performance energy dimension space, analyzes the contribution of each lower-level indicator to the intersection, and obtains the weight coefficient of each lower-level indicator relative to its upper-level indicator.

[0155] ADC performance evaluation method: By analyzing the system's performance in three dimensions—availability, reliability, and capability—the contribution ratio of each lower-level indicator to the overall system performance is calculated, and the weight coefficient of each lower-level indicator relative to its upper-level indicator is obtained.

[0156] By integrating multiple professional weight calculation methods and providing detailed implementation specifications, the system achieves professionalization and standardization of weight calculation, enabling it to select the most suitable weight allocation strategy according to evaluation needs, thus ensuring the scientific nature and authority of the weight calculation results.

[0157] Step S7: Based on the standardized assignment results of performance indicators and the weighting coefficient system, the evaluation value of the effectiveness indicator is obtained through indicator aggregation calculation, which serves as the effectiveness evaluation result of the flight test mission.

[0158] By performing layer-by-layer aggregation calculations based on standardized data and a weighting system, the performance evaluation value is obtained, realizing the quantitative derivation from the bottom-level indicators to the top-level performance. This ensures that the evaluation results can comprehensively reflect the overall performance of the system and provide accurate numerical basis for performance optimization.

[0159] In some specific embodiments, step S7 includes:

[0160] Based on the standardized performance index assignment results and weight coefficient system of each performance index, the evaluation value of the upper-level index is calculated layer by layer until the evaluation value of the performance index is obtained.

[0161] The evaluation value of each upper-level indicator is calculated using the following formula:

[0162]

[0163] in, This indicates the current evaluation value of the upper-level indicator;

[0164] This indicates the first [level] under this higher-level indicator. The weighting coefficients of each lower-level indicator;

[0165] This indicates the first [level] under this higher-level indicator. The evaluation value of each lower-level indicator; when the lower-level indicator is a performance indicator. For the first Standardized assignment results of each lower-level indicator;

[0166] This indicates the total number of lower-level indicators under this upper-level indicator.

[0167] By defining specific formulas for layer-by-layer aggregation and clarifying data transmission paths, traceable calculation of evaluation values ​​is achieved, ensuring that each upper-level indicator value accurately reflects the performance of the lower-level indicators and guaranteeing the mathematical rigor and logical transparency of the evaluation results.

[0168] In some specific embodiments, step S8 is also included: generating visualization charts and evaluation reports based on the evaluation results, and displaying them visually.

[0169] By generating diverse visualizations and structured reports based on the evaluation results, the evaluation results are presented in a multi-dimensional way, enabling complex evaluation data to be presented in an intuitive form, which greatly improves the efficiency of understanding the results and their value in supporting decision-making.

[0170] In some specific embodiments, the visualization chart includes at least one of the following:

[0171] The hierarchical structure diagram of the indicator system is generated based on the topological relationships of the multi-level indicator system;

[0172] Capability assessment radar chart, generated based on the assessment values ​​and weighting coefficients of each indicator in a multi-level indicator system;

[0173] The pie chart showing the weight distribution of each higher-level indicator is generated based on the weight coefficients of each level of indicator in the multi-level indicator system, and shows the weight allocation ratio of each lower-level indicator under the higher-level indicator.

[0174] Indicator mapping relationship table;

[0175] The evaluation report includes at least one of the following:

[0176] Effectiveness evaluation results of the flight test mission;

[0177] Definitions of each indicator;

[0178] Evaluation values ​​and weighting coefficients for each indicator;

[0179] The specific parameters corresponding to each performance index in the test data.

[0180] By specifying the types of visualization charts and the composition of report content, the system achieves standardized customization of evaluation output, enabling different users to obtain analytical views that meet their needs, thus enhancing the practicality and relevance of the system output.

[0181] The following are embodiments of the flight test mission performance evaluation system provided in this application. This flight test mission performance evaluation system and the performance evaluation methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the performance evaluation system, please refer to the embodiments of the flight test mission performance evaluation methods described above.

[0182] like Figure 2 As shown, the performance evaluation system for flight test missions includes:

[0183] The meta-model library construction module is used to build a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements for describing flight test missions. The meta-model elements include test object meta-models and test process meta-models.

[0184] The computable task model generation module is used to instantiate meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test task, generate a target flight test task flowchart, and convert the target flight test task flowchart into a computable task model.

[0185] The multi-level indicator system construction module is used to match candidate indicators in a predefined indicator algorithm library based on a computable task model, and to construct a multi-level indicator system based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is the various performance indicators.

[0186] The indicator mapping relationship table construction module is used to match the corresponding experimental data source for each performance indicator and establish an indicator mapping relationship table between the performance indicator and the experimental data source.

[0187] The standardization assignment module is used to acquire the test data of the target flight test mission, extract specific parameters related to each performance index from the test data based on the index mapping relationship table, and perform standardization assignment on each performance index to obtain the standardization assignment result of each performance index.

[0188] The weight coefficient system construction module is used to perform data feature analysis and weight allocation based on the standardized assignment results, and obtain the weight coefficient system of the multi-level indicator system.

[0189] The performance evaluation result generation module is used to calculate the performance index evaluation value based on the standardized performance index assignment results and weight coefficient system, and to serve as the performance evaluation result of the flight test mission.

[0190] The performance evaluation system in this embodiment is used to implement a performance evaluation method for flight test missions.

[0191] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.

[0192] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0193] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0194] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0195] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.

[0196] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0197] This application also provides a storage medium storing a program product capable of implementing a performance evaluation method for flight test missions. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.

[0198] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0199] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for evaluating the effectiveness of a flight test mission, characterized in that, include: S1. Construct a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements used to describe flight test missions. The meta-model elements include test object meta-models and test process meta-models. The meta-model of the test object includes the attribute definitions of the detection equipment, processing equipment, and execution equipment involved in the flight test; The experimental process meta-model includes the process definitions for target detection, signal processing, and mission execution involved in the flight test; Metamodel elements are formally described using the Unified Modeling Language, including metadata such as name, type, input parameters, output parameters, and constraints. S2. Instantiate the meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test mission, generate the target flight test mission flowchart, and convert the target flight test mission flowchart into a computable mission model. The computable mission model is a machine-readable structured data representation that contains a complete description of the test object, the test process, and their relationships. Step S2 includes: S201. Instantiate meta-model elements by dragging and dropping them from the flight test concept meta-model library using a graphical modeling tool, then configure the attribute parameters of each instance, and define the logical relationships between instances through connecting lines to generate a visual target flight test mission flowchart. S202. Analyze the semantic information of each graphic element in the target flight test mission flowchart, then map the graphic elements to computing nodes, map the connection relationships to data streams, and generate a computable mission model; S3. Based on the computable task model, candidate indicators are matched in a predefined indicator algorithm library, and a multi-level indicator system is constructed based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is various performance indicators. Step S3 includes: S301. In the computable task model, the test object type and test process type are parsed, and then semantic matching is performed in the indicator algorithm library using the test object type and test process type as keywords to obtain relevant candidate indicators; S302. Based on the hierarchical relationship of candidate indicators in the computable task model, a topological sorting algorithm is used to construct the dependency relationship between indicators, forming a multi-level indicator system with a tree structure. S4. Match the corresponding test data source for each performance indicator and establish a mapping relationship table between the performance indicators and the test data source; Step S4 includes: S401. Analyze the data requirements for each performance metric, including parameter type, data format, and sampling frequency; S402. Locate the data source that meets the data requirements in the flight test data source list. The flight test data source list is a list of test data types that can be collected in the flight test mission, including flight status data, radar detection data, and action execution records. S403. Establish the mapping relationship between each performance indicator and its corresponding data source, forming an indicator mapping relationship table; S5. Obtain the test data of the target flight test mission, extract the specific parameters of each performance index from the test data based on the index mapping relationship table, and standardize and assign values ​​to each performance index to obtain the standardized assignment results of each performance index. S6. Based on the standardized assignment results, perform data feature analysis and weight allocation to obtain the weight coefficient system of the multi-level indicator system; S7. Based on the standardized performance index assignment results and weighting coefficient system, the evaluation value of the effectiveness index is obtained through index aggregation calculation, which serves as the effectiveness evaluation result of the flight test mission.

2. The performance evaluation method as described in claim 1, characterized in that, Step S5 includes: S501. Obtain the current test dataset for the target flight test mission; S502. Based on the index mapping relationship table, locate and extract the specific parameters corresponding to each performance index from the current test dataset; S503. Determine the raw values ​​for each performance metric, where: When a performance metric directly corresponds to a specific parameter, the corresponding specific parameter is directly used as the original value of the performance metric. When a performance metric needs to be calculated based on specific parameters, the corresponding specific parameters are used to perform the calculation to obtain the original value of the performance metric. S504. Standardize the raw values ​​of each performance metric to obtain the corresponding standardized values, where: For performance indicators where a larger original value indicates better performance, the standardization formula is: ； For performance indicators where smaller original values ​​indicate better performance, the standardization formula is: ； in, This represents the standardized value assigned to the current performance metric. Represents the raw value of the current performance metric; This indicates the preset upper limit of the original value of the current performance indicator; This indicates the lower limit of the original value of the current performance indicator.

3. The performance evaluation method as described in claim 1, characterized in that, Step S6 includes: S601. For each upper-level indicator with lower-level indicators, perform feature analysis on the standardized assignment results of all its lower-level indicators to obtain data features, including the correlation coefficient between each lower-level indicator and the coefficient of variation of each lower-level indicator. S602. Based on data characteristics, select the weight calculation method for the lower-level indicators for each upper-level indicator through a predefined decision rule base. The decision rule base contains rules for selecting weight calculation methods based on data characteristics. S603. Using the selected weight calculation method, calculate the weight value of each lower-level indicator under each upper-level indicator to form a complete weight coefficient system. The sum of the weight values ​​of all lower-level indicators under a given upper-level indicator is 1.

4. The performance evaluation method as described in claim 1, characterized in that, Step S7 includes: Based on the standardized performance index assignment results and weight coefficient system of each performance index, the evaluation value of the upper-level index is calculated layer by layer until the evaluation value of the performance index is obtained. The evaluation value of each upper-level indicator is calculated using the following formula: in, This indicates the current evaluation value of the upper-level indicator; This indicates the first [level] under this higher-level indicator. The weighting coefficients of each lower-level indicator; This indicates the first [level] under this higher-level indicator. The evaluation value of each lower-level indicator; when the lower-level indicator is a performance indicator. For the first Standardized assignment results of each lower-level indicator; This indicates the total number of lower-level indicators under this upper-level indicator.

5. The performance evaluation method as described in claim 1, characterized in that, It also includes step S8: generating visualization charts and evaluation reports based on the evaluation results, and displaying them visually.

6. A performance evaluation system for flight test missions, characterized in that, To implement the performance evaluation method as described in any one of claims 1-5, the method includes: The meta-model library construction module is used to build a flight test concept meta-model library. The flight test concept meta-model library contains multiple meta-model elements for describing flight test missions. The meta-model elements include test object meta-models and test process meta-models. The computable task model generation module is used to instantiate meta-model elements in the flight test concept meta-model library based on the requirements of the target flight test task, generate a target flight test task flowchart, and convert the target flight test task flowchart into a computable task model. The multi-level indicator system construction module is used to match candidate indicators in a predefined indicator algorithm library based on a computable task model, and to construct a multi-level indicator system based on the candidate indicators. The top level of the multi-level indicator system is the efficiency indicator, and the bottom level is the various performance indicators. The indicator mapping relationship table construction module is used to match the corresponding experimental data source for each performance indicator and establish an indicator mapping relationship table between the performance indicator and the experimental data source. The standardization assignment module is used to acquire the test data of the target flight test mission, extract specific parameters related to each performance index from the test data based on the index mapping relationship table, and perform standardization assignment on each performance index to obtain the standardization assignment result of each performance index. The weight coefficient system construction module is used to perform data feature analysis and weight allocation based on the standardized assignment results, and obtain the weight coefficient system of the multi-level indicator system. The performance evaluation result generation module is used to calculate the performance index evaluation value based on the standardized performance index assignment results and weight coefficient system, and to serve as the performance evaluation result of the flight test mission.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When a processor is used to execute a computer program, it implements the steps of the performance evaluation method as described in any one of claims 1-5.

8. A storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the steps of the performance evaluation method as described in any one of claims 1-5.

Images