Effectiveness evaluation method of vehicle equipment and system for building effectiveness evaluation system

Abstract

The present invention relates to a vehicle equipment performance evaluation method, including: obtaining the environmental conditions and performance target attributes of the task action phase and its application, constructing a full set of performance evaluation indicators based on the target task; decomposing the task profile and establishing subtasks, and constructing a full set of performance evaluation indicator systems based on the capability level according to the required capability analysis of the subtasks; selecting the corresponding full set of performance evaluation indicators to customize the evaluation indicators according to the evaluation object and the performance evaluation target; filtering the corresponding evaluation indicators, and manually selecting the corresponding indicator items; repeating the selection and filtering process to obtain the application project performance evaluation indicator set; normalizing the indicator set, assigning weights and performing aggregation calculations to obtain the comprehensive performance of the vehicle equipment. The performance evaluation indicator set obtained by the method is highly matched with the performance evaluation target, and the correlation between the indicators is small and the indicator differentiation is high; the evaluation method fully considers the application environment of the vehicle equipment and the performance target attributes of the task, and has strong operability.

Description

Technical Field

[0001] The present invention relates to the field of weapon equipment effectiveness evaluation system, in particular to a vehicle equipment effectiveness evaluation method and an effectiveness evaluation system construction system. Background Art

[0002] System performance is the ability of a designated person to use a designated application system to complete a specified task under specified environmental conditions. System performance evaluation is the work of evaluating the performance of a specific application system. In system performance evaluation, the final step is to develop a performance indicator system, that is, to determine an evaluation standard for carrying out system performance evaluation work.

[0003] At present, there are two main methods for constructing an effectiveness evaluation index system: the first is to simply replace the effectiveness evaluation index with the function and performance index set by the system; the second is to invite relevant authoritative experts in the industry to determine the evaluation index system based on past experience. For the first method, it is shown that there are many deviations. The function and performance indicators of the system have been determined during the development of the system, and most of the application systems have been evaluated by the function and performance indicators. This method of constructing an effectiveness evaluation index system determines that the significance of its evaluation results will not be too great; for the second method, most of the invited experts are only experts in the industry, not the actual users of the specific application system. They do not have a deep understanding of the specific contents of the application system, such as the task objectives, environmental conditions, important nodes, task processes, and capability requirements. They often propose indicators based on personal experience, and the experience is far more than the scientific nature. Therefore, there will be a large deviation between the evaluation results and the real effectiveness of the system. It often takes multiple evaluation practices and a long time of correction to gradually approach the real results.

[0004] From the above performance evaluation index system construction process, we can see the main problems of the two current methods: First, the system performance evaluation indicators may seriously deviate from the real performance of the system, resulting in directional deviations in the evaluation results. Second, the system performance evaluation indicators may not cover the real performance of the system, resulting in the evaluation results only reflecting individual attributes of the system performance. Third, there may be a large inclusion relationship between individual evaluation indicators, resulting in an imbalance and inaccuracy in the overall evaluation system. The end point of system performance evaluation is to scientifically describe the ability of the application system to complete specific tasks under specific environmental conditions, and provide a basic basis for the upgrade, improvement and scientific use of the system. The defects of the aforementioned methods will undoubtedly greatly reduce the actual role of system performance evaluation, and even have negative effects under certain extreme conditions. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a vehicle equipment performance evaluation method and a system construction system thereof. The method and system include the characteristics of "one system, two levels, two main lines, and one output" in terms of architecture, thereby improving the accuracy and practicality of system performance evaluation.

[0006] To achieve the above object, the technical solution adopted by the present invention is a method for evaluating the effectiveness of vehicle equipment, the method comprising the following steps:

[0007] (1) Obtaining the environmental conditions and attributes of each task action phase and its application according to the category or subcategory of the vehicle equipment assessment task, and constructing a complete set of effectiveness assessment indicators A based on the target task according to these parameters;

[0008] (2) Decompose the task profile of the vehicle equipment assessment task and establish subtasks, analyze the capabilities required by the subtasks, and build a complete set of effectiveness assessment indicator systems based on capability levels B;

[0009] (3) According to the vehicle equipment and its effectiveness evaluation target of the newly established target evaluation task item, the relevant evaluation indicators are selected from the effectiveness evaluation indicator set A based on the target task and / or the effectiveness evaluation indicator system set B based on the capability level, and the corresponding evaluation indicators are filtered out from the indicator set to obtain the application project effectiveness evaluation indicator system that is compatible with the vehicle equipment and its effectiveness evaluation target;

[0010] The application project effectiveness evaluation index system includes any one of a complete set A of effectiveness evaluation indexes based on target tasks and a complete set B of effectiveness evaluation indexes based on capability levels, or a mixture of the two complete sets of indexes.

[0011] A preferred technical solution, after step (3), further comprises:

[0012] (4) According to the task purpose, task focus and indicator contribution of the target evaluation task item, weights are assigned to the evaluation indicators at all levels in the application project effectiveness evaluation indicator system through expert scoring and system calculation methods;

[0013] (5) selecting and editing an indicator normalization model and an indicator aggregation model based on the set evaluation model library according to the data item categories of the evaluation basic indicators in the application project effectiveness evaluation indicator system, so as to form an effectiveness evaluation model system for the target task;

[0014] (6) Designing and generating an indicator data collection form based on the data collection requirements of the basic evaluation indicators in the application project effectiveness evaluation indicator system and in combination with the environmental conditions and properties of the vehicle equipment and its application;

[0015] (7) Connect the performance evaluation model system of the target task, the evaluation rule algorithm and the indicator data items of the indicator data collection form, perform performance evaluation calculations, obtain the evaluation calculation result data, and perform statistical analysis to form an evaluation report.

[0016] Furthermore, the step (1) specifically includes the following steps:

[0017] A1. According to the specific evaluation items, select or directly create new target tasks in the target task library to complete the determination of performance evaluation target tasks;

[0018] A2. Select or directly create and determine the category or subcategory of the target task in the target task category library;

[0019] A3. Select or directly establish and determine each task action phase of the target task in the target task action library;

[0020] A4. Determine the environmental conditions and related attributes existing in each task action stage. The environmental conditions include the natural geographical environment, meteorological and hydrological environment, humanistic and social environment, economic and political environment and conditions associated with the user's task; the related attributes are the attributes of the performance target, such as timeliness, accuracy, precision, etc.;

[0021] A5. Based on each task action phase, environmental conditions and related attributes, construct evaluation indicator items that can be measured by qualitative grades or quantitative values;

[0022] In step A2, when the target task is selected from the target task library, the system also automatically recommends the target task category of the corresponding stage; in step A3, when the target task is selected from the target task library, the system also automatically recommends the task action stage of the corresponding stage.

[0023] Furthermore, the step (2) specifically includes the following steps:

[0024] B1. Decompose the task profile from different angles, select and determine subtasks in the target task profile library, or create new task profiles and subtasks;

[0025] B2. Analyze the capabilities required to complete the subtasks according to their nature, select the capabilities and their levels required from the basic capability library, or directly establish new capability items and capability levels;

[0026] B3. Based on the capability requirements, construct evaluation indicator items measured by qualitative grades or quantitative values.

[0027] Furthermore, in step (3), the corresponding evaluation indicators filtered out from the entire set of indicators are specifically:

[0028] If a customized evaluation indicator is selected from the full set A of the effectiveness evaluation indicator system based on the target task, the corresponding evaluation indicator is filtered out by selecting the task action;

[0029] If a customized evaluation indicator is selected from the complete set B of the effectiveness evaluation indicator system based on capability level, the corresponding evaluation indicator is filtered out by selecting the capability requirement.

[0030] Furthermore, the weighting of the evaluation indicators at all levels in the application project performance evaluation index system by expert scoring and system calculation methods in step (4) specifically includes: expert direct scoring method, fuzzy ranking method of the degree of performance importance based on the understanding of expert weighted scoring experience, hierarchical analysis method of constructing a judgment matrix, and entropy weight method weighting method;

[0031] The hierarchical analysis method includes:

[0032] 1) Relying on the application project effectiveness evaluation index system and the determined evaluation indicators, establish a hierarchical structure model of the evaluation index system for the corresponding project;

[0033] 2) Constructing a judgment matrix based on the hierarchical structure model of the evaluation index system and generating a comparison matrix; specifically including the following steps;

[0034] 2.1) Select the level of constructing the judgment matrix, and divide all evaluation indicators into three levels, first select the high level, then the middle level, and finally the low level;

[0035] 2.2) Set the reference target O of this layer. The target predetermined weight and the corresponding number of subordinate indicators are automatically given by the system; construct a judgment matrix for each target.

[0036] 2.3) Construct the element values ​​aij of the comparison matrix, select two indicators for constructing the comparison matrix from the options of indicator i and indicator j, and input the importance coefficient scales of the two indicators;

[0037] 2.4) Perform comparative matrix operations on the selected level and analyze and obtain the weights of the relevant indicators at the selected level;

[0038] 2.5) According to the above steps, return to step 1.2), select another reference target O, and repeat steps 2.2) to 2.4) until all reference targets at the selected level are analyzed;

[0039] 3) Rely on the system weight calculation method to calculate the weights of the hierarchical single sorting and total sorting and check their consistency.

[0040] In a preferred technical solution, the evaluation rule algorithm of step (7) includes: weighting and aggregation calculation of the indicator data after normalization; and comprehensive processing and membership calculation of multiple sampling data of fixed value interval, threshold interval, qualitative grade, and logical binary type indicator of single indicator data;

[0041] The evaluation report includes analysis to obtain the following indicator levels: basic evaluation indicators, attribute capability level indicators, specific task level indicators and action level indicators.

[0042] Furthermore, the evaluation rule algorithm also includes one or more of the following:

[0043] Automatically reduce the weight of performance evaluation indicators that have no data support and distribute the reduced weight to other indicators;

[0044] Automatically sort out the weights of indicators at all levels above the grassroots level based on the weighting results of grassroots indicators in the performance evaluation indicators;

[0045] Carry out correlation and resolution calculations between various indicators in the application project effectiveness evaluation indicator system;

[0046] Identify the correlation between performance evaluation indicators and aggregate similar indicators.

[0047] The second aspect of the present invention discloses a system for constructing a vehicle equipment performance evaluation system using the above method, the system comprising:

[0048] A target task effectiveness evaluation index construction module is configured to obtain each task action phase and its existing environmental conditions and attributes according to the category or subcategory of the vehicle equipment evaluation task, and construct a full set A of effectiveness evaluation indicators based on the target task according to these parameters;

[0049] The capability level effectiveness evaluation indicator construction module is configured as the task profile of the vehicle equipment evaluation task and establishes subtasks, and analyzes the required capabilities of the subtasks to construct a complete set of effectiveness evaluation indicator systems based on capability levels B;

[0050] The effectiveness evaluation index customization module is configured to customize the evaluation index based on the vehicle equipment of the newly established target evaluation task item and its effectiveness evaluation target, and select the evaluation index related thereto to belong to the effectiveness evaluation index set A based on the target task and / or the effectiveness evaluation index system set B based on the capability level;

[0051] The performance evaluation index filtering module is configured to filter out corresponding evaluation indicators according to the selected performance evaluation index set;

[0052] The application project effectiveness evaluation indicator generation module is configured to select and determine corresponding indicator items from the filtered evaluation indicator set to obtain an application project effectiveness evaluation indicator set that is compatible with the vehicle equipment and its effectiveness evaluation objectives; wherein the application project effectiveness evaluation indicator system includes any one of the effectiveness evaluation indicator set A based on the target task and the effectiveness evaluation indicator system set B based on the capability level, or a mixture of the two indicator system sets.

[0053] In a preferred technical solution, the system further comprises:

[0054] A weight assignment module is configured to assign weights to evaluation indicators at all levels in the application project effectiveness evaluation indicator system according to the task purpose, task focus and indicator contribution of the target evaluation task item through expert scoring and system calculation methods;

[0055] A data collection form generation module is configured to design and generate an indicator data collection form according to the demand for collection data of the basic evaluation indicators in the application project effectiveness evaluation indicator system, combined with the environmental conditions and properties of the vehicle equipment and its application;

[0056] An evaluation model system construction module is configured to select and edit an indicator normalization model and an indicator aggregation model based on the data item category of the evaluation base indicators in the application project performance evaluation indicator system and the set evaluation model library to form a performance evaluation model system for the target task;

[0057] An indicator data item docking module is configured to dock the evaluation indicator items, indicator data items with the normalization model and the indicator aggregation model to form an effectiveness evaluation task plan;

[0058] The performance evaluation calculation module is configured to perform performance evaluation calculation using the performance evaluation index system, evaluation rule algorithm and collected index data of the application project; and

[0059] The performance evaluation result analysis module is configured to perform statistical analysis based on the calculation result data to form an evaluation report; the evaluation report includes the analysis of the following indicator levels: the most basic evaluation indicators, attribute capability level indicators, specific task level indicators and action level indicators.

[0060] Compared with the prior art, the performance evaluation method of vehicle equipment based on target tasks and capability levels of the present invention can be to move the creation and preparation of the evaluation index system to daily life, and form a complete set of evaluation index systems through daily information collection. The performance evaluation method and system comprehensively describe the performance improvement of the system or system from two different aspects: the completion of the target task and the execution capability level requirements, so that the final specific application project performance evaluation index set is not only highly matched with the performance evaluation target, but also has extremely low correlation between the indicators and high indicator differentiation, which improves the accuracy of the performance evaluation results.

[0061] The method and system of the present invention have normalization model support that covers all types of indicator matching capabilities of various evaluation indicators; the system is embedded with a dynamic grade quantization model, which can support dynamic quantization of grade values ​​when graded indicators are normalized; it has the ability to intelligently allocate weights of indicators that should be devalued or lost, which plays a role in eliminating invalid data and other interference and increasing the differentiation of effective indicators; it has the ability to highly automatically generate performance evaluation reports, and can automatically generate performance evaluation reports and corresponding charts, greatly improving the accuracy and speed of analysis of results in the later stage of evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and a person skilled in the art can also obtain other drawings based on these drawings.

[0063] Figure 1 An overall flow chart of the performance evaluation method;

[0064] Figure 2 A flowchart for constructing a complete set of performance evaluation indicator systems based on target tasks;

[0065] Figure 3 A flowchart for constructing a complete set of performance evaluation indicator systems based on capability levels;

[0066] Figure 4 A diagram of the generation process of the overall effectiveness evaluation indicator set;

[0067] Figure 5 is an overall flow chart of another embodiment of the performance evaluation method;

[0068] Figure 6 One implementation method for building a system for effectiveness evaluation system;

[0069] Figure 7 Another implementation of a system for building a performance evaluation system. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and beneficial effects of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] The vehicle equipment effectiveness evaluation method based on target tasks and capability levels of the present invention includes at least the characteristics of "one system, two levels, two main lines, and one output" in the architecture method. Among them:

[0072] “One system” refers to the performance evaluation indicator system based on target tasks and capability levels;

[0073] “Two levels” means that the performance evaluation index system based on target tasks and capability levels has at least two levels, namely, the performance evaluation index system full set layer and the performance evaluation index application layer;

[0074] "Two main lines": one is to start from the target task to be completed, and build an effectiveness evaluation index system based on the target task according to the task category, action process, condition attributes, and task achievement. The other is to start from the task completion capability requirements, and build an effectiveness evaluation index system based on the capability level according to the specific tasks, required capabilities, and capability levels of the task profile;

[0075] "One output" means that no matter what specific performance evaluation application the performance evaluation index system based on target tasks and capability levels is applied to, the output method of the performance evaluation index of the specific application is the same, that is, the same program and data structure can be used to implement it in the specific application system.

[0076] The performance evaluation method described in this application is based on the above operating principle and mainly includes three stages: Figure 1 As shown: the first is the stage of constructing a set of performance evaluation indicator systems based on target tasks; the second is the stage of constructing a set of performance evaluation indicator systems based on capability levels; and the third is the stage of customizing and generating performance evaluation indicators for specific projects.

[0077] An embodiment of the method for evaluating the effectiveness of vehicle equipment according to the present invention will be described below with reference to the accompanying drawings.

[0078] (1) According to the category or subcategory of the vehicle equipment evaluation task, obtain each task action phase and its existing environmental conditions and attributes, and construct a complete set of effectiveness evaluation indicators A based on the target task based on these parameters, such as Figure 2 As shown, the specific method includes the following steps:

[0079] A1. According to the specific evaluation items of the vehicle equipment evaluation task, select or directly create a new target task in the target task library to complete the determination of the performance evaluation target task; wherein the target task library is constructed based on the task information recorded daily by the system.

[0080] When creating a new target task, the user needs to assign a unique number to the newly created task project, enter the test task, test unit, start and end date, report template, test description, accompanying test equipment, experimental subjects and test equipment, and complete the establishment of the new target task. The task library is responsible for storing and managing typical task lists, and its tasks can be broken down into multiple levels, such as the hierarchical categories that can include army campaigns or tactical battles; the task description is closely related to the task item, the task conditions are managed in the environment library and associated by selection during implementation, and the task indicators are managed in the indicator system library and the appropriate indicators are determined by selection during implementation.

[0081] A2. Based on the new target task, determine its category or subcategory; the category or subcategory of the target task can be selected from the target task category library accumulated daily, or a new target task category can be directly established. If and only if the target task is selected from the target task library, the system can automatically recommend the associated target task category.

[0082] A3. Based on the new target task, determine its various task action stages, i.e., subtasks; the task action stages can be selected from the daily accumulated task action library, or a new task action can be established. If and only if the target task is selected from the target task library, the system can automatically recommend the associated task action stages.

[0083] A4. Determine the environmental conditions and related attributes that exist in each task action stage. The environmental conditions include the natural geographical environment, meteorological and hydrological environment, humanistic and social environment, economic and political environment and conditions and corresponding levels associated with the user's task; the related attributes refer to the attributes of the performance goals, such as timeliness, accuracy, precision, etc.

[0084] A5. Based on each task action stage, environmental conditions and related attributes, construct evaluation index items that can be measured by qualitative grades or quantitative values; for example, the collection result value is quantified through "customized interval" and "threshold interval", and the collection result is qualitatively measured through registration descriptions of excellent, good, medium and poor. Based on the above method, the evaluation index items have three characteristics: first, they can be measured by qualitative grades or quantitative values; second, they have the ability to associate the task action stage in step A3 with the environmental conditions, related attributes and other information in step A4; third, there is as little correlation as possible between the evaluation index items.

[0085] The whole process of constructing the target task-based performance evaluation index system is as follows: Figure 2 It is a tree-like parsing process with gradual branching and expansion.

[0086] (2) Decompose the task profile of the vehicle equipment assessment task and establish subtasks, analyze the capabilities required by the subtasks, and build a complete set of effectiveness assessment indicator systems based on capability levels, such as Figure 3 As shown, the method comprises the following steps:

[0087] B1. Decompose the task profile from different angles and determine the subtask. The subtask is a task of a certain action link of the selected equipment. The subtask can be selected and determined from the target task profile library accumulated daily, or a new task profile and subtask can be established.

[0088] B2. According to the nature of the subtask, analyze the capabilities required to complete the subtask and select the capabilities and their levels required. The capabilities and their levels can be selected and determined from the basic capability library accumulated daily, or completely new capability items and capability levels can be established.

[0089] B3. Based on the capability requirements, construct evaluation index items measured by qualitative grades or quantitative values. The evaluation index items have three characteristics: first, they can be measured by qualitative grades or quantitative values; second, they directly reflect the capability requirements in step B2; and third, there is as little or no correlation between the evaluation index items as possible.

[0090] Similarly, the whole process of constructing the performance evaluation indicator system based on capability level is as follows: Figure 3 It is a tree-like parsing process with gradual branching and expansion.

[0091] According to the above steps (2) and (3), we obtain the complete set of performance evaluation index system A based on target tasks and the complete set of performance evaluation index system B based on capability level. The following is the processing and selection of specific index system sets. The specific method is combined with Figure 4 Instructions include the following steps.

[0092] (3) According to the vehicle equipment and its performance evaluation target of the newly established target evaluation task item, select the relevant evaluation indicators that belong to the performance evaluation indicator set A based on the target task and / or the performance evaluation indicator system set B based on the capability level; and filter out the corresponding evaluation indicators in the indicator set to obtain the application project performance evaluation indicator system that is compatible with the vehicle equipment and its performance evaluation target. If a customized evaluation indicator is selected in the performance evaluation indicator system set A based on the target task, the corresponding evaluation indicator is filtered out by selecting the task action; if a customized evaluation indicator is selected in the performance evaluation indicator system set B based on the capability level, the corresponding evaluation indicator is filtered out by selecting the capability requirement.

[0093] During the operation, the customized evaluation indicators are repeatedly selected according to the evaluation indicator task items, and the corresponding evaluation indicators are filtered and limited, so as to obtain the performance evaluation indicator set of the application project corresponding to the specific evaluation indicator task and adapted to the performance evaluation target. The application project performance evaluation indicator set includes any one of the performance evaluation indicator set A based on the target task and the performance evaluation indicator system set B based on the capability level, or a mixture of the two indicator system sets.

[0094] In the design stage of the indicator system, the method of the present invention takes the vehicle equipment evaluation task list and equipment task set as the basis, and constructs a task-based evaluation indicator system from top to bottom according to the mission-task hierarchical relationship of "task-condition-attribute-indicator"; and focuses on the equipment task and capability decomposition, and constructs a capability-based evaluation indicator system from top to bottom according to the equipment capability hierarchical relationship of "task-capability-indicator-condition"; and displays and manages the evaluation indicator system from two aspects: task-based and capability-based.

[0095] It should be noted that in order to ensure the implementation of the indicator system design and the following evaluation plan design, it is necessary to collect and analyze the original data and basic information in the early stage. The specific operations include: inputting a standard data dictionary to form a data dictionary table, designing an evaluation report template to form an evaluation template library, entering equipment type data to form an equipment type table, entering combat personnel to form a combat personnel table, entering environmental condition categories to form an environmental condition table, analyzing, decomposing, and entering combat task lists to form a combat entry table, and analyzing, decomposing, and entering equipment tasks to form an equipment task set table; at the same time, the above forms need to be maintained and managed.

[0096] In some improved examples, the present invention further optimizes and improves the system based on the above-mentioned performance evaluation index system, and the specific scheme is described as follows.

[0097] Figure 5 In the method flow shown, after step (3), the following steps are further included:

[0098] (4) According to the task purpose, task focus and indicator contribution of the target evaluation task item, weights are assigned to the evaluation indicators at all levels in the application project effectiveness evaluation indicator system through expert scoring and system calculation methods.

[0099] Among them, the weighting of the evaluation indicators at all levels in the application project effectiveness evaluation indicator system through expert scoring and system calculation methods specifically includes the following methods:

[0100] Expert direct scoring method: Experts use the "Expert Empowerment Scoring Table" to understand the evaluation index system and score each evaluation index based on their experience; the system checks the expert scoring to check whether the scoring result input is correct, whether there are any missing items, etc.; and calculates the expert scoring to obtain the final scoring result calculation value of each index.

[0101] Fuzzy ranking method: Experts use the "Expert Weighted Scoring Table" to understand the evaluation index system and judge the importance of each evaluation index to performance based on their experience. It can generally be expressed as an integer from 1 to 9, but if there are too many indicators, it can exceed 9.

[0102] Analytical Hierarchy Method: It is to decompose the problem into different components according to the nature of the problem and the overall goal to be achieved, and to cluster and combine the factors at different levels according to the interrelationships and affiliations between the factors, so that the problem is ultimately reduced to the determination of the relative importance of the lowest-level evaluation indicator items relative to the highest-level performance evaluation overall goal, and this is used as the ranking standard for the weights.

[0103] The steps include:

[0104] 1. The chromatography analysis method comprises:

[0105] 1) Relying on the performance evaluation index system and the determined evaluation indicators, establish a hierarchical structure model of the evaluation index system for the corresponding project;

[0106] 2) Constructing a judgment matrix based on the hierarchical structure model of the evaluation index system and generating a comparison matrix; specifically including the following steps;

[0107] 2.1) Select the level of constructing the judgment matrix, and divide all evaluation indicators into three levels, first select the high level, then the middle level, and finally the low level;

[0108] 2.2) Select the reference target O of this layer. The optional targets of different levels are automatically listed by the system for users to choose; after the user selects a target O, the target predetermined weight and the corresponding number of subordinate indicators are automatically given by the system for users to refer to. In this process, a judgment matrix needs to be constructed for each target. Therefore, no matter which level, a judgment matrix must be constructed for each optional target listed by the system.

[0109] 2.3) Construct the element values ​​aij of the comparison matrix. The user selects two indicators for constructing the comparison matrix from the options of indicator i and indicator j, and inputs the importance coefficient scale (1-9) of the two indicators.

[0110] In an example, the scaling method of the judgment matrix is ​​as follows: Scale 1 indicates that two factors are equally important; Scale 3 indicates that one factor is slightly more important than the other factor; Scale 5 indicates that one factor is significantly more important than the other factor; Scale 7 indicates that one factor is strongly more important than the other factor; Scale 9 indicates that one factor is extremely more important than the other factor; Scales 2, 4, 6, and 8 indicate the median value between two adjacent judgments.

[0111] After inputting the comparison values ​​of the two indicators, the system saves and generates the elements aij and aji of the comparison matrix.

[0112] An example is used to describe the specific method: for example, compared with ai and aj, indicator i is obviously more important than indicator j, that is, ai:aj=5, which means that when the importance coefficient of ai is 5, aj is only 1. At this time, you only need to enter 5 in the aij column, and aji will automatically get 1 / 5, that is, 0.2.

[0113] On the contrary, if ai:aj=1 / 5, it means that when the importance coefficient of ai is 1, aj is 5. At this time, you only need to enter 5 in the aji column, and aij will automatically become 1 / 5, that is, 0.2.

[0114] 2.4) The system performs the comparison matrix operation of the selected level and analyzes and obtains the weights of the relevant indicators of the selected level. After the analysis and operation of the selected level is completed, it may not pass the consistency test. The user can choose to accept or reject the result according to the situation. If the calculation result is not accepted, it is necessary to modify the element values ​​of the comparison matrix again and calculate again.

[0115] 2.5) According to the above steps, return to step 1.2), select another reference target O, and repeat steps 2.2) to 2.4) until all reference targets at the selected level are analyzed;

[0116] 3) Rely on the weight calculation method to calculate the weights of the hierarchical single ranking and the total ranking and verify their consistency. In this step, the weight calculation and verification calculation are completed by the background calculation model of the system. Therefore, here, the weighting process of the hierarchical analysis method is mainly the operation process of the user constructing the judgment matrix.

[0117] Entropy weight method: According to the degree of variation of each evaluation indicator, the entropy weight of each indicator is calculated using information entropy, and then the weight of each indicator is corrected by entropy weight to obtain a more objective indicator weight.

[0118] (5) According to the data item categories of the basic evaluation indicators in the application project performance evaluation indicator system, the indicator normalization model and the indicator aggregation model are selected and edited based on the set evaluation model library to form the performance evaluation model system of the target task.

[0119] The indicator normalization model is one of the most basic and important models of the evaluation system. It has two main functions: first, it judges each indicator according to the evaluation rules to obtain the evaluation score of the indicator; second, it normalizes the judged indicator score to a standard scale. By unifying the benchmark scales of many indicators, it can ensure that the score of each evaluation indicator can be aggregated to form the total value of the performance evaluation. In the present invention, the system has several types of built-in general indicator normalization models, which can cover the normalization of all performance evaluation indicators; at the same time, it can also support users to define new normalization models. When the user creates a new indicator normalization model, it should exist in the form of DLL, which can include one or more functions.

[0120] (6) Based on the data collection requirements of the basic evaluation indicators in the application project performance evaluation indicator system, combined with the vehicle equipment and the environmental conditions and properties in which it exists, design and generate an indicator data collection form.

[0121] The data collection form is used to enter the collected sampling data corresponding to the indicator system into the system to support performance evaluation; the content that needs to be collected mainly includes the following columns:

[0122] {Equipment Category}: refers to the category of the equipment under test in the test, that is, the category of the equipment under test collected.

[0123] {Equipment Name}: refers to the name of the test equipment in the test, that is, the name of the test equipment collected.

[0124] {Experimental subject}: refers to the name of the experimental subject in the experiment, that is, the name of the experimental subject for which experimental data collection is being carried out.

[0125] {Personnel Number}: refers to the number of the operator who uses the test equipment in the test, that is, the number of the operator who uses the test equipment being collected.

[0126] {Collection start time}: refers to the starting astronomical time when the test data collection starts using this watch.

[0127] {Collection end time}: refers to the astronomical end time for starting to collect test data using this watch.

[0128] {Sampling period}: refers to whether the main period for collecting test data using this table is daytime or nighttime.

[0129] {Indicator Name}: Lists the name of the evaluation indicator, which is also the main reference item for data collection.

[0130] {Data type}: is the data type of the sampling data corresponding to this indicator. It must be collected according to the specified type, and the collection results must be filled in the test collection result column corresponding to the data type.

[0131] {Dimension}: It is the dimension of the test result data. It is important to note that the collected data value should be filled in according to the dimension specified in the table. If the time is tight on site and the dimension needs to be converted, you can check and correctly fill in the data according to the dimension specified in the table before submitting the table.

[0132] {Test collection results}: Test collection results description includes the following three methods:

[0133] When the {Data Type} of the indicator is "Fixed Value Interval" or "Threshold Interval", the collection result is a numerical value and is filled in the {Quantitative Result} column.

[0134] When the {Data Type} of the indicator is "Qualitative Grade", select one of the four grades of collection results: excellent, good, medium, and poor, and check one of the corresponding grades () in the {Qualitative Grade} column.

[0135] When the {Data Type} of the indicator is "Logical Binary Value", the collection result is to select one of the two values ​​​​(Yes) and No). Just check one of the two values ​​​​(Yes or No) in the {Logical Binary Value} column.

[0136] (7) Perform performance evaluation calculations based on the performance evaluation model system, evaluation rule algorithms, and collected indicator data of the target task, obtain evaluation calculation result data, and perform statistical analysis to form an evaluation report;

[0137] The evaluation rule algorithm includes: weighting and aggregation calculation of indicator data after normalization; and comprehensive processing and membership calculation of multiple sampling data of fixed value interval, threshold interval, qualitative grade and logical binary type indicator of single indicator data.

[0138] On the basis of the above-mentioned necessary rule algorithms, the evaluation rule algorithms also include one or more of the following:

[0139] Automatically reduce the weight of performance evaluation indicators that have no data support and distribute the reduced weight to other indicators;

[0140] Automatically sort out the weights of indicators at all levels above the grassroots level based on the weighting results of grassroots indicators in the performance evaluation indicators;

[0141] Carry out correlation and resolution calculations between the indicators in the performance evaluation indicator system;

[0142] Identify the correlation between performance evaluation indicators and aggregate similar indicators.

[0143] The evaluation report analysis mainly displays and analyzes the evaluation result data effectively according to different needs and purposes. The indicator levels of the analysis process include the following four levels:

[0144] The most basic evaluation indicators refer to the last level indicators in the indicator system, and are also indicators that are directly connected to the evaluation data items and evaluation models. They are used to analyze and view the evaluation results of each basic indicator and the overall comparison.

[0145] The attribute and capability level indicators refer to the capabilities and attributes of the last level indicators in the indicator system, and are also the superior attributes or capabilities of the basic level indicators. They are used to analyze and view the evaluation results and overall comparison of various capabilities and attributes.

[0146] Specific task-level indicators refer to the tasks or subtasks to which the last-level indicators in the indicator system belong, and are also the superior indicators of the basic indicators. They are used to analyze and view the evaluation results and overall comparison of various tasks or subtasks.

[0147] Action-level indicators refer to the actions or tasks to which the last-level indicators in the indicator system belong, and are also the superior indicators of the basic-level indicators. They are used to analyze and view the evaluation results and overall comparison of various task actions or task links.

[0148] The evaluation result data obtained by analysis and calculation include weighted results and unweighted results. The weighted results are evaluation result data that have been weighted according to the assigned weights. The unweighted results are evaluation result data that have not been weighted. Which of the two types of data to analyze can be selected according to the needs of the evaluator.

[0149] On the other hand, the present invention also provides a vehicle equipment evaluation index system construction system, such as Figure 6 As shown, the system comprises:

[0150] A target task effectiveness evaluation index construction module is configured to obtain each task action phase and its existing environmental conditions and attributes according to the category or subcategory of the vehicle equipment evaluation task, and construct a full set A of effectiveness evaluation indicators based on the target task according to these parameters;

[0151] The capability level effectiveness evaluation indicator construction module is configured as the task profile of the vehicle equipment evaluation task and establishes subtasks, and analyzes the required capabilities of the subtasks to construct a complete set of effectiveness evaluation indicator systems based on capability levels B;

[0152] The effectiveness evaluation index customization module is configured to customize the evaluation index based on the vehicle equipment of the newly established target evaluation task item and its effectiveness evaluation target, and select the evaluation index related thereto to belong to the effectiveness evaluation index set A based on the target task and / or the effectiveness evaluation index system set B based on the capability level;

[0153] The performance evaluation index filtering module is configured to filter out corresponding evaluation indicators according to the selected performance evaluation index set;

[0154] The application project effectiveness evaluation indicator generation module is configured to select and determine corresponding indicator items from the filtered evaluation indicator set to obtain an application project effectiveness evaluation indicator set that is compatible with the vehicle equipment and its effectiveness evaluation objectives; wherein the application project effectiveness evaluation indicator system includes any one of the effectiveness evaluation indicator set A based on the target task and the effectiveness evaluation indicator system set B based on the capability level, or a mixture of the two indicator system sets.

[0155] In an improved example, Figure 7 As shown, the system also includes:

[0156] A weight assignment module is configured to assign weights to evaluation indicators at all levels in the application project effectiveness evaluation indicator system according to the task purpose, task focus and indicator contribution of the target evaluation task item through expert scoring and system calculation methods;

[0157] A data collection form generation module is configured to design and generate an indicator data collection form according to the demand for data collection of the basic evaluation indicators in the application project effectiveness evaluation indicator system, combined with the vehicle equipment and the environmental conditions and properties in which it exists;

[0158] An evaluation model system construction module is configured to select and edit an indicator normalization model and an indicator aggregation model based on the data item category of the evaluation base indicators in the application project performance evaluation indicator system and the set evaluation model library to form a performance evaluation model system for the target task;

[0159] An indicator data item docking module is configured to dock the evaluation indicator items, indicator data items with the normalization model and the indicator aggregation model to form an effectiveness evaluation task plan;

[0160] The performance evaluation calculation module is configured to perform performance evaluation calculation on the performance evaluation index system, evaluation rule algorithm and collected index data in the performance evaluation task plan; and

[0161] The performance evaluation result analysis module is configured to perform statistical analysis based on the calculation result data to form an evaluation report; the evaluation report includes the analysis of the following indicator levels: the most basic evaluation indicators, attribute capability level indicators, specific task level indicators and action level indicators.

[0162] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0163] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, including several programs to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0164] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for evaluating the performance of vehicle equipment, the method comprising the following steps: (1) Obtain the environmental conditions and attributes of each task action phase and its application according to the category or subcategory of the vehicle equipment evaluation task, and construct a complete set of effectiveness evaluation indicators A based on the target task based on these parameters; (2) Decompose the task profile of the vehicle equipment assessment task and establish subtasks, analyze the capabilities required by the subtasks, and build a complete set of effectiveness assessment indicator systems based on capability levels B; (3) According to the vehicle equipment and its effectiveness evaluation target of the newly established target evaluation task item, the relevant evaluation indicators are selected from the effectiveness evaluation indicator set A based on the target task and / or the effectiveness evaluation indicator system set B based on the capability level, and the corresponding evaluation indicators are filtered out from the indicator set to obtain the application project effectiveness evaluation indicator system that is compatible with the vehicle equipment and its effectiveness evaluation target; The application project effectiveness evaluation index system includes any one of the effectiveness evaluation index set A based on target tasks and the effectiveness evaluation index set B based on capability levels, or a mixture of the two index system sets; (4) According to the task purpose, task focus and indicator contribution of the target evaluation task item, weights are assigned to the evaluation indicators at all levels in the application project effectiveness evaluation indicator system through expert scoring and system calculation methods; (5) selecting and editing an indicator normalization model and an indicator aggregation model based on the set evaluation model library according to the data item categories of the evaluation basic indicators in the application project effectiveness evaluation indicator system, so as to form an effectiveness evaluation model system for the target task; (6) Designing and generating an indicator data collection form based on the data collection requirements of the basic evaluation indicators in the application project effectiveness evaluation indicator system and in combination with the environmental conditions and properties of the vehicle equipment and its application; (7) Connect the performance evaluation model system of the target task, the evaluation rule algorithm and the indicator data items of the indicator data collection form, perform performance evaluation calculations, obtain the evaluation calculation result data, and perform statistical analysis to form an evaluation report; Wherein, the step (1) specifically includes the following steps: A1. According to the specific evaluation items, select or directly create new target tasks in the target task library to complete the determination of performance evaluation target tasks; A2. Select or directly create and determine the category or subcategory of the target task in the target task category library; A3. Select or directly establish and determine each task action phase of the target task in the target task action library; A4. Determine the environmental conditions and related attributes applied in each task action stage. The environmental conditions include the natural geographical environment, meteorological and hydrological environment, humanistic and social environment, economic and political environment and conditions associated with the user's task; the related attributes refer to the attribute values ​​or levels of the task objectives; A5. Based on each task action phase, environmental conditions and related attributes, construct evaluation indicator items measured by qualitative grades or quantitative values; In step A2, when the target task is selected from the target task library, the system also automatically recommends the target task category of the corresponding stage; in step A3, when the target task is selected from the target task library, the system also automatically recommends the task action stage of the corresponding stage.

2. The vehicle equipment performance evaluation method according to claim 1, characterized in that: The step (2) specifically comprises the following steps: B1. Decompose the task profile from different angles, select and determine subtasks in the target task profile library, or create new task profiles and subtasks; B2. Analyze the capabilities required to complete the subtasks according to their nature, select the capabilities and their levels required from the basic capability library, or directly establish new capability items and capability levels; B3. Based on the requirements of the capabilities and their levels, construct evaluation indicator items measured by qualitative grades or quantitative values.

3. The vehicle equipment performance evaluation method according to claim 1, characterized in that: In the step (3), the corresponding evaluation indicators are filtered out from the entire set of indicators: If a customized evaluation indicator is selected from the full set A of the effectiveness evaluation indicator system based on the target task, the corresponding evaluation indicator is filtered out by selecting the task action; If a customized evaluation indicator is selected from the complete set B of the effectiveness evaluation indicator system based on capability level, the corresponding evaluation indicator is filtered out by selecting the capability requirement.

4. The vehicle equipment performance evaluation method according to claim 1, characterized in that: The step (4) of assigning weights to the evaluation indicators at all levels in the application project performance evaluation index system by expert scoring and system calculation methods specifically includes: an expert direct scoring method, a fuzzy ranking method of the degree of performance importance based on the understanding of the expert weighted scoring experience, a hierarchical analysis method for constructing a judgment matrix, and an entropy weight method weighting method; The hierarchical analysis method includes: 1) Relying on the application project effectiveness evaluation index system and the determined evaluation indicators, establish a hierarchical structure model of the evaluation index system for the corresponding project; 2) Constructing a judgment matrix based on the hierarchical structure model of the evaluation index system and generating a comparison matrix; specifically including the following steps; 2.1) Select the level of constructing the judgment matrix, divide all evaluation indicators into three levels, first select the high level, then the middle level, and finally the lower level; 2.2) Set the reference target O of this layer, and the target predetermined weight and the corresponding number of subordinate indicators are automatically given by the system; construct a judgment matrix for each target; 2.3) Construct the element values ​​aij of the comparison matrix, select two indicators for constructing the comparison matrix from the options of indicator i and indicator j, and input the importance coefficient scales of the two indicators; 2.4) Perform comparative matrix operations on the selected level and analyze and obtain the weights of the relevant indicators at the selected level; 2.5) According to the above steps, return to step 1.2, select another reference target O, and repeat steps 2.2 to 2.4 until all reference targets at the selected level are analyzed; 3) Rely on the system weight calculation method to calculate the weights of the hierarchical single sorting and total sorting and check their consistency.

5. The vehicle equipment performance evaluation method according to claim 1, characterized in that: The evaluation rule algorithm in step (7) includes: weighting and aggregation calculation of the indicator data after normalization; and comprehensive processing and membership calculation of multiple sampling data of the fixed value interval, threshold interval, qualitative grade, and logical binary type indicator of the single indicator data; The evaluation report includes analysis to obtain the following indicator levels: basic evaluation indicators, attribute capability level indicators, specific task level indicators and action level indicators.

6. The vehicle equipment performance evaluation method according to claim 5, characterized in that: The evaluation rule algorithm also includes one or more of the following: Automatically reduce the weight of performance evaluation indicators without data support and distribute the reduced weight to other indicators; Automatically sort out the weights of indicators at all levels above the grassroots level based on the weighting results of grassroots indicators in the performance evaluation indicators; Carry out correlation and resolution calculations between various indicators in the application project effectiveness evaluation indicator system; Identify the correlation between performance evaluation indicators and aggregate similar indicators.

7. A system for constructing a vehicle equipment performance evaluation system using the method described in any one of claims 1 to 6, characterized in that: The system comprises: A target task effectiveness evaluation index construction module is configured to obtain environmental conditions and attributes of each task action phase and its application according to the category or subcategory of the vehicle equipment evaluation task, and to construct a full set of effectiveness evaluation indicators A based on the target task according to these parameters; The capability level effectiveness evaluation indicator construction module is configured as the task profile of the vehicle equipment evaluation task and establishes subtasks, and analyzes the required capabilities of the subtasks to construct a complete set of effectiveness evaluation indicator systems based on capability levels B; The effectiveness evaluation index customization module is configured to customize the evaluation index based on the vehicle equipment of the newly established target evaluation task item and its effectiveness evaluation target, and select the evaluation index related thereto to belong to the effectiveness evaluation index set A based on the target task and / or the effectiveness evaluation index system set B based on the capability level; The performance evaluation index filtering module is configured to filter out corresponding evaluation indicators according to the selected performance evaluation index set; The application project effectiveness evaluation indicator generation module is configured to select and determine corresponding indicator items from the filtered evaluation indicator set to obtain an application project effectiveness evaluation indicator set that is compatible with the vehicle equipment and its effectiveness evaluation objectives; wherein the application project effectiveness evaluation indicator system includes any one of the effectiveness evaluation indicator set A based on the target task and the effectiveness evaluation indicator system set B based on the capability level, or a mixture of the two indicator system sets.

8. The vehicle equipment performance evaluation system construction system according to claim 7, characterized in that: The system further comprises: A weight assignment module is configured to assign weights to evaluation indicators at all levels in the application project effectiveness evaluation indicator system according to the task purpose, task focus and indicator contribution of the target evaluation task item through expert scoring and system calculation methods; A data collection form generation module is configured to design and generate an indicator data collection form according to the demand for data collection of the basic evaluation indicators in the application project effectiveness evaluation indicator system, combined with the vehicle equipment and the environmental conditions and properties in which it exists; An evaluation model system construction module is configured to select and edit an indicator normalization model and an indicator aggregation model based on the data item category of the evaluation base indicators in the application project performance evaluation indicator system and the set evaluation model library to form a performance evaluation model system for the target task; An indicator data item docking module is configured to dock the evaluation indicator items, indicator data items with the normalization model and the indicator aggregation model to form an effectiveness evaluation task plan; The performance evaluation calculation module is configured to connect the application project performance evaluation index system, the evaluation rule algorithm and the collected index data to perform performance evaluation calculation; and The performance evaluation result analysis module is configured to perform statistical analysis based on the calculation result data to form an evaluation report; the evaluation report includes the analysis of the following indicator levels: the most basic evaluation indicators, attribute capability level indicators, specific task level indicators and action level indicators.

Images