Evaluation method for aviation emergency simulation rescue comprehensive training

Abstract

The invention relates to an evaluation method for aviation emergency simulation rescue comprehensive training, belongs to the technical field of emergency rescue task training evaluation, and solves the problem that in the prior art, aviation emergency rescue training lacks multi-role collaborative comprehensive evaluation. A task process training evaluation model is established and used for executing task completion condition evaluation, including individual completion condition evaluation and unit completion condition evaluation, and individual completion condition evaluation and unit completion condition evaluation are obtained; s2, establishing a task cooperative training evaluation model for obtaining a unit cooperative training evaluation value; and S3, obtaining task training parameters through VR training, providing the task training parameters to the established task process training evaluation model and the task cooperative training evaluation model, obtaining individual completion condition evaluation, unit completion condition evaluation and unit cooperative training evaluation values as evaluation results, and providing the evaluation results to an analysis process and a decision process.

Description

Technical Field

[0001] This invention relates to the field of emergency rescue mission training and evaluation technology, specifically to an evaluation method for comprehensive aviation emergency simulation rescue training. Background Technology

[0002] In recent years, my country has frequently experienced natural disasters such as fires, floods, and earthquakes. Timely and effective aviation emergency rescue can greatly reduce casualties and property losses caused by accidents. Aviation emergency rescue missions involve complex environments and intricate flight procedures, requiring high levels of coordination and cooperation between crews, multiple aircraft, and between air and ground operations. Crew members need to receive appropriate training before actually carrying out rescue missions. Effective training requires scientific assessment and timely feedback to help trainees improve their emergency rescue capabilities and refine their emergency rescue strategies.

[0003] Currently, the main forms of aviation emergency rescue training include theoretical lectures, simulation training, practical training, and joint drills. Corresponding assessment methods include questionnaires, examinations, and practical assessments. Written or online tests can evaluate trainees' grasp of basic aviation emergency rescue knowledge; questionnaires can provide feedback on training content, methods, and effectiveness; and trainees' performance in virtual and real-world scenarios can assess their skill mastery. Ultimately, the goal is to reduce crew errors and lack of proficiency in aviation emergency rescue missions and minimize casualties from disasters.

[0004] However, the above training and evaluation methods have certain limitations. Common flight simulation training and evaluation mainly focus on flight operations and lack the functions of multi-role collaborative training and comprehensive evaluation.

[0005] Chinese invention patent application, publication number CN113537738A, entitled "A Method for Evaluating the Effectiveness of Aviation Emergency Rescue by Integrating Multiple Cycles and Constraints," discloses a method that determines the weights of indicators based on mission training requirements, then determines the cycle probability and cycle difficulty coefficient based on mission completion, and obtains the evaluation values ​​of five indicators to arrive at the evaluation result. However, this method does not consider that the scale and complexity of scenarios involved in different levels of missions vary.

[0006] There is a need in this field for a comprehensive evaluation method for multi-role training in aviation emergency rescue missions, so as to facilitate effective collaborative training for aviation emergency rescue missions. Summary of the Invention

[0007] In view of the above problems, the present invention provides an evaluation method for comprehensive training of aviation emergency simulation rescue, which can be used for collaborative training of aviation emergency rescue missions, reflecting both mission execution operations and crew collaboration.

[0008] An evaluation method for comprehensive aviation emergency simulation rescue training provided according to an embodiment of the present invention includes the following steps: Step S1: Based on the pre-set processes and training points in the task flow, as well as the crew members in the task flow, establish a task flow training and evaluation model for evaluating task completion. This includes establishing the weights of each process based on the operational difficulty of each process and the degree of influence of each process on the task result; and conducting individual and crew completion evaluations based on the process weights and statistical data from the training process to obtain individual and crew completion evaluations. Step S2: Establish a task collaborative training evaluation model to obtain crew collaborative training evaluation values. This includes determining the primary indicators of the task and the secondary indicators included in each primary indicator. The primary indicators include reporting, evaluation, and execution. The weights of reporting, evaluation, and execution of the primary indicators are determined using the analytic hierarchy process (AHP), as well as the weights of the secondary indicators included in each primary indicator. The evaluation indicator scoring results are obtained. Based on the weights of each primary indicator, the weights of each secondary indicator included in each primary indicator, and the evaluation indicator scoring results, the crew collaborative training evaluation score and the scores of each process in the task flow of the training process are obtained. Step S3: Obtain task training parameters through VR training, and provide them to the established task process training evaluation model and task collaboration training evaluation model. The individual performance evaluation, crew performance evaluation, and crew collaboration training evaluation values ​​are obtained as evaluation results, which are then provided to the analysis and decision-making process of aviation emergency rescue collaboration training.

[0009] Optionally, step S1 specifically includes: Step S1.1: Based on the pre-set processes and training points in the mission flow, establish the weights of each process using the analytic hierarchy process (AHP). This includes setting the crew in the mission flow as captain, co-pilot, flight engineer, bucket safety officer, and flight observer; setting four training processes as fire reconnaissance, water source collection, water spraying for firefighting, and return to base; setting two factors: operational difficulty and impact on results; comparing each of the two factors and the four training processes pairwise with each of the two factors as a premise to construct a judgment matrix, including a criterion-level judgment matrix, an operational difficulty judgment matrix, and an impact on results judgment matrix; and performing consistency checks, weighted calculations, and normalization on the judgment matrices to obtain the weights of each process. Step S1.2: Establish an individual performance evaluation. Based on the weights of each process and the statistical data of the training process, obtain the individual training point completion rate and individual stage score as the evaluation of individual performance. Step S1.3: Establish a unit completion assessment. Based on the weights of each process and the statistical data of the training process, obtain the unit training point completion rate and unit stage score as the evaluation of unit completion.

[0010] Optionally, step S1 specifically includes: Step S1.1: Based on the pre-set processes and training points in the mission flow, establish the weights of each process using the analytic hierarchy process (AHP). This includes setting the crew in the mission flow as captain, co-pilot, winch operator, and lifeguard; setting four training processes as five processes: heading to the rescue point, airdropping supplies, rescuing people in distress, and returning to base. Using the first four processes as indicators, and comprehensively considering factors affecting the outcome, the weights of each process are obtained by solving the AHP problem. Step S1.2: Establish an individual performance evaluation. Based on the weights of each process and the statistical data of the training process, the individual training point completion rate and individual stage score are obtained as the evaluation of individual performance. Step S1.3: Establish a unit completion assessment. Based on the weights of each process and the statistical data of the training process, obtain the unit training point completion rate and unit stage score as the evaluation of unit completion.

[0011] Optionally, step S1.2 specifically includes: Completion rate of individual training points of members in the task process and individual stage scores :

[0012]

[0013] in, For the index of the process of the task and , The total number of processes included in the rescue mission. Indexes to the members of the group; processes Members The number of relevant training points and progress of the correct operation. Zhongyu The number of relevant training points is obtained by counting based on the preset training point content; Based on the individual training point completion rate and individual stage score, an evaluation of individual performance is obtained:

[0014] in, For process The weight.

[0015] Optionally, step S1.3 specifically includes: Crew training point completion rate during the construction task process and unit stage score :

[0016]

[0017] Among them, process The number of training points and progress that were correctly operated in the process The number of training points included is obtained by counting based on the preset training point content; Based on the obtained crew training point completion rate and crew phase score, the crew completion status is evaluated as follows: 。

[0018] Optionally, step S2 specifically includes: Step S2.1: Determine the primary indicators of the task and the secondary indicators included in each primary indicator. The primary indicators include three items: reporting, evaluation, and execution. Step S2.2: Determine the weights of the primary indicators reporting, evaluation, and execution, as well as the weights of the secondary indicators included in reporting, evaluation, and execution, respectively, using the analytic hierarchy process (AHP). Step S2.3: Obtain the scores of the secondary indicators under the primary indicators in each process of the task, and obtain the crew collaborative training evaluation value.

[0019] Optionally, in step S2.1: The report, as a primary indicator, includes the following secondary indicators: clarity of reporting, accuracy of information, and collaborative content. As a primary indicator, the evaluation includes the following secondary indicators: the ability to formulate solutions, the rationality of the selection, and the decisiveness of decision-making. The execution of primary indicators includes the following secondary indicators: correctness of execution and decisiveness of execution.

[0020] Optionally, step S2.3 specifically includes: The scores of each secondary indicator under the primary indicator in each process of the task are obtained. ,in, Represents the index of the process and , For the index of the first-level indicator and , For the index of secondary indicators and , Indicates the first-level indicator The number of sub-secondary indicators; The following evaluation values ​​were obtained for the crew collaborative training:

[0021] in, This represents the total number of processes included in the task. The first step for instructors to complete the scoring The weights corresponding to each secondary indicator Indicates the first The weight of each primary indicator.

[0022] Optionally, step S2.3 further includes: Based on the scores of the secondary indicators under the primary indicators in each process of the task, the scores of each process in the task flow are obtained: 。

[0023] Compared with existing technologies, the evaluation method for comprehensive aviation emergency simulation rescue training provided by the embodiments of the present invention has at least the following beneficial effects: considering both proficiency in task procedures and crew collaboration, the completion status of training points reflects the task completion capabilities of trainees and crews, and the RAD (Report, Evaluate, Execute) assessment method scores the crew's collaborative cooperation during the task; this method can quantitatively and accurately evaluate rescue training, and through scores such as training point completion, task progress score, and average RAD score of task progress, it reflects weak task links and weaknesses in collaborative cooperation, enabling targeted learning and practice. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The RAD model structure is provided in the evaluation method for comprehensive training of aviation emergency simulation rescue according to an embodiment of the present invention.

[0026] Figure 2 A hierarchical model for evaluating helicopter firefighting mission training is provided in an embodiment of an evaluation method for comprehensive aviation emergency simulation rescue training according to an embodiment of the present invention.

[0027] Figure 3 A hierarchical structure model for evaluating helicopter flood rescue mission training is provided in an embodiment of an evaluation method for comprehensive aviation emergency simulation rescue training according to an embodiment of the present invention. Detailed Implementation

[0028] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0029] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein. Therefore, the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0030] The following detailed description, with reference to the accompanying drawings, illustrates an evaluation method for comprehensive aviation emergency simulation rescue training according to an embodiment of the present invention. This invention provides an evaluation method for comprehensive aviation emergency simulation rescue training, serving as a general assessment method for aviation emergency rescue training in disasters such as forest fires and floods. Compared to traditional personnel training assessment methods applied to aviation emergency rescue, this method simultaneously considers task completion and crew coordination, reflecting aspects that both trainees and crew need improvement.

[0031] like Figure 1 As shown, an evaluation method for comprehensive aviation emergency simulation rescue training provided according to an embodiment of the present invention includes the following steps.

[0032] Step S1 involves establishing a task flow training and evaluation model based on the pre-set processes and training points in the task flow, and performing task content evaluation, including individual performance evaluation and crew performance evaluation. To improve the crew's ability to complete tasks, the task processes in the rescue mission, divided according to time sequence and key task areas, can be used as indicators. Taking into account both operational difficulty and impact on task results, the weights of each process are calculated using the Analytic Hierarchy Process (AHP), which represents the weight values ​​of the training points within that process. Emergency rescue task training points are set according to the order of task execution. Each process in the task flow includes a certain number of training points, and each training point involves different execution members. The total number of training points is independent of the number of roles. Step S1 specifically includes the following steps.

[0033] Step S1.1: Based on the pre-set processes and training points in the task flow, establish the weights of each process using the Analytic Hierarchy Process (AHP). In this step, the weight values ​​of each process can be obtained using AHP according to the set processes, and used for subsequent score calculation. Specifically, in this step, the process of calculating the process weights using AHP involves first comparing two factors: operational difficulty and impact on task results. Then, considering these two factors, pairwise comparisons are made between the four training processes, that is, pairwise comparisons are made between the operational difficulty of the four training processes and between the degree of impact of the four training processes on the task results. Processes and training points can be set according to the order of task execution. Each process in the task flow includes a certain number of training points, and each training point involves different execution members.

[0034] Step S1.2: Establish an individual performance evaluation to obtain an individual performance assessment.

[0035] The individual training point completion rate and individual stage score of each member in each process of the task construction:

[0036]

[0037] in, For process index and , This represents the total number of processes contained in the task. Index the members of the unit. For the first In the process of the first Individual training completion rate of each member For the first In the process of the first Individual stage scores for each member. Among them, process... Members The number of relevant training points and progress of the correct operation. Zhongyu The number of relevant training points can be obtained by counting the preset training point content, which is a statistical data of the training process.

[0038] Based on individual training point completion rate and individual stage score, the individual performance evaluation of each member of the crew is obtained. :

[0039] in, For the first The weight of each process, This represents the total number of processes involved in the rescue mission. The weight of each process can be derived from the weight values ​​of each process obtained in the previous step S1.2.

[0040] The individual performance evaluation obtained can be used as an individual score.

[0041] Step S1.3: Establish a unit performance assessment to obtain an evaluation of the unit's performance.

[0042] In each stage of the task construction process, the crew training point completion rate and the crew phase score are as follows:

[0043]

[0044] in, Indicates the first The completion rate of the crew training points in each process. Indicates the first The unit stage score for each process. The number of training points that were correctly executed in each process and the total number of training points in each process can be obtained by counting based on the preset training point content and the training completion status.

[0045] Based on the obtained crew training point completion rate and crew phase score, the crew completion status is evaluated as follows: 。

[0046] The evaluation of the unit's performance can be used as the unit's score.

[0047] Step S2: Establish a task collaboration training evaluation model to obtain crew collaboration training evaluation values. The task collaboration training evaluation score is obtained by judging whether the communication and collaboration among personnel are appropriate under different situations.

[0048] Step S2.1: Determine the primary indicators of the task and the secondary indicators included in each primary indicator. In this implementation, the primary indicators include three items: reporting, evaluation, and execution. The secondary indicators included in the primary indicators can be set in the following manner.

[0049] like Figure 1As shown, for hierarchical, multi-scenario team tasks, the scale and complexity of the scenarios involved in different levels of tasks vary, but they all share the characteristic of sequential task training and the need for team collaboration. Therefore, tasks at each level are uniformly divided into three stages: Report, Assessment, and Do, which form a loop. The RAD (Report, Assessment, Do) model is not only the minimum loop for completing a task but also an important indicator for evaluating member capabilities. This serves as a primary indicator to establish a mapping relationship between the member's task execution process and the assessment indicator. The RAD method divides the process of the crew responding to external changes into three stages: Report, Assessment, and Do. That is, any operation performed by the crew includes reporting (having a shared understanding of changes in the task environment), assessment (generating multiple alternative solutions and making a selection), and do (executing the task according to plan). Therefore, for each primary indicator (i.e., Report, Assessment, Do), at least one corresponding secondary indicator score is required to ensure the integrity of these three stages (i.e., the three primary indicators) in the task execution process. The secondary indicators set under each primary indicator can be several comprehensive requirements for the stages of that primary indicator, selected based on experience.

[0050] Reporting is fundamental to information transmission and sharing, ensuring all crew members have a consistent understanding of the mission status and environment. This stage requires the reporter to possess considerable information transmission, communication, and collaboration skills. Reporting can be considered a primary indicator, and secondary indicators may include: clarity of reporting, accuracy of information, and collaborative content. Clarity of reporting refers to whether the report content is clear and concise, avoiding ambiguity; accuracy of content refers to whether the reported information is accurate and error-free, using standard terminology, and free of errors or omissions to prevent misunderstandings; collaborative communication refers to whether it provides support to the recipient and promotes mutual understanding within the team.

[0051] Assessment is the process of analyzing the current mission status, environmental information, and potential risks to develop feasible solutions and select the best course of action. Assessment serves as a primary indicator and may include, for example, the following secondary indicators: solution development capability, selection rationality, and decision-making decisiveness. Solution development capability refers to the recipient's ability to predict the future mission environment based on current information and thus develop a reasonable response plan. Selection rationality refers to the ability to weigh the pros and cons of different options and make decisions based on the current mission status, ensuring safety while addressing the upcoming mission. Decision-making decisiveness refers to the ability to make decisions under time pressure; for example, saving time is crucial in forest fire fighting missions.

[0052] Execution is the process of translating decisions into concrete actions to ensure the achievement of mission objectives. Execution is considered a primary indicator, and can include secondary indicators such as correctness and decisiveness. For example, in forest firefighting operations, the ability of team members to quickly and correctly execute each decision contributes to minimizing losses of manpower and resources. Therefore, correctness and decisiveness are used as evaluation criteria, requiring trainees to operate equipment or execute procedures correctly according to orders without panic or delay.

[0053] Step S2.2: Determine the weights of the primary indicator reports, assessments, and actions. And the weights of the secondary indicators contained in each primary indicator. Optionally, in practical applications, the weights of the primary and secondary indicators can be set to preset values ​​or calculated using the analytic hierarchy process, depending on the needs.

[0054] Step S2.3: Obtain the scores of the secondary indicators under the primary indicators in each process of the task, and calculate the evaluation results of the crew collaborative training.

[0055] First, the performance of the crew was scored using the Santy 1-9 scale method shown in Table 1. If this scoring item is not involved, it will be marked as "Not Applicable". This scoring method involves evaluating and scoring the content of crew coordination (rather than the correctness of individual operations) based on training parameters obtained during the training process, and using the Sany 1-9 scale. Optionally, the instructor can score the content of crew coordination (rather than content with clear right or wrong judgments, such as the correctness of individual operations) based on audio and video playback information from the aviation emergency rescue mission coordination training process, according to Table 1 below.

[0056] Table 1. Quantitative Scale for Qualitative Indicators

[0057] In Table 1 above, for example, score 2 indicates a degree between the range of score 1 and the difference of score 3, score 4 indicates a degree between the difference of score 3 and the general of score 5, and so on.

[0058] For example, for a report as a primary indicator, its secondary indicator—reporting clarity—is scored based on whether the report's content is clear, concise, and avoids ambiguity, using the Sany 1-9 scale method in Table 1 above. Similarly, the secondary indicators included in each primary indicator are also scored.

[0059] The evaluation method requires instructors to score at least one subordinate secondary indicator for each primary indicator when evaluating crew members. In practical applications, if a primary indicator has a secondary indicator that is not applicable (e.g., a secondary indicator is not applicable in the scenario), and the instructor did not score that secondary indicator, the weights of the secondary indicators with scores are recalculated and redistributed according to their original proportions. This method avoids the following problems when secondary indicators are not applicable: setting unscored items uniformly to 0 results in scores lower than the instructor's evaluation; setting unscored items uniformly to full marks results in scores higher than the instructor's evaluation; and only removing unscored items from the calculation changes the maximum score, making the meaning of the scores less intuitive.

[0060] Based on the scores of the secondary indicators under the primary indicators in each process of the task acquisition, an evaluation value for crew collaborative training is established:

[0061] in, For the teacher regarding the first First-level indicators in each process Secondary indicators The rating, Represents the index of the process and , For the index of the first-level indicator and , For the index of secondary indicators and , Indicates the first-level indicator The number of sub-secondary indicators The first step for instructors to complete the scoring The weights corresponding to each secondary indicator Indicates the first The weights of each primary indicator. The secondary indicators involved in the calculation of the above unit collaborative training evaluation values ​​are all secondary indicators used to complete the scoring.

[0062] in, An index for primary indicator reporting, evaluation, and execution (RAD), including but not limited to: configurable For the report, then As for the weight of the report, The number of secondary indicators under the report; For evaluation, then For the evaluation weights, The number of secondary indicators under the assessment; To execute, then Weights for execution, The number of secondary indicators under execution.

[0063] Step S3 involves obtaining task training parameters through VR training and providing them to the established task process training evaluation model and task collaboration training evaluation model. This yields individual performance evaluations, crew performance evaluations, and crew collaboration training evaluation values ​​as evaluation results, which are then provided to the analysis and decision-making processes. In this embodiment, the obtained task training parameters may include the execution status of training points at each stage of the training process, task audio and video playback, etc. In the analysis and decision-making processes, the evaluation results can be analyzed based on errors in individual operations during the task, errors in crew task execution, and weaknesses in crew collaboration. Based on this analysis, corresponding decisions are made, providing optimization processes for training and solutions for actual emergency rescue.

[0064] Example 1 The following describes Example 1 of an evaluation method for integrated training in aviation emergency simulation rescue according to an embodiment of the present invention. In Example 1, the method is applied to the collaborative training evaluation of a VR helicopter bucket firefighting and rescue mission, specifically including the following steps.

[0065] Step S1: Based on the pre-set processes and training points in the task flow, establish a task flow training and evaluation model.

[0066] like Figure 2 As shown in Example 1, the processes and training points in the task flow are set as follows. In the VR collaborative training system, the crew performing helicopter bucket firefighting missions typically includes five training roles: captain, co-pilot, air mechanic, bucket safety officer, and observer. All trainees wear VR devices to conduct mission training in a virtual simulation environment. To improve the crew's ability to complete firefighting missions, four training processes are set up: fire reconnaissance, water source collection, water spraying for firefighting, and return flight. Taking into account both operational difficulty and impact on the outcome (i.e., the impact on the mission result), the weight values ​​of each process are calculated using the analytic hierarchy process (AHP).

[0067] Step S1.1: Based on the pre-set processes and training points in the task flow, establish the weights of each process using the Analytic Hierarchy Process (AHP). In this step, based on the processes and factors set above, the weight values ​​of each process are obtained using the AHP.

[0068] By comparing each of the two factors and four training processes pairwise with two factors as a prerequisite, the following judgment matrix is ​​constructed:

[0069] in, It is the first A judgment matrix, Let be the judgment term in this judgment matrix, representing the th . The first metric (such as the factors mentioned above or the training process) relative to the second metric The importance of each metric (such as the factors mentioned above or the training process), This indicates the number of indicators compared in a given judgment. Judgment items can be represented using Santy's 1-9 scale method as shown in Table 1 above.

[0070] In this embodiment 1, the judgment matrix is ​​constructed through expert experience to obtain the weight of each process. This includes first comparing the two factors of "operational difficulty" and "impact on results". Then, taking these two factors as a premise, the four training processes are compared pairwise. That is, the operation difficulty of the four training processes is compared pairwise, and the degree of impact on the results of the four training processes is compared pairwise.

[0071] The criterion-level judgment matrix is ​​as follows: Table 2 Criterion Layer Judgment Matrix

[0072] By calculating the weights of each item using the summation method, the column vector normalization of the above criterion layer judgment matrix is ​​first performed to obtain... ,right The sum of the rows is then normalized to obtain the criterion layer weights, where each row element represents the weight of the corresponding position indicator. 。

[0073] Since the above criterion-level judgment matrix only considers two indicators, namely, operational difficulty and impact on results, there is no need to perform a consistency check.

[0074] Next, an operation difficulty judgment matrix and an impact result judgment matrix are established. The four task processes are ranked hierarchically using two indicators of the criterion layer matrix. Specifically, the operation difficulty of the four training processes is compared pairwise, and the degree of impact on the results of the four training processes is compared pairwise. The results are shown in Table 3 (Operation Difficulty Judgment Matrix) and Table 4 (Influence Result Judgment Matrix).

[0075] Table 3 Operational Difficulty Judgment Matrix

[0076] Table 4 Influence Result Judgment Matrix

[0077] Similarly, first, the column vectors of the operation difficulty judgment matrix and the influence result judgment matrix are normalized, and then the row sum is calculated and normalized again to obtain the operation difficulty weight value and the influence result weight value, respectively:

[0078] .

[0079] A consistency check was performed on the above matrices to determine whether there were any logical problems in the constructed criterion-level judgment matrix, operational difficulty judgment matrix, and result impact judgment matrix. The specific verification process is as follows.

[0080] Consistency Indicators The value is calculated as follows:

[0081] in, Represents the largest eigenvalue. Indicates the matrix dimension.

[0082] The random consistency index is obtained by looking up a table. Value, based on consistency index Values ​​and random consistency index The consistency ratio is obtained by solving the value. value.

[0083] The consistency ratio is obtained from the solution. The value determines whether consistency is achieved, specifically including when... If the consistency of the judgment matrix is ​​within acceptable limits, it fails. Based on the judgment result, if it passes, the weight values ​​calculated above can be used; otherwise, the data in the judgment matrix needs to be corrected and the judgment matrix reconstructed. In this embodiment 1, the consistency of the judgment matrices constructed based on Tables 2 to 4 above all meet the requirements.

[0084] The operational difficulty weight value and the impact result weight value obtained above are calculated by weighting and normalizing to obtain the indicator weights as shown in Table 5 below.

[0085] Table 5 Indicator Weight Matrix

[0086] The weight matrix of this indicator can be represented as:

[0087] The index weights obtained from Table 5 above are used as the weights of the four processes in this embodiment 1. ,Right now, to Substitute these values ​​into steps S1.2 and S1.3 below.

[0088] Step S1.2: Establish an individual performance evaluation.

[0089] After the training is completed, the individual and crew performance will be evaluated based on the results of the training point execution judgments output by the VR training system.

[0090] In this Example 1, there were a total of 72 training points. 20 were completed correctly, 6 were due to operational errors, and 7 were not completed.

[0091] In the task process In the middle, members Individual training point completion rate and individual stage scores The calculation is as follows:

[0092]

[0093] Therefore, an individual performance evaluation, or individual score, can be obtained. :

[0094] In this embodiment 1, the crew consists of 5 members: the captain's training score is 64.46, the co-pilot's training score is 49.67, the flight engineer's training score is 49.85, the bucket safety officer's training score is 66.76, and the flight observer's training score is 32.96.

[0095] Step S1.3: Establish a unit completion assessment.

[0096] In the task process In the middle, the completion rate of the crew training points and unit stage score The calculation is as follows:

[0097]

[0098] Therefore, the evaluation of the unit's performance, i.e., the unit score, can be obtained:

[0099] In this Example 1, the crew training score was 63.02.

[0100] Step S2: Establish a task collaborative training and evaluation model.

[0101] The VR collaborative training system can replay the crew's mission process and record all actions and voice communications of all roles. Instructors use RAD models to evaluate the crew's performance of helicopter bucket firefighting missions based on the playback and recordings of the training process.

[0102] Step S2.1: Determine the primary indicators of the task, including three items: reporting, evaluation, and execution, and the secondary indicators included in each primary indicator.

[0103] For the primary indicator R (reporting) stage, its secondary indicators include the assessment scores for clarity, accuracy, and collaborativeness. "Clarity" is scored on whether the report content is clear and easy to understand, with 1 point indicating "very vague and incomprehensible" and 9 points indicating "fully comprehensible." "Accuracy" is scored on whether technical terms are used in the report and whether the content is accurate, necessary, and complete, with 1 point indicating "no technical terms used, incorrect content, or omission of key information," and 9 points indicating "appropriate use of technical terms, accurate and complete content, and no redundancy." "Collaborativeness" is scored on whether it includes support information for other members, i.e., a comparison of the level of support information for other members in the report to the expected level, with 1 point indicating "almost none" and 9 points indicating "sufficient."

[0104] For the primary indicator A (assessment) stage, its secondary indicators include assessment values ​​for plan formulation capability, selection rationality, and decision-making decisiveness. "Plan formulation capability" scores the crew's ability to respond to different environments; based on the crew's discussions, 1 point indicates "helpless," and 9 points indicate "able to fully consider all possible scenarios and formulate a plan." "Selection rationality" is judged based on the content of the decision, such as whether the water intake point is the optimal choice or whether the direction of entry into the fire is correct. "Decision-making decisiveness" scores the crew's ability to make immediate decisions; based on the time from proposing a feasible plan to the captain's final order, 1 point indicates "hesitant, excessive discussion and indecisiveness," and 9 points indicate "quickly completing the selection after discussion."

[0105] For the primary indicator D (Execution), its secondary indicator is the instructor's observation of the crew completing the task from a third-person perspective outside the helicopter. Scores are given based on the crew's execution, including: an evaluation value for correctness and an evaluation value for efficiency. "Correctness" refers to whether the operation was correct when observing the crew completing the current task from a third-person perspective outside the helicopter. "Efficiency" refers to whether the operation was sluggish or lacked proficiency when observing the crew completing the current task from a third-person perspective outside the helicopter.

[0106] Step S2.2: Determine the weights of the primary indicators Report, Assess, and Do using the Analytic Hierarchy Process (AHP). And the weights of the secondary indicators contained in each primary indicator. As shown in Table 6 below. In this implementation, the evaluation index weights in Table 6 are used for each process.

[0107] Table 6 Weights of RAD Evaluation Indicators

[0108] The scoring results of Example 1 are shown in Table 7 below. The evaluation method requires instructors to score at least one subordinate secondary indicator for each primary indicator. If an instructor fails to score all secondary indicators under a primary indicator, the weights of the secondary indicators with scores are recalculated according to their original proportions.

[0109] Table 7. RAD Evaluation Index Scoring Results

[0110] Based on the weights of each primary indicator and each secondary indicator under each primary indicator in Table 6, and the RAD evaluation indicator scoring results in Table 7, a crew collaborative training evaluation score is established:

[0111] in, For the teacher Primary indicators in the process Secondary indicators The score given The weights corresponding to the secondary indicators used to score instructors are specified. In this Example 1, the crew collaborative training evaluation score is 6.17.

[0112] As shown in Table 6, in this embodiment 1, for For the report, The number of secondary indicators included in the report, as weighted by the report's criteria. There are 3 items; For evaluation, The number of secondary indicators included for the evaluation weights. There are 3 items; For execution, The weighting for execution, including the number of secondary indicators. There are 2 items.

[0113] Further analysis of the RAD assessment scores can clarify the task segments and collaborative aspects that the crew needs to strengthen during training. For example, the score for a certain task progress can be obtained as follows: 。

[0114] Step S3, as shown in Table 7, uses the established task flow training evaluation model and task collaboration training evaluation model to obtain individual scores, crew scores, and crew collaboration training evaluation values ​​as evaluation results. These results are provided to the helicopter bucket firefighting mission analysis and decision-making processes. By comparing the total scores of each task process, the aspects of crew collaboration that are good and poor during the mission can be reflected. Furthermore, the average scores for reporting, evaluation, and execution in each process can reflect the weaknesses in crew collaboration in each task process. The average scores for reporting, evaluation, and execution in the overall mission flow can reflect the weaknesses in crew collaboration during helicopter bucket firefighting missions. Example 2 See below Figure 3 Example 2 of an evaluation method for integrated training in aviation emergency simulation rescue according to an embodiment of the present invention will be described. In this example 2, the method is applied to the collaborative training evaluation of VR flood rescue missions, specifically including the following steps.

[0115] Step S1: Based on the pre-set processes and training points in the task flow, establish a task flow training and evaluation model.

[0116] In this embodiment 2, the processes and training points in the task flow are set as follows. In the VR collaborative training system, the crew performing flood relief missions typically includes four training members: captain, co-pilot, winch operator, and lifeguard. All trainees wear VR devices to train in a virtual simulation environment. The entire mission is divided into five processes: flight preparation, heading to the rescue point, airdropping supplies, rescuing those in distress, and returning to base. To improve the crew's ability to complete flood relief missions, this embodiment 2 uses the four training processes—heading to the rescue point, airdropping supplies, rescuing those in distress, and returning to base—as indicators, comprehensively considering factors affecting the outcome (i.e., the impact on the mission result), and solving for the weight values ​​using the analytic hierarchy process (AHP). It should be understood that although this embodiment considers factors affecting the outcome and establishes an outcome judgment matrix, operational difficulty factors can also be considered simultaneously, and an operational difficulty judgment matrix can be established according to actual needs.

[0117] Step S1.1: Based on the pre-set processes and training points in the task flow, establish the weights of each process using the Analytic Hierarchy Process (AHP). In this step, based on the processes and factors set above, the weight values ​​of each process are obtained using the AHP.

[0118] The weights of the indicators affecting the judgment matrix are obtained by weighted calculation and normalization, as shown in Table 8 below.

[0119] Table 8 Influence Result Judgment Matrix

[0120] The weights obtained from Table 8 above are used as the weights of the four processes in this embodiment 2. ,Right now, to .

[0121] After the training is completed, the individual and crew performance will be evaluated based on the results of the training point execution judgments output by the VR training system.

[0122] In this second embodiment, there were a total of 48 training points. 23 were completed correctly, 3 were due to operational errors, and 2 were not completed.

[0123] Step S1.2: Establish an individual performance evaluation model to calculate the individual performance evaluation of the task.

[0124] Calculate the individual training point completion rate of each member at each stage of the task. and individual stage scores :

[0125]

[0126] Therefore, an individual performance evaluation, or individual score, can be obtained. :

[0127] in, For process The weights are obtained from Table 8.

[0128] In this embodiment 2, the individual scores of the four crew members were obtained as follows: captain training score 44.5, co-pilot training score 100, lifeguard training score 72, and winch operator training score 100.

[0129] Step S1.3: Establish a unit performance evaluation model for evaluating the unit's performance.

[0130] Calculate the crew training point completion rate at each stage of the mission. and unit stage score :

[0131]

[0132] Therefore, the unit's performance evaluation, i.e., the unit score, can be obtained: .

[0133] In this second embodiment, the unit score is 88.57.

[0134] Step S2: Establish a task collaborative training and evaluation model.

[0135] The VR collaborative training system can replay the crew's mission process and record all actions and voice communications of all roles. Instructors use the RAD assessment method to evaluate the crew's performance in helicopter flood relief missions based on the playback and recordings of the training process.

[0136] Step S2.1: Determine the primary indicators of the task, including three items: reporting, evaluation, and execution, and the secondary indicators included in each primary indicator.

[0137] For the primary indicator R (reporting) stage, its secondary indicators include evaluation values ​​for clarity, accuracy, and collaborative content. "Clarity" is scored on whether the report content is clear and easy to understand, with 1 point indicating "very vague and incomprehensible" and 9 points indicating "fully comprehensible." "Accuracy" is scored on whether technical terms are used in the report and whether the content is correct, necessary, and complete, with 1 point indicating "no technical terms used, incorrect content, or omission of key information," and 9 points indicating "appropriate use of technical terms, accurate and complete content, and no redundancy." "Collaborative content" is scored on whether it includes support information for other members, i.e., a comparison of the level of support information for other members in the report with the expected level, with 1 point indicating "almost none" and 9 points indicating "sufficient."

[0138] For the primary indicator A (assessment) stage, its secondary indicators include assessment values ​​for plan formulation capability, selection rationality, and decision-making decisiveness. "Plan formulation capability" scores the crew's ability to respond to different environments; based on the crew's discussions, 1 point indicates "helpless," and 9 points indicate "able to fully consider all possible scenarios and formulate a plan." "Selection rationality" is judged based on the content of the decision, such as whether the choice of airdrop hovering location is appropriate, and whether the actions taken when rescuing distressed personnel are correct. "Decision-making decisiveness" scores the crew's ability to make immediate decisions; based on the time from the crew proposing a feasible plan to the captain's final order, 1 point indicates "hesitant, excessive discussion and indecisiveness," and 9 points indicate "quickly completing a selection after discussion."

[0139] For the primary indicator D (Execution) stage, its secondary indicator is the instructor's observation of the crew completing the task from a third-person perspective outside the helicopter. Scores are given based on the crew's execution, including assessments of correctness and efficiency. "Correctness" refers to whether the operation was correct when observing the crew completing the current task from a third-person perspective outside the helicopter; "Efficiency" refers to whether the operation was sluggish or lacked proficiency when observing the crew completing the current task from a third-person perspective outside the helicopter.

[0140] Step S2.2: Determine the weights of the primary indicators Report, Assess, and Do using the Analytic Hierarchy Process (AHP). And the weights of the secondary indicators contained in each primary indicator. As shown in Table 9 below. In this implementation, the evaluation index weights in Table 8 are used for each process.

[0141] Table 9 Weights of RAD Evaluation Indicators

[0142] In this Example 2, the scoring results are shown in Table 10 below. The evaluation method requires instructors to score at least one subordinate secondary indicator for each primary indicator. If an instructor fails to score all secondary indicators under a primary indicator, the weights of the secondary indicators with scores are recalculated according to their original proportions.

[0143] Table 10. RAD Evaluation Index Scoring Results

[0144] Based on the weights of each primary indicator and each secondary indicator under each primary indicator in Table 9, and the results of the RAD evaluation indicator scoring section in Table 10, a crew collaborative training evaluation score is established. for:

[0145] In this embodiment 2, the task includes 4 processes. For the teacher Primary indicators in the process Secondary indicators The score given The weights corresponding to the secondary indicators for evaluating instructors.

[0146] In this second embodiment, the unit collaborative training evaluation score was 6.47.

[0147] As shown in Table 9, in this embodiment 2, for For the report, The number of secondary indicators included in the report, as weighted by the report's criteria. There are 3 items; For evaluation, The number of secondary indicators included for the evaluation weights. There are 3 items; For execution, The weighting for execution, including the number of secondary indicators. There are 2 items.

[0148] Further analysis of the RAD assessment scores can clarify the task segments and collaboration aspects that the crew needs to strengthen during training. For example, a score for the progress of a specific task can be obtained. for: 。

[0149] Step S3, as shown in Table 10, uses the established task flow training evaluation model and task collaboration training evaluation model to obtain individual scores, crew scores, and crew collaboration training evaluation values ​​as evaluation results. These results are provided to the helicopter flood relief mission analysis and decision-making processes. By comparing the total scores of each task process, the aspects of crew collaboration that are good and poor during the mission can be reflected. Furthermore, the average scores for reporting, evaluation, and execution in each process can reflect the weaknesses in crew collaboration in each task process. The average scores for reporting, evaluation, and execution in the overall mission flow can reflect the weaknesses in crew collaboration during helicopter flood relief missions.

[0150] In this embodiment, the task involves aviation emergency rescue operations for disasters such as forest fires and floods, and the evaluation method is a general evaluation method for collaborative training in aviation emergency rescue missions. By adopting the above-disclosed technical solution, the following beneficial results can be obtained: An evaluation method for aviation emergency rescue training is provided, which simultaneously considers the task completion capabilities of individuals and crews, as well as crew collaboration performance, reflecting aspects for improvement for trainees and crews from multiple dimensions. This method can quantitatively and accurately evaluate rescue training, reflecting weak task links and weaknesses in collaborative cooperation through both objective scoring and subjective evaluation. It allows for targeted learning and practice and can also serve as a reference for developing actual emergency rescue plans.

[0151] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0152] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An evaluation method for comprehensive aviation emergency simulation rescue training, characterized in that, Includes the following steps: Step S1: Based on the pre-set processes and training points in the task flow, as well as the crew members in the task flow, establish a task flow training and evaluation model for evaluating task completion. This includes establishing the weights of each process based on the operational difficulty of each process and the degree of influence of each process on the task result; and conducting individual and crew completion evaluations based on the process weights and statistical data from the training process to obtain individual and crew completion evaluations. Step S2: Establish a task collaborative training evaluation model to obtain crew collaborative training evaluation values. This includes determining the primary indicators of the task and the secondary indicators included in each primary indicator. The primary indicators include reporting, evaluation, and execution. The weights of reporting, evaluation, and execution of the primary indicators are determined using the analytic hierarchy process (AHP), as well as the weights of the secondary indicators included in each primary indicator. The evaluation indicator scoring results are obtained. Based on the weights of each primary indicator, the weights of each secondary indicator included in each primary indicator, and the evaluation indicator scoring results, the crew collaborative training evaluation score and the scores of each process in the task flow of the training process are obtained. Step S3: Obtain task training parameters through VR training, and provide them to the established task process training evaluation model and task collaboration training evaluation model. The individual performance evaluation, crew performance evaluation, and crew collaboration training evaluation values ​​are obtained as evaluation results, which are then provided to the analysis and decision-making process of aviation emergency rescue collaboration training.

2. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 1, characterized in that, Step S1 specifically includes: Step S1.1: Based on the pre-set processes and training points in the mission flow, establish the weights of each process using the analytic hierarchy process (AHP). This includes setting the crew in the mission flow as captain, co-pilot, flight engineer, bucket safety officer, and flight observer; setting four training processes as fire reconnaissance, water source collection, water spraying for firefighting, and return to base; setting two factors: operational difficulty and impact on results; comparing each of the two factors and the four training processes pairwise with each of the two factors as a premise to construct a judgment matrix, including a criterion-level judgment matrix, an operational difficulty judgment matrix, and an impact on results judgment matrix; and performing consistency checks, weighted calculations, and normalization on the judgment matrices to obtain the weights of each process. Step S1.2: Establish an individual performance evaluation. Based on the weights of each process and the statistical data of the training process, obtain the individual training point completion rate and individual stage score as the evaluation of individual performance. Step S1.3: Establish a unit completion assessment. Based on the weights of each process and the statistical data of the training process, obtain the unit training point completion rate and unit stage score as the evaluation of unit completion.

3. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 1, characterized in that, Step S1 specifically includes: Step S1.1: Based on the pre-set processes and training points in the mission flow, establish the weights of each process using the analytic hierarchy process (AHP). This includes setting the crew in the mission flow as captain, co-pilot, winch operator, and lifeguard; setting four training processes as five processes: heading to the rescue point, airdropping supplies, rescuing people in distress, and returning to base. Using the first four processes as indicators, and comprehensively considering factors affecting the outcome, the weights of each process are obtained by solving the AHP problem. Step S1.2: Establish an individual performance evaluation. Based on the weights of each process and the statistical data of the training process, the individual training point completion rate and individual stage score are obtained as the evaluation of individual performance. Step S1.3: Establish a unit completion assessment. Based on the weights of each process and the statistical data of the training process, obtain the unit training point completion rate and unit stage score as the evaluation of unit completion.

4. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 2 or 3, characterized in that, Step S1.2 specifically includes: Completion rate of individual training points of members in the task process and individual stage scores : in, For the index of the process of the task and , The total number of processes included in the rescue mission. Indexes to the members of the group; processes Members The number of relevant training points and progress of the correct operation. Zhongyu The number of relevant training points is obtained by counting based on the preset training point content; Based on the individual training point completion rate and individual stage score, an evaluation of individual performance is obtained: in, For process The weight.

5. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 4, characterized in that, Step S1.3 specifically includes: Crew training point completion rate during the construction task process and unit stage score : Among them, process The number of training points and progress that were correctly operated in the process The number of training points included is obtained by counting based on the preset training point content; Based on the obtained crew training point completion rate and crew phase score, the crew completion status is evaluated as follows: 。 6. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 1, characterized in that, Step S2 specifically includes: Step S2.1: Determine the primary indicators of the task and the secondary indicators included in each primary indicator. The primary indicators include three items: reporting, evaluation, and execution. Step S2.2: Determine the weights of the primary indicators reporting, evaluation, and execution, as well as the weights of the secondary indicators included in reporting, evaluation, and execution, respectively, using the analytic hierarchy process (AHP). Step S2.3: Obtain the scores of the secondary indicators under the primary indicators in each process of the task, and obtain the crew collaborative training evaluation value.

7. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 6, characterized in that, In step S2.1: The report, as a primary indicator, includes the following secondary indicators: clarity of reporting, accuracy of information, and collaborative content. The evaluation of primary indicators includes the following secondary indicators: ability to formulate solutions, rationality of selection, and decisiveness of decision-making. The execution of primary indicators includes the following secondary indicators: correctness of execution and decisiveness of execution.

8. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 6, characterized in that, Step S2.3 specifically includes: The scores of each secondary indicator under the primary indicator in each process of the task are obtained. ,in, Represents the index of the process and , For the index of the first-level indicator and , For the index of secondary indicators and , Indicates the first-level indicator The number of sub-secondary indicators; The following evaluation values ​​were obtained for the crew collaborative training: in, This represents the total number of processes included in the task. The first step for instructors to complete the scoring The weights corresponding to each secondary indicator Indicates the first The weight of each primary indicator.

9. The evaluation method for comprehensive aviation emergency simulation rescue training according to claim 8, characterized in that, Step S2.3 also includes: Based on the scores of the secondary indicators under the primary indicators in each process of the task, the scores of each process in the task flow are obtained: 。

Images