A multi-machine cooperative rescue efficiency evaluation method fusing weight sensitivity analysis and robustness test

Abstract

The application provides a multi-machine cooperative rescue efficiency evaluation method combining weight sensitivity analysis and robustness test. The method solves the problems of insufficient representation of the cooperative characteristics and insufficient robustness of the fixed weight in the prior art by constructing an index system which more fully reflects the multi-machine cooperative characteristics and exploring the stability of the quantitative evaluation conclusion based on the weight space.

Description

Technical Field

[0001] This invention relates to the field of aviation emergency rescue technology, and more specifically to an evaluation method for multi-aircraft collaborative missions in aviation emergency rescue. Background Technology

[0002] Aviation emergency rescue effectiveness assessment is the basis for the selection of rescue equipment and mission planning.

[0003] Existing indicator systems are mostly constructed using decomposition methods based on task objectives or task profiles, breaking down tasks into stages such as preparation, deployment, and operation, and selecting observable physical quantities or performance parameters from them; in specific paths, they mainly rely on the statistics of single-machine performance in historical cases, and obtain indicator values ​​through expert scoring or consulting manuals.

[0004] After obtaining the indicator data, existing technologies generally use a hierarchical weighted scoring model with fixed weights to evaluate the scheme. Common weighting methods include using the Analytic Hierarchy Process (AHP) to calculate subjective weights, or using the entropy weight method to calculate objective weights based on the statistical differences in the sample data.

[0005] However, the aforementioned existing technologies have certain shortcomings when facing complex and dynamic scenarios such as multi-machine collaboration.

[0006] For example, the existing rescue indicator system focuses on the superimposed description of the static performance of a single machine, and lacks a quantitative characterization of the system redundancy and mission reliability in multi-machine rescue collaborative operations, which makes the evaluation indicators unable to truly reflect the coordination of the formation in complex interference environments.

[0007] In addition, traditional methods often treat indicator weights as fixed constants, which on the one hand ignores the ambiguity of expert subjective judgment and the deviation of indicator contribution caused by changes in the rescue environment, making the ranking of the schemes susceptible to the influence of weight fine-tuning; on the other hand, existing technologies lack robustness testing mechanisms, focusing more on the highest score under specific weights during evaluation, without quantifying the stability of the evaluation conclusion itself within the weight space, and may not be able to identify robust schemes that perform well under multiple weight configurations in practical applications. Summary of the Invention

[0008] The purpose of this invention is to provide a method for evaluating the effectiveness of multi-machine collaborative rescue by integrating weighted sensitivity analysis and robustness testing. This method constructs an index system that more fully reflects the characteristics of multi-machine collaboration and quantifies the stability of the evaluation conclusions based on weight space analysis.

[0009] To achieve the above objectives, the technical solution applied in this invention is as follows.

[0010] A method for designing diffuse reflective surfaces based on random grid structures includes the following steps:

[0011] Step 1: Construct a multi-aircraft collaborative rescue effectiveness evaluation index system: Referring to the actual multi-aircraft collaborative rescue mission flow and core concerns, the system is deconstructed from four dimensions: timeliness, execution effectiveness, cost-effectiveness, and support capability, and further subdivided into more specific physical and logical indicators. Taking fire rescue as an example, nine indicators can be established, including multi-aircraft assembly time, multi-aircraft collaborative response time, fire extinguishing completion time, fire extinguishing completion rate, accuracy of fire extinguishing agent delivery, collaborative fuel efficiency, aircraft cost, multi-aircraft reliability, and multi-aircraft maintainability. Based on the physical meaning of each indicator, they are categorized into cost-oriented indicators (smaller values ​​are better) or benefit-oriented indicators (larger values ​​are better), forming a multi-aircraft collaborative rescue effectiveness evaluation index system that can characterize the collaborative characteristics in relatively complex environments.

[0012] Step 2: Obtain the original simulation data of the scheme and perform normalization preprocessing: Perform simulation examples for multiple multi-machine cluster configuration schemes to be evaluated, obtain the original simulation observations of each scheme under specific indicators, and construct a multi-scheme-indicator decision matrix. The matrix is ​​then normalized and oriented, and a normalized decision matrix in the range [0,1] is output. .

[0013] Step 3: Calculate the combined weights based on AHP-entropy weight combination: The combined weighting process consists of three sub-steps: subjective weight calculation, objective entropy weight calculation, and combined weight synthesis. First, AHP subjective weight calculation is performed by collecting judgment matrices from domain experts who compare indicators pairwise using the 1-9 scaling method, and then passing a consistency check. After the value is less than 0.1, the mean vector of subjective weights reflecting expert experience preferences is calculated using the eigenvector method. Secondly, the objective weight of entropy is calculated using a normalized decision matrix. Calculate the contribution ratio and information entropy of each indicator among different schemes. and coefficient of variation The final output is an objective weight vector that reflects the data's ability to distinguish between different data. Finally, a combined weighting process is performed: Based on requirements, a linear weighting operator (e.g., 50% subjective and 50% objective) is introduced to combine the weights. and The coupling is performed, and the final combined weight vector is output, which serves as the origin of the combined weights for subsequent space exploration. .

[0014] Step 4: Calculate and rank the deterministic performance scores of each scheme based on the combined weights: Calculate and rank the normalized decision matrix obtained in Step 2. Combined with the weight vector calculated in step 3 Perform matrix weighted summation. Calculate the comprehensive effectiveness score of each multi-aircraft collaborative rescue scheme at the baseline point. They were then ranked and used as the initial performance evaluation conclusion.

[0015] Step 5: Construct a weighted uncertainty space and perform Monte Carlo random sampling: To eliminate the interference of expert subjective judgment and scenario disturbances on the evaluation conclusion to a certain extent, the combined weights from Step 3 are used. Set the disturbance radius using the center coordinates. (e.g., ±50%), construct the uncertainty space for the weights. Within this space, use Monte Carlo sampling. Multiple (e.g., 5000) independent random multiplication perturbations. In each sampling, the program generates a set of random weights and normalizes them to ensure that the sum of the randomly generated weight vectors is equal to 1, providing a basis for subsequent robustness testing.

[0016] Step 6: Multi-dimensional Robustness Testing and Performance Satisfaction Loss Assessment: The results of numerous random sampling simulations using the Monte Carlo method are evaluated. The performance score and ranking of each scheme are calculated iteratively, and the evaluation is analyzed from two aspects: ① Ranking Transition Probability Matrix Statistics: The frequency of each scheme achieving 1st, 2nd, and 3rd place in 5000 simulations is statistically analyzed to calculate the dominance probability of each scheme, used to assess the robustness of the ranking; ② Performance Satisfaction Loss Assessment: In each random simulation, the difference between the highest-scoring scheme and other schemes and that highest-scoring scheme (Performance Satisfaction Loss Value, ESL) is taken. After traversing each simulation, the average performance loss and maximum loss value of each scheme are statistically analyzed. This is to quantify the potential decision-making risks caused by weight bias.

[0017] Step 7: Extract the robust optimal solution and output the optimal decision support suggestion: Evaluate the two aspects evaluated in Step 6 and formulate robust optimality judgment criteria. If a solution has a high probability of dominance within a large weight fluctuation space, and its average efficiency loss and maximum efficiency loss are both low, then the solution is determined to meet the anti-interference dominance condition. This solution is extracted as the robust optimal decision solution and used as the final recommended fleet formation for evaluation under uncertain fire rescue environments. If a solution is high in one aspect and low in another, the optimal solution judgment is adjusted according to actual preference requirements, and other solutions with similar conditions are provided as alternatives.

[0018] Compared with the prior art, the beneficial effects of this invention are:

[0019] 1. At the level of indicator system, the indicators proposed by this method do not only focus on the superposition description of the static performance of a single machine, but also analyze from four dimensions: timeliness, execution effect, cost and guarantee capability of multi-machine collaborative tasks. They are further subdivided into more specific physical and logical indicators, which can better reflect the characteristics of multi-machine collaboration.

[0020] 2. In terms of weight evaluation, in view of the fact that traditional fixed-value weighting methods often treat weights as constant values, which makes the ranking susceptible to subjective ambiguity and the influence of weight fine-tuning, this invention uses AHP-entropy weight combination as the initial weight and introduces Monte Carlo random sampling to establish a multi-dimensional weight perturbation space. Through large-scale random iteration, the jitter interference caused by subjective weighting bias is eliminated to a certain extent, and the robustness and anti-interference ability of the evaluation conclusion under wide-area preference perturbation are improved.

[0021] 3. At the decision optimization level: The "efficiency satisfaction loss" index is introduced into the decision risk measurement of multi-machine collaborative rescue. By quantifying the average and maximum efficiency loss in random simulation, robust optimal decision schemes are extracted from the two directions of "dominance probability" and "low satisfaction loss". This provides quantitative decision support that can reduce risks for the evaluation of heterogeneous cluster schemes in dynamic emergency rescue environments. Attached Figure Description

[0022] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. Examples of the embodiments are shown in the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and are only used to explain the technical solutions of the present invention, and should not be construed as limiting the present invention.

[0023] Figure 1 Taking firefighting missions as an example, this paper explains the structure and content of each evaluation dimension and specific indicator in the multi-machine collaborative rescue effectiveness evaluation methodology system that integrates weighted sensitivity analysis and robustness testing.

[0024] Figure 2 This is a flowchart of the overall technical route of the present invention.

[0025] Figure 3 This is a bar chart showing the comprehensive performance evaluation scores of each scheme under combined weights.

[0026] Figure 4 This is a box plot showing the distribution of performance evaluation scores for each scheme under weighted perturbation.

[0027] Figure 5 It is a multi-machine fire fighting and rescue effectiveness evaluation index system and its classification.

[0028] Figure 6 This is the original data table for multi-machine simulation experiments.

[0029] Figure 7 It is a numerical table of the original data after normalizing each indicator.

[0030] Figure 8 This is the expert AHP subjective weighting result table.

[0031] Figure 9 This is a table showing the results of the objective weight calculation using the entropy weight method.

[0032] Figure 10 This is a table showing the combined weights of various evaluation indicators.

[0033] Figure 11 It is the ranking transition matrix table after simulation using Monte Carlo random sampling.

[0034] Figure 12 This is a table of performance satisfaction loss statistics after simulation using Monte Carlo random sampling. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. Examples of the embodiments are shown in the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and are only used to explain the technical solutions of the present invention, and should not be construed as limiting the present invention.

[0036] Based on a real-world case, this invention was developed using an experimental background, and three different multi-aircraft firefighting and rescue mission scenarios were proposed for subsequent performance evaluation. The real-world case background was as follows: On the morning of April 7, 2019, a forest fire in Lier Village, Yalongjiang Town, was aggravated by strong winds. Hidden smoldering spots on a cliff within the northeastern fire zone, previously covered by artificial rainmaking, cooling, and snowfall, reignited. Burning, rotten logs rolled down the cliff, igniting unburnt trees within the fire zone, creating crown fires. Flying embers were blown to the eastern edge of the fire zone, causing further fires. The terrain was complex and steep, making it difficult for personnel to reach the fire. Based on information provided by Xichang Station, rescue personnel dispatched helicopters from Xichang Yanyuan Helicopter Airport and Xichang Qingshan Airport to the fire site to participate in firefighting efforts.

[0037] In the real-world case, the multi-aircraft formation deployed for this rescue consisted of two K32s and one Mi26. In this experiment, different aerial firefighting and rescue mission plans were formed by changing the mission aircraft types, with the helicopters located at airports based on actual deployment conditions. The original simulation decision matrix and corresponding index weights were obtained through system simulation and information collection. The multi-aircraft collaborative rescue effectiveness evaluation method proposed in this paper, which integrates weighted sensitivity analysis and robustness testing, was then used to evaluate and analyze the three hypothetical rescue plans, and a robust optimal decision solution was provided. Option 1: Two K-32s and one Mi-26 (one K-32 and Mi-26 depart from Xichang Qingshan Airport, and one K-32 departs from Xichang Yanyuan Helicopter Airport); Option 2: Two K-32s and one Mi-171 (one K-32 and Mi-171 depart from Xichang Qingshan Airport, and one K-32 departs from Xichang Yanyuan Helicopter Airport); Option 3: Two Z-8s and one Mi-26 (one Z-8 and Mi-26 depart from Xichang Qingshan Airport, and one Z-8 departs from Xichang Yanyuan Helicopter Airport).

[0038] Step 1: Construct a multi-aircraft collaborative rescue effectiveness evaluation index system. Referring to the actual multi-aircraft collaborative rescue mission flow and core concerns, the system is deconstructed from four criteria: timeliness, execution effectiveness, cost-effectiveness, and support capability, and further subdivided into more specific physical and logical indicators. In this case, fire rescue, it can be subdivided into nine secondary indicators, including multi-aircraft assembly time, multi-aircraft collaborative response time, fire extinguishing completion time, fire extinguishing completion rate, accuracy of fire extinguishing agent delivery, collaborative fuel efficiency, aircraft cost, multi-aircraft reliability, and multi-aircraft maintainability. Based on the physical meaning of each indicator, they are further divided into cost-based indicators and benefit-based indicators, such as… Figure 5 As shown.

[0039] Step 2: Obtain the original simulation data of the scheme and perform normalization preprocessing. Simulation examples are performed for multiple multi-machine cluster configuration schemes to be evaluated, obtaining the original simulation observations for each scheme under the nine indicators proposed in Step 1, such as... Figure 6 As shown. Construct a multi-option-indicator decision matrix. The matrix is ​​then normalized and oriented, and a normalized decision matrix in the range [0,1] is output. ,like Figure 7 As shown.

[0040] Step 3: Calculate the combined weights based on AHP-entropy weight combination. The combined weighting process consists of three sub-steps: subjective weight calculation, entropy weight objective weight calculation, and combined weight synthesis. Based on the multi-aircraft rescue effectiveness evaluation index system established above, four relevant industry experts (including front-line crew members and related research experts) were invited to construct judgment matrices for four dimensions and nine criteria indicators, respectively. After obtaining the average subjective weights of the experts, the objective weights were calculated using the entropy weight method, and the combined weights were obtained using the deviation minimization method. The obtained weights are listed in the following table. First, the AHP subjective weights were calculated by collecting the judgment matrices of field experts who compared the indicators pairwise using the 1-9 scaling method and performing consistency checks. Results with a value <0.1 and no logical problems, such as Figure 8 As shown, the mean vector of subjective weights reflecting expert experience preferences is calculated using the eigenvector method. Secondly, the objective weight of entropy is calculated using a normalized decision matrix. Calculate the contribution ratio and information entropy of each indicator among different schemes. and coefficient of variation The final output is an objective weight vector that reflects the data's ability to distinguish between different data. List the calculation results Figure 9 Finally, the combined weights are synthesized, and... and The coupling is performed, and the final combined weight vector is output, as shown in the figure. Figure 10 As shown.

[0041] Step 4: Calculate and rank the deterministic performance scores of each scheme based on the combined weights. The subjective weighting results of AHP obtained in Step 2 and the objective weighting results of entropy weight obtained in Step 3 are combined using a linear weighted combination method to calculate the combined weights of each indicator. The comprehensive performance scores of each scheme are then obtained through these combined weights, and the schemes are ranked and analyzed. The scoring is as follows: Figure 3 As shown. Under the current weight configuration, Scheme 2 is superior to Schemes 1 and 3, and this result is consistent with the historical scheme comparison analysis results. Analyzing the results, in this simulation experiment, since the fire extinguishing completion rate of all three schemes reached the target state, the difference coefficient of this indicator in the entropy weight calculation was 0. However, from the perspective of actual rescue experience, fire extinguishing efficiency is always the core evaluation dimension. The combined weighting introduces expert subjective experience, allowing this indicator to retain a 14.42% share in the final weight, which to some extent compensates for the limitation of the entropy weight method relying solely on sample differences and ignoring the essential importance of the indicator. Meanwhile, the combined weight results show that the final weights of coordinated fuel efficiency and multi-aircraft assembly time are 10.50% and 8.97% respectively, significantly higher than the expert's subjective expectations. This indicates that in this multi-aircraft coordinated rescue simulation scenario, different aircraft formation schemes exhibit strong dispersion in these two indicators, reflecting the difference between expert experience and simulation statistical characteristics, leading to the effectiveness evaluation conclusion potentially being sensitive to the selection of specific weights. Therefore, a single ranking conclusion derived solely from the combined weights may not be sufficient to support highly reliable decision-making. It is necessary to construct a weight uncertainty space and conduct weight sensitivity analysis and scheme robustness testing in subsequent steps.

[0042] Step 5: Construct the uncertainty space of weight fluctuations and perform Monte Carlo random sampling. Using the combined weights from Step 3... Set the disturbance radius using the center coordinates. (±50%), constructing an uncertainty space for the weights. Within this space, 5000 independent random multiplication perturbations are performed using Monte Carlo sampling. In each sampling, the program generates a set of random weights and normalizes them.

[0043] Step 6: Multi-dimensional robustness testing and performance satisfaction loss assessment of the proposed solutions. The sampling simulation results from Step 5 are evaluated separately. The performance score and ranking of each solution are calculated iteratively, the probability of dominance for each solution is calculated, and the average performance loss and maximum performance loss for each solution are statistically analyzed. The results are as follows: Figure 11 and Figure 12As shown in the figure, Scheme 2 consistently ranked first in almost all experimental samples, demonstrating extremely strong stability. While Scheme 3 surpassed Scheme 1 to reach second place in a very few weight configurations, it ranked worse than Scheme 1 in most cases. This indicates that the preliminary evaluation conclusions have a high degree of resistance to interference when the weights fluctuate slightly. Observing the average efficiency loss and maximum efficiency loss results, Scheme 2 also consistently ranked first even when the weights fluctuated, with both the average efficiency loss and maximum efficiency loss being 0. This shows that even when the weights deviate most severely from their nominal values, the efficiency loss caused by choosing Scheme 2 is minimal. In contrast, the average efficiency loss of Schemes 1 and 3 is relatively high. This means that if decision-makers choose these two schemes under the same conditions, they may face significant efficiency losses when the weights fluctuate. Therefore, from a risk perspective, Scheme 2 is superior to Scheme 1, and Scheme 1 is superior to Scheme 3.

[0044] Step 7: Extract the robust optimal solution and output the optimal decision support recommendation. The evaluation is based on the two aspects evaluated in Step 6: From the perspective of decision stability, in a random space with a weight perturbation amplitude as high as 50%, Solution 2 has a 100.00% probability of ranking first. This indicates that regardless of significant shifts in the decision-maker's preferences for timeliness, execution effectiveness, cost, or guarantee capability, Solution 2 remains optimal and possesses extremely strong anti-interference capabilities. From the perspective of risk, Solution 2 has the lowest average efficiency loss and maximum efficiency loss value, while Solutions 1 and 3 have average efficiency losses of 0.3183 and 0.3414, respectively, and maximum efficiency losses of 0.4038 and 0.4458. Although Solution 3 has a local advantage in response speed, its efficiency loss risk in the global weight space is too high. This means that choosing Solution 1 or Solution 3 may result in an efficiency loss risk of over 40% in an uncertain weight environment. In conclusion, Solution 2 is determined to be the robust optimal solution in multi-machine collaborative fire rescue scenarios, providing the most robust efficiency support in highly uncertain emergency rescue environments.

Claims

1. A method for evaluating the effectiveness of multi-machine collaborative rescue operations by integrating weighted sensitivity analysis and robustness testing, characterized in that: The method includes the following steps: Step 1: Based on actual mission processes and requirements, construct a multi-machine collaborative rescue effectiveness evaluation index system; Step 2: Obtain the original simulation data of the rescue plan and perform normalization preprocessing: Perform simulation examples for multiple multi-aircraft cluster configuration plans to be evaluated, obtain the original simulation observation values ​​of specific indicators of each plan and process them. Step 3: Calculate the combined weight based on AHP-entropy weight combination: Based on the data in Step 2, calculate the subjective weight, the objective entropy weight, and synthesize the combined weight, which will serve as the origin of the combined weight for subsequent spatial exploration. Step 4: Calculate and rank the deterministic performance scores of each scheme based on the combined weights. Step 5: Construct the weighted fluctuation uncertainty space and perform Monte Carlo random sampling. Step 6: Multi-dimensional robustness test and performance satisfaction loss assessment of the scheme: The sampling simulation results of Step 5 are evaluated from two aspects respectively. The performance score and ranking of each scheme are calculated in cycles, and the average performance loss and maximum loss value of each scheme in each cycle are calculated. Step 7: Extract the robust optimal solution and output the optimal rescue decision support suggestion.

2. The method of claim 1, wherein the method further comprises: In step 1, the multi-machine collaborative rescue effectiveness evaluation index system includes at least four primary indicators: timeliness, execution effect, cost and support capability, with corresponding secondary indicators set under each primary indicator.

3. The method of claim 1, wherein the method further comprises: In step 3, when the combined weight is obtained by assigning weights to the AHP-entropy weight combination, the ratio of subjective weights to objective weights can be dynamically adjusted according to the type of aviation rescue mission.

4. The method of claim 1, wherein the method further comprises: In step 5, the uncertainty space of weight fluctuation is set based on the combined weight in step 3, and the number of Monte Carlo samplings is no less than 2000 to ensure the statistical significance of the sampling results.

5. The method of claim 1, wherein the method further comprises: The scheme evaluation in step 6 considers both the stability of the scheme's ranking and the loss of effectiveness. When both evaluation conditions meet the proposed requirements, the scheme is adopted as the recommended scheme.

Images