Fire-fighting reconnaissance robot system performance comprehensive evaluation and weak link identification method

Abstract

This invention discloses a method for comprehensive performance evaluation and weak link identification of a fire reconnaissance robot system. The method includes: collecting performance test data of the fire reconnaissance robot system under six standardized test scenarios; obtaining a standardized score matrix for each test scenario; determining the objective weight of the evaluation indicators within each dimension based on the standardized score matrix, ultimately obtaining the global weight of each evaluation indicator relative to the overall target; obtaining the overall comprehensive performance score and dimensional comprehensive performance scores of the system; classifying warning levels and calculating the accuracy rate of weak link identification in the fire reconnaissance robot system. This method enables multi-angle and comprehensive evaluation of the application performance of fire reconnaissance robots, providing quantifiable evidence for the research and development iteration and quality supervision of fire reconnaissance robots.

Description

Technical Field

[0001] This invention relates to the field of robot performance evaluation and fault diagnosis technology, specifically to a method for comprehensive performance evaluation and weak link identification of a fire reconnaissance robot system. Background Technology

[0002] Firefighting reconnaissance robots are core equipment for performing tasks such as fire scene environmental detection, hazard identification, and search and rescue of trapped personnel. Due to the typical characteristics of fire scenes, such as high temperatures, dense smoke, complex spatial structures, and limited communication conditions, robots must simultaneously meet multiple challenges in actual operation, including hardware tolerance, operating system stability, and data analysis accuracy.

[0003] Currently, some patents have explored performance evaluation methods for firefighting robots. For example, patent CN112743556B focuses on optimizing the mobility and load capacity of firefighting robots in the structural design stage by establishing kinematic and dynamic models, but does not involve comprehensive performance evaluation at the task execution level; patent CN202111091792.X discloses a performance evaluation method for autonomous firefighting robots, mainly targeting the qualification evaluation of functions such as autonomous navigation and obstacle avoidance, but does not incorporate hardware reliability and data analysis accuracy into the comprehensive evaluation system, nor does it establish a quantitative identification mechanism for weak links.

[0004] Therefore, the evaluation of the performance of such robots still has the following shortcomings:

[0005] (1) Regarding the construction of the evaluation system, the fire scene environment is complex and ever-changing. The performance of the robot in different tasks such as explosion identification, hazardous chemical detection, and personnel search and rescue may vary significantly in terms of hardware, operating system, and data analysis capabilities. If the evaluation indicators do not establish a clear correspondence with specific task scenarios, it will be difficult to fully reflect the robot's comprehensive application capabilities in real fire scene environments.

[0006] (2) Regarding the identification of weak links, existing evaluations rarely delve into the dimensional or indicator levels for refined diagnosis. When the overall system performance declines, it is difficult to quickly pinpoint whether the decline is due to insufficient hardware reliability, slow operating system response, or decreased data analysis accuracy. The lack of an effective mechanism for identifying weak links limits the guiding role of evaluation results in system optimization and improvement;

[0007] (3) In determining diagnostic thresholds, existing methods mostly use fixed thresholds or empirical values, lacking an optimization process based on real data distribution and diagnostic efficacy. There are still few studies that introduce statistical indicators such as sensitivity and specificity to quantitatively evaluate diagnostic effectiveness, and the scientific nature and interpretability of threshold setting need to be further improved.

[0008] Therefore, how to construct a comprehensive evaluation method that can cover multi-task scenarios, weak link identification, and threshold optimization is a research direction worthy of attention in this field. Summary of the Invention

[0009] The purpose of this invention is to provide a method for comprehensive performance evaluation and weak point identification of a fire reconnaissance robot system, which can provide a scientific and systematic solution for comprehensive performance evaluation and weak point identification of a fire reconnaissance robot system.

[0010] Technical solution: The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system described in this invention includes the following steps:

[0011] Step 1: Collect performance test data of the fire reconnaissance robot system under six pre-designed standardized test scenarios. The six standardized test scenarios are used to simulate different typical tasks and environmental conditions in a fire scene.

[0012] Step 2: Standardize the system performance test data recorded in the standardized test scenarios of the fire reconnaissance robot to obtain the standardized score matrix for each test scenario;

[0013] Step 3: Divide the system performance test data into three dimensions according to their properties, determine the weights of the three dimensions, and then determine the objective weights of the evaluation indicators in each dimension based on the standardized score matrix. Finally, obtain the global weight of each evaluation indicator relative to the overall goal.

[0014] Step 4: Based on the standardized score matrix and global weights, calculate the comprehensive performance index and comprehensive performance score of each dimension of the fire reconnaissance robot system. Then, take the average of the comprehensive performance index and comprehensive performance score of each dimension of each test sample to obtain the overall comprehensive performance score and dimensional comprehensive performance score of the system.

[0015] Step 5: Define the real weak link based on the dimensional comprehensive performance score. Quantitatively characterize the drag on the overall system performance of each index by constructing a bottleneck coefficient that considers cascading failures. Use the maximum bottleneck coefficient within the dimension as the predictive index of the weak link. Determine the optimal diagnostic threshold through cost-sensitive weighted Youden index co-optimization, classify the early warning levels, and calculate the accuracy of weak link identification in the fire reconnaissance robot system.

[0016] Furthermore, in step 1, the six standardized test scenarios include explosion identification test under different smoke concentrations, hazardous chemical identification test, carbon monoxide identification test, trapped personnel identification test, building collapse identification test, and temperature resistance performance test; the system performance test data includes hardware reliability data, operating system effectiveness data, and data analysis accuracy data.

[0017] Furthermore, in step 2, the specific steps for obtaining the standardized score matrix for each experimental scenario are as follows:

[0018] Step 2.1: Based on the design scheme of the six standardized test scenarios, each basic condition formed by the combination of test elements in each standardized test scenario is defined as a test environment. Each test environment corresponds to a set of test parameters. Under each set of test parameters, five repeated tests are conducted to obtain five experimental results. The average value of the five experimental results is then calculated to obtain the test mean of each test environment.

[0019] Step 2.2: Collect the test mean values ​​for six standardized test scenarios to obtain n test mean values ​​under n test environments. Treat each test mean value as a test sample, and each test sample contains m evaluation indicators. Then, denote the data matrix X as... In the formula, x ij Let be the test mean of the i-th test sample for the j-th evaluation index;

[0020] Step 2.3: Perform Min-Max standardization on each evaluation indicator to eliminate the influence of dimensions. For extremely large indicators, the standardization formula is as follows: If the index is extremely small, the standardization formula is: In the formula, x ij Let x be the test mean of the i-th test sample for the j-th evaluation index, min(x) j and max(x) j ) represent the minimum and maximum values ​​of the evaluation index in all experimental samples, respectively. ij The standardized score ranges from [0,1], with larger values ​​indicating better performance. The resulting standardized score matrix is: .

[0021] Furthermore, in step 3, the specific steps for determining the weights of the three dimensions are as follows:

[0022] Step 3.1: Based on expert knowledge in the field of fire reconnaissance robots, the importance of the three dimensions—hardware reliability, operating system effectiveness, and data analysis accuracy—is compared pairwise using the interval type-two fuzzy scaling method. An interval type-two fuzzy judgment matrix D is then constructed. In the formula, and Let the lower and upper bound scale values ​​represent the importance of the i-th dimension relative to the j-th dimension, respectively. Their assignment follows a scaling rule from 1 to 9. The interval type-II fuzzy judgment matrix D is then defuzzified to obtain the initial dimension weights for each dimension. ;

[0023] Step 3.2: Based on the design schemes of the six standardized test scenarios, statistically analyze the frequency of occurrence or the proportion of hazard levels of each standardized test scenario in fire scene missions. k Given k=1,…,6, calculate the environmental entropy of the fire scene. , Then construct environmental regulation functions for each dimension. , In the formula, β p The p-th dimension is the environmental sensitivity coefficient, which is determined by the failure statistics of the fire reconnaissance robot in a real fire scene.

[0024] Step 3.3, initial dimension weights Environmental entropy correction is performed to obtain dimensional weights. , Then construct a three-dimensional weight vector. , In the formula, w d1 w d2 and w d3 The weights are for each of the three dimensions;

[0025] Step 3.4: Perform interval consistency test, calculate the interval consistency index of the interval judgment matrix, and then calculate the consistency ratio. If the consistency ratio is less than 0.10, the consistency of the interval judgment matrix is ​​accepted; otherwise, the interval judgment matrix is ​​adjusted.

[0026] Furthermore, in step 3, the specific steps for determining the objective weights of the evaluation indicators within each dimension based on the standardized score matrix are as follows:

[0027] Step 3.5, let the p-th dimension contain q evaluation indicators, corresponding to the standardized data submatrix S. p S p ∈R n×q Then calculate the standard deviation of the j-th evaluation index under the p-th dimension. , In the formula, For the standardized data submatrix S p The element in the i-th row and j-th column, Let j be the mean of the j-th evaluation index across all experimental samples;

[0028] Step 3.6: Calculate the correlation coefficient matrix between the evaluation indicators in the p-th dimension. , , In the formula, r uv Let be the correlation coefficient between the u-th evaluation index and the v-th evaluation index. as well as From the standardized score matrix respectively The standardized scores of the i-th experimental sample in the p-th dimension on the u and v evaluation metrics are extracted. These are the mean values ​​of the u-th and v-th indicators in the current dimension across all n experimental samples;

[0029] Step 3.7, based on the correlation coefficient matrix The degree of information overlap between the indicators is measured, and the conflict coefficient between the j-th evaluation indicator and other evaluation indicators is calculated. , In the formula, r ju Correlation coefficient matrix The element in the j-th row and u-th column;

[0030] Step 3.8, convert the standard deviation With conflict coefficient By combining these, we can obtain the comprehensive information content carried by the j-th evaluation index. , This product simultaneously reflects the discriminative power (standard deviation) and independence (conflict resolution) of the indicators. Then, the information content of all evaluation indicators under the p-th dimension is normalized to obtain the objective weights of each evaluation indicator. , In the formula, It is the sum of the information content of all evaluation indicators under the p-th dimension.

[0031] Furthermore, in step 3, the specific steps for obtaining the global weight of each evaluation indicator relative to the overall goal are as follows:

[0032] Step 3.9: Calculate the global weight for the j-th evaluation index belonging to the p-th dimension. , In the formula, w dp The subjective weights for the p-th dimension determined in step 3, Let be the objective weight of the j-th evaluation indicator under the p-th dimension; this multiplicative fusion method achieves the complementarity between expert prior knowledge and the inherent laws of test data, which not only preserves the emphasis and preference of fire scene tasks on different dimensions, but also refines the weight through the distinguishability and independence of data-driven indicators.

[0033] Step 3.10: Perform global weight calculation on all evaluation indicators across all dimensions to obtain the complete global weight vector. , In the formula, q1, q2, and q3 represent the number of evaluation indicators under the three dimensions, respectively. This is the weight vector of all evaluation indicators in the first dimension. This is the weight vector of all evaluation indicators in the second dimension. This is the weight vector for all evaluation indicators in the third dimension.

[0034] Furthermore, in step 4, the specific steps for obtaining the overall comprehensive performance score and dimensional comprehensive performance score of the system are as follows:

[0035] Step 4.1: Calculate the comprehensive performance index of the i-th test sample. , Then calculate the overall performance score of the system. , In the formula, s ij Let be the standardized score of the i-th test sample on the j-th evaluation index. Let be the global weight of the j-th evaluation indicator, and k be the total number of secondary indicators;

[0036] Step 4.2: Calculate the overall performance score of the i-th test sample on the p-th dimension. , Then calculate the system's overall dimensional performance score for the current dimension. , In the formula, J p This is the set of secondary evaluation indexes contained in the p-th dimension, where the dimension numbers p=1,2,3 correspond to the three dimensions of hardware reliability, operating system effectiveness, and data analysis accuracy, respectively.

[0037] Furthermore, in step 5, the specific steps for defining the true weak link based on the dimensional comprehensive performance score, quantitatively characterizing the drag on the overall system performance of each indicator by constructing a bottleneck coefficient that considers cascading failures, and using the maximum bottleneck coefficient within the dimension as the predictive indicator of the weak link are as follows:

[0038] Step 5.1: For the j-th evaluation index of the i-th experimental sample, calculate the initial bottleneck coefficient. , , , In the formula, C ij (p) Let s be the actual contribution of the i-th experimental sample to the j-th evaluation index. ij (p) w is the standardized score of the i-th test sample on the j-th evaluation index. gj (p) Let j be the global weight of the j-th evaluation index. The average contribution of all evaluation indicators for all test samples;

[0039] Step 5.2, construct the index coupling influence matrix in the p-th dimension. , In the formula, This represents the drag coefficient of the failure of the u-th evaluation index on the v-th evaluation index. It is calibrated using experimental data on the fire fault propagation mechanism, and then the cascading failure bottleneck coefficient is calculated. , In the formula, λ is the cascade amplification factor, which is determined by the statistical data of fault propagation intensity under six standardized test scenarios;

[0040] Step 5.3: For the p-th dimension, take the maximum value of the cascading failure bottleneck coefficient of all evaluation indicators as the predictive indicator of potential weaknesses in the p-th dimension. , In the formula, J p This is the set of secondary indicator indices contained in the p-th dimension;

[0041] Step 5.4: Based on the task standards or industry specifications for fire reconnaissance robots, preset the target performance score η for each dimension. The target performance score η for each dimension is set independently. For the i-th test sample, the comprehensive performance score in the p-th dimension is... If D ip If η < 1, then the environment-dimensional sample is determined to be a true vulnerability and labeled as such. ip =1; otherwise, mark y ip =0;

[0042] Step 5.5, with As a predictor variable, all experimental samples were selected. As a candidate threshold space T∈[0,1], for each threshold τ, Samples with values ​​greater than τ are predicted as having weak dimensionality; therefore, predictive labels should be added. =1, otherwise it is predicted as non-weak and marked. =0;

[0043] Step 5.6, Calculate sensitivity and specificity They are respectively: , In the formula, TP is the number of correctly identified weak samples, TN is the number of correctly identified non-weak samples, FP is the number of non-weak samples that were misclassified as weak, and FN is the number of weak samples that were missed.

[0044] Furthermore, in step 5, the specific steps for determining the optimal diagnostic threshold through cost-sensitive weighted Youden index co-optimization are as follows:

[0045] Step 5.7, based on the cost C of mission failure caused by the fire reconnaissance robot failing to identify weak points during fire scene operations. FN The cost of resource waste caused by misjudging weak points C FP Construct a cost-sensitive weighted Youden index , In the formula, To calculate sensitivity, For specificity, ,and ∈(0.5,1), reflecting a higher penalty for weak links that are missed in judgment;

[0046] Step 5.8: Use the fuzzy C-means clustering algorithm to cluster all experimental samples. Unsupervised clustering is performed to obtain two cluster centers c1 and c2. The threshold [min(c1,c2),max(c1,c2)] is used as the fine-grained search interval, and within this interval, the cluster centers are... Maximize the dimensionality search for the objective and select the option that makes The largest threshold is used as the optimal diagnostic threshold τ for the p-th dimension. p If multiple thresholds reach their maximum simultaneously Then, the median value of each threshold is taken as the optimal diagnostic threshold τ for the p-th dimension. p .

[0047] Furthermore, in step 5, the specific steps for classifying early warning levels and calculating the accuracy rate of weak point identification in the fire reconnaissance robot system are as follows:

[0048] Step 5.9, using the optimal diagnostic threshold τ p Perform dimensional weakness prediction to obtain predicted labels. and with real label y ip Compare and calculate the accuracy A of weak link identification. ;

[0049] Step 5.10, based on the binary classification diagnosis, analyze the results obtained through... τ p For test samples predicted to be weak, the warning level is further divided according to the degree of bottleneck coefficient exceedance to indicate the severity of the weakness. For test samples in the p-th dimension, the warning level L is defined. ip for: In the formula, τ is the maximum bottleneck coefficient of the i-th experimental sample in the p-th dimension, with a value range of [0,1]. p For the optimal diagnostic threshold in the p-th dimension, 1-τ p This represents potential room for improvement in the bottleneck coefficient.

[0050] Compared with the prior art, the beneficial effects of this invention are:

[0051] (1) A three-dimensional comprehensive evaluation index system covering multiple task scenarios was constructed. Based on six types of standardized test scenarios, evaluation indexes of three dimensions, namely hardware reliability, operating system effectiveness and data analysis accuracy, were systematically designed. This enabled a multi-angle and all-round evaluation of the application performance of the fire reconnaissance robot. Compared with traditional methods that focus on a single task, the index system of this application is more comprehensive and task-oriented, and can more realistically reflect the robot's comprehensive performance in complex fire scene environments.

[0052] (2) The quantitative identification of weak links and the optimization of diagnostic thresholds have been realized. By constructing a cascading failure bottleneck index model, the degree of "drag" of each indicator on the overall system performance is quantified, and the weak links are predicted based on the maximum bottleneck index within the dimension. At the same time, the cost-sensitive Youden index is used to optimize the diagnostic thresholds of each dimension, a three-level hierarchical early warning system is constructed, and the best judgment criteria are determined. This mechanism can not only accurately locate the key indicators that restrict the system performance, but also significantly improve the accuracy of weak link identification, providing a direction for subsequent system optimization.

[0053] (3) The evaluation results are intuitive and have longitudinal comparability. By calculating the comprehensive performance index and the comprehensive performance score of each dimension, and mapping the results to the percentage range, the evaluation conclusions are intuitive and easy to understand. Multiple tests of the same system can be compared longitudinally, providing a quantifiable basis for the research and development iteration and quality supervision of fire reconnaissance robots. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0055] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0056] like Figure 1 As shown, the method for comprehensive performance evaluation and weak point identification of the fire reconnaissance robot system disclosed in this invention includes the following steps:

[0057] Step 1: Collect performance test data of the fire reconnaissance robot system under six pre-designed standardized test scenarios. The six standardized test scenarios are used to simulate different typical tasks and environmental conditions in a fire scene.

[0058] Step 2: Standardize the system performance test data recorded in the standardized test scenarios of the fire reconnaissance robot to obtain the standardized score matrix for each test scenario;

[0059] Step 3: Divide the system performance test data into three dimensions according to their properties, determine the weights of the three dimensions, and then determine the objective weights of the evaluation indicators in each dimension based on the standardized score matrix. Finally, obtain the global weight of each evaluation indicator relative to the overall goal.

[0060] Step 4: Based on the standardized score matrix and global weights, calculate the comprehensive performance index and comprehensive performance score of each dimension of the fire reconnaissance robot system. Then, take the average of the comprehensive performance index and comprehensive performance score of each dimension of each test sample to obtain the overall comprehensive performance score and dimensional comprehensive performance score of the system.

[0061] Step 5: Define the real weak link based on the dimensional comprehensive performance score. Quantitatively characterize the drag on the overall system performance of each index by constructing a bottleneck coefficient that considers cascading failures. Use the maximum bottleneck coefficient within the dimension as the predictive index of the weak link. Determine the optimal diagnostic threshold through cost-sensitive weighted Youden index co-optimization, classify the early warning levels, and calculate the accuracy of weak link identification in the fire reconnaissance robot system.

[0062] Furthermore, in step 1, the six standardized test scenarios specifically include explosion identification test under different smoke concentrations, hazardous chemical identification test, carbon monoxide identification test, trapped personnel identification test, building collapse identification test, and temperature resistance performance test; system performance test data includes hardware reliability data, operating system effectiveness data, and data analysis accuracy data.

[0063] In the explosion recognition test under different smoke concentrations, the test elements mainly included four levels of smoke concentration (no smoke, 10 mg / m³, 20 mg / m³, 30 mg / m³), four explosions, and a total of 16 basic condition combinations. The main records included: the simulated explosion location, number, and smoke concentration; start and end times; recognition time; data reported by the robot: the number of explosions recognized and the smoke concentration recognized; recognition results (success / failure); and any abnormal situations or false alarms.

[0064] In the identification test of hazardous chemicals (packaging), the test elements mainly include 4 types of targets (ethanol stainless steel cans, nitrocellulose steel cans, liquid ammonia cylinders, and interfering objects), 2 spatial locations (behind a door, in a corner), and 2 target postures (upright, tilted), totaling 16 combinations of basic conditions. The main records include: the simulated hazardous object's location and posture; start and end times; identification time; identification result (success / failure); and any abnormal situations or false alarms.

[0065] In the carbon monoxide identification test, the test elements mainly included three target concentrations (5ppm, 10ppm, and 15ppm), corresponding to three measurement distances (1m, 4m, and 7m), for a total of three basic condition combinations. The main records were as follows: the reconnaissance operator used emergency rescue reconnaissance techniques to collect CO concentrations at distances of 1m, 4m, and 7m from the fire source; the robot operator controlled a fire reconnaissance robot to measure CO concentrations at distances of 1m, 4m, and 7m from the fire source and relayed the results to the data recorder via walkie-talkie.

[0066] In the trapped person identification experiment, the experimental elements mainly included 3 target types (dummy, interference A, and interference B), 3 spatial locations (under the bed, behind the door, and in a corner), and 3 dummy postures (prone, side-lying, and curled up), totaling 27 basic condition combinations. The main records included: the simulated trapped person's position and posture; start and end times; identification time; vital sign data (body temperature) reported by the robot; identification result (success / failure); and any abnormal situations or false alarms.

[0067] In the building collapse recognition experiment, the main test elements included two types of targets (simulated collapse devices and interference objects), four spatial locations (front, back, left, and right of the robot), and a total of eight basic condition combinations. The main records included: the set location of the reconnaissance target; the start and end times; the recognition time; the recognition result (success / failure); and any abnormal situations or false alarms.

[0068] In the temperature resistance test, the test elements mainly include four temperatures (room temperature, 50℃, 80℃ and 100℃), which correspond to four smoke environments (no smoke, low concentration, medium concentration and high concentration), and a total of four basic condition combinations. The main records are: test conditions; start and end time; robot status, test results, any abnormalities or false alarms.

[0069] Based on the data recorded from six standardized tests, specific quantifiable indicators can be obtained. Among these, the specific indicators for explosion identification tests under different smoke concentrations include: explosion count identification accuracy, smoke concentration identification accuracy, explosion identification response time, smoke environment positioning error, and communication packet loss rate.

[0070] Specific indicators in the hazardous chemicals (packaging) identification test include: hazardous chemical identification accuracy, hazardous chemical interference false alarm rate, hazardous chemical identification response time, hazardous chemical target positioning accuracy, and hazardous chemical multi-pose identification success rate;

[0071] Specific indicators in the carbon monoxide identification test include: CO concentration measurement error, CO sensor measurement repeatability, CO sensor response time, distance attenuation compensation capability, and CO sensor stability.

[0072] Specific indicators in the trapped person identification experiment include: trapped person identification accuracy, trapped person interference false alarm rate, body temperature measurement error, trapped person identification response time, trapped person multi-pose recognition success rate, and concealed location recognition capability;

[0073] Specific indicators in the building collapse recognition test include: collapse recognition accuracy, false alarm rate of interference objects in collapse recognition, collapse recognition response time, and collapse recognition direction sensitivity;

[0074] Specific indicators in the temperature resistance performance test include: extreme tolerance temperature, high temperature working time, temperature field plotting accuracy, thermal imaging performance attenuation rate, and shell thermal insulation.

[0075] The indicators and their properties obtained from the six standardized tests are classified according to three dimensions: hardware reliability, operating system effectiveness, and data analysis accuracy. The specific indicator system is shown in Table 1.

[0076] Table 1, Indicator System Table

[0077] Furthermore, in step 2, the specific steps for obtaining the standardized score matrix for each experimental scenario are as follows:

[0078] Step 2.1: Based on the design scheme of the six standardized test scenarios, each basic condition formed by the combination of test elements in each standardized test scenario is defined as a test environment. Each test environment corresponds to a set of test parameters (such as smoke concentration level, number of explosions, target type, spatial location, etc.). Under each set of test parameters, five repeated tests are conducted to obtain five experimental results data. The average value of the five experimental results data is then calculated to obtain the test mean value of each test environment.

[0079] Step 2.2: Collect the test mean values ​​for six standardized test scenarios to obtain n test mean values ​​under n test environments. Treat each test mean value as a test sample, and each test sample contains m evaluation indicators. Then, denote the data matrix X as... In the formula, x ij Let be the test mean of the i-th test sample for the j-th evaluation index;

[0080] Step 2.3: Perform Min-Max standardization on each evaluation indicator to eliminate the influence of dimensions. For extremely large indicators, the standardization formula is as follows: If the index is extremely small, the standardization formula is: In the formula, x ij Let x be the test mean of the i-th test sample for the j-th evaluation index, min(x) j and max(x) j) represent the minimum and maximum values ​​of the evaluation index in all experimental samples, respectively. ij The standardized score ranges from [0,1], with larger values ​​indicating better performance. The resulting standardized score matrix is: .

[0081] Furthermore, in step 3, the specific steps for determining the weights of the three dimensions are as follows:

[0082] Step 3.1: Based on expert knowledge in the field of fire reconnaissance robots, the importance of the three dimensions—hardware reliability, operating system effectiveness, and data analysis accuracy—is compared pairwise using the interval type-two fuzzy scaling method. An interval type-two fuzzy judgment matrix D is then constructed. In the formula, and Let the lower and upper bound scale values ​​represent the importance of the i-th dimension relative to the j-th dimension, respectively. Their assignment follows a scaling rule from 1 to 9. The interval type-II fuzzy judgment matrix D is then defuzzified to obtain the initial dimension weights for each dimension. ;

[0083] Step 3.2: Based on the design schemes of the six standardized test scenarios, statistically analyze the frequency of occurrence or the proportion of hazard levels of each standardized test scenario in fire scene missions. k Given k=1,…,6, calculate the environmental entropy of the fire scene. , Then construct environmental regulation functions for each dimension. , In the formula, β p The p-th dimension is the environmental sensitivity coefficient, which is determined by the failure statistics of the fire reconnaissance robot in a real fire scene.

[0084] Step 3.3, initial dimension weights Environmental entropy correction is performed to obtain dimensional weights. , Then construct a three-dimensional weight vector. , In the formula, w d1 w d2 and w d3 The weights are for each of the three dimensions;

[0085] Step 3.4: Perform an interval consistency test. Calculate the interval consistency index of the interval judgment matrix, and then calculate the consistency ratio. If the consistency ratio is less than 0.10, the consistency of the interval judgment matrix is ​​accepted; otherwise, find the element with the largest interval deviation. The ratio is then corrected towards the theoretical interval ratio and recalculated until it passes the consistency test.

[0086] Furthermore, in step 3, the specific steps for determining the objective weights of the evaluation indicators within each dimension based on the standardized score matrix are as follows:

[0087] Step 3.5, let the p-th dimension contain q evaluation indicators, corresponding to the standardized data submatrix S. p S p ∈R n×q Then calculate the standard deviation of the j-th evaluation index under the p-th dimension. , In the formula, For the standardized data submatrix S p The element in the i-th row and j-th column, Let j be the mean of the j-th evaluation index across all experimental samples;

[0088] Step 3.6: Calculate the correlation coefficient matrix between the evaluation indicators in the p-th dimension. , , In the formula, r uv Let be the correlation coefficient between the u-th evaluation index and the v-th evaluation index. as well as From the standardized score matrix respectively The standardized scores of the i-th experimental sample in the p-th dimension on the u and v evaluation metrics are extracted. The u-th and v-th indicators in the current dimension are the means of all n test samples, and the correlation coefficient r is the mean of the u-th and v-th indicators. uv The value range is [-1, 1], and the larger the absolute value, the stronger the correlation between the two evaluation indicators.

[0089] Step 3.7, based on the correlation coefficient matrix The degree of information overlap between the indicators is measured, and the conflict coefficient between the j-th evaluation indicator and other evaluation indicators is calculated. , In the formula, r ju Correlation coefficient matrix The element in the j-th row and u-th column has a higher conflict coefficient, which indicates that the information redundancy between this indicator and other indicators is lower, and it should be given a higher weight.

[0090] Step 3.8, convert the standard deviation With conflict coefficient By combining these, we can obtain the comprehensive information content carried by the j-th evaluation index. , This product simultaneously reflects the discriminative power (standard deviation) and independence (conflict resolution) of the indicators. Then, the information content of all evaluation indicators under the p-th dimension is normalized to obtain the objective weights of each evaluation indicator. , In the formula, It is the sum of the information content of all evaluation indicators under the p-th dimension.

[0091] Furthermore, in step 3, the specific steps for obtaining the global weight of each evaluation indicator relative to the overall goal are as follows:

[0092] Step 3.9: Calculate the global weight for the j-th evaluation index belonging to the p-th dimension. , In the formula, w dp The subjective weights for the p-th dimension determined in step 3, Let be the objective weight of the j-th evaluation indicator under the p-th dimension; this multiplicative fusion method achieves the complementarity between expert prior knowledge and the inherent laws of test data, which not only preserves the emphasis and preference of fire scene tasks on different dimensions, but also refines the weight through the distinguishability and independence of data-driven indicators.

[0093] Step 3.10: Perform global weight calculation on all evaluation indicators across all dimensions to obtain the complete global weight vector. , In the formula, q1, q2, and q3 represent the number of evaluation indicators under the three dimensions, respectively. This is the weight vector of all evaluation indicators in the first dimension. This is the weight vector of all evaluation indicators in the second dimension. This is the weight vector for all evaluation indicators in the third dimension. The three parts together form the complete global weight vector.

[0094] Furthermore, in step 4, the specific steps for obtaining the overall comprehensive performance score and dimensional comprehensive performance score of the system are as follows:

[0095] Step 4.1: Calculate the comprehensive performance index of the i-th test sample. , Then calculate the overall performance score of the system. , In the formula, s ij Let be the standardized score of the i-th test sample on the j-th evaluation index. is the global weight of the j-th evaluation index, and k is the total number of secondary indicators. The larger the value of k, the better the overall application performance of the test conditions represented by the test sample.

[0096] Step 4.2: Calculate the overall performance score of the i-th test sample on the p-th dimension. , Then calculate the system's overall dimensional performance score for the current dimension. , In the formula, J p This is the set of secondary evaluation indexes contained in the p-th dimension, where the dimension numbers p=1,2,3 correspond to the three dimensions of hardware reliability, operating system effectiveness, and data analysis accuracy, respectively.

[0097] Furthermore, in step 5, the specific steps for defining the true weak link based on the dimensional comprehensive performance score, quantitatively characterizing the drag on the overall system performance of each indicator by constructing a bottleneck coefficient that considers cascading failures, and using the maximum bottleneck coefficient within the dimension as the predictive indicator of the weak link are as follows:

[0098] Step 5.1: For the j-th evaluation index of the i-th experimental sample, calculate the initial bottleneck coefficient. , , , In the formula, C ij (p) Let s be the actual contribution of the i-th experimental sample to the j-th evaluation index. ij (p) w is the standardized score of the i-th test sample on the j-th evaluation index. gj (p) Let j be the global weight of the j-th evaluation index. The average contribution of all evaluation indicators for all test samples;

[0099] Step 5.2: Fire reconnaissance robots often experience cascading failures between indicators in fire scenes. Construct the indicator coupling influence matrix within the p-th dimension. , In the formula, This represents the drag coefficient of the failure of the u-th evaluation index on the v-th evaluation index. It is calibrated using experimental data on the fire fault propagation mechanism, and then the cascading failure bottleneck coefficient is calculated. , In the formula, λ is the cascade amplification factor, determined by statistical data on fault propagation intensity under six standardized test scenarios. Cascade failure bottleneck coefficient. It not only measures the degree of lag of a single indicator, but also quantifies its drag effect on other indicators within the dimension through coupling relationships, accurately locating the source of cascading failures;

[0100] Step 5.3: For the p-th dimension, take the maximum value of the cascading failure bottleneck coefficient of all evaluation indicators as the predictive indicator of potential weaknesses in the p-th dimension. , In the formula, J p This is the set of secondary indicator indices contained in the p-th dimension;

[0101] Step 5.4: Based on the task standards or industry specifications for fire reconnaissance robots, preset the target performance score η for each dimension (hardware reliability, operating system effectiveness, data analysis accuracy). The target performance score η for each dimension is set independently. For the i-th test sample, the comprehensive performance score in the p-th dimension is... If D ip If η < 1, then the environment-dimensional sample is determined to be a true vulnerability and labeled as such. ip =1; otherwise, mark y ip =0;

[0102] Step 5.5, with As a predictor variable, all experimental samples were selected. As a candidate threshold space T∈[0,1], for each threshold τ, Samples with values ​​greater than τ are predicted as having weak dimensionality; therefore, predictive labels should be added. =1, otherwise it is predicted as non-weak and marked. =0;

[0103] Step 5.6, calculate sensitivity (true positive rate). And specificity (true negative rate) They are respectively: , In the formula, TP (true positive) is the number of weak samples correctly identified, TN (true negative) is the number of non-weak samples correctly identified, FP (false positive) is the number of non-weak samples that were misjudged as weak, and FN (false negative) is the number of weak samples that were missed.

[0104] Furthermore, in step 5, the specific steps for determining the optimal diagnostic threshold through cost-sensitive weighted Youden index co-optimization are as follows:

[0105] Step 5.7: In the application of fire reconnaissance robots, failing to identify a weak point (FN) may result in the robot being damaged in a fire, with a cost C. FN The cost of failure, C, due to a fire reconnaissance robot missing a weak point in a fire scene mission is far greater than the maintenance resource waste caused by misjudgment (FP). FN The cost of resource waste caused by misjudging weak points C FP Construct a cost-sensitive weighted Youden index , In the formula, To calculate sensitivity, For specificity, ,and ∈(0.5,1), reflecting a higher penalty for missed weak points, the cost of task failure C FN Let C be the expected loss value of the robot being damaged in the fire due to a weak point that was not identified, representing the cost of resource waste.FP The resource consumption value that triggers redundant maintenance due to misjudgment of weaknesses is determined based on the task risk level and equipment damage loss level specified in the fire reconnaissance robot task standard or industry specification.

[0106] Step 5.8: Use the fuzzy C-means clustering algorithm to cluster all experimental samples. Unsupervised clustering is performed to obtain two cluster centers, c1 (non-weak class) and c2 (weak class). The threshold interval [min(c1,c2), max(c1,c2)] is used for fine-grained search. Within this interval, the cluster centers are... Maximize the dimensionality search for the objective and select the option that makes The largest threshold is used as the optimal diagnostic threshold τ for the p-th dimension. p If multiple thresholds reach their maximum simultaneously Then, the median value of each threshold is taken as the optimal diagnostic threshold τ for the p-th dimension. p .

[0107] Furthermore, in step 5, the specific steps for classifying early warning levels and calculating the accuracy rate of weak point identification in the fire reconnaissance robot system are as follows:

[0108] Step 5.9, using the optimal diagnostic threshold τ p Perform dimensional weakness prediction to obtain predicted labels. and with real label y ip Compare and calculate the accuracy A of weak link identification. The accuracy rate A in identifying weak links reflects the diagnostic efficacy of the bottleneck coefficient for weak dimensions (based on the definition of dimensional comprehensive score).

[0109] Step 5.10, based on the binary classification diagnosis, analyze the results obtained through... τ p For test samples predicted to be weak, the warning level is further divided according to the degree of bottleneck coefficient exceedance to indicate the severity of the weakness. τ is known. p ∈[0,1], when ≤τ p When, it is considered that there are no weaknesses in this dimension (no warning); when >τ p The greater the degree of exceedance, the more serious the weakness, and the higher the warning level should be. To avoid failing to trigger severe warnings, an unequal interval division strategy with a tighter initial range followed by a looser one is adopted to ensure that each level has practical coverage. For the test sample in the p-th dimension, the warning level L is defined. ip for: In the formula, τ is the maximum bottleneck coefficient of the i-th experimental sample in the p-th dimension, with a value range of [0,1]. pFor the optimal diagnostic threshold in the p-th dimension, 1-τ p To represent the potential increase in the bottleneck coefficient, coefficients 0.2 and 0.5 are empirical parameters, when τ p Even when the values ​​are extreme, this classification method can still ensure the effectiveness of the early warning system. The early warning level is used as an auxiliary information output and is not included in the accuracy calculation.

[0110] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for comprehensive evaluation of performance and identification of weak links of a fire-fighting reconnaissance robot system, characterized in that, Includes the following steps: Step 1: Collect performance test data of the fire reconnaissance robot system under six pre-designed standardized test scenarios. The six standardized test scenarios are used to simulate different typical tasks and environmental conditions in a fire scene. Step 2: Standardize the system performance test data recorded in the standardized test scenarios of the fire reconnaissance robot to obtain the standardized score matrix for each test scenario; Step 3: Divide the system performance test data into three dimensions according to their properties, determine the weights of the three dimensions, and then determine the objective weights of the evaluation indicators in each dimension based on the standardized score matrix. Finally, obtain the global weight of each evaluation indicator relative to the overall goal. Step 4: Based on the standardized score matrix and global weights, calculate the comprehensive performance index and comprehensive performance score of each dimension of the fire reconnaissance robot system. Then, take the average of the comprehensive performance index and comprehensive performance score of each dimension of each test sample to obtain the overall comprehensive performance score and dimensional comprehensive performance score of the system. Step 5: Define the real weak link based on the dimensional comprehensive performance score. Quantitatively characterize the drag on the overall system performance of each index by constructing a bottleneck coefficient that considers cascading failures. Use the maximum bottleneck coefficient within the dimension as the predictive index of the weak link. Determine the optimal diagnostic threshold through cost-sensitive weighted Youden index co-optimization, classify the early warning levels, and calculate the accuracy of weak link identification in the fire reconnaissance robot system.

2. The method according to claim 1, wherein, In step 1, the six standardized test scenarios include explosion identification test under different smoke concentrations, hazardous chemical identification test, carbon monoxide identification test, trapped personnel identification test, building collapse identification test, and temperature resistance performance test; the system performance test data includes hardware reliability data, operating system effectiveness data, and data analysis accuracy data.

3. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 1, characterized in that, In step 2, the specific steps to obtain the standardized score matrix for each experimental scenario are as follows: Step 2.1: Based on the design scheme of the six standardized test scenarios, each basic condition formed by the combination of test elements in each standardized test scenario is defined as a test environment. Each test environment corresponds to a set of test parameters. Under each set of test parameters, five repeated tests are conducted to obtain five experimental results. The average value of the five experimental results is then calculated to obtain the test mean of each test environment. Step 2.2: Collect the test mean values ​​for six standardized test scenarios to obtain n test mean values ​​under n test environments. Treat each test mean value as a test sample, and each test sample contains m evaluation indicators. Then, denote the data matrix X as... In the formula, x ij Let be the test mean of the i-th test sample for the j-th evaluation index; Step 2.3: Perform Min-Max standardization on each evaluation indicator to eliminate the influence of dimensions. For extremely large indicators, the standardization formula is as follows: If the index is extremely small, the standardization formula is: In the formula, x ij Let x be the test mean of the i-th test sample for the j-th evaluation index, min(x) j and max(x) j ) represent the minimum and maximum values ​​of the evaluation index in all experimental samples, respectively. ij The standardized score ranges from [0,1], with larger values ​​indicating better performance. The resulting standardized score matrix is: .

4. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 1, characterized in that, In step 3, the specific steps for determining the weights of the three dimensions are as follows: Step 3.1: Based on expert knowledge in the field of fire reconnaissance robots, the importance of the three dimensions—hardware reliability, operating system effectiveness, and data analysis accuracy—is compared pairwise using the interval type-two fuzzy scaling method. An interval type-two fuzzy judgment matrix D is then constructed. In the formula, and Let the lower and upper bound scale values ​​represent the importance of the i-th dimension relative to the j-th dimension, respectively. Their assignment follows a scaling rule from 1 to 9. The interval type-II fuzzy judgment matrix D is then defuzzified to obtain the initial dimension weights for each dimension. ; Step 3.2: Based on the design schemes of the six standardized test scenarios, statistically analyze the frequency of occurrence or the proportion of hazard levels of each standardized test scenario in fire scene missions. k Given k=1,…,6, calculate the environmental entropy of the fire scene. , Then construct environmental regulation functions for each dimension. , In the formula, β p The p-th dimension is the environmental sensitivity coefficient, which is determined by the failure statistics of the fire reconnaissance robot in a real fire scene. Step 3.3, initial dimension weights Environmental entropy correction is performed to obtain dimensional weights. , Then construct a three-dimensional weight vector. , In the formula, w d1 w d2 and w d3 The weights are for each of the three dimensions; Step 3.4: Perform interval consistency test, calculate the interval consistency index of the interval judgment matrix, and then calculate the consistency ratio. If the consistency ratio is less than 0.10, the consistency of the interval judgment matrix is ​​accepted; otherwise, the interval judgment matrix is ​​adjusted.

5. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 3, characterized in that, In step 3, the specific steps for determining the objective weights of the evaluation indicators within each dimension based on the standardized score matrix are as follows: Step 3.5, let the p-th dimension contain q evaluation indicators, corresponding to the standardized data submatrix S. p S p ∈R n×q Then calculate the standard deviation of the j-th evaluation index under the p-th dimension. , In the formula, For the standardized data submatrix S p The element in the i-th row and j-th column, Let j be the mean of the j-th evaluation index across all experimental samples; Step 3.6: Calculate the correlation coefficient matrix between the evaluation indicators in the p-th dimension. , , In the formula, r uv Let be the correlation coefficient between the u-th evaluation index and the v-th evaluation index. as well as From the standardized score matrix respectively The standardized scores of the i-th experimental sample in the p-th dimension on the u and v evaluation metrics are extracted. These are the mean values ​​of the u-th and v-th indicators in the current dimension across all n experimental samples; Step 3.7, based on the correlation coefficient matrix The degree of information overlap between the indicators is measured, and the conflict coefficient between the j-th evaluation indicator and other evaluation indicators is calculated. , In the formula, r ju Correlation coefficient matrix The element in the j-th row and u-th column; Step 3.8, convert the standard deviation With conflict coefficient By combining these, we can obtain the comprehensive information content carried by the j-th evaluation index. , This product simultaneously reflects the discriminative power (standard deviation) and independence (conflict resolution) of the indicators. Then, the information content of all evaluation indicators under the p-th dimension is normalized to obtain the objective weights of each evaluation indicator. , In the formula, It is the sum of the information content of all evaluation indicators under the p-th dimension.

6. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 5, characterized in that, In step 3, the specific steps for obtaining the global weight of each evaluation indicator relative to the overall goal are as follows: Step 3.9: Calculate the global weight for the j-th evaluation index belonging to the p-th dimension. , In the formula, w dp The subjective weights for the p-th dimension determined in step 3, Let be the objective weight of the j-th evaluation indicator under the p-th dimension; this multiplicative fusion method achieves the complementarity between expert prior knowledge and the inherent laws of test data, which not only preserves the emphasis and preference of fire scene tasks on different dimensions, but also refines the weight through the distinguishability and independence of data-driven indicators. Step 3.10: Perform global weight calculation on all evaluation indicators across all dimensions to obtain the complete global weight vector. , In the formula, q1, q2, and q3 represent the number of evaluation indicators under the three dimensions, respectively. This is the weight vector of all evaluation indicators in the first dimension. This is the weight vector of all evaluation indicators in the second dimension. This is the weight vector for all evaluation indicators in the third dimension.

7. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 1, characterized in that, In step 4, the specific steps to obtain the overall comprehensive performance score and dimensional comprehensive performance score of the system are as follows: Step 4.1: Calculate the comprehensive performance index of the i-th test sample. , Then calculate the overall performance score of the system. , In the formula, s ij Let be the standardized score of the i-th test sample on the j-th evaluation index. Let be the global weight of the j-th evaluation indicator, and k be the total number of secondary indicators; Step 4.2: Calculate the overall performance score of the i-th test sample on the p-th dimension. , Then calculate the system's overall dimensional performance score for the current dimension. , In the formula, J p This is the set of secondary evaluation indexes contained in the p-th dimension, where the dimension numbers p=1,2,3 correspond to the three dimensions of hardware reliability, operating system effectiveness, and data analysis accuracy, respectively.

8. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 7, characterized in that, In step 5, the actual weak links are defined based on the dimensional comprehensive performance score. The extent to which each indicator drags down the overall system performance is quantitatively characterized by constructing a bottleneck coefficient that considers cascading failures. The specific steps for using the maximum bottleneck coefficient within a dimension as the predictive indicator of the weak link are as follows: Step 5.1: For the j-th evaluation index of the i-th experimental sample, calculate the initial bottleneck coefficient. , , , In the formula, C ij (p) Let s be the actual contribution of the i-th experimental sample to the j-th evaluation index. ij (p) w is the standardized score of the i-th test sample on the j-th evaluation index. gj (p) Let j be the global weight of the j-th evaluation index. The average contribution of all evaluation indicators for all test samples; Step 5.2, construct the index coupling influence matrix in the p-th dimension. , In the formula, This represents the drag coefficient of the failure of the u-th evaluation index on the v-th evaluation index. It is calibrated using experimental data on the fire fault propagation mechanism, and then the cascading failure bottleneck coefficient is calculated. , In the formula, λ is the cascade amplification factor, which is determined by the statistical data of fault propagation intensity under six standardized test scenarios; Step 5.3: For the p-th dimension, take the maximum value of the cascading failure bottleneck coefficient of all evaluation indicators as the predictive indicator of potential weaknesses in the p-th dimension. , In the formula, J p This is the set of secondary indicator indices contained in the p-th dimension; Step 5.4: Based on the task standards or industry specifications for fire reconnaissance robots, preset the target performance score η for each dimension. The target performance score η for each dimension is set independently. For the i-th test sample, the comprehensive performance score in the p-th dimension is... If D ip If η < 1, then the environment-dimensional sample is determined to be a true vulnerability and labeled as such. ip =1; otherwise, mark y ip =0; Step 5.5, with As a predictor variable, all experimental samples were selected. As a candidate threshold space T∈[0,1], for each threshold τ, Samples with values ​​greater than τ are predicted as having weak dimensionality; therefore, predictive labels should be added. =1, otherwise it is predicted as non-weak and marked. =0; Step 5.6, Calculate sensitivity and specificity They are respectively: , In the formula, TP is the number of correctly identified weak samples, TN is the number of correctly identified non-weak samples, FP is the number of non-weak samples that were misclassified as weak, and FN is the number of weak samples that were missed.

9. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 8, characterized in that, In step 5, the specific steps for determining the optimal diagnostic threshold through cost-sensitive weighted Youden exponent co-optimization are as follows: Step 5.7, based on the cost C of mission failure caused by the fire reconnaissance robot failing to identify weak points during fire scene operations. FN The cost of resource waste caused by misjudging weak points C FP Construct a cost-sensitive weighted Youden index , In the formula, To calculate sensitivity, For specificity, ,and ∈(0.5,1), reflecting a higher penalty for weak links that are missed in judgment; Step 5.8: Use the fuzzy C-means clustering algorithm to cluster all experimental samples. Unsupervised clustering is performed to obtain two cluster centers c1 and c2. The threshold [min(c1,c2),max(c1,c2)] is used as the fine-grained search interval, and within this interval, the cluster centers are... Maximize the dimensionality search for the objective and select the option that makes The largest threshold is used as the optimal diagnostic threshold τ for the p-th dimension. p If multiple thresholds reach their maximum simultaneously Then, the median value of each threshold is taken as the optimal diagnostic threshold τ for the p-th dimension. p .

10. The method for comprehensive performance evaluation and weak link identification of the fire reconnaissance robot system according to claim 8, characterized in that, Step 5, the specific steps for classifying early warning levels and calculating the accuracy of weak point identification in the fire reconnaissance robot system are as follows: Step 5.9, using the optimal diagnostic threshold τ p Perform dimensional weakness prediction to obtain predicted labels. and with real label y ip Compare and calculate the accuracy A of weak link identification. ; Step 5.10, based on the binary classification diagnosis, analyze the results obtained through... τ p For test samples predicted to be weak, the warning level is further divided according to the degree of bottleneck coefficient exceedance to indicate the severity of the weakness. For test samples in the p-th dimension, the warning level L is defined. ip for: In the formula, τ is the maximum bottleneck coefficient of the i-th experimental sample in the p-th dimension, with a value range of [0,1]. p For the optimal diagnostic threshold in the p-th dimension, 1-τ p This represents potential room for improvement in the bottleneck coefficient.

Images