ATC Speech Recognition Evaluation Using Multi-Layer Index Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems for air traffic control lack a unified quantitative evaluation method, leading to inconsistent performance and recognition quality, which can compromise flight safety and communication efficiency.
Innovation Solution
An evaluation method and system that grades ATC speech data based on quality levels, using a multi-layer index system with utility and support indexes, and applies the improved group-G2 and CRITIC methods to determine weights and scores, ultimately providing a comprehensive score for recognition performance through the TOPSIS method.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If different speech recognition systems emphasize different evaluation indexes (response time, accuracy), then each system can be optimized for its specific focus, but there is no unified quantitative evaluation method to compare them consistently
Solution Approach 1:
The patent establishes a universal evaluation index system that can assess multiple aspects of speech recognition systems (accuracy, response time, resource consumption) through a unified quantitative framework. This allows different systems with varying optimizations to be compared consistently using the same evaluation methodology, resolving the contradiction between flexible system optimization and consistent evaluation measurement.
2Speed
If speech recognition systems prioritize speed (fast speech speed handling), then response time improves, but recognition accuracy may deteriorate
Solution Approach 1:
The patent implements a dynamic evaluation approach where the weight coefficients of different evaluation indexes can be adjusted based on specific application scenarios and requirements. This allows the system to dynamically balance between speed and accuracy priorities, enabling optimization for response time when needed while maintaining the capability to prioritize accuracy when required, thus resolving the static trade-off between these conflicting parameters.
3Loss of information
If speech recognition systems use complex evaluation criteria to capture all performance aspects, then evaluation comprehensiveness improves, but evaluation complexity and difficulty increase
Solution Approach 1:
The patent segments the evaluation system into two distinct layers: the utility layer (first evaluation index set) containing directly measurable performance metrics, and the support layer (second evaluation index set) containing underlying system attributes. This segmentation allows comprehensive evaluation of multiple performance aspects while organizing complexity into manageable, structured modules with clear relationships between layers, reducing the difficulty of implementation and interpretation.
Data Source
AI summary
An evaluation method and system of an air traffic control (ATC) speech recognition system are provided, the method includes: S1, obtaining, grading and inputting ATC speech data; S2, constructing an evaluation index system; S3, determining weights of utility layer indexes and support layer indexes under different levels ATC speech data; S4, calculating a utility layer score and a support layer score, and adding the scores of the utility layer and the support layer to obtain comprehensive scores of the utility layer and the support layer; S5, determining, weights of the utility layer and the support layer; S6, adding a product of the utility layer weight with the utility layer comprehensive score and a product of the support layer weight with the support layer comprehensive score to obtain a comprehensive score of the speech recognition system; and S7, determining a level of the recognition performance of the speech recognition system.