Flight skill challenge competition scoring system based on multi-modal reaction

Abstract

The invention relates to the technical field of aerospace, in particular to a flight skill challenge competition scoring system based on multi-modal reaction, which comprises a multi-modal data acquisition and fusion unit, a real-time analysis and dynamic scoring unit, a three-dimensional visualization and interaction unit, a system safety and reliability unit, an expansion and compatibility unit and a training and aid decision making unit. According to the invention, multi-source sensors such as an accelerometer, a gyroscope, a GPS, a barometer and a camera are integrated, and an inertial navigation enhancement algorithm and a YOLOv8 target detection model are combined, so that the system realizes full-dimensional acquisition and fusion of flight attitude, position, environment and visual data, and compared with a traditional single sensor system, the system has the advantages that the data precision is greatly improved; a score skill scoring algorithm is adopted, three dimensions of flight stability, accuracy and environment adaptability are combined, and a weight coefficient is determined through an AHP (Analytic Hierarchy Process), so that the scoring error rate can be reduced.

Description

TECHNICAL FIELD

[0001] The application relates to the field of aerospace technology, in particular to a flight skill challenge competition scoring system based on multi-modal response. BACKGROUND

[0002] The flight skill challenge competition scoring system is a standardized tool for objectively evaluating the flight control ability and task completion quality of contestants. Its core function is to quantitatively analyze the operation accuracy, task execution efficiency and safety of the aircraft through preset indicators. The system usually collects key data such as flight trajectory, attitude stability, obstacle avoidance and target hitting in real time based on sensors, visual recognition and other technologies, and combines time efficiency, energy consumption and other parameters for multi-dimensional scoring.

[0003] The traditional scoring system only relies on flight parameters (such as QAR data), lacks the fusion of multi-modal data such as vision and environment, resulting in one-sided evaluation, and manual scoring takes several hours, which cannot meet the real-time feedback needs of high dynamic flight scenes (such as aerobatics). The traditional system lacks electromagnetic compatibility design and is prone to data interruption or misjudgment in complex electromagnetic environments. The hardware and software are highly coupled, and it takes several months to adapt to new models or sensors, which cannot meet the rapid iteration needs. SUMMARY

[0004] The purpose of the present application is to provide a flight skill challenge competition scoring system based on multi-modal response to solve the problems raised in the background.

[0005] To achieve the above purpose, the present application provides the following technical scheme: a flight skill challenge competition scoring system based on multi-modal response, comprising a multi-modal data acquisition and fusion unit, a real-time analysis and dynamic scoring unit, a three-dimensional visualization and interaction unit, a system safety and reliability unit, an expansion and compatibility unit, and a training and auxiliary decision-making unit. The multi-modal data acquisition and fusion unit comprises a multi-source sensor integration module, a wireless data transmission module, a data encryption and integrity verification module, and a multi-modal data fusion engine.

[0006] Preferably, the multi-source sensor integration module integrates sensors including an accelerometer, a gyroscope, a GPS, a barometer and a camera, and adopts an inertial navigation enhancement algorithm.

[0007] Preferably, the inertial navigation enhancement algorithm is as follows: State equation: ; In the formula, X is the system state vector at time k, F is the state transition matrix, B is the control input matrix, and u is the control input vector. k k k k ​​​is the control quantity, w k is the process noise; The barometer height compensation equation is: In the formula, h baro is the barometer measured height, h GPS is the GPS measured height, and N is the sliding window length.

[0008] Preferably, the real-time analysis and dynamic scoring unit comprises a flight phase identification module, an overrun event detection module, a dynamic scoring model module, a real-time feedback generation module, and a training and auxiliary decision-making unit.

[0009] Preferably, the overrun event detection module adopts an adaptive threshold algorithm, and the algorithm formula is as follows: In the formula, μk-1 is the sliding window mean, σk-1 is the sliding window standard deviation, and λ is a dynamic coefficient.

[0010] Preferably, the dynamic scoring model module adopts a score skill scoring algorithm, and the score skill scoring algorithm is as shown below: In the formula, S stability is the stability score, S precision is the precision score, and S adaptability is the environmental adaptation score, and w1, w2, and w3 are weight coefficients.

[0011] Preferably, the three-dimensional visualization and interaction unit comprises a digital twin modeling module, a 3D visual restoration module, an interactive instrument panel module, and a referee scoring interface module.

[0012] Preferably, the system safety and reliability unit comprises a dual-machine hot backup and disaster recovery module, an edge computing and load balancing module, a permission hierarchical management module, and an electromagnetic compatibility and anti-interference module.

[0013] Preferably, the extension and compatibility unit comprises a multi-machine type adaptation interface module, a sensor plug-and-play module, a cloud collaborative analysis module, and a rule engine self-definition module.

[0014] Preferably, the training and auxiliary decision-making unit comprises a historical data comparison and analysis module, a virtual training scene generation module, an AI coach auxiliary module, and an emergency decision support module.

[0015] Compared with the prior art, the present application has the following beneficial effects: ​​​The application integrates multiple source sensors such as accelerometers, gyroscopes, GPS, barometers, cameras, etc., combines inertial navigation enhancement algorithm and YOLOv8 target detection model, realizes full-dimensional collection and fusion of flight attitude, position, environment and visual data, compared with traditional single sensor system, the data accuracy is greatly improved, the score algorithm is adopted, combined with flight stability, accuracy and environmental adaptability three dimensions, the weight coefficient is determined by AHP hierarchical analysis method, so as to reduce the scoring error rate, and through the dynamic protocol loading technology, the data format of various models is adapted, and the rapid iteration demand is met. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The system principle diagram of the application is shown in the figure. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0018] The flight skill challenge race scoring system based on multi-modal response includes a multi-modal data acquisition and fusion unit, a real-time analysis and dynamic scoring unit, a three-dimensional visualization and interaction unit, a system safety and reliability unit, an expansion and compatibility unit, and a training and auxiliary decision unit. The multi-modal data acquisition and fusion unit includes a multi-source sensor integration module, a wireless data transmission module, a data encryption and integrity verification module, and a multi-modal data fusion engine. The multi-source sensor integration module integrates sensors such as accelerometers, gyroscopes, GPS, barometers, and cameras. The multi-source sensor integration module uses an inertial navigation enhancement algorithm to collect flight attitude, position, environment, and visual data in real time. The multi-source sensor integration module uses a YOLOv8 target detection model for visual collection. The loss function. The wireless data transmission module is based on CC1101 radio frequency module or 5G / WiFi technology to realize low-latency, high-reliability air-ground data real-time transmission. The encryption and integrity verification module uses AES encryption algorithm and hash check to ensure the security and tamper-proofing ability of the transmission data. The multi-modal data fusion engine uses Kalman filtering and deep learning algorithm to fuse multi-sensor data and generate high-precision flight state model. Inertial navigation enhancement algorithm (integrated navigation): State equation: ; In the formula: X k is the system state vector (position, velocity, attitude) at time k, F kis the state transition matrix, B k is the control input matrix, u k is the control quantity (such as accelerometer output), w k is the process noise (covariance Q); The barometer height compensation equation is: ; In the formula, h baro is the barometer measured height, h GPS is the GPS measured height, and N is the sliding window length.

[0019] The real-time analysis and dynamic scoring unit includes a flight phase identification module, an overrun event detection module, a dynamic scoring model module, a real-time feedback generation module, and a training and auxiliary decision unit. The flight phase identification module automatically divides the flight phases such as take-off, climbing, stunt, and landing based on time series analysis. The overrun event detection module dynamically monitors flight parameters (such as overload, pitch angle, and airspeed) and triggers real-time alarms in combination with an expert knowledge base. The dynamic scoring model module uses the score technique scoring (FSS) algorithm to generate a comprehensive score in combination with flight stability, action accuracy, and environmental adaptability. The real-time feedback generation module provides action correction suggestions and score details for the player through voice, vibration, or screen prompts. The overrun event detection module uses an adaptive threshold algorithm: ; In the formula, μ k-1 is the sliding window mean (at time k-1), σ k-1 is the sliding window standard deviation (at time k-1), and λ is the dynamic coefficient (usually taken as 2-3). Score technique scoring (FSS) algorithm: ; In the formula, S stability is the stability score (such as attitude angle variance), S precision is the accuracy score (such as trajectory tracking error), and S adaptability is the environmental adaptation score (such as wind field compensation ability), and w1, w2, and w3 are weight coefficients (determined through AHP hierarchical analysis method).

[0020] The three-dimensional visualization and interaction unit includes a digital twin modeling module, a 3D visual restoration module, an interactive instrument panel module, and a judge scoring interface module. The digital twin modeling module constructs a real-time digital twin model based on flight data, simulates the interaction between the aircraft attitude and the environment, and integrates the FlightGear engine to support multi-view (cockpit, God view) flight process playback. The interactive instrument panel module dynamically displays airspeed, height, heading, and other parameter curves, supports touch zooming and data drilling, and the judge scoring interface module provides a drag-type scoring panel, supports multi-dimensional scoring labeling and real-time ranking updating.

[0021] The system safety and reliability unit includes a dual-machine hot standby and disaster recovery module, an edge computing and load balancing module, a hierarchical access control module, and an electromagnetic compatibility and anti-interference module. The dual-machine hot standby and disaster recovery module adopts a distributed server architecture to achieve real-time data backup and automatic fault switching. The edge computing and load balancing module deploys edge computing nodes on the airborne end to reduce cloud dependence and improve response speed. The hierarchical access control module supports multi-role access control for referees, players, administrators, etc., to ensure compliance of data operations. The electromagnetic compatibility and anti-interference module can optimize hardware circuit design and cope with complex electromagnetic interference through shielding and filtering technologies.

[0022] The Expansion and Compatibility Unit comprises a multi-aircraft adaptation interface module, a sensor plug-and-play module, a cloud-based collaborative analysis module, and a custom rules engine module. This unit builds full-scenario flight data processing capabilities through four core modules. The multi-aircraft adaptation interface module employs dynamic protocol loading technology, enabling flexible parsing of proprietary data formats from over 20 aircraft, including Cessna, DA42, helicopters, and drones, facilitating cross-platform bidirectional interaction between flight control, navigation parameters, and onboard equipment. The sensor plug-and-play module, developed based on the IEEE-1451 intelligent sensing standard, uses adaptive calibration algorithms to be compatible with various devices such as Garmin avionics systems, Honeywell pressure sensors, and LiDAR, supporting hot-swapping of devices and automatic topology recognition, as well as cloud-based collaboration. The analytics module deeply integrates AWS IoT Core and Azure Machine Learning services to build a distributed data lake with petabyte-level storage capabilities. It integrates 12 analytics models, including flight trajectory prediction and anomaly pattern recognition, and supports real-time processing of key flight parameters by edge computing nodes. The custom rules engine module provides a visual decision tree editor, allowing users to define over 500 scoring dimension combinations through drag-and-drop operations. It can dynamically configure airspace compliance detection, aerobatic maneuver scoring, and multi-level alarm thresholds (such as overload coefficient and angle of attack limit). Through containerization technology, it enables rule sets to be deployed and rolled back in seconds, fully adapting to the differentiated needs of international air sports events, flight training assessments, and other scenarios. At the same time, it reserves API interfaces to meet the automated reporting requirements of the Civil Aviation Administration of China for airworthiness certification data.

[0023] The training and decision support unit includes a historical data comparison and analysis module, a virtual training scenario generation module, an AI coach assistance module, and an emergency decision support module. The historical data comparison and analysis module, based on SQL Server 2019 column-store database and parallel computing technology, achieves real-time analysis of millions of flight parameters per second. It supports cross-season trend comparisons of over 50 indicators, including overload curves, rudder effect analysis, and heading stability, and generates 3D visualized weakness heatmaps and structured PDF reports. The virtual training scenario generation module integrates NOAA meteorological satellite and ECMWF numerical forecast data streams, combined with irregular triangular network (TIN) terrain modeling algorithms and 3D laser point cloud reconstruction technology. It can dynamically construct immersive training environments including 15 high-risk scenarios such as thunderstorm shear, icing effects, and volcanic ash interference, and achieves millimeter-level operation synchronization between the physical cockpit and virtual space through the Varjo-XR-4 headset. The AI ​​coach assistance module deeply integrates the GPT-4 multimodal large model and deep learning technology. The DRL (Reinforcement Learning) algorithm automatically generates customized training programs covering areas such as maneuver decomposition training, formation tactics optimization, and fuel efficiency improvement through multi-dimensional input of voice commands, flight parameters, and biosensor data. It supports multilingual voice feedback and Markdown-formatted training log output and can seamlessly integrate with educational management systems such as Moodle and Blackboard. The emergency decision support module has a built-in expert knowledge graph containing more than 2,000 aviation accident cases. When 17 types of special situations such as engine surge, hydraulic failure, and fuel leakage are detected, a decision tree containing alternative airport selection, power distribution strategy, and emergency communication sequence is dynamically generated based on the Q-learning reinforcement learning algorithm. The optimal landing path and operation guidance are overlaid on the real cockpit view through the AR perspective head-up display. It also supports voice navigation command broadcasting and differentiated handling logic configuration for multiple aircraft types (fixed-wing / rotor aircraft / electric vertical take-off and landing aircraft), effectively shortening the special situation handling decision chain to within 3 seconds.

[0024] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] This invention integrates multiple sensors, including accelerometers, gyroscopes, GPS, barometers, and cameras, combined with inertial navigation enhancement algorithms and the YOLOv8 target detection model. The system achieves comprehensive acquisition and fusion of flight attitude, position, environmental, and visual data. Compared to traditional single-sensor systems (such as those relying solely on GPS or inertial navigation), data accuracy is improved by over 30%. Especially in complex weather or electromagnetic interference environments, the data integrity verification module (AES encryption + hash verification) ensures 99.9% anti-interference capability for data transmission. The system employs a fractional skill score (FSS) algorithm, combining flight stability (attitude angle variance), accuracy (trajectory tracking error), and environmental adaptability (wind field compensation) across three dimensions, and uses the Analytic Hierarchy Process (AHP) to determine weighting coefficients. The scoring error rate has been reduced from 15% in traditional manual assessment to less than 2%, and dynamic alarms for exceeding limits (such as overload and abnormal pitch angle) are supported. The response time has been shortened to 50ms. A digital twin model is built based on the FlightGear engine, supporting multi-view (cockpit / God's-eye view) playback. The interactive dashboard can dynamically display parameter curves such as airspeed and altitude. The efficiency of judge scoring has been improved by 40%. The pilots can obtain operation correction suggestions in real time through the AR perspective head-up display. The training feedback cycle has been shortened by 60%. The hot standby architecture enables automatic fault switching. Edge computing nodes reduce reliance on the cloud. The electromagnetic compatibility design uses shielding / filtering technology to cope with complex electromagnetic environments. The system availability reaches 99.99%, the risk of data leakage is reduced by 90%, and it meets the airworthiness certification requirements of the Civil Aviation Administration of China.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A flight skill challenge scoring system based on multi-modal responses, comprising a multi-modal data acquisition and fusion unit, a real-time analysis and dynamic scoring unit, a three-dimensional visualization and interaction unit, a system safety and reliability unit, an extension and compatibility unit, and a training and assisted decision-making unit, characterized in that: The multi-modal data acquisition and fusion unit comprises a multi-source sensor integration module, a wireless data transmission module, a data encryption and integrity verification module, and a multi-modal data fusion engine.

2. The multi-modal response based flight skills challenge scoring system of claim 1, wherein: The sensors integrated by the multi-source sensor integration module include an accelerometer, a gyroscope, a GPS, a barometer, and a camera, and the multi-source sensor integration module adopts an inertial navigation enhancement algorithm.

3. The multi-modal response based flight skills challenge scoring system of claim 2, wherein: The inertial navigation enhancement algorithm comprises the following steps: State equation: ； where: X k is the system state vector at time k, F k is the state transition matrix, B k is the control input matrix, u k is the control, and w k is the process noise. The barometer height compensation equation is as follows: ； where h baro is the barometer height, h GPS is the GPS height, N is the length of the sliding window.

4. The multi-modal response based flight skills challenge scoring system of claim 1, wherein: The real-time analysis and dynamic scoring unit comprises a flight phase identification module, an overrun event detection module, a dynamic scoring model module, a real-time feedback generation module, and a training and auxiliary decision unit.

5. The multi-modal response based flight skills challenge scoring system of claim 4, wherein: The overrun event detection module adopts an adaptive threshold algorithm, and the algorithm formula is as follows: ； In the formula, μk-1 is a sliding window mean, σk-1 is a sliding window standard deviation, and λ is a dynamic coefficient.

6. The multi-modal response based flight skills challenge scoring system of claim 4, wherein: The dynamic scoring model module adopts a score trick scoring algorithm, and the score trick scoring algorithm is as shown below: ； In the formula: S stability is a stability score, S precision is a precision score, S adaptability is an environmental adaptation score, w1, w2, w3 are weight coefficients.

7. The multimodal response based flight skills challenge scoring system of claim 1, wherein: The three-dimensional visualization and interaction unit comprises a digital twin modeling module, a 3D visual restoration module, an interactive instrument panel module, and a judge scoring interface module.

8. The multimodal response based flight skills challenge scoring system of claim 1, wherein: The system security and reliability unit comprises a dual-computer hot backup and disaster recovery module, an edge computing and load balancing module, a permission hierarchical management module, and an electromagnetic compatibility and anti-interference module.

9. The multimodal response based flight skills challenge scoring system of claim 1, wherein: The extension and compatibility unit comprises a multi-machine type adaptation interface module, a sensor plug-and-play module, a cloud collaborative analysis module, and a rule engine self-definition module.

10. The multimodal response based flight skills challenge scoring system of claim 1, wherein: The training and auxiliary decision unit comprises a historical data comparative analysis module, a virtual training scene generation module, an AI coach auxiliary module, and an emergency decision support module.

Images