Multilingual training system and training method for unmanned aerial vehicle

Abstract

This invention belongs to the field of UAV education and training technology, specifically a multilingual UAV training system and method. It includes a multilingual interaction module, a UAV simulation training module, a multilingual teaching resource library, an adaptive training management module, a multilingual command teaching and evaluation module, and a data storage module. This invention integrates multilingual support throughout all stages of theoretical teaching, command teaching, simulated flight training, and evaluation feedback. Users can select the training language according to their own language proficiency, and the system automatically adapts to the interface language, voice interaction language, and teaching resource language. This solves the problem of single-language support in existing UAV training systems and facilitates the international export of UAV training resources.

Description

Technical Field

[0001] This invention relates to the field of drone education and training technology, specifically to a multilingual drone training system and training method. Background Technology

[0002] With the rapid development of the low-altitude economy and the continuous expansion of drone application scenarios, the demand for drone operator training is experiencing explosive growth. According to the Civil Aviation Administration of China, the market size of China's low-altitude economy will reach 1.5 trillion yuan by 2025, with a shortage of 1 million drone operators alone. As of early June 2025, the number of drone training institutions nationwide had exceeded 2,500. At the same time, the internationalization of drone education and training is accelerating, with overseas training programs for drone application technology already implemented in several countries, including Laos.

[0003] However, existing drone training systems have the following technical shortcomings: Limited language support: Existing drone training systems mostly present their user interfaces, voice commands, and teaching materials in a single language, which cannot meet the training needs of multilingual learners and limits their international promotion.

[0004] The voice control and training systems are disconnected: Existing drone voice control technology is mainly applied to flight control scenarios, such as wearable custom voice remote controllers and AI-based airborne voice control systems. However, multilingual voice interaction technology has not been systematically integrated into the entire training and teaching process, and there is a lack of voice command teaching and evaluation mechanisms oriented towards training scenarios.

[0005] Lack of multilingual adaptive training capabilities: Existing training systems mostly adopt a uniform teaching process and assessment standards, which cannot dynamically adjust training strategies according to the learners' language proficiency and learning progress, resulting in inconsistent learning outcomes.

[0006] Insufficient integration of simulation training and language teaching: Existing UAV simulation training systems mainly focus on the simulation training of flight control skills, such as UAV application simulation training systems and VR-based remote controller training methods. However, they are relatively weak in the function of simultaneously teaching multilingual professional terminology and training command comprehension during the training process.

[0007] Therefore, there is an urgent need to develop a drone training system and method that can support multilingual interaction and achieve personalized adaptive training. Summary of the Invention

[0008] The purpose of this invention is to provide a multilingual training system and method for unmanned aerial vehicles (UAVs) to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a multilingual training system for unmanned aerial vehicles (UAVs), comprising a multilingual interaction module, a UAV simulation training module, a multilingual teaching resource library, an adaptive training management module, a multilingual instruction teaching and evaluation module, and a data storage module; The multilingual interaction module serves as the system's human-computer interaction entry point. It is directly connected to the UAV simulation training module, the adaptive training management module, and the multilingual instruction teaching and evaluation module. It is responsible for receiving multilingual voice commands from users and issuing operational intentions, while broadcasting system feedback to users in multilingual form. The multilingual teaching resource library provides unified multilingual courseware, demonstration videos, test questions, and terminology comparisons as basic resources for the adaptive training management module and the multilingual instruction teaching and assessment module, and is the core support for the system's teaching content; The UAV simulation training module receives control commands from the multilingual interaction module, executes virtual flight, and transmits flight status data back to the adaptive training management module in real time as a basis for evaluating trainees' abilities. The multilingual instruction teaching and assessment module collects students' pronunciation through the multilingual interaction module, completes the assessment by combining standard instruction samples from the multilingual teaching resource library, and uploads the assessment results to the adaptive training management module. The adaptive training management module coordinates the overall situation and connects to the multilingual teaching resource library, the UAV simulation training module, the multilingual instruction teaching and evaluation module, and the data storage module. It dynamically plans training paths and adjusts training content based on student data. The data storage module uniformly receives and stores student files, training records, flight data, evaluation results, and system logs from all other modules, enabling full-process data traceability, analysis, and reuse.

[0010] Preferably, the multilingual interaction module includes a language selection unit, a voice acquisition unit, a voice recognition unit, a semantic understanding unit, and a voice synthesis unit; the language selection unit is used to receive the training language set by the user; the voice acquisition unit is used to acquire voice commands issued by the user; the voice recognition unit recognizes the acquired voice commands based on a multilingual voice recognition model and outputs the corresponding text information; the semantic understanding unit uses natural language processing technology to perform intent parsing on the text information and generate structured operation commands; the voice synthesis unit is used to broadcast the system feedback information in the selected language.

[0011] Preferably, the UAV simulation training module includes a 3D scene construction unit, a UAV physical simulation unit, a flight control simulation unit, and a flight status monitoring unit; the 3D scene construction unit is used to construct a 3D virtual flight scene; the UAV physical simulation unit constructs a virtual UAV model based on an aerodynamic model and a UAV dynamics model; the flight control simulation unit receives structured operation commands generated by the multilingual interaction module and controls the virtual UAV to perform corresponding flight operations in the 3D virtual flight scene; the flight status monitoring unit collects flight status data of the virtual UAV in real time.

[0012] Preferably, the multilingual teaching resource library includes a multilingual theoretical courseware library, a multilingual operation demonstration library, a multilingual question bank, and a terminology comparison library; the multilingual theoretical courseware library stores UAV theoretical knowledge courseware written in multiple languages; the multilingual operation demonstration library stores UAV operation demonstration videos dubbed in multiple languages; the multilingual question bank stores test questions presented in multiple languages; and the terminology comparison library stores the mapping relationship between UAV professional terms in different languages.

[0013] Preferably, the adaptive training management module includes a student profile building unit, a competency assessment unit, a training path planning unit, and a progress tracking unit. The student profile building unit collects basic information and initial language proficiency of students to build a student profile model. The competency assessment unit dynamically assesses students' comprehensive competency level based on multi-dimensional data such as theoretical test scores, accuracy of operation command recognition, and flight operation completion rate during the training process. The training path planning unit matches suitable learning resources from the multilingual teaching resource library based on the student profile and competency assessment results to dynamically generate personalized training paths. The progress tracking unit records students' training progress and adjusts subsequent training content in real time based on competency assessment results.

[0014] Preferably, the multilingual instruction teaching and evaluation module includes a standard instruction library, an instruction pronunciation teaching unit, an instruction recognition evaluation unit, and a feedback guidance unit. The standard instruction library stores the standard pronunciation text and audio samples of standard UAV operation instructions in multiple languages. The instruction pronunciation teaching unit is used to display and play the pronunciation of standard instructions to students, guiding them to practice reading along. The instruction recognition evaluation unit collects the voice instructions issued by students, compares them with the standard pronunciation in the standard instruction library, and evaluates the accuracy, speed, and completeness of the students' instruction pronunciation. The feedback guidance unit generates personalized pronunciation correction suggestions and reinforcement practice content for students based on the evaluation results.

[0015] A multilingual training method for drones, the specific steps of which are as follows: S1: Initialization and Language Configuration: The user selects the target training language through the multilingual interaction module. The system loads the corresponding user interface language pack, speech recognition model and speech synthesis engine for that language, and extracts the teaching resource index for that language from the multilingual teaching resource library. S2: Student Profile Construction: The system collects basic information, initial language level and drone operation skills of students to construct an initial student profile model; S3: Multilingual theoretical teaching: Based on the student profile, the system selects suitable theoretical courseware from the multilingual teaching resource library, presents the theoretical knowledge of UAVs to the students in the selected language, and conducts voice explanation and interactive Q&A through the multilingual interactive module; S4: Multilingual instruction teaching: The system loads the UAV operation instructions required for the current training stage from the standard instruction library, and demonstrates the pronunciation of the standard instructions to the trainees through the instruction pronunciation teaching unit. After the trainees practice reading along, the instruction recognition and evaluation unit conducts real-time evaluation and provides feedback and correction suggestions. S5: Multilingual Simulated Flight Training: Trainees issue operation commands via voice. The voice acquisition unit collects the trainee's voice and generates structured operation commands through voice recognition and semantic understanding to control the virtual drone to perform flight operations. At the same time, the flight status monitoring unit collects flight data in real time and feeds it back to the trainee's interface. S6: Comprehensive Ability Assessment: The ability assessment unit integrates the trainees' performance in theoretical teaching, command evaluation, and simulated flight training to generate a multi-dimensional ability assessment report; S7: Adaptive Path Adjustment: The training path planning unit dynamically adjusts subsequent training content based on the ability assessment results. It adds reinforcement training modules for weak links and automatically jumps to advanced content for content that has been mastered well. S8: Iterative training: Repeat steps S3 to S7 until the trainee completes all training content and passes the final assessment.

[0016] Preferably, in step S6, the multi-dimensional capability assessment report includes indicators such as theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. In step S5, the speech recognition adopts a multilingual end-to-end speech recognition model based on the Transformer architecture, and the semantic understanding adopts a large language model to map the recognized text into structured operation commands.

[0017] Preferably, in step S6, the capability assessment includes a comprehensive evaluation of three dimensions: theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. Each dimension is weighted according to a preset weight to calculate a comprehensive score.

[0018] Preferably, in step S7, the dynamic adjustment of the training path includes: reducing the training difficulty level when the trainee's comprehensive ability score is lower than a first preset threshold, and automatically increasing the training difficulty level when the trainee's comprehensive ability score increases by more than a second preset threshold twice in a row.

[0019] Compared with the prior art, the beneficial effects of the present invention are: 1) This invention integrates multilingual support into all aspects of theoretical teaching, instruction teaching, simulated flight training, and evaluation feedback. Users can choose the training language according to their own language ability. The system automatically adapts the interface language, voice interaction language, and teaching resource language, which solves the problem of the single language support of existing UAV training systems and is conducive to the international output of UAV training resources. 2) This invention innovatively applies multilingual speech recognition and semantic understanding technology to the drone simulation training scenario. Trainees can directly issue operation commands through voice. The system recognizes, parses and drives the virtual drone to perform operations in real time, realizing a "mouth instead of hand" training interaction method, reducing the dependence on traditional remote control operation, and providing a more intuitive and low-threshold learning experience for multilingual trainees. This invention organically integrates multilingual pronunciation teaching of standard UAV operating instructions with real-time recognition and evaluation. It not only solves the problem of multilingual learners having difficulty mastering UAV professional terminology, but also realizes a "learn-practice-test" closed loop through real-time evaluation and feedback mechanisms, effectively improving learners' mastery efficiency of multilingual instructions. 3) This invention constructs a trainee profile and conducts competency assessment by integrating multi-dimensional data such as theoretical test scores, command recognition accuracy, and flight operation completion rate. Based on this, the training path is dynamically adjusted, realizing personalized training tailored to individual needs, which significantly improves training efficiency and trainee pass rate. 4) The terminology database set up in this invention stores the mapping relationship between UAV professional terms in multiple languages. Trainees can look up the cross-language comparison of terms at any time during the training process, which helps to cultivate compound UAV talents with cross-language professional communication skills. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall architecture of the UAV multilingual training system of the present invention; Figure 2 A structural diagram of the multilingual interaction module; Figure 3 This is a structural block diagram of the UAV simulation training module; Figure 4 A structural diagram of a multilingual teaching resource database; Figure 5 This is a structural diagram of the adaptive training management module; Figure 6This is a structural diagram of the multilingual instruction teaching and assessment module. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example: Please see Figure 1-6 The present invention provides a technical solution: The UAV multilingual training system includes a multilingual interactive module, a UAV simulation training module, a multilingual teaching resource library, an adaptive training management module, a multilingual instruction teaching and evaluation module, and a data storage module; The multilingual interaction module serves as the system's human-computer interaction entry point. It is directly connected to the UAV simulation training module, the adaptive training management module, and the multilingual instruction teaching and evaluation module. It is responsible for receiving multilingual voice commands from users and issuing operational intentions, while broadcasting system feedback to users in multilingual form. The multilingual teaching resource library provides unified multilingual courseware, demonstration videos, test questions, and terminology comparisons as basic resources for the adaptive training management module and the multilingual instruction teaching and assessment module, and is the core support for the system's teaching content; The UAV simulation training module receives control commands from the multilingual interaction module, executes virtual flight, and transmits flight status data back to the adaptive training management module in real time as a basis for evaluating trainees' abilities. The multilingual instruction teaching and assessment module collects students' pronunciation through the multilingual interaction module, completes the assessment by combining standard instruction samples from the multilingual teaching resource library, and uploads the assessment results to the adaptive training management module. The adaptive training management module coordinates the overall situation and connects to the multilingual teaching resource library, the UAV simulation training module, the multilingual instruction teaching and evaluation module, and the data storage module. It dynamically plans training paths and adjusts training content based on student data. The data storage module uniformly receives and stores student files, training records, flight data, evaluation results, and system logs from all other modules, enabling full-process data traceability, analysis, and reuse.

[0023] The multilingual interaction module includes a language selection unit, a voice acquisition unit, a voice recognition unit, a semantic understanding unit, and a voice synthesis unit. The language selection unit receives the training language set by the user and provides a language selection interface. Supported languages ​​include, but are not limited to, Chinese, English, Arabic, French, Spanish, Russian, Japanese, and Korean. After the user selects a training language, the system switches to the corresponding language's user interface. The voice acquisition unit collects voice commands issued by the user using a microphone array and performs noise reduction and echo cancellation processing. The voice recognition unit recognizes the collected voice commands based on a multilingual voice recognition model and outputs the corresponding text information. The voice recognition unit uses a Transformer-based multilingual end-to-end voice recognition model. This model is trained using a large-scale multilingual voice corpus during the pre-training phase and supports voice recognition for more than eight languages, including Chinese, English, Arabic, French, Spanish, Russian, Japanese, and Korean. For less common languages ​​or specific accents, the system supports incremental learning and model fine-tuning. Testing showed that the model achieves a recognition accuracy of over 95% in quiet environments and over 88% in moderately noisy environments. The semantic understanding unit uses natural language processing (NLP) technology to parse the text information into intent, generating structured operation commands. The semantic understanding unit employs a Large Language Model (LLM) for natural language understanding, mapping the recognized text commands into structured operation commands executable by the UAV. For example, when the trainee says "ascent two meters," the semantic understanding unit parses it as {action: "ascent", parameter: 2, unit: "m"}; when the trainee says "fly forward five meters," it parses it as {action: "forward", parameter: 5, unit: "m"}. The speech synthesis unit is used to broadcast system feedback information in the selected language. The speech synthesis unit adopts text-to-speech synthesis technology based on neural networks, supports multilingual and multi-tone natural speech broadcasting, and is used to read system prompts, teaching content and feedback suggestions to students.

[0024] The UAV simulation training module includes a 3D scene construction unit, a UAV physical simulation unit, a flight control simulation unit, and a flight status monitoring unit; The 3D scene construction unit is used to build 3D virtual flight scenes. Based on Unity 3D or Unreal Engine, it constructs high-fidelity 3D virtual flight scenes, including various typical flight environments such as cities, villages, mountains, and water areas, and simulates different weather conditions such as day and night, sunny and rainy days, and fog and snow. The UAV physical simulation unit constructs virtual UAV models based on aerodynamic models and UAV dynamic models. The UAV physical simulation unit constructs virtual UAV models based on six-degree-of-freedom rigid body dynamics equations and multi-rotor aerodynamic models. Taking a quadcopter UAV as an example, its dynamic model is as follows: State variables: position vector p = [x, y, z]^T, velocity vector v = [v_x, v_y, v_z]^T, Euler angle Φ = [φ, θ, ψ]^T (roll, pitch, yaw), angular velocity ω = [ω_x, ω_y, ω_z]^T.

[0025] Dynamic equations: m·dv / dt=F_g+F_t+F_d; I·dω / dt+ω×I·ω=M; Where m is the mass of the UAV, F_g is gravity, F_t is the total lift generated by the four rotors, F_d is air resistance, I is the inertia tensor matrix, and M is the torque generated by the rotors; The flight control simulation unit receives structured operation commands generated by the multilingual interaction module, controlling the virtual drone to perform corresponding flight operations in a three-dimensional virtual flight scene. The flight control simulation unit also receives structured operation commands generated by the semantic understanding unit, calls the drone physical simulation unit for real-time calculation, and drives the virtual drone to perform corresponding actions in the three-dimensional scene. The command response latency is controlled within 200ms to meet real-time interaction requirements. The flight status monitoring unit collects flight status data of the virtual drone in real time. The flight status monitoring unit collects flight status data such as attitude, position, speed, altitude, and battery level of the virtual drone at a frequency of more than 30 frames per second, and displays it on the student interface in real time.

[0026] The multilingual teaching resource database includes a multilingual theoretical courseware database, a multilingual operation demonstration database, a multilingual question bank, and a terminology comparison database; The multilingual theoretical courseware library stores courseware on UAV theoretical knowledge written in multiple languages; The multilingual operation demonstration library stores drone operation demonstration videos with voiceovers in multiple languages; The multilingual question bank stores test questions presented in multiple languages; The terminology database stores the mapping relationships between UAV technical terms in different languages.

[0027] Multilingual theoretical courseware library: In accordance with the Civil Aviation Administration of China's "Regulations on the Management of Civil Unmanned Aerial Vehicle Pilots" (CCAR-61-R4), the core course content such as UAV regulations and airspace management, flight principles, meteorological knowledge, flight operation techniques, and mission planning has been translated and produced into multilingual courseware. The courseware in each language has been reviewed and proofread by native language experts. Multilingual Operation Demonstration Library: Records standard operation demonstration videos of drones, such as takeoff, hovering, forward and backward flight, left and right lateral movement, rotation, and landing, and provides multilingual voice-over narration for each operation; Multilingual question bank: Theoretical test questions are stored in categories according to difficulty level (beginner, intermediate, advanced), and each question is available in multiple languages; Terminology Lookup Database: This database uses a relational database or knowledge graph to store the mapping relationships between UAV technical terms in multiple languages, allowing students to look them up at any time. Terminology entries include, but are not limited to: lift, thrust, hovering, attitude, yaw, pitch, roll, etc. The adaptive training management module includes a student profile building unit, a competency assessment unit, a training path planning unit, and a progress tracking unit. The student profile building unit collects basic information and initial language proficiency data from students to construct a student profile model. The competency assessment unit dynamically evaluates students' comprehensive competency levels based on multi-dimensional data such as theoretical test scores, accuracy of operational command recognition, and flight operation completion rates during the training process. The training path planning unit matches suitable learning resources from the multilingual teaching resource library based on the student profile and competency assessment results, dynamically generating personalized training paths. The progress tracking unit records students' training progress and adjusts subsequent training content in real time based on the competency assessment results.

[0028] The student profile building unit collects information such as name, age, native language, second language proficiency, and drone operation experience when a student logs in for the first time, constructing an initial student profile. For example, student A's profile might be: native language Arabic, intermediate English level, and zero drone operation experience. The competency assessment unit collects multi-dimensional performance data from students at the end of each training phase. Theoretical test score (weight 30%) Command recognition accuracy (weight 25%) Flight operation completion rate (weight 30%) Reaction time (weight 15%) Based on the above data, a comprehensive ability score is calculated and divided into four levels according to a percentage system: beginner (0-40 points), basic (41-65 points), intermediate (66-85 points), and proficient (86-100 points). The training path planning unit employs a knowledge graph-based path planning algorithm. The drone training knowledge points are constructed as a directed acyclic graph, where nodes represent knowledge points (e.g., "takeoff," "hovering," "landing," "flight planning," etc.) and edges represent prerequisite dependencies between knowledge points. Based on the trainee's current skill level, the optimal next knowledge point is selected from the set of knowledge points that the trainee has not yet mastered but whose prerequisites are met. The push strategy balances maximizing learning efficiency (prioritizing knowledge points with high credit values) and maintaining confidence (reducing difficulty when there are more than 3 consecutive failures). The progress tracking unit records the learner's learning trajectory in a time-series format, including learned knowledge points, earned credits, and test scores, and adjusts subsequent training content in real time based on the competency assessment results. When the competency assessment score improves by more than 10 points twice consecutively, the system automatically increases the training difficulty level; when the competency assessment score decreases by more than 10 points twice consecutively, the system automatically decreases the difficulty level and provides auxiliary prompts.

[0029] The multilingual instruction teaching and assessment module includes a standard instruction library, an instruction pronunciation teaching unit, an instruction recognition assessment unit, and a feedback and guidance unit. The standard instruction library stores the standard pronunciation text and audio samples of the standard operating instructions for drones in multiple languages; The instruction pronunciation teaching unit is used to demonstrate and play the pronunciation of standard instructions to students, and guide students to practice reading along. The instruction recognition and evaluation unit collects the voice instructions issued by the student, compares them with the standard pronunciations in the standard instruction library, and evaluates the accuracy, speed, and completeness of the student's instruction pronunciation. The feedback and guidance unit generates personalized pronunciation correction suggestions and reinforcement exercises for trainees based on the assessment results.

[0030] The standard instruction library stores multilingual standard pronunciation text and audio samples of more than 30 standard drone operation instructions. Each instruction includes: instruction name (e.g., "ascend"), instruction code (e.g., "CMD_ASC"), operation description (e.g., "lift the drone vertically to a certain height"), multilingual pronunciation text, and native speaker audio recordings.

[0031] The instruction pronunciation teaching unit uses visual waveform diagrams to display the acoustic characteristics of standard pronunciation, and students can practice reading along by playing the demonstration audio multiple times.

[0032] The instruction recognition evaluation unit evaluates students' pronunciation based on a speech recognition model. The evaluation metrics include: Accuracy: The similarity of the acoustic features of the student's pronunciation to the standard pronunciation, calculated using the Dynamic Time Warping (DTW) algorithm; Speech speed: The degree to which the duration of a speech matches the standard duration; Completeness: Whether all syllables in the instruction are read out completely.

[0033] The evaluation scores are presented on a 100-point scale and are given four levels: “Excellent (≥90 points)”, “Good (75-89 points)”, “Pass (60-74 points)”, and “Needs Improvement (<60 points)”.

[0034] The feedback and guidance unit generates personalized feedback based on the assessment results. For instructions with a score below 75, the system marks the specific location of the pronunciation error, provides targeted pronunciation practice materials, and guides the learner into a dedicated intensive training mode for that instruction.

[0035] The data storage module uses a MySQL or PostgreSQL relational database to store student profiles, training records, competency assessment data, and system operation logs. Student profiles include basic information such as user ID, name, registration time, native language, and selected training language; training records include detailed data such as learning timestamps, learning content identifiers, completion status, and scores.

[0036] A multilingual training method for drones, the specific steps of which are as follows: S1: Initialization and Language Configuration: The user selects the target training language through the multilingual interaction module. The system loads the corresponding user interface language pack, speech recognition model and speech synthesis engine for that language, and extracts the teaching resource index for that language from the multilingual teaching resource library. S2: Student Profile Construction: The system collects basic information, initial language proficiency, and drone operation skills of students to construct an initial student profile model; the system guides users to fill out a basic information form, collecting the following information: Personal basic information (name, age, nationality); Language proficiency (native language, second language, self-assessment of language level); Drone operation experience (whether you have operating experience, flight time, existing certificate level); Based on the above information, the system constructs an initial student profile model and provides a basis for subsequent selection of teaching resources according to language proficiency. S3: Multilingual Theoretical Teaching: Based on the student's profile, the system selects suitable theoretical courseware from the multilingual teaching resource library, presenting UAV theoretical knowledge to the student in the selected language, and providing voice explanations and interactive Q&A through the multilingual interactive module. The system also selects suitable theoretical courseware from the multilingual theoretical courseware library based on the student's language proficiency and UAV experience level in their profile. The theoretical teaching adopts a three-in-one model of "text and image explanation, voice broadcast, and interactive Q&A": the courseware content is presented in text and image format, while the voice synthesis unit provides synchronous voice explanations in the selected language; after each chapter, the system generates interactive Q&A questions for students to answer via voice or text. S4: Multilingual Command Instruction Teaching: The system loads the required UAV operation commands for the current training phase from the standard command library. The command pronunciation teaching unit demonstrates the pronunciation of the standard commands and plays them aloud. After students practice reading along, the command recognition and evaluation unit provides real-time evaluation and feedback. For each command, the command pronunciation teaching unit displays the standard pronunciation text and plays a demonstration audio. Students practice reading along into the microphone. The command recognition and evaluation unit collects students' pronunciation and performs real-time evaluation, displaying the results visually on the interface in the form of waveform graphs and score bars. When a student scores below 60 points for a command in three consecutive evaluations, the feedback and guidance unit automatically switches to the specialized reinforcement training mode for that command, providing more detailed pronunciation guidance. S5: Multilingual Simulated Flight Training: Trainees issue operation commands via voice. The voice acquisition unit collects the trainee's voice and generates structured operation commands through voice recognition and semantic understanding to control the virtual drone to perform flight operations. At the same time, the flight status monitoring unit collects flight data in real time and feeds it back to the trainee's interface. After the trainee completes the flight operation of this stage, the system generates a flight operation score based on the degree of operation completion, response accuracy, and compliance with safety regulations. S6: Comprehensive Ability Assessment: The ability assessment unit integrates the trainees' performance in theoretical teaching, command evaluation, and simulated flight training to generate a multi-dimensional ability assessment report; the ability assessment unit summarizes the trainees' performance data in this stage, including: theoretical test scores, command pronunciation evaluation scores (weighted average of accuracy, speech rate, and completeness), flight operation scores, and average command response time. The above indicators are weighted according to preset weights to generate a comprehensive ability assessment score of 0-100, and the ability distribution of each dimension is displayed to the trainees in the form of a radar chart.

[0037] S7: Adaptive Path Adjustment: The training path planning unit dynamically adjusts subsequent training content based on the ability assessment results. It adds reinforcement training modules for weak links and automatically jumps to advanced content for content that has been mastered well. The training path planning unit reads the trainee's comprehensive ability assessment score and ability radar chart, and dynamically adjusts the subsequent training path based on the trainee profile and the system's preset knowledge point map. If the trainee scores low in the "command pronunciation" dimension, the system will increase the weight of the command pronunciation-specific training module in the next stage; if the trainee scores low in the "flight operation" dimension, the system will increase the frequency and duration of simulated flight training in the next stage; if the trainee scores above 85 points in all dimensions and has completed all knowledge points in the current stage, the system will automatically unlock the advanced training content for the next stage. S8: Iterative training: Repeat steps S3 to S7 until the trainee completes all training content and passes the final assessment.

[0038] The final assessment consists of three parts: a theoretical exam (100 questions, with a correct accuracy rate of ≥80%), a command pronunciation evaluation (all core commands must score ≥75 points), and a simulated flight assessment (completing the designated flight mission within the specified time without any major violations). Upon passing all three parts, the system will automatically generate a training completion certificate and enter it into the trainee's training file.

[0039] In S6, the multi-dimensional capability assessment report includes indicators such as theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. In S5, the speech recognition adopts a multilingual end-to-end speech recognition model based on the Transformer architecture, and the semantic understanding uses a large language model to map the recognized text into structured operation commands.

[0040] In S6, the capability assessment includes a comprehensive evaluation of three dimensions: theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. Each dimension is weighted according to a preset weight to calculate a comprehensive score.

[0041] In S7, the dynamic adjustment of the training path includes: reducing the training difficulty level when the trainee's comprehensive ability score is lower than the first preset threshold, and automatically increasing the training difficulty level when the trainee's comprehensive ability score increases by more than the second preset threshold twice in a row.

[0042] Example 2: This embodiment, based on embodiment 1, further adds a multilingual real-time translation assistance function.

[0043] The multilingual interaction module also includes a real-time translation unit, which employs a neural machine translation model based on the Transformer architecture to support real-time translation between the source and target languages. During training, if trainees encounter unfamiliar terms or instructions, they can initiate translation requests via voice or text. The system then invokes the real-time translation unit to translate the text and broadcasts it through a speech synthesis unit. Furthermore, the system can translate and broadcast guidance information input by the instructor into the trainee's selected language in real time, enabling cross-language remote teaching and guidance.

[0044] Example 3: This embodiment, based on embodiment 1, further adds mixed reality (MR) assisted training functionality.

[0045] The drone simulation training module also includes a mixed reality display unit, which connects to a head-mounted MR glasses device to blend a 3D virtual flight scene with the real environment. Wearing the MR glasses, trainees can see a virtual drone and flight path indicators superimposed on the real space, and control the virtual drone to "fly" in the real space via voice commands. The mixed reality display unit uses Simultaneous Localization and Mapping (SLAM) technology for spatial positioning and scene understanding, ensuring accurate spatial mapping between the virtual drone and the real environment. This function can also be used for drone assembly and maintenance training: trainees can see a 3D structural model of the virtual drone in the MR glasses, and control the model to disassemble and assemble via voice commands. The system provides real-time guidance through gesture recognition and voice interaction.

[0046] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multilingual training system for unmanned aerial vehicles (UAVs), characterized in that: It includes a multilingual interactive module, a UAV simulation training module, a multilingual teaching resource library, an adaptive training management module, a multilingual instruction teaching and evaluation module, and a data storage module; The multilingual interaction module serves as the system's human-computer interaction entry point. It is directly connected to the UAV simulation training module, the adaptive training management module, and the multilingual instruction teaching and evaluation module. It is responsible for receiving multilingual voice commands from users and issuing operational intentions, while broadcasting system feedback to users in multilingual form. The multilingual teaching resource library provides unified multilingual courseware, demonstration videos, test questions, and terminology comparisons as basic resources for the adaptive training management module and the multilingual instruction teaching and assessment module, and is the core support for the system's teaching content; The UAV simulation training module receives control commands from the multilingual interaction module, executes virtual flight, and transmits flight status data back to the adaptive training management module in real time as a basis for evaluating trainees' abilities. The multilingual instruction teaching and assessment module collects students' pronunciation through the multilingual interaction module, completes the assessment by combining standard instruction samples from the multilingual teaching resource library, and uploads the assessment results to the adaptive training management module. The adaptive training management module coordinates the overall situation and connects to the multilingual teaching resource library, the UAV simulation training module, the multilingual instruction teaching and evaluation module, and the data storage module. It dynamically plans training paths and adjusts training content based on student data. The data storage module uniformly receives and stores student files, training records, flight data, evaluation results, and system logs from all other modules, enabling full-process data traceability, analysis, and reuse.

2. The UAV multilingual training system according to claim 1, characterized in that: The multilingual interaction module includes a language selection unit, a voice acquisition unit, a voice recognition unit, a semantic understanding unit, and a voice synthesis unit. The language selection unit receives the training language set by the user. The voice acquisition unit acquires voice commands issued by the user. The voice recognition unit recognizes the acquired voice commands based on a multilingual voice recognition model and outputs corresponding text information. The semantic understanding unit uses natural language processing technology to parse the intent of the text information and generate structured operation commands. The voice synthesis unit broadcasts the system feedback information in the selected language.

3. The UAV multilingual training system according to claim 1, characterized in that: The UAV simulation training module includes a 3D scene construction unit, a UAV physical simulation unit, a flight control simulation unit, and a flight status monitoring unit. The 3D scene construction unit is used to construct a 3D virtual flight scene. The UAV physical simulation unit constructs a virtual UAV model based on an aerodynamic model and a UAV dynamics model. The flight control simulation unit receives structured operation commands generated by the multilingual interaction module and controls the virtual UAV to perform corresponding flight operations in the 3D virtual flight scene. The flight status monitoring unit collects the flight status data of the virtual UAV in real time.

4. The UAV multilingual training system according to claim 1, characterized in that: The multilingual teaching resource library includes a multilingual theoretical courseware library, a multilingual operation demonstration library, a multilingual question bank, and a terminology comparison library; the multilingual theoretical courseware library stores UAV theoretical knowledge courseware written in multiple languages; the multilingual operation demonstration library stores UAV operation demonstration videos dubbed in multiple languages; and the multilingual question bank stores test questions presented in multiple languages. The terminology database stores the mapping relationships between UAV technical terms in different languages.

5. The UAV multilingual training system according to claim 1, characterized in that: The adaptive training management module includes a student profile building unit, a competency assessment unit, a training path planning unit, and a progress tracking unit. The student profile building unit collects basic information and initial language proficiency data from students to construct a student profile model. The competency assessment unit dynamically evaluates students' comprehensive competency levels based on multi-dimensional data such as theoretical test scores, accuracy of operational command recognition, and flight operation completion rates during the training process. The training path planning unit matches suitable learning resources from the multilingual teaching resource library based on the student profile and competency assessment results, dynamically generating personalized training paths. The progress tracking unit records students' training progress and adjusts subsequent training content in real time based on the competency assessment results.

6. The UAV multilingual training system according to claim 1, characterized in that: The multilingual instruction teaching and evaluation module includes a standard instruction library, an instruction pronunciation teaching unit, an instruction recognition evaluation unit, and a feedback guidance unit. The standard instruction library stores the standard pronunciation text and audio samples of standard UAV operation instructions in multiple languages. The instruction pronunciation teaching unit is used to demonstrate and play the pronunciation of standard instructions to students, guiding them to practice reading along. The instruction recognition evaluation unit collects the voice instructions issued by students, compares them with the standard pronunciation in the standard instruction library, and evaluates the accuracy, speed, and completeness of the students' instruction pronunciation. The feedback guidance unit generates personalized pronunciation correction suggestions and reinforcement practice content for students based on the evaluation results.

7. The UAV multilingual training method according to any one of claims 1-6, characterized in that: The specific steps of this training method are as follows: S1: Initialization and Language Configuration: The user selects the target training language through the multilingual interaction module. The system loads the corresponding user interface language pack, speech recognition model and speech synthesis engine for that language, and extracts the teaching resource index for that language from the multilingual teaching resource library. S2: Student Profile Construction: The system collects basic information, initial language level and drone operation skills of students to construct an initial student profile model; S3: Multilingual theoretical teaching: Based on the student profile, the system selects suitable theoretical courseware from the multilingual teaching resource library, presents the theoretical knowledge of UAVs to the students in the selected language, and conducts voice explanation and interactive Q&A through the multilingual interactive module; S4: Multilingual instruction teaching: The system loads the UAV operation instructions required for the current training stage from the standard instruction library, and demonstrates the pronunciation of the standard instructions to the trainees through the instruction pronunciation teaching unit. After the trainees practice reading along, the instruction recognition and evaluation unit conducts real-time evaluation and provides feedback and correction suggestions. S5: Multilingual Simulated Flight Training: Trainees issue operation commands via voice. The voice acquisition unit collects the trainee's voice and generates structured operation commands through voice recognition and semantic understanding to control the virtual drone to perform flight operations. At the same time, the flight status monitoring unit collects flight data in real time and feeds it back to the trainee's interface. S6: Comprehensive Ability Assessment: The ability assessment unit integrates the trainees' performance in theoretical teaching, command evaluation, and simulated flight training to generate a multi-dimensional ability assessment report; S7: Adaptive Path Adjustment: The training path planning unit dynamically adjusts subsequent training content based on the ability assessment results. It adds reinforcement training modules for weak links and automatically jumps to advanced content for content that has been mastered well. S8: Iterative training: Repeat steps S3 to S7 until the trainee completes all training content and passes the final assessment.

8. The UAV multilingual training method according to claim 7, characterized in that: In S6, the multi-dimensional capability assessment report includes indicators such as theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. In S5, the speech recognition adopts a multilingual end-to-end speech recognition model based on the Transformer architecture, and the semantic understanding uses a large language model to map the recognized text into structured operation commands.

9. The UAV multilingual training method according to claim 7, characterized in that: In S6, the capability assessment includes a comprehensive evaluation of three dimensions: theoretical knowledge mastery, command recognition accuracy, and flight operation proficiency. Each dimension is weighted according to a preset weight to calculate a comprehensive score.

10. The UAV multilingual training method according to claim 7, characterized in that: In S7, the dynamic adjustment of the training path includes: reducing the training difficulty level when the trainee's comprehensive ability score is lower than the first preset threshold, and automatically increasing the training difficulty level when the trainee's comprehensive ability score increases by more than the second preset threshold twice in a row.

Images