An air traffic control oriented virtual-real fusion multi-role cooperative command simulation system

Abstract

This invention relates to the field of air traffic control simulation and training technology, and in particular to a virtual-real fusion multi-role collaborative command simulation system for air traffic control. Based on a three-layer linkage architecture of semantic layer-logic layer-physical layer, it includes: a data fusion engine that generates an airspace situation semantic graph; a distributed collaborative simulation core that performs graph analysis and multi-agent inference; a multimodal immersive interactive subsystem that generates differentiated command views; and a collaborative command chain management module that models command levels and permissions. This invention achieves deep virtual-real fusion, multi-role collaborative command, human-machine intelligent collaboration, and distributed multi-terminal scenario consistency, and is suitable for controller collaborative training and emergency drills.

Description

Technical Field

[0001] This invention relates to the field of air traffic control simulation and training technology, and in particular to a virtual-real fusion multi-role collaborative command simulation system for air traffic control. Background Technology

[0002] Air traffic control is a core component of ensuring aviation safety and efficiency. With the continuous growth of global air transport, airspace resources are becoming increasingly scarce, and the complexity and pressure of air traffic control are constantly rising. The professional competence of radar controllers directly affects the quality and effectiveness of air traffic control. Radar control simulation training systems follow ICAO standards, employing graphical, digital, and high-precision simulation methods to simulate and train in radar control operations. Aviation colleges and air traffic control agencies typically introduce radar control simulation training systems to train students in professional skills, thereby improving controllers' ability to handle complex air traffic situations and emergencies.

[0003] Current air traffic control simulation systems mainly suffer from the following technical limitations: The integration of virtual and real data is insufficient. Most existing systems adopt a pure virtual simulation mode, that is, generating aircraft models and airspace environments in a completely virtual scene. It is difficult to achieve high-fidelity real-time fusion of real operational data (such as real-time radar tracks, meteorological information, flight dynamics, etc.) with the virtual simulation scene. Although some technical solutions propose to collect real-time video footage of the airport through front-end cameras and fuse it with virtual aircraft, this is only applicable to tower visual training scenarios. When facing wide airspace such as large-area area control, it lacks the ability to generate dynamic airspace situation based on real-time operational data, and there is a significant gap between the training scenario and the real operating environment.

[0004] Limited role collaboration capabilities. Existing simulation systems typically build training scenarios around a single controller, lacking in-depth modeling of collaborative command relationships among multiple roles (tower control, approach control, area control, flight services, airport operations, etc.). While radar control simulation training systems linking multiple control areas enable interaction between simulated controllers and between simulated controllers and simulated pilots, they are essentially simple communication between homogeneous roles, failing to reflect the differentiated needs of different roles in the air traffic control chain in terms of decision-making levels, information views, and operational permissions. Furthermore, the simulated captain's end typically relies on automatic system responses, lacking modeling of real pilot behavior and crew resource management factors. Especially for authorized operations involving the captain's final decision-making authority (such as diversions, emergency descents, and rerouting), existing systems cannot fully present the closed-loop command chain of instruction transmission, authorization confirmation, operation execution, and feedback verification in the simulation environment.

[0005] The lack of human-machine collaboration mechanisms is a significant issue. Next-generation air traffic management systems will rely on seamless collaboration between humans and increasingly intelligent systems. However, existing simulation systems neither integrate decision-making agents nor design strategies for human-machine collaboration, failing to support controller training needs under human-machine collaborative conditions. Controllers require collaborative decision-making with automated systems in actual operations, but current simulation training systems cannot provide this collaborative training environment for mixed human-machine formations.

[0006] Ensuring scene consistency is challenging. In multi-role collaborative simulations, different roles use different display devices and interactive terminals. Ensuring scene consistency (including temporal consistency, spatial consistency, and state consistency) between these terminals is a technical challenge. Existing solutions have failed to effectively address the problem of high-precision spatiotemporal alignment and state synchronization among multiple terminals in distributed simulation environments, leading to information discrepancies between different roles and affecting collaborative command efficiency and simulation training effectiveness.

[0007] Therefore, there is an urgent need for an air traffic control simulation system that can achieve deep integration of virtual and real systems, support differentiated collaborative command for multiple roles, integrate human-machine intelligent fusion mechanisms, and ensure consistency in distributed scenarios. Summary of the Invention

[0008] In order to overcome the above-mentioned defects of the prior art, the present invention provides a virtual-real fusion multi-role collaborative command simulation system for air traffic control, so as to solve the problems existing in the background art.

[0009] This invention provides the following technical solution: a virtual-real fusion multi-role collaborative command simulation system for air traffic control, implemented based on a three-layer linkage simulation architecture of "semantic layer—logic layer—physical layer," the system comprising: The data fusion engine, located in the semantic layer, is used to collect, clean, spatiotemporally align, and semantically fused multi-source heterogeneous air traffic control operation data in real time to generate an airspace situation semantic map. The distributed collaborative simulation core, located in the logic layer, communicates with the semantic layer through a standardized semantic interface. It is used to perform the parsing of the spatial situation semantic graph, the simulation logic deduction based on multi-agent collaboration, and the modeling and arbitration of multi-role collaborative command relationships. The multimodal immersive interaction subsystem, located in the physical layer, communicates with the logic layer through a standardized presentation interface. It is used to generate personalized display interfaces according to the differentiated command view requirements of multiple roles and to realize the rendering overlay of virtual simulation elements and real operation data. The collaborative command chain management module is connected to the distributed collaborative simulation core and the multimodal immersive interactive subsystem, respectively, and is used to model and maintain the command hierarchy relationship, information flow path and decision authority matrix among multiple roles.

[0010] Furthermore, the data fusion engine includes: The multi-source data access adapter is configured to receive multi-source heterogeneous data streams, including ADS-B broadcast automatic dependent surveillance data, radar track data, flight plan data, meteorological observation data, NOTAM air traffic notification data, and airport surface surveillance data. The spatiotemporal alignment module is configured to unify the above-mentioned multi-source heterogeneous data into the same spatiotemporal reference coordinate system. The spatiotemporal alignment includes timestamp normalization processing and spatial coordinate transformation. The semantic association and fusion module is configured to perform semantic annotation and association on spatiotemporally aligned data based on the air traffic control domain knowledge graph to generate an airspace situation semantic graph. The airspace situation semantic graph takes aircraft entities as core nodes, associates waypoint nodes, airspace unit nodes, meteorological feature nodes and control instruction nodes, and uses directed edges with semantic labels to represent the spatial relationships, prediction relationships and control relationships between entities.

[0011] Furthermore, the core of the distributed collaborative simulation includes: The semantic graph parser is configured to extract key semantic feature vectors of the current simulation scenario from the airspace situation semantic graph. The key semantic feature vectors include airspace complexity indicators, potential conflict hotspots, traffic flow density distribution, and meteorological influencing factors. The multi-agent collaborative inference engine is configured to run multiple types of simulated agents, including: virtual aircraft agent, virtual pilot agent, decision support agent, and scenario disturbance agent; wherein, the virtual pilot agent is used to simulate receiving and responding to authorization instructions issued by the controller, and the decision support agent provides conflict detection and resolution suggestions to the controller based on a deep reinforcement learning strategy. The collaborative relationship arbitrator is configured to perform conflict detection and priority sorting on command instructions issued by multiple roles based on the command hierarchy relationship and decision authority matrix defined by the collaborative command chain management module. The distributed synchronization controller is configured to ensure state consistency between each simulated agent and each role terminal. It achieves targeted distribution of simulation state updates through a publish / subscribe model and realizes simulation clock synchronization between terminals through a distributed spatiotemporal remapping mechanism.

[0012] Furthermore, the distributed synchronization controller uses the RTI soft bus as the underlying communication support, and the distributed spatiotemporal remapping mechanism includes: maintaining a local simulation clock and state cache in each role terminal; when any terminal or simulation agent undergoes a state change, a state update event carrying a global timestamp is published to all terminals subscribed to the event via the RTI soft bus; after receiving the state update event, each terminal performs spatiotemporal remapping based on the deviation between the global timestamp and the local simulation clock. For events with a time deviation within the allowable threshold, the local state is directly updated; for events with a time deviation exceeding the allowable threshold, the local state is updated after a smooth state transition using an interpolation algorithm.

[0013] Furthermore, the multimodal immersive interaction subsystem includes: The role view generator is configured to obtain the corresponding command view template and information permission configuration from the collaborative command chain management module based on the role type of the currently logged-in role, and dynamically generate a personalized immersive interactive interface. The virtual-real rendering fusion processor is configured to blend real running data layers and virtual simulation layers in the same 3D geographic information scene, and distinguish between real data and simulation data through color level mapping and transparency layering strategies. The multi-channel natural human-computer interaction module is configured to support users to interact with the simulation system through one or more interaction channels, including gestures, voice, eye tracking, and touch operations.

[0014] Furthermore, the collaborative command chain management module includes: The role-permission mapping table stores predefined multiple role types and their corresponding command permissions and accessible information domains; The command chain topology manager maintains a directed acyclic graph of command hierarchy relationships among multiple roles and supports instructors in dynamically adjusting the command chain structure through a visual interactive interface. The information flow controller is configured to control the routing and distribution of coordination instructions, handover requests, and permission confirmation information among various role terminals according to the command chain topology.

[0015] Furthermore, it also includes a collaborative performance evaluation module, which is configured to record the operation sequence, communication records and decision delay of multiple roles in real time during the simulation process, and generate a collaborative performance evaluation report based on a preset evaluation index system after the simulation ends.

[0016] Furthermore, the inter-layer data flow of the three-layer linkage simulation architecture includes: Semantic-logical data flow from the semantic layer to the logical layer: The data fusion engine transmits the generated spatial situation semantic graph to the semantic graph parser of the distributed collaborative simulation core through a standardized semantic interface; Logical-presentation data flow from the logical layer to the physical layer: The core of distributed collaborative simulation transmits simulation results to the multimodal immersive interactive subsystem through a standardized presentation interface; Interaction-feedback data flow from the physical layer to the logical layer: The multimodal immersive interaction subsystem feeds back user operation events to the collaborative relationship arbitrator of the core of the distributed collaborative simulation; Update-writeback data flow from the logical layer to the semantic layer: The core of the distributed collaborative simulation writes back the new states generated during the simulation process to the data fusion engine to update the spatial situation semantic graph, forming a closed-loop iterative update mechanism.

[0017] Furthermore, the multimodal immersive interactive subsystem also includes a global view of the instructor / evaluator role, which provides the following functions: real-time viewing of the operation status and interaction records of each training role during the simulation; adjustment of the simulation speed; quick jump to specific training scene nodes; setting multi-role collaborative task objectives; and real-time viewing of the collaborative effectiveness score trend chart of any training role.

[0018] Furthermore, the behavioral decision-making model of the virtual pilot agent is trained based on real pilot behavior data, and can simulate the response latency and decision-making style of pilots with different experience levels. When a diversion instruction is received, the virtual pilot agent calculates a reasonable response latency based on the current aircraft status, the weather conditions of the diversion airport, and the aircraft's remaining fuel, and generates authorization confirmation information that conforms to standard air-to-ground communication specifications. For emergency descent operations, the virtual pilot agent automatically generates a sequence of coordinated crew actions based on a preset emergency procedure model, including oxygen mask donning, transponder coding settings, and descent profile planning.

[0019] The technical effects and advantages of this invention are as follows: This invention achieves deep virtual-real fusion through three-layer linkage. The proposed three-layer linkage architecture—"semantic layer—logic layer—physical layer"—decouples data fusion, logical deduction, and interactive presentation into independent layers and establishes an efficient inter-layer communication mechanism. This solves the problems of tight coupling between data flow and control flow and poor scalability in traditional simulation systems. The semantic layer outputs a spatial situation semantic map, the logic layer drives multi-agent simulation deduction based on the semantic map, and the physical layer is responsible for differentiated presentation to the user. The three layers achieve loosely coupled collaboration through standardized interfaces, enabling the system to flexibly adapt to different spatial scenarios and training requirements.

[0020] This invention supports differentiated collaborative command across multiple roles. Through a collaborative command chain management module, it accurately models the hierarchical relationships and decision-making authority matrices of multiple roles within the air traffic control command chain, providing differentiated command views and operational permissions for different roles. This realistically simulates collaborative scenarios involving multiple roles across the "tower—approach—area—airport" system. Compared to the homogeneous role settings of existing multi-control area linkage systems, this invention offers significant improvements in role heterogeneity, authority modeling, and dynamic command chain management.

[0021] This invention introduces a human-machine collaborative mechanism. It integrates multiple types of intelligent agents, including virtual aircraft agents, virtual pilot agents, and decision-making support agents, embedding AI-assisted decision-making capabilities into the simulation scenario. This realistically reflects the collaborative operation mode of human controllers and automated systems in next-generation air traffic control systems. The introduction of the virtual pilot agent effectively compensates for the shortcomings of existing simulation systems in operations involving captain decision-making authority, making the simulation of authorized operations such as diversions, emergency descents, and rerouting more complete and realistic. Simultaneously, the decision-making support agent provides conflict resolution suggestions to controllers based on deep reinforcement learning strategies, helping to improve controllers' adaptability to the human-machine collaborative mode.

[0022] Ensuring consistency in distributed multi-terminal scenarios. This invention solves the problem of real-time synchronization of simulation states between multi-role terminals through a distributed spatiotemporal remapping mechanism and an RTI soft bus publish / subscribe mode. It can control the time synchronization deviation between terminals within a preset accuracy range, effectively avoiding information discrepancies and coordination errors caused by time inconsistencies. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention.

[0024] Figure 2 This is a schematic diagram of the internal structure of the data fusion engine.

[0025] Figure 3 This is a schematic diagram of the internal structure of the core of distributed collaborative simulation.

[0026] Figure 4 This is an example of a directed acyclic graph representing the command hierarchy in the collaborative command chain management module.

[0027] Figure 5 This is a flowchart illustrating the simulation process of multi-role collaborative command.

[0028] Figure 6 This is a schematic diagram of the inter-layer data flow in a three-layer linkage simulation architecture. Detailed Implementation

[0029] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention.

[0030] Example 1: System Architecture like Figure 1 As shown, the virtual-real fusion multi-role collaborative command simulation system for air traffic control of the present invention includes: Data fusion engine 10, located in semantic layer L1, is used to collect, clean, spatiotemporally align and semantically fused multi-source heterogeneous air traffic control operation data in real time to generate an airspace situation semantic map. The distributed collaborative simulation core 20, located in the logic layer L2, communicates with the semantic layer L1 through a standardized semantic interface. It is used to perform the parsing of the spatial situation semantic map, the simulation logic deduction based on multi-agent collaboration, and the modeling and arbitration of multi-role collaborative command relationships. The multimodal immersive interaction subsystem 30, located in the physical layer L3, communicates with the logic layer L2 through a standardized presentation interface and is used to generate personalized display interfaces according to the differentiated command view requirements of multiple roles. The collaborative command chain management module 40 is connected to the distributed collaborative simulation core 20 and the multimodal immersive interactive subsystem 30, respectively, and is used to model and maintain the command hierarchy relationship, information flow path and decision authority matrix among multiple roles.

[0031] The inter-layer data flow relationship of the three-layer linkage simulation architecture is as follows: Figure 6 As shown.

[0032] Example 2: Detailed Structure of the Data Fusion Engine like Figure 2 As shown, the data fusion engine 10 includes: Multi-source data access adapter 11 is configured to receive the following multi-source heterogeneous data streams: ADS-B broadcast automatic dependent surveillance data is obtained through ADS-B ground stations or data service networks and includes information such as aircraft identification, three-dimensional position (latitude and longitude, barometric altitude / geometric altitude), and velocity vector. Radar track data is acquired through the radar data interface of the air traffic control automation system and includes fused track information from primary and secondary radars. Flight plan data is obtained through a flight data processing system and includes information such as flight number, departure and arrival airports, planned route, and estimated time. Meteorological observation data is obtained through the meteorological information service system, including METAR / SPECI routine / special weather reports, TAF airport weather forecasts, and important meteorological intelligence SIGMET, etc. NOTAM (Notification of Flight) data is obtained through the Aeronautical Information Service (AIS) system. Airport surface surveillance data is acquired through airport surface surveillance radar or ADS-B surface equipment.

[0033] The spatiotemporal alignment module 12 is configured to unify the aforementioned multi-source heterogeneous data into the same spatiotemporal reference coordinate system. The spatiotemporal alignment includes: Timestamp normalization: Coordinated Universal Time (UTC) is used as the global unified time base to standardize the format and align the precision of timestamps from different data sources. Considering the differences in time precision among data sources—radar track data typically updates every 4-12 seconds, ADS-B data typically broadcasts every 0.5-2 seconds, and meteorological data typically updates every 30 minutes (METAR) to several hours (TAF)—the spatiotemporal alignment module performs interpolation or extrapolation based on the time frequency characteristics of each data source to ensure that spatiotemporal alignment values ​​from each data source can be obtained at any given time.

[0034] Spatial coordinate transformation: The geospatial references of various data sources are uniformly mapped to the UTM projection coordinate system under the WGS-84 ellipsoid datum, and the elevation datum (EGM96 geoid model) is corrected.

[0035] The semantic association and fusion module 13 is configured to perform semantic annotation and association on spatiotemporally aligned data based on the air traffic control domain knowledge graph, generating an airspace situation semantic graph. This airspace situation semantic graph uses aircraft entities as core nodes, and associates waypoint nodes, airspace unit nodes (sector nodes, danger zone nodes, restricted zone nodes), meteorological feature nodes, and control instruction nodes. It uses directed edges with semantic labels to represent the spatial relationships, prediction relationships, and control relationships between entities.

[0036] Example 3: Detailed Structure of the Distributed Cooperative Simulation Core like Figure 3 As shown, the distributed collaborative simulation core 20 includes: Semantic graph parser 21 is configured to extract key semantic feature vectors of the current simulation scenario from the airspace situation semantic graph. These key semantic feature vectors include: airspace complexity indices (calculated based on aircraft density and traffic flow intersection), potential conflict hotspots (identified based on predicted relationship edges in the semantic graph), traffic flow density distribution, and meteorological influence factors.

[0037] The multi-agent collaborative inference engine 22 is configured to run multiple types of simulated agents, specifically including: Virtual Aircraft Agent 221: Each virtual aircraft agent corresponds to a simulated aircraft, maintains the aircraft's four-dimensional trajectory (three-dimensional position plus time), evolves its state according to the flight plan, aircraft performance model, meteorological conditions and received control instructions, and perceives surrounding aircraft through interaction with semantic graphs to maintain autonomous intervals.

[0038] Virtual Pilot Agent 222: Each virtual pilot agent corresponds to an aircraft requiring simulated captain decision-making authority. Internally, this agent maintains a behavioral decision-making model capable of responding to authorization commands issued by air traffic controllers (including but not limited to diversion commands, emergency descent commands, rerouting commands, holding line commands, and altitude change commands), performing authorization confirmation (via simulated voice or data link communication), and executing operations and providing feedback verification. The virtual pilot agent's decision-making model is trained based on real pilot behavioral data (e.g., parameterized models based on behavioral cloning or inverse reinforcement learning), enabling it to simulate the response latency and decision-making styles of pilots at different experience levels. For operations requiring rapid response, such as emergency descents, the virtual pilot agent can also automatically generate crew coordination sequences based on a pre-set emergency procedure model, thus providing controllers with a more realistic interactive experience.

[0039] Decision-Making Auxiliary Agent 223: This agent provides conflict detection and resolution suggestions to controllers based on a deep reinforcement learning strategy. It takes an airspace situational semantic graph as input and outputs conflict resolution strategy suggestions (including suggestions for prioritizing speed adjustments, heading adjustments, and altitude changes). These suggestions are then pushed to controllers via a multimodal immersive interactive subsystem 30 using visual cues (highlighting conflicting aircraft and plotting suggested flight paths) and voice prompts. The training of this agent can be achieved by combining existing aircraft conflict modeling methods based on BP neural network models and aircraft conflict resolution techniques based on deep deterministic policy gradient algorithms.

[0040] Scene Disturbance Agent 224: Used to dynamically inject sudden events during simulation. This agent maintains a configurable library of disturbance events, including: aircraft emergencies (in-flight mechanical failure, cabin depressurization, sudden passenger illness), sudden weather changes (sudden thunderstorms, wind shear, low visibility), communication failures (single aircraft communication interruption, sector frequency interference), etc., and is activated during simulation according to preset triggering conditions (time-triggered, airspace complexity-triggered, random probability-triggered).

[0041] The collaborative relationship arbitrator 23 is configured to perform conflict detection and priority ranking of command instructions issued by multiple roles based on the command hierarchy and decision authority matrix defined by the collaborative command chain management module 40. When a potential conflict is detected between instructions issued by different roles for the same aircraft—for example, when tower control and approach control issue different instructions to an aircraft in the handover area at the same time—arbitration is performed according to preset priority rules (responsibility sector priority takes precedence over adjacent sectors, and emergency instructions take precedence over routine instructions).

[0042] The distributed synchronization controller 24 is configured to ensure state consistency between each simulated agent and each role terminal. This controller implements publish / subscribe mode targeted distribution of simulation state updates via the RTI soft bus (based on the HLA high-level architecture standard) and employs the distributed spatiotemporal remapping mechanism described above to achieve simulation clock synchronization between terminals.

[0043] Example 4: Multi-role chain of command modeling The collaborative command chain management module 40 implements formal modeling of multi-role command relationships through a role-permission mapping table and a command chain topology manager.

[0044] like Figure 4 As shown, the command chain topology manager maintains a directed acyclic graph representing the command hierarchy among multiple roles. In this graph, each node represents a role instance, and directed edges represent the hierarchical relationship between commanders and subordinates. Specific role types include, but are not limited to: Tower control role: Responsible for the control of aircraft on the airport surface and during the takeoff / landing phases, and has command authority over runway and taxiway resources; Approach Control Role: Responsible for the sequencing and guidance of aircraft arriving and departing in the terminal area, and has command authority over the approach and departure sectors; Area control role: Responsible for controlling aircraft along high-altitude routes, and has command authority over regional sectors; Airport operations role: Responsible for parking stand allocation and ground service coordination, with scheduling authority over parking stands and ground resources; Flight service role: Responsible for flight plan processing and aeronautical information services, with authority to process flight data; Instructor / Evaluator Role: Has global monitoring and evaluation permissions, but does not participate in direct command by default (unless an intervention or takeover operation is performed).

[0045] The command chain topology manager also supports instructors in dynamically adjusting the command chain structure through a visual interactive interface. For example, when a new intermediate-level role is added (such as "Chief Controller" as a coordinating node for tower control and approach control), instructors can modify the directed acyclic graph topology of the command chain on the visual interactive interface by dragging and dropping. The system updates the routing table of the information flow controller in real time, thereby enabling online reorganization of the command chain without restarting the simulation session.

[0046] Example 5: Complete Simulation Process like Figure 5 As shown, the multi-role collaborative command simulation process includes the following steps: Step S1: Simulation Scene Initialization Instructors configure simulation scenario parameters through the global view of the instructor / evaluator role, including training scenario type selection (such as routine operation training, high-traffic operation training, emergency response training, and multi-role collaborative emergency drills), airspace range (training sector boundary vertices and upper and lower altitude limits), weather conditions, traffic flow levels, participating roles, and their initial configurations. Instructors can also set the initial simulation time, simulation speed, and scoring weights for multi-role collaborative tasks—for example, for a training scenario primarily aimed at multi-role collaborative emergency drills, instructors can increase the scoring weights for "emergency response capability" and "collaboration efficiency" to highlight the training focus of that scenario.

[0047] The data fusion engine 10 begins receiving real operational data (if the simulation scenario is selected and the real data playback mode is used, then real flight paths and meteorological data from the historical database are read as the operational baseline; if a fully virtual scenario is selected, then the scenario generator generates completely virtual aircraft operational data), and generates an initial airspace situation semantic map. The multi-agent collaborative inference engine 22 creates virtual aircraft agents 221 corresponding to the number of simulated aircraft, and virtual pilot agents 222 configured according to the number of roles.

[0048] Step S2: Multi-character immersive scene generation The multimodal immersive interaction subsystem 30 generates personalized immersive interactive interfaces based on the role type. Tower controllers see a visual interface centered on a 3D panoramic view of the airport through VR headsets or panoramic stitching displays; approach controllers see a radar display interface centered on the terminal area situation through radar control workstations; and area controllers see a monitoring interface centered on the high-altitude airway network through flight data display terminals. In other words, each role's command view is configured differently based on its scope of airspace focus and decision-making responsibilities. Access to each role's command view is controlled according to the information accessibility domains defined in the collaborative command chain management module 40, ensuring that each role can only see information matching its command authority.

[0049] The multi-channel natural human-computer interaction module initializes the speech recognition engine, gesture recognition engine, and eye-tracking device to ensure that each role can operate the system naturally through multi-channel interaction.

[0050] Step S3: Simulation Run and Real-time Deduction During the simulation, the multi-agent collaborative inference engine 22 continuously runs the simulation inference closed loop: The decision-making auxiliary intelligent agent 223 scans the airspace situation semantic map in real time. When it detects that the minimum distance between two or more aircraft may be lower than the safety threshold, it generates a conflict alarm and a recommended resolution plan, which is pushed to the corresponding control role through the multimodal immersive interaction subsystem 30. Command instructions from each role are sent to the target virtual aircraft agent 221 and the virtual pilot agent 222 after conflict detection and priority sorting by the collaboration relationship arbitrator 23. For authorized operations (such as diversion instructions), the virtual pilot agent 222 simulates the decision-making process of a real pilot—after receiving the diversion instruction, it calculates the response delay based on the behavioral decision model (simulating the pilot's assessment of the weather at the diversion airport, checking the fuel level, confirming the diversion procedure, etc.), and then sends authorization confirmation information through simulated voice communication and executes the diversion operation; Scene disturbance agent 224 dynamically injects sudden events based on preset trigger conditions to assess the collaborative response capabilities of multiple roles; The distributed synchronization controller 24 synchronizes the simulation state of all agents and terminals through a publish / subscribe pattern.

[0051] Step S4: Collaborative Effectiveness Assessment After the simulation, the collaborative effectiveness evaluation module generates a collaborative effectiveness evaluation report based on the real-time recorded operation sequence, communication records, and decision latency, providing quantitative evaluation results and a comprehensive score for each dimension. Evaluation indicators include: collaborative efficiency indicators (such as handover time and instruction response latency), conflict resolution success rate, communication standardization indicators (standardization of terminology and completeness of repetition), and emergency response capability (first response time and correct handling rate). Instructors can replay the training process through the global view of the instructor / evaluator role, manually calibrate each evaluation indicator, and supplement with subjective evaluations, ultimately generating a comprehensive collaborative effectiveness report for the trainee.

[0052] Example 6: Three-layer linked data flow like Figure 6 As shown, the inter-layer data flow of the three-layer linkage simulation architecture is as follows: Semantic layer L1 to logical layer L2 (semantic-logical data flow DL1): The data fusion engine 10 transmits the generated spatial situational semantic graph in a structured message format to the semantic graph parser 21 in the distributed co-simulation core 20 through a standardized semantic interface. This message contains a serialized representation of all nodes and edges of the semantic graph.

[0053] From the logical layer L2 to the physical layer L3 (Logic-Presentation Data Flow DL2): The distributed collaborative simulation core 20 transmits the simulation results (aircraft position updates, conflict alarms, command execution status, etc.) to the role view generator of the multimodal immersive interactive subsystem 30 through a standardized presentation interface.

[0054] Physical layer L3 to logical layer L2 (interaction-feedback data flow DL3): The multimodal immersive interaction subsystem 30 feeds back user operation events (command instructions, view switching requests, etc.) to the collaborative relationship arbitrator 23 of the distributed collaborative simulation core 20 through a standardized presentation interface.

[0055] From the logical layer L2 to the semantic layer L1 (update-writeback data flow DL4): The distributed collaborative simulation core 20 writes back the new states (aircraft state updates, execution results of control commands) generated during the simulation to the semantic association fusion module 13 of the data fusion engine 10 to update the corresponding semantic nodes and edges in the airspace situation semantic graph, forming a closed-loop iterative update mechanism.

[0056] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A virtual-real fusion multi-role collaborative command simulation system for air traffic control, characterized in that, The system is implemented based on a three-layer linkage simulation architecture of "semantic layer - logical layer - physical layer". The system includes: The data fusion engine (10), located in the semantic layer, is used to collect, clean, spatiotemporally align and semantically fused multi-source heterogeneous air traffic control operation data in real time to generate an airspace situation semantic map. The distributed collaborative simulation core (20), located in the logic layer, communicates with the semantic layer through a standardized semantic interface and is used to perform the parsing of the spatial situation semantic map, the simulation logic deduction based on multi-agent collaboration, and the modeling and arbitration of multi-role collaborative command relationships; The multimodal immersive interaction subsystem (30), located in the physical layer, communicates with the logic layer through a standardized presentation interface. It is used to generate personalized display interfaces according to the differentiated command view requirements of multiple roles and to realize the rendering overlay of virtual simulation elements and real running data. The collaborative command chain management module (40) is connected to the distributed collaborative simulation core (20) and the multimodal immersive interaction subsystem (30) respectively, and is used to model and maintain the command hierarchy relationship, information flow path and decision authority matrix among multiple roles.

2. The virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The data fusion engine (10) includes: The multi-source data access adapter (11) is configured to receive multi-source heterogeneous data streams, including ADS-B broadcast automatic dependent surveillance data, radar track data, flight plan data, meteorological observation data, NOTAM air traffic notification data, and airport surface surveillance data. The spatiotemporal alignment module (12) is configured to unify the above multi-source heterogeneous data into the same spatiotemporal reference coordinate system. The spatiotemporal alignment includes timestamp normalization processing and spatial coordinate transformation. The semantic association fusion module (13) is configured to perform semantic annotation and association on the spatiotemporally aligned data based on the knowledge graph of the air traffic control domain, and generate an airspace situation semantic graph. The airspace situation semantic graph takes aircraft entities as core nodes, associates waypoint nodes, airspace unit nodes, meteorological feature nodes and control instruction nodes, and uses directed edges with semantic labels to represent the spatial relationship, prediction relationship and control relationship between entities.

3. The virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The distributed collaborative simulation core (20) includes: The semantic graph parser (21) is configured to extract key semantic feature vectors of the current simulation scenario from the airspace situation semantic graph. The key semantic feature vectors include airspace complexity index, potential conflict hotspots, traffic flow density distribution and meteorological influence factors. The multi-agent collaborative inference engine (22) is configured to run multiple types of simulated agents, including: virtual aircraft agent (221), virtual pilot agent (222), auxiliary decision agent (223), and scene disturbance agent (224); wherein, the virtual pilot agent (222) is used to simulate receiving and responding to authorization instructions issued by the controller, and the auxiliary decision agent (223) provides conflict detection and resolution suggestions to the controller based on a deep reinforcement learning strategy; The collaborative relationship arbitrator (23) is configured to perform conflict detection and priority sorting on command instructions issued by multiple roles based on the command hierarchy relationship and decision authority matrix defined by the collaborative command chain management module (40). The distributed synchronization controller (24) is configured to ensure the consistency of state between each simulation agent and each role terminal, implement the targeted distribution of simulation state updates through the publish / subscribe mode, and realize the synchronization of simulation clocks between terminals through the distributed spatiotemporal remapping mechanism.

4. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 3, characterized in that, The distributed synchronization controller (24) uses the RTI soft bus as the underlying communication support. The distributed spatiotemporal remapping mechanism includes: maintaining a local simulation clock and state cache in each role terminal; when any terminal or simulation agent changes its state, it publishes a state update event carrying a global timestamp to all terminals that subscribe to the event through the RTI soft bus; after receiving the state update event, each terminal performs spatiotemporal remapping based on the deviation between the global timestamp and the local simulation clock. For events with a time deviation within the allowable threshold, the local state is updated directly. For events with a time deviation exceeding the allowable threshold, the local state is updated after a smooth state transition through an interpolation algorithm.

5. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The multimodal immersive interaction subsystem (30) includes: The role view generator is configured to obtain the corresponding command view template and information permission configuration from the collaborative command chain management module (40) according to the role type of the currently logged-in role, and dynamically generate a personalized immersive interactive interface. The virtual-real rendering fusion processor is configured to blend real running data layers and virtual simulation layers in the same 3D geographic information scene, and distinguish between real data and simulation data through color level mapping and transparency layering strategies. The multi-channel natural human-computer interaction module is configured to support users to interact with the simulation system through one or more interaction channels, including gestures, voice, eye tracking, and touch operations.

6. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The collaborative command chain management module (40) includes: The role-permission mapping table stores predefined multiple role types and their corresponding command permissions and accessible information domains; The command chain topology manager maintains a directed acyclic graph of command hierarchy relationships among multiple roles and supports instructors in dynamically adjusting the command chain structure through a visual interactive interface. The information flow controller is configured to control the routing and distribution of coordination instructions, handover requests, and permission confirmation information among various role terminals according to the command chain topology.

7. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, It also includes a collaborative performance evaluation module, which is configured to record the operation sequence, communication records and decision delay of multiple roles in real time during the simulation process, and generate a collaborative performance evaluation report based on a preset evaluation index system after the simulation ends.

8. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The inter-layer data flow of the three-layer linkage simulation architecture includes: Semantic-logical data flow from semantic layer to logical layer: The data fusion engine (10) transmits the generated spatial situation semantic graph to the semantic graph parser (21) of the distributed collaborative simulation core (20) through a standardized semantic interface. Logical-presentation data flow from the logic layer to the physical layer: The distributed collaborative simulation core (20) transmits the simulation results to the multimodal immersive interactive subsystem (30) through a standardized presentation interface. Interaction-feedback data flow from physical layer to logical layer: The multimodal immersive interaction subsystem (30) feeds back user operation events to the collaborative relationship arbitrator (23) of the distributed collaborative simulation core (20); Update-writeback data flow from the logical layer to the semantic layer: The distributed collaborative simulation core (20) writes back the new state generated during the simulation to the data fusion engine (10) to update the spatial situation semantic map, forming a closed-loop iterative update mechanism.

9. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The multimodal immersive interactive subsystem (30) also includes a global view of the instructor / evaluator role, which provides the following functions: real-time viewing of the operation status and interaction records of each trainee role during the simulation; and adjustment of the simulation speed. Quickly jump to a specific training scene node; Set multi-role collaborative task objectives; And view the collaborative performance rating trend chart of any trained role in real time.

10. A virtual-real fusion multi-role collaborative command simulation system for air traffic control according to claim 1, characterized in that, The behavior decision-making model of the virtual pilot agent (222) is trained based on real pilot behavior data and can simulate the response delay and decision-making style of pilots with different experience levels. When a diversion instruction is received, the virtual pilot agent calculates a reasonable response delay based on the current aircraft status, the weather conditions of the diversion airport and the aircraft's remaining fuel, and generates authorization confirmation information that conforms to the standard air-to-ground communication specifications. For emergency descent operations, the virtual pilot agent automatically generates a sequence of coordinated crew actions based on a preset emergency procedure model, including oxygen mask donning, transponder coding settings and descent profile planning.

Images