Real-time flight data driven variable scale unmanned swarm three-dimensional visualization system

Abstract

Real-time flight data driven variable scale unmanned cluster three-dimensional visualization system belongs to the technical field of unmanned aerial vehicle simulation and visualization, and particularly relates to a system for updating and visualizing three-dimensional states of unmanned clusters based on external real-time flight data; solves the problems of poor interface generality of existing unmanned cluster visualization systems, difficulty in supporting large-scale variable cluster simulation, and inflexible parameter configuration and real-time adjustment; the system comprises: a flight data driving module for driving corresponding visualized unmanned aerial vehicle models in a three-dimensional scene to update states according to data converted to a unified system internal coordinate system. The real-time flight data driven variable scale unmanned cluster three-dimensional visualization system is suitable for various application scenarios such as simulation training, task planning deduction, flight control algorithm visualization verification, and real-time monitoring and demonstration based on digital twinning of unmanned aerial vehicle clusters.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) simulation and visualization technology, and in particular to a system for updating and visualizing the three-dimensional status of unmanned aerial vehicle (UAV) swarms based on external real-time flight data. Background Technology

[0002] Currently, in the field of UAV swarm simulation and training, real-time updates and visualization of swarm motion states are fundamental to effective simulation, evaluation, and operational training. Existing technologies typically rely on offline or simulation data generated from pre-built UAV physical or dynamic models to drive the swarm's motion within the visualized scene. While these devices can meet basic simulation demonstration needs to some extent, they still exhibit significant limitations and shortcomings in practical applications, particularly in high-fidelity, large-scale, and real-time training and testing scenarios.

[0003] Specifically, existing technologies mainly face the following key issues:

[0004] First, existing devices lack versatility in accessing and processing flight data. Most devices employ highly customized data interfaces, typically designed to receive and process formatted data output from specific simulation environments (such as specific flight control software or dynamics simulators). This design results in a single interface type, poor versatility, and difficulty in compatibility with real-time flight data from different data sources (e.g., flight simulators based on different protocols, telemetry data output from onboard sensors of real UAVs, etc.). When new data sources need to be accessed, extensive interface adaptation and data parsing code modifications are often required, significantly limiting the applicability and deployment efficiency of the device, making it difficult to serve as a universal unmanned swarm visualization and verification platform.

[0005] Secondly, existing devices suffer from bottlenecks in supporting cluster sizes and struggle to achieve flexible scaling. Many systems, in their initial design, did not adequately consider performance and resource management issues in large-scale cluster scenarios for their data processing, state updates, and 3D rendering engine architectures. This results in a limited number of drones that can be stably simulated, typically only a few to dozens. Furthermore, these systems often struggle to dynamically and easily adjust the cluster size during runtime. Changing the number of drones simulated usually requires reconfiguring complex parameter files or even modifying program code, making "one-click" configuration and switching of cluster sizes impossible. This makes it extremely difficult to conduct simulation verification at different scales, from squad-level to cluster-level, using the same system, reducing simulation efficiency and flexibility.

[0006] Secondly, existing devices lack convenience and real-time adjustment capabilities in configuring and managing simulation parameters. Simulations of unmanned swarms typically involve numerous parameters, including the initial number of swarm members, the initial pose (position and attitude) of each UAV, coordinate system definition, data communication ports, and 3D viewing angles. In existing technologies, the initial configuration process for these parameters is usually cumbersome, requiring modification of complex configuration files or inconvenient graphical interfaces. More importantly, during simulation operation, operators often find it difficult to make real-time, dynamic adjustments to certain key parameters (such as viewing angles and display attributes of specific UAVs). This inconvenience in parameter configuration and inflexibility in adjustment severely impacts the efficiency of simulation experiments and the operator's ability to intervene and observe the simulation process in real time.

[0007] In summary, due to shortcomings such as poor interface universality, limited supported cluster size and difficulty in adjustment, inconvenient parameter configuration, and lack of real-time adjustment functions, existing unmanned cluster state update and 3D rendering devices are usually used in single scenarios and are inefficient. They are unable to achieve efficient state synchronization and immersive 3D rendering of large-scale, variable-scale unmanned clusters driven by diverse external real-time data sources, and cannot meet the growing demand for simulation, testing, and training of complex unmanned aerial vehicle clusters.

[0008] Therefore, this application aims to provide an innovative solution to overcome the shortcomings of the prior art. Summary of the Invention

[0009] This invention proposes a real-time flight data-driven 3D visualization system for variable-scale unmanned swarms, which solves the problems of poor interface universality, difficulty in supporting large-scale variable swarm simulation, and inflexible parameter configuration and real-time adjustment in existing unmanned swarm visualization systems.

[0010] The real-time flight data-driven variable-scale unmanned swarm 3D visualization system of the present invention includes:

[0011] The flight data acquisition module is used to connect to external data sources and acquire real-time flight data of the drones from those sources.

[0012] The unmanned cluster status update server module is communicatively connected to the flight data acquisition module, and is used to receive and process the real-time flight data, and send the processed data through the network.

[0013] The unmanned cluster status update interface module communicates with the unmanned cluster status update server module via a network, and is used to receive processed data sent by the network and convert it into a unified three-dimensional scene reference coordinate system within the system.

[0014] The flight data driving module is communicatively connected to the unmanned cluster state update interface module and is used to drive the corresponding visualized UAV model in the three-dimensional scene to perform state updates based on the data after being transformed into the three-dimensional scene reference coordinate system.

[0015] The simulation parameter configuration module is used to centrally manage the system's configurable parameters through an external configuration file, and the configurable parameters include at least the drone swarm size;

[0016] The unmanned swarm state 3D presentation module is communicatively connected to the simulation parameter configuration module and the flight data driving module. It is used to generate a 3D scene and a corresponding number of visualized unmanned swarm models according to the size of the unmanned swarm, and to render the state update of the visualized unmanned swarm models in real time according to the driving of the flight data driving module.

[0017] Furthermore, in a preferred embodiment, the simulation parameter configuration module uses an external configuration file that is a script file with a key-value pair structure.

[0018] The configurable parameters include initialization parameters and runtime adjustable parameters;

[0019] The initialization parameters include at least the drone cluster size, the initial pose of each drone, the coordinate system definition, and the data port configuration;

[0020] The runtime adjustable parameters include at least the three-dimensional viewing angle parameters.

[0021] Furthermore, a preferred embodiment is provided, wherein the unmanned cluster state update interface module supports the conversion of input data in multiple coordinate systems, including:

[0022] The latitude, longitude, and altitude data in the WGS-84 coordinate system are converted into geocentric and geofixed coordinate system data, then into local coordinate system data, and finally into the system's 3D scene reference coordinate system data.

[0023] And converting the local coordinate system data of Northeast Sky and Northeast Earth to the reference coordinate system of the three-dimensional scene;

[0024] The transformation includes unifying the coordinate axis directions and unifying the left-handed and right-handed coordinate systems.

[0025] Furthermore, a preferred embodiment is provided in which the unmanned cluster status update server module allocates a set of independent data sending ports for each UAV in the cluster, and the unmanned cluster status update interface module configures a corresponding data receiving port for each UAV, thereby achieving data flow isolation through port mapping.

[0026] Furthermore, in a preferred embodiment, the unmanned cluster state update server module is also used to convert the acquired flight data into a data representation format, including converting attitude quaternion data into three-axis attitude angle data in the body coordinate system.

[0027] Furthermore, a preferred embodiment is provided, wherein the unmanned cluster state 3D presentation module includes:

[0028] The model instantiation unit is used to dynamically generate a corresponding number of visualized UAV models in a 3D scene according to the cluster number parameter in the simulation parameter configuration module.

[0029] The multi-view presentation unit has multiple built-in 3D presentation components to provide various fixed viewing angles, including top view, front view, back view, left view, right view and bottom view.

[0030] The perspective switching unit is used to switch the observation perspective between a first-person perspective and a third-person free perspective.

[0031] Furthermore, a preferred implementation is provided in which the flight data driving module maintains an independent data storage space for each visualized UAV model and periodically reads standardized position and attitude data from the corresponding data storage space, directly mapping it to the pose of the corresponding visualized UAV model in the three-dimensional scene, thereby achieving concurrent updates of the cluster state.

[0032] This invention also proposes a real-time flight data-driven 3D visualization method for variable-scale unmanned swarms, applicable to a real-time flight data-driven 3D visualization system for variable-scale unmanned swarms as described in any of the above embodiments. The method includes the following steps:

[0033] The simulation parameter configuration module obtains and parses the external configuration file to determine the configurable parameters; the configurable parameters include at least the drone swarm size and initial parameters.

[0034] The unmanned swarm state 3D presentation module generates a corresponding number of visualized unmanned swarm models in a 3D scene based on the swarm size and initial parameters.

[0035] The flight data acquisition module acquires real-time flight data of the drone from an external data source.

[0036] The real-time flight data is processed by the unmanned cluster status update server module, and the processed data is sent via the network.

[0037] The unmanned cluster status update interface module receives the processed data sent by the network, performs decoding and coordinate system-1 conversion, and stores the coordinate system-1 converted data separately for each unmanned drone.

[0038] The flight data-driven module reads the coordinate system-transformed data stored by each UAV and drives the corresponding visualized UAV model in the 3D scene to update its position and attitude.

[0039] The updated 3D scene is rendered in real time using the unmanned cluster state 3D rendering module.

[0040] Furthermore, a preferred implementation is provided, wherein the coordinate system transformation via the unmanned cluster state update interface module includes:

[0041] If the input data is latitude, longitude, and altitude data in the WGS-84 coordinate system, then the conversion from the WGS84 coordinate system to the Earth-centered and Earth-fixed coordinate system, from the Earth-centered and Earth-fixed coordinate system to the local coordinate system, and from the local coordinate system to the 3D scene reference coordinate system will be performed in sequence.

[0042] If the input data is in a local coordinate system, then a transformation from that local coordinate system to the 3D scene reference coordinate system is performed.

[0043] The transformation process includes adjusting the definition of the positive direction of the coordinate axes and processing the left-handed and right-handed coordinate systems.

[0044] The present invention also proposes a computer storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed, it performs the real-time flight data-driven variable-scale unmanned swarm 3D visualization method described in any of the above embodiments.

[0045] The present invention has the following beneficial effects:

[0046] 1. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system of this invention constructs an end-to-end pipeline from external data access, protocol parsing, coordinate unification, state-driven operation to 3D display through a complete collaborative architecture including a simulation parameter configuration module, a flight data acquisition and server module, a generalized state update interface module, a flight data driving module, and a 3D presentation module. This architecture decouples the data flow from the configuration flow, enabling the system to process real-time flight data from various heterogeneous external data sources using a unified and standardized internal data processing flow. Furthermore, it flexibly controls each stage of the pipeline through parameterized configuration, ultimately achieving accurate and real-time 3D reproduction of the unmanned swarm's motion state, significantly improving the system's versatility, scalability, and processing efficiency.

[0047] 2. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system of this invention achieves flexible and convenient configuration and efficient resource management of the simulation swarm size through a simulation parameter configuration module based on lightweight structured files such as JSON, and through the technical means of instantiating and encapsulating each UAV in the swarm and allocating independent data storage space for each instance. Users only need to modify key-value pair parameters such as the number of UAVs in the configuration file, and the system can dynamically generate a specified number of visualized UAV entities during initialization and allocate independent communication and data buffers to them. This "parameter-driven + instance isolation" design allows the system to easily support swarm simulations of different sizes, from single UAVs to hundreds of UAVs, without requiring modification of the core code during adjustment, avoiding resource conflicts and greatly enhancing the system's flexibility and configurability.

[0048] 3. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system described in this invention utilizes a universal unmanned swarm state update interface module. This module integrates support for various commonly used geographic and local coordinate systems, such as WGS-84, Northeast-Sky, and Northeast-Earth, and incorporates a data conversion chain from the global coordinate system to the local coordinate system, then to the scene reference coordinate system, and finally to the left-handed or right-handed coordinate system. This allows for efficient and accurate processing of heterogeneous flight data from real UAVs or various simulators. Regardless of the coordinate system (global or local) and chirality rule (left-handed or right-handed) of the external input data, this interface module can uniformly convert it into the standard coordinate system data used by the system's internal 3D scene through a preset conversion algorithm. This shields the differences in data sources, enabling the system to seamlessly access and utilize various internal and external data sources to drive 3D visualization, thus expanding the system's application scenarios and data compatibility.

[0049] 4. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system described in this invention supports real-time modification of runtime parameters through its simulation parameter configuration module, and provides multi-view, switchable observation components through its unmanned swarm state 3D presentation module. Visualization parameters and observation perspectives can be dynamically adjusted during simulation. Operators can switch between first-person and third-person perspectives in real time, or adjust the observation window layout to focus on specific UAVs, without interrupting the simulation process. This "adjustable during simulation" capability greatly enhances operators' real-time monitoring and situational awareness of complex swarm flight states, improving the effectiveness of simulation verification and training.

[0050] 5. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system described in this invention, through the tight coupling of the flight data-driven module and the 3D presentation module, directly maps the real-time position and attitude data, after standardization processing by the interface module, to the position and rotation state of the corresponding UAV model in the 3D scene. This enables high-fidelity, low-latency synchronous presentation of the motion state of real or simulated UAVs in a 3D virtual environment. This provides an intuitive and realistic visualization environment for UAV swarm control training, mission simulation, and algorithm testing based on digital twin technology.

[0051] The real-time flight data-driven variable-scale unmanned swarm 3D visualization system described in this invention is suitable for various application scenarios such as unmanned swarm simulation training, mission planning and simulation, flight control algorithm visualization verification, and real-time monitoring and demonstration based on digital twins. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the overall structure of a real-time flight data-driven variable-scale unmanned swarm 3D visualization system, as described in one embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the internal structure of the flight data acquisition module in one embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram illustrating the data transmission relationship between the unmanned cluster status update server module and the unmanned cluster status update interface module in one embodiment of the present invention.

[0056] Figure 4 This is a schematic diagram of the working principle of the flight data driving module in one embodiment of the present invention;

[0057] Figure 5 An example diagram illustrating the parameter configuration and reading relationship between the simulation parameter configuration module and the unmanned cluster state 3D presentation module in one embodiment of the present invention;

[0058] Figure 6 This is an example diagram of the flight path of an unmanned swarm (i.e., a drone swarm) in one embodiment of the present invention.

[0059] Figure 7An example diagram of the initial state of an unmanned swarm, as shown in one embodiment of the present invention;

[0060] Figure 8 This is a three-dimensional schematic diagram of a certain flight state after the unmanned swarm takes off, as described in one embodiment of the present invention.

[0061] Figure 9 This is a three-dimensional schematic diagram of another flight state after the unmanned swarm takes off, as described in one embodiment of the present invention.

[0062] Figure 10 This is a schematic diagram showing the distribution and flight status of a cluster of 100 drones in a three-dimensional scene, as one embodiment of the present invention.

[0063] Figure 11 In one embodiment of the present invention, a schematic diagram showing the distribution and flight status of a 100-unmanned swarm in a three-dimensional scene from another perspective;

[0064] Figure 12 This is a schematic diagram of the interface conversion principle of the unmanned cluster state update interface module in one embodiment of the present invention. Detailed Implementation

[0065] To make the technical solutions and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail and completely below with reference to the accompanying drawings. The various embodiments described below are only some preferred embodiments of the present invention, and not all of them; the various embodiments described below are intended to explain the present invention and should not be construed as limiting the present invention; reasonable combinations of the technical features defined in the various embodiments of the present invention, as well as all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort, are all within the scope of protection of the present invention.

[0066] Implementation Method 1: Real-time flight data-driven 3D visualization system for variable-scale unmanned swarms:

[0067] The system adopts a modular design and can be logically divided into three core parts: the acquisition, processing, and transmission of unmanned swarm flight data; the reception, status update, and 3D visualization of unmanned swarm flight data; and the top-level configuration of parameters related to status updates and 3D visualization. These three parts work together through clearly defined interfaces and data flows to achieve a complete process from external data input to internal 3D visualization.

[0068] The unmanned swarm flight data acquisition, processing, and transmission section mainly includes a flight data acquisition module and an unmanned swarm status update server module. In this system, unmanned aerial vehicles (UAVs) are categorized into two types: "UAVs from external data sources" serving as data sources, and digital twins used for display in a 3D scene, i.e., "visualized UAVs (models)." The core function of the flight data acquisition module is to establish connections with external data sources (such as real UAV flight control systems, third-party flight simulation software, etc.) and acquire raw flight data such as position and attitude of the UAVs in real time. The unmanned swarm status update server module acts as a data relay and preprocessing hub, responsible for receiving raw data from the flight data acquisition module, performing necessary format conversions, data type adjustments, and other preprocessing operations, and sending out the processed standard format data via network communication. This part not only provides a real-time data stream for subsequent visualization but also provides a basis for configuring the initial state (such as initial position and orientation) of the visualized UAVs.

[0069] The receiving, status updating, and 3D rendering of unmanned swarm flight data is the core processing and display component of the system, comprising an unmanned swarm status update interface module, a flight data driving module, and an unmanned swarm status 3D rendering module. The unmanned swarm status update interface module listens to the network port and receives standardized flight data packets from the server module. Its key function is to decode the received data and, according to preset coordinate system transformation rules, uniformly transform data from different coordinate systems (such as the WGS-84 geodetic coordinate system, the Northeast-Northeast Local Coordinate System, etc.) to the internal reference coordinate system used by the system's 3D scene, thus unifying the data spatial reference. The flight data driving module obtains the unified data from the interface module and directly applies this data (including 3D position coordinates and three-axis attitude angles) to the corresponding visualized UAV model, driving it to perform position movement and attitude rotation in the 3D scene synchronized with the data source, thereby completing real-time status updates. The unmanned swarm status 3D rendering module constructs a 3D virtual environment, responsible for loading scene elements such as terrain and skyboxes, and instantiating and rendering a specified number of visualized UAV models according to the configuration. It receives drive commands from the flight data drive module, updates the poses of all drone models in the scene in real time, and presents the motion status of the entire cluster to the user in a multi-view, switchable manner.

[0070] The top-level configuration of the state update and 3D rendering parameters is handled independently by the simulation parameter configuration module. This module centrally manages all configurable parameters of the system through an easily readable and writable external configuration file (such as a JSON file). These parameters are mainly divided into two categories: one is initialization parameters, such as cluster size (number of drones, cluster size), initial pose of each drone, coordinate system definition, data receiving port, etc. These parameters are read when the system starts up to initialize the entire simulation environment; the other is runtime adjustable parameters, such as 3D viewing angle, window layout, display labels for specific drones, etc. These parameters allow users to dynamically modify them during the simulation to adjust the viewing experience. After deserialization and parsing, the configured parameters are applied to the state update and 3D rendering parts to configure information such as the initial state and 3D rendering status of variable-scale drone clusters. All functional modules obtain their respective operating parameters by reading and parsing the configuration module's file during startup or operation, thereby achieving flexible control of system behavior.

[0071] To address the shortcomings of existing technologies, this invention provides a systematic solution through the aforementioned modular architecture:

[0072] To address the issue of poor data interface universality, this invention connects to diverse data sources through a flight data acquisition module (classifying flight data by type), and establishes a standardized data reception, processing, and conversion pathway consisting of a server module and an interface module. This transforms heterogeneous data into an internal standard format, achieving universal data access and processing capabilities.

[0073] To address the issues of fixed cluster size (supporting a limited number of drones) and difficulty in adjustment (making it hard to achieve variable-scale cluster simulation), this invention centrally manages key parameters such as cluster size through a simulation parameter configuration module. The 3D rendering module parses these parameters upon startup (reading parameters through deserialization and adjusting simulation functions based on these parameters) and dynamically generates the corresponding number of drone instances, thus achieving "parameterized variable" cluster size. For example, the drone cluster 3D rendering module, by deserializing and parsing parameters related to the number of drones and their generation from the simulation parameter configuration module, can generate a specified number of drones in the 3D rendering scene after the device is running, enabling simulation configuration for variable-scale drone clusters.

[0074] To address the issues of cumbersome parameter configuration and weak real-time adjustment capabilities, this invention provides a unified configuration file interface through a simulation parameter configuration module. This allows most parameters (those affecting state updates and 3D rendering) to be conveniently and dynamically set and modified through file editing. Furthermore, the system supports hot updates for some visualization parameters during runtime, thereby significantly improving configuration efficiency and operational flexibility.

[0075] In this embodiment, a complete architecture is designed, which includes simulation parameter configuration, flight data acquisition and processing, data transmission and reception, data processing and transformation, flight data-driven state updates, and 3D visual presentation, supporting cluster simulations of variable scale.

[0076] In this implementation, large-scale variable swarm flight simulation in a 3D scene is supported. The individual state update process in the swarm is abstracted and encapsulated, and an independent data storage space is allocated for each instantiated UAV. A simulation parameter configuration module is constructed to facilitate the configuration of swarm simulation parameters. A 3D presentation module of the UAV swarm state enables the generation of swarms of variable scale and the visual presentation of diverse UAV state updates. A UAV swarm state update interface module is used to receive swarm flight data, satisfying the input of common simulation and real swarm flight data. At the same time, the data reception, processing, and transformation processes of individuals in the swarm are isolated to avoid resource conflicts. A flight data-driven module completes the state update of the swarm, realizing large-scale swarm simulation in a 3D scene.

[0077] In this embodiment, external flight simulation data input is supported to drive the unmanned swarm to achieve state updates. The unmanned swarm state update interface module is used to receive, process, and convert flight data output from inertial navigation sensors, and can support flight data output from both simulated and real UAVs. The swarm state is updated through the device's flight data driving module, and diverse visual presentations are achieved through the unmanned swarm state 3D presentation module. During use, the input data of the interface module is monitored according to simulation requirements, which can easily complete the swarm state update in the 3D visual environment and the diverse visual presentation of the UAV simulation process.

[0078] In summary, this invention constructs a complete, closed-loop technical solution system, encompassing "top-level parameter configuration," "external data acquisition and processing," "data transmission and reception," "data parsing and coordinate unification," and "data-driven state updates," ultimately achieving "3D visual presentation" and supporting "real-time adjustment of some parameters." The modules within this system have clearly defined responsibilities and low coupling, working together to address key deficiencies in existing technologies.

[0079] Implementation Method 2: Flight Data Acquisition Module

[0080] The flight data acquisition module serves as the entry point for the system's interaction with the external data environment. It is primarily responsible for determining the data source and acquiring the UAV flight data required to drive 3D visualization. Its functions cover two main stages: acquiring and determining the initial state of the UAV before takeoff, and acquiring real-time data during flight.

[0081] In the initial state acquisition and determination phase, this module needs to complete the initial configuration work before simulation. Specifically, the operator or the upper-level configuration system needs to set a series of basic parameters through this module (or its associated configuration file) to clarify the characteristics of the data source and the benchmark for subsequent processing. These parameters include, but are not limited to: selecting the type of UAV platform used as the data source (such as multi-rotor, fixed-wing), clarifying the specific data fields to be subscribed to for state updates (such as latitude, longitude, altitude, speed, quaternions, Euler angles, etc.), and defining the type of coordinate system used for the data, the origin position, and the positive direction of each coordinate axis. In addition, the type of coordinate system expected by the data receiving end (i.e., within the system) also needs to be specified. These initial parameters are not only used to guide how the flight data acquisition module interfaces with the data source, but more importantly, they will serve as a reference for determining the initial position and attitude when generating a visualized UAV digital twin in the subsequent 3D scene, and as reference values ​​for the UAV configuration in the 3D presentation. In this phase, the flight path planning of the UAV swarm also needs to be completed, that is, pre-setting the flight path points or flight tasks for each data source UAV in the swarm to form a complete flight plan.

[0082] During the real-time data acquisition phase, once the simulation starts, the drones operate according to the planned flight trajectory of the drone swarm, and the flight data acquisition module begins continuous operation. Based on the parameters set in the previous phase, it establishes a data link with the data source (which could be the physical drone's flight control system or a software-in-the-loop simulation environment). The module acquires real-time flight data from each data source drone at a specific, configurable frequency (e.g., 50Hz or 100Hz) through mechanisms such as data subscription, callback functions, or polling. The core data consists of the drone's position and attitude information in space. The acquired raw flight data is typically timestamped and retains its original format from the data source. This data is then immediately sent to the drone swarm status update server module for further processing and forwarding. This module ensures that external real-time data can be stably and efficiently acquired and injected into the system's internal processing pipeline.

[0083] Implementation Method 3: Unmanned Cluster Status Update Server Module:

[0084] The unmanned cluster status update server module, located after the flight data acquisition module, plays the role of data processing and distribution. Its main function is to standardize the raw flight data acquired upstream and reliably transmit it to the downstream interface module. The design of this module fully considers the actual conditions of network transmission and the diversity of raw data.

[0085] In terms of data processing, this module primarily performs two key operations. First is data type conversion. Because the raw data (especially floating-point numbers) obtained by the flight data acquisition module from different data sources may have different memory representation formats than the byte order or standard formats required for network transmission, this module needs data type conversion capabilities. For example, it can convert floating-point numbers of different precisions to network transmission standard formats, or adjust the byte order of multi-byte data to ensure correct parsing in heterogeneous computing environments. Second is data representation format conversion. There are various ways to describe UAV attitude, commonly quaternions and Euler angles. While quaternions are computationally efficient and have no singularities, their physical meaning is not intuitive and they are not convenient for directly driving attitude updates of 3D models. Therefore, an important function of this module is to convert attitude data represented in quaternion form into more intuitive and easily usable three-axis attitude angles (roll, pitch, and yaw) in the body coordinate system if received. Additionally, it may include standardization of other data formats, such as unifying position data in different units.

[0086] In terms of data transmission, this module employs a socket-based network communication mechanism to send processed data to the unmanned swarm status update interface module. To ensure the accuracy and orderliness of data transmission and avoid confusion between different drones or different data fields, this module adopts a port splitting strategy. Specifically, it allocates a set of independent network ports to each drone in the swarm for sending different categories of status data for that drone. For example, a drone is allocated six consecutive ports to send its X, Y, and Z position coordinates and roll, pitch, and yaw angles, respectively. Correspondingly, the interface module at the receiving end also presets a corresponding port group for each drone. This point-to-point, data-category-isolated port mapping mechanism effectively reduces the risk of data corruption (ensuring accurate data transmission and reducing the risk of drones receiving incorrect data), ensuring data integrity and the accuracy of correspondences during large-scale swarm data transmission.

[0087] This module operates continuously during simulation, and its workflow can be summarized as the following cyclical steps:

[0088] 1) Data processing: Receive raw data packets from the flight data acquisition module, convert data types and representations as needed, and generate standardized data packets.

[0089] 2) Data transmission: According to the preset port allocation rules, standardized data packets are sent to the designated port of the interface module via Ethernet or other networks.

[0090] 3) Loop execution: The above data processing and transmission process is repeated continuously at a preset fixed frequency (synchronized with or adapted to the data acquisition frequency) to form a stable, real-time downlink data stream.

[0091] Implementation Method 4: Unmanned Cluster Status Update Interface Module:

[0092] The unmanned cluster state update interface module is a key component for achieving data compatibility and universality in this system. It is responsible for receiving network data from the server module, decoding it, and performing core processing such as coordinate system unification, providing directly usable standardized data for the final state-driven process. This module addresses the problem of existing technologies having highly specialized interfaces and being unable to handle data from multiple external coordinate systems by designing a universal data processing and conversion path.

[0093] When the interface module starts, it first completes the initial configuration of network communication according to the agreement with the server module. This includes setting the corresponding socket receiving port for each type of data (such as the position and attitude of each UAV) and configuring an appropriate receiving buffer size to cope with possible data flow fluctuations. During simulation, the module polls the receiving buffer of each socket at a specific frequency. Once new data is detected in the buffer, it is immediately extracted and decoded to restore the original state information.

[0094] Decoded data processing is the core of this module. Since external data sources may use different spatial reference systems, this module has a powerful built-in coordinate system transformation capability to ensure that all input data is unified to the reference coordinate system used by the system's internal 3D scene. When external data is input, the module will process it according to its category. This module divides the input data into two main categories based on its spatial reference system for targeted processing: one is global data based on the WGS-84 coordinate system (World Geodetic System 1984), which typically uses longitude, latitude, and altitude to represent the UAV's absolute position globally, used to describe the UAV's state in the global coordinate system; the other is data based on various local coordinate systems, such as the "Northeast-East-South" (ENU) or "Northeast-East-West" (NED) coordinate systems. These coordinate systems use a specific point as the origin to describe the UAV's relative position, used to describe the UAV's state in the local reference system. These two types of coordinate systems can cover common flight data types. The module will determine the processing method based on the type of input data. The device is configured to support processing both global UAV data in the WGS-84 coordinate system and data in local coordinate systems.

[0095] For latitude, longitude, and altitude data in the WGS-84 coordinate system, the module performs a three-step transformation chain: First, it applies a geographic coordinate system transformation algorithm to convert the latitude, longitude, and altitude data into 3D coordinates in a geocentric rectangular coordinate system. Second, it selects a suitable local coordinate system origin (usually the scene center or the starting point of the first UAV) and converts the geocentric coordinates into coordinates in that local coordinate system. Finally, it transforms the data in this local coordinate system into the predefined "scene reference coordinate system" ultimately used by the system's 3D scene.

[0096] For local coordinate system data such as Northeast Sky and Northeast Earth, the global coordinate transformation step is skipped, and the transformation from the input local coordinate system to the system's internal "scene reference coordinate system" is performed directly.

[0097] After completing the coordinate system transformation described above, the positive directions of the position components and attitude angles of the data on the three coordinate axes may still not be completely consistent with the scene reference coordinate system. Therefore, the module still needs to perform a unified adjustment of the coordinate axis directions.

[0098] Furthermore, different 3D graphics engines or data sources may employ different chirality rules (left-handed or right-handed coordinate systems). This module supports unifying chirality. For example, if the system scene reference frame is left-handed, while the input data is right-handed, the module converts from right-handed to left-handed by changing the sign of one or more coordinate axis components (e.g., inverting the Z-axis). This step is unnecessary if the chirality of the input data is already consistent with the scene reference frame.

[0099] In summary, the interface module's data processing flow ensures that regardless of the global or local coordinate system from which the input data originates, or what chirality rules are used, it can ultimately be transformed into a unified format that perfectly matches the 3D visualization scene. When all the necessary data (such as position XYZ, attitude angle RPY) for a particular UAV has been received and processed at a given moment, this standardized data will be stored by type in a data storage area specifically allocated for that UAV. This completes one round of data reception, decoding, conversion, and storage for a single UAV. The data in this storage area will serve as direct input to the flight data-driven module, used to update the corresponding UAV's state in the 3D scene. The above data polling, reception, and processing process operates at a certain frequency. If all data in the same batch has been received and processed, it signifies the completion of one round of state data transmission and reception processing.

[0100] It should be noted that some existing research has developed interfaces that are suitable for processing UAV status data within the system, which are highly specialized and lack the ability to process and transform external flight data.

[0101] In this embodiment, external flight simulation data input processing and conversion are supported, thereby driving the unmanned swarm to achieve state updates.

[0102] In this embodiment, the interface module receives, processes, and transforms flight data output from real and simulated UAVs via inertial navigation sensors. Based on the transformed data, the flight data driving module of the device updates the cluster's state, and the 3D presentation module of the UAV cluster state provides diverse visual representations. During use, by focusing on the input data of the interface module according to simulation requirements, cluster state updates in a 3D visual environment can be easily completed.

[0103] Implementation Method 5: Flight Data Driven Module

[0104] The flight data driving module is a bridge connecting data processing and 3D rendering. Its core function is to use flight data that has been standardized by the interface module to directly drive the corresponding visualized UAV model in the 3D scene, so as to realize the real-time and accurate update of its motion status.

[0105] The module works by continuously monitoring the data storage space allocated by the interface module for each drone. When it detects that the data in a certain storage space has been updated by the interface module (i.e., a new batch of standardized flight data has been received), the drive module immediately initiates the drive process for updating the drone's status. This process includes two distinct steps:

[0106] The first step is data extraction and preprocessing. The drive module extracts the latest position coordinates and attitude angles from the specified UAV's data storage space, categorized by data type. These data have already been processed by the interface module, and their values ​​directly correspond to the spatial position (X, Y, Z) and attitude angles (roll, pitch, yaw) relative to that coordinate system or the aircraft's body coordinate system in the system's 3D scene reference coordinate system. In this step, the module may also perform some simple data validity checks or interpolation smoothing to ensure data stability and continuity, but the core data itself is already in a directly usable format.

[0107] The second step is state mapping and updating. The driver module directly sets the extracted position coordinate data (X, Y, Z) as the new position of the corresponding UAV model in world space (or scene coordinate system) in the 3D scene. At the same time, it converts the extracted attitude angle data (Roll, Pitch, Yaw) into the rotation representation required by the 3D graphics engine (such as Euler angles or quaternions) and sets it as the new attitude (or rotation state) of the UAV model's body coordinate system. Through this direct, one-to-one data mapping, the real-time spatial pose of the UAV from the external data source is accurately reproduced on the digital twin in the 3D visualization environment.

[0108] This process is executed cyclically on each visualized drone at an extremely high frequency (typically synchronized with the data reception frequency or following the graphics rendering frame rate). For large-scale clusters, the driving module polls and updates the data storage space of all drones in the cluster in a parallel or efficient manner, thereby achieving synchronous and concurrent driving of the motion state of the entire drone cluster. This ensures that the cluster motion presented in the 3D scene is consistent with the external data source, achieving a real-time visualization effect.

[0109] Implementation Method 6: 3D Presentation Module for Unmanned Cluster Status:

[0110] The unmanned swarm status 3D presentation module is the system's user-facing graphical output terminal. It is responsible for constructing a realistic 3D virtual environment and dynamically rendering the motion of the entire unmanned swarm based on the status information provided by the flight data-driven module, providing users with intuitive visual feedback and an immersive observation experience.

[0111] This module's functions mainly cover two aspects: instantiation (generation) and deployment of visualized drone models, and diverse visual presentations during the simulation process.

[0112] In terms of instantiation and deployment of visualized UAV models, the core task of this module is to respond to configuration and generate UAV entities in the 3D scene. Upon system startup, this module reads parameters regarding cluster size (e.g., the number of UAVs N) from the simulation parameter configuration module and calls the instantiation interface of the 3D graphics engine to dynamically create N UAV 3D models in the scene. Each model is an independent, individually controllable visualized entity (i.e., a digital twin). The module encapsulates the logic for updating the UAV state (i.e., receiving instructions from the driver module and updating its own pose) within each entity or its associated controller. As the simulation runs, the flight data driver module continuously provides new pose data, and the corresponding UAV model in the scene updates its position and rotation state in real time, thus achieving a mapping from data to 3D animation and completing the 3D visualization of the UAV state.

[0113] Regarding visualization during the simulation process, this module provides rich and flexible observation methods, specifically including:

[0114] Scene environment deployment: By importing resources such as 3D models, terrain elevation data, and texture maps, a virtual flight environment containing elements such as terrain, landforms, sky, and buildings is constructed to enhance the realism of the simulation.

[0115] Multi-view (window) presentation: The module incorporates multiple 3D presentation components (which can be understood as virtual cameras or observation windows) for observing the drone from different angles (presentation and changes in window perspective). In a preferred implementation, the system constructs six such sets of components. One set of components is deployed on the bottom of the drone to simulate the payload's perspective, presenting changes in the ground scene during flight and outputting an image stream. The remaining five sets of components correspond to the drone's top, front, rear, left, and right views, respectively, providing users with a comprehensive view of the drone's own status. These views can be tiled across multiple display windows.

[0116] Perspective Switching: The module also integrates a perspective-following component, enabling free switching of the observation perspective. In the "third-person" perspective, users can freely control the observation camera to move and rotate arbitrarily within the 3D scene, observing the entire cluster or a specific drone from a macroscopic external perspective. When switching to the "first-person" perspective, the observation view is automatically bound to the virtual camera carried by a selected drone. At this point, the user's view will be entirely determined by the drone's pose and the orientation of its onboard camera. Through the simulation parameter configuration module, users can switch between the currently active observation perspective or perspective mode in real time, thus flexibly monitoring the simulation process.

[0117] Implementation Method 7: Simulation Parameter Configuration Module:

[0118] The simulation parameter configuration module is the control center and information hub of the entire system. It adopts an externalized and structured approach to centrally and uniformly manage and configure all parameters that affect the system's behavior and performance, which is the key to achieving system flexibility, scalability, and ease of use.

[0119] Physically, this module typically manifests as one or more readable and writable configuration files, preferably in JSON format. JSON files store data using a key-value pair structure, treating each configuration parameter as an attribute with its corresponding numerical value. For example, "uav_count": 10 indicates a count of 10 drones. This structure offers high readability, facilitating human reading and editing, while also being easily deserialized and parsed by programs. When system behavior needs adjustment, users or upper-level management tools only need to modify the values ​​corresponding to the keys in the configuration file. Each functional module of the system rereads these values ​​upon startup or runtime, enabling dynamic adjustment of functionality without requiring modification or recompilation of the core code.

[0120] The parameters managed in this module are mainly divided into two categories, corresponding to different stages of system operation:

[0121] Initialization configuration parameters: These parameters are read during system startup to complete the initial construction of the entire simulation environment. They mainly include: the number of visualized drones to be generated in the 3D scene, the initial position and attitude angle of each drone, the definition of the coordinate system used for various data types (e.g., origin, positive directions of each axis), the expected coordinate system type when the interface module receives data, and the network port configuration related to data reception. These parameters directly determine the initial scale, layout, and basic communication settings of the cluster simulation. When the system starts, the drone cluster state 3D rendering module, interface module, etc., read these parameters and accordingly create the corresponding number of drone entities, initialize the scene coordinate system, bind network ports, etc., completing the environment setup from scratch.

[0122] Real-time parameter adjustment: These parameters can be dynamically modified during simulation, primarily focusing on adjusting the visualization presentation without affecting the core state update logic. For example, users can switch the viewpoint of the current main observation window in real time (e.g., from a global top-down view to the first-person view of a specific drone), adjust the layout of the display window, show or hide drone labels and trajectory lines, or highlight specific drones. At this stage, the initial state of the cluster and the data flow are already determined, and the real-time adjustment function enhances the user's interactivity and observation flexibility during the simulation process.

[0123] Based on the above introduction of each module, the complete workflow of this system for updating the status of unmanned clusters and rendering in 3D can be summarized as follows:

[0124] (1) Initialization Phase: Operators use the flight data acquisition module or its configuration tool to set initial parameters and plan flight routes for the UAV cluster from the data source. Simultaneously, based on the reference information provided by the data source, the operator sets parameters such as the number of UAVs and their initial pose in the JSON file of the simulation parameter configuration module. After system startup, the UAV cluster state 3D rendering module parses these parameters, dynamically generates a specified number of UAV models in the 3D scene, and sets their initial positions and attitudes, completing the initial deployment of the scene and cluster.

[0125] (2) Data acquisition phase: The simulation starts running, and the data source UAV (real or simulated) moves along the preset route. The flight data acquisition module continuously subscribes to and acquires their real-time flight data at a specific frequency.

[0126] (3) Data processing and forwarding stage: The server module performs format and type conversion on the acquired raw data (such as quaternion to Euler angle), and then sends the processed data out through the network according to the port pre-assigned to each UAV.

[0127] (4) Data Reception and Standardization Stage: The interface module listens to the corresponding port and receives data. The received data is decoded and converted into standard coordinate system data used by the system's 3D scene using the built-in transformation algorithm according to its original coordinate system (which may be WGS-84 or some local coordinate system). Then, the standardized data is stored in their respective independent storage spaces according to the UAV number.

[0128] (5) State-driven and presentation stage: The flight data-driven module periodically reads the latest standardized position and attitude data from the data storage space of each UAV. This data is directly set as the pose of the corresponding UAV model in the 3D scene. The UAV cluster state 3D presentation module renders the entire scene in real time, displaying the new state of each UAV after being driven in the form of 3D graphics, thereby completing the closed-loop update from external data to internal visualized state. These steps (2) to (5) are continuously executed in a loop during the simulation process to achieve continuous real-time update and presentation of the state.

[0129] In this embodiment, the simulation parameter configuration module is mainly used to efficiently and conveniently configure the initial state and related parameters of the unmanned cluster.

[0130] In this embodiment, the simulation parameter configuration module mainly consists of readable and writable files, such as JavaScript Object Notation files. Based on the characteristics of the parameters, they can be divided into two parts: attributes and values. Changing the values ​​corresponding to the attributes affects the parameters' role in the simulation process. JavaScript Object Notation files often store object attributes and values ​​in key-value pairs. This storage method offers high readability while facilitating data reading and writing. Therefore, the device achieves rapid parameter configuration based on the above process. When parameter configuration is required, the parameter values ​​are first modified based on the key-value pair information stored in the file. The functional module then reads and parses the parameter values, ultimately adjusting the simulation function.

[0131] In this embodiment, the configurable parameters mainly include two types: one is the initialization configuration for unmanned cluster state update and 3D rendering (initialization configuration parameters), including the initial position and attitude of the unmanned cluster; the other is the real-time adjustment for unmanned cluster state update and 3D rendering simulation (real-time adjustment parameters).

[0132] In this embodiment, the initialization configuration parameters, which can be configured, are mainly as follows:

[0133] The number of drones generated in the 3D scene, the initial position and attitude of each drone, the zero point and positive direction of various data, the coordinate system type of the interface receiving data, and the relevant configuration of the data receiving port, etc. These parameters will affect the initial state of the drone swarm, and thus affect the subsequent drone swarm state update and 3D rendering process. When this module starts, other related modules read the parameters in the corresponding files in the module through deserialization parsing and other methods, and complete the relevant initialization deployment in the 3D scene.

[0134] In this embodiment, the real-time parameter adjustment is mainly used for parameter adjustment during UAV status updates:

[0135] At this stage, the data sources and methods for the initial and subsequent status updates of the drones have been set up. Therefore, this section focuses more on adjusting the visualization of the drone flight status, including presenting the flight status of a specified number of drones from multiple perspectives.

[0136] This implementation supports variable-scale swarm flight simulation in a 3D scene. The individual state update process within the swarm is abstracted and encapsulated, with independent data storage space allocated for each instantiated UAV. Simultaneously, the data reception, processing, and transformation processes of individuals within the swarm are isolated to avoid resource conflicts. A simulation parameter configuration module is built based on JSON script files for convenient configuration of swarm simulation parameters. Parameter data is stored in key-value pairs for easy reading and modification. When in use, parameters in the simulation parameter configuration module are read or modified, and combined with the UAV swarm state 3D presentation module, variable-scale swarm generation and diverse visual presentation of UAV state updates are achieved.

[0137] Implementation Method 8: Providing Example 1: Small-scale (3 aircraft) unmanned swarm 3D visualization example:

[0138] The implementation of this system will be further explained in detail below with a specific small-scale cluster simulation example. This example simulates a small cluster consisting of 3 fixed-wing UAVs.

[0139] Step 1: Data source configuration and route planning:

[0140] The initial configuration of the three UAVs serving as the data source is performed using the flight data acquisition module or its associated configuration interface. In the configuration file, the coordinate system used by the data source UAVs to provide flight data is set to the "Northeast-Sky" (ENU) local coordinate system. Simultaneously, the expected coordinate system for receiving data by the system's internal interface module is set to a spatial Cartesian coordinate system, specified as a left-handed system, with each axis's positive direction defined according to the global direction of the 3D scene (e.g., X-axis East, Y-axis North, Z-axis Sky). The UAVs are configured as fixed-wing, with a quantity of three. Their initial attitude is set to face due east. Their initial positions are based on the origin of the scene coordinate system, arranged sequentially along the due north direction (Y-axis), with offsets from the origin of 0 meters, +30 meters, and +60 meters, respectively. After completing the basic parameter configuration, flight paths are planned for each of the three data source UAVs to ensure they can perform predetermined formation or maneuvering actions in the air. After takeoff, each UAV moves along its corresponding flight path, achieving flight path planning for the UAV swarm. Finally, configure the relevant communication components of the flight data acquisition module so that it can subscribe to and acquire the real-time flight data of these three drones through a specific API or protocol after the simulation starts.

[0141] Step 2: Visualize the cluster initialization configuration:

[0142] Based on the initial state parameters of the data source drones in Step 1, the operator edits the JSON configuration file for the simulation parameter configuration module. The configured parameters include the number of drones, their initial state, and, according to simulation needs, the positive directions of the four cardinal directions in the scene environment. For example, in this file, the "Number of Drones" parameter is set to 3. Simultaneously, the reference coordinate system of the 3D scene is set: the positive direction of the X-axis in the local coordinate system is defined as due west, and the positive direction of the Y-axis as due north (corresponding to the north direction in the "Northeast Sky" coordinate system of the data source). When the system main program starts, the 3D rendering module of the drone swarm state reads this configuration file when loading the scene. By deserializing and parsing parameters such as the "Number of Drones," this module dynamically generates three visualized drone models in the 3D scene, serving as digital twins of the data source drones. At this moment, these three visualized drones have completed the initialization of their network interfaces and the setting of their initial poses, and are in a standby state, ready to receive data.

[0143] Step 3: 3D Scene Launch and Rendering:

[0144] The main rendering loop of the unmanned swarm state 3D rendering module is run. This module loads a 3D scene including terrain and sky, and places the three UAV models generated in step 2 into the initial position of the scene (i.e., 0, 30, 60 meters on the Y-axis, with the nose facing east). At this point, the user can see a static, ready three-UAV formation in the 3D view, indicating that the visualization swarm initialization is complete and it has the ability to receive data and update its status.

[0145] Step 4: Simulation Execution and Data Transmission

[0146] Unlock and takeoff commands are executed on the data source UAV (in the simulation environment). After the simulation starts, the flight data acquisition module acquires flight data (position, attitude, etc.) of the three UAVs in real time at a set frequency (e.g., 50Hz). Subsequently, the UAV cluster status update server module processes this data (e.g., necessary format conversion) and sends it over the network. Data is transmitted to the ports specified by the interface module according to its type. In this embodiment, a port splitting strategy is adopted: ports 8000-8005 are assigned to UAV number 1, where ports 8000, 8001, and 8002 send its X, Y, and Z position coordinates in the local scene coordinate system, respectively; ports 8003, 8004, and 8005 send its roll, pitch, and yaw angles in the body coordinate system, respectively. UAVs numbered 2 and 3 use ports 8010-8015 and 8020-8025, respectively, with the same port-data type correspondence as UAV number 1.

[0147] Step 5: Data reception and coordinate unification:

[0148] The unmanned cluster status update interface module listens to the aforementioned port. When data arrives, the interface module receives and decodes it. Since the data source in this embodiment uses a "north-southeast" local coordinate system, while the scene reference system is a custom left-handed Cartesian coordinate system, the interface module performs a conversion calculation from the "north-southeast" system to the scene's left-handed Cartesian coordinate system. After the conversion is complete, the obtained standard coordinate data (position XYZ, attitude angle RPY) is stored in the data storage space allocated to each of the three UAVs.

[0149] Step 6: State-Driven and Multi-Perspective Observation:

[0150] The flight data-driven module monitors changes in the data storage space of each UAV in real time. Once new data is stored, it is immediately read. For UAV 1, it reads the latest data transmitted from ports 8000-8005 in its storage space and sets its position (X,Y,Z) and attitude angles (R,P,Y) to the position and rotation of the UAV 1 visualization model in the scene, respectively. The same operation is performed on UAVs 2 and 3. Thus, the three UAV models in the 3D scene begin to synchronously reproduce the flight actions of the data source UAV. Through the simulation parameter configuration module, users can adjust the viewing angle in real time on the graphical interface, such as switching from a global top-down view in one window to a first-person view of a specific UAV in another window, achieving multi-angle, immersive observation of the cluster status.

[0151] Step 7: Continue running:

[0152] The processes from steps 4 to 6 above—namely, the acquisition, processing, transmission, reception, coordinate unification, state-driven, and 3D rendering of real-time flight data—are continuously executed in a loop as the simulation progresses, thereby achieving continuous, real-time, and high-fidelity 3D visualization of the motion state of the three UAV clusters.

[0153] Implementation Method 9: Providing Example 2: 3D visualization example of a large-scale (100 aircraft) unmanned swarm:

[0154] This embodiment is used to illustrate the system's ability to support large-scale variable-scale cluster simulation, taking a cluster of 100 drones as an example.

[0155] Step 1: Configure the visualization cluster based on external data:

[0156] In this embodiment, the data originates from an external cluster system containing 100 simulated drones. First, based on the initial cluster state information provided by the external system, the operator configures the parameters in the JSON configuration file of the simulation parameter configuration module. The main configuration parameter is the "number of drones," which is set to 100. Simultaneously, based on the characteristics of the external data and simulation requirements, the spatial coordinate system reference origin of the 3D scene is set in the configuration file (e.g., the center point of the cluster or a geographical reference point). After system startup, the 3D rendering module of the drone cluster state reads the configuration file and, by parsing the "number of drones" parameter, dynamically generates 100 visualized drone models in the 3D scene. These models, during initialization, complete the binding of network communication interfaces and the setting of basic states, becoming digital twins corresponding to the 100 external data source drones.

[0157] Step 2: Initialization of a large-scale cluster 3D scene:

[0158] Run the 3D rendering module for the unmanned swarm status. This module loads the 3D terrain environment and, based on the configuration in step 1, instantiates and renders 100 drone models in the area corresponding to the initial position of the external swarm in the scene. At this point, a large-scale static drone swarm is presented in the 3D viewport, marking the initialization and readiness of the large-scale visualization swarm. All 100 visualization entities are now capable of receiving and processing independent data streams.

[0159] Step 3: Receiving and processing large-scale external data:

[0160] The system initiates the operation of a swarm of 100 simulated drones and begins receiving their flight data streams. In this embodiment, the external data uses the WGS-84 global coordinate system, describing the position of each drone in the form of longitude, latitude, and altitude. The drone swarm status update interface module simultaneously listens to hundreds of data ports allocated to these 100 drones (one set of ports per drone). When a data packet arrives, the interface module executes a critical data conversion process: First, the received latitude, longitude, and altitude data of each drone are converted to geocentric Cartesian coordinates using a geographic coordinate conversion algorithm. Then, using the scene-defined reference origin as a reference, the geocentric Cartesian coordinates are converted to a local Cartesian coordinate system. Finally, the data is uniformly converted to the system's internal 3D scene reference coordinate system. Attitude data is also uniformly processed according to the coordinate axis directions. After the conversion is complete, the standard position and attitude data of each of the 100 drones are stored in 100 independent data storage spaces pre-allocated for them. This "one drone, one storage" isolation design ensures order and efficiency during large-scale concurrent data processing.

[0161] Step 4: Driving and Presenting Large-Scale Cluster Movement:

[0162] The flight data-driven module monitors and reads these 100 data storage spaces in parallel. For each UAV, the module directly assigns the latest converted position coordinates and attitude angle data from its storage space to the corresponding numbered visualized UAV model in the 3D scene. Because the driving and rendering processes are highly optimized, the system can update the state of these 100 models simultaneously at a high frame rate. Thus, in the 3D view, the user can observe a large-scale swarm of 100 UAVs performing synchronized and coordinated flight exercises according to motion commands from an external data source. During the simulation, data acquisition, processing, transmission, reception, conversion, driving, and presentation form a highly efficient real-time pipeline, continuously enabling 3D visualization of the motion state of a swarm of hundreds of UAVs.

[0163] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time flight data-driven, variable-scale unmanned swarm 3D visualization system, characterized in that, include: The flight data acquisition module is used to connect to external data sources and acquire real-time flight data of the drones from those sources. The unmanned cluster status update server module is communicatively connected to the flight data acquisition module, and is used to receive and process the real-time flight data, and send the processed data through the network. The unmanned cluster status update interface module communicates with the unmanned cluster status update server module via a network, and is used to receive processed data sent by the network and convert it into a unified three-dimensional scene reference coordinate system within the system. The flight data driving module is communicatively connected to the unmanned cluster state update interface module and is used to drive the corresponding visualized UAV model in the three-dimensional scene to perform state updates based on the data after being transformed into the three-dimensional scene reference coordinate system. The simulation parameter configuration module is used to centrally manage the system's configurable parameters through an external configuration file, and the configurable parameters include at least the drone swarm size; The unmanned swarm state 3D presentation module is communicatively connected to the simulation parameter configuration module and the flight data driving module. It is used to generate a 3D scene and a corresponding number of visualized unmanned swarm models according to the size of the unmanned swarm, and to render the state update of the visualized unmanned swarm models in real time according to the driving of the flight data driving module.

2. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The simulation parameter configuration module uses an external configuration file that is a script file with a key-value pair structure. The configurable parameters include initialization parameters and runtime adjustable parameters; The initialization parameters include at least the drone cluster size, the initial pose of each drone, the coordinate system definition, and the data port configuration; The runtime adjustable parameters include at least the three-dimensional viewing angle parameters.

3. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The unmanned cluster state update interface module supports the transformation of input data in multiple coordinate systems, including: The latitude, longitude, and altitude data in the WGS-84 coordinate system are converted into geocentric and geofixed coordinate system data, then into local coordinate system data, and finally into the system's 3D scene reference coordinate system data. And converting the local coordinate system data of Northeast Sky and Northeast Earth to the reference coordinate system of the three-dimensional scene; The transformation includes unifying the coordinate axis directions and unifying the left-handed and right-handed coordinate systems.

4. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The unmanned cluster status update server module assigns an independent set of data sending ports to each drone in the cluster, and the unmanned cluster status update interface module configures a corresponding data receiving port for each drone, thereby achieving data flow isolation through port mapping.

5. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The unmanned cluster state update server module is also used to convert the acquired flight data into a different data representation format, including converting attitude quaternion data into three-axis attitude angle data in the body coordinate system.

6. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The unmanned cluster status 3D presentation module includes: The model instantiation unit is used to dynamically generate a corresponding number of visualized UAV models in a 3D scene according to the cluster number parameter in the simulation parameter configuration module. The multi-view presentation unit has multiple built-in 3D presentation components to provide various fixed viewing angles, including top view, front view, back view, left view, right view and bottom view. The perspective switching unit is used to switch the observation perspective between a first-person perspective and a third-person free perspective.

7. The real-time flight data-driven variable-scale unmanned swarm 3D visualization system according to claim 1, characterized in that, The flight data-driven module maintains an independent data storage space for each visualized UAV model and periodically reads standardized position and attitude data from the corresponding data storage space, directly mapping it to the pose of the corresponding visualized UAV model in the three-dimensional scene, thereby achieving concurrent updates of the cluster state.

8. A real-time flight data-driven 3D visualization method for variable-scale unmanned swarms, characterized in that, The method, applied to a real-time flight data-driven, variable-scale unmanned swarm 3D visualization system as described in any one of claims 1-7, comprises the following steps: The simulation parameter configuration module obtains and parses the external configuration file to determine the configurable parameters; the configurable parameters include at least the drone swarm size and initial parameters. The unmanned swarm state 3D presentation module generates a corresponding number of visualized unmanned swarm models in a 3D scene based on the swarm size and initial parameters. The flight data acquisition module acquires real-time flight data of the drone from an external data source. The real-time flight data is processed by the unmanned cluster status update server module, and the processed data is sent via the network. The unmanned cluster status update interface module receives the processed data sent by the network, performs decoding and coordinate system-1 conversion, and stores the coordinate system-1 converted data separately for each unmanned drone. The flight data-driven module reads the coordinate system-transformed data stored by each UAV and drives the corresponding visualized UAV model in the 3D scene to update its position and attitude. The updated 3D scene is rendered in real time using the unmanned cluster state 3D rendering module.

9. The real-time flight data-driven 3D visualization method for variable-scale unmanned swarms according to claim 8, characterized in that, The coordinate system transformation via the unmanned cluster state update interface module includes: If the input data is latitude, longitude, and altitude data in the WGS-84 coordinate system, then the conversion from the WGS84 coordinate system to the Earth-centered and Earth-fixed coordinate system, from the Earth-centered and Earth-fixed coordinate system to the local coordinate system, and from the local coordinate system to the 3D scene reference coordinate system will be performed in sequence. If the input data is in a local coordinate system, then a transformation from that local coordinate system to the 3D scene reference coordinate system is performed. The transformation process includes adjusting the definition of the positive direction of the coordinate axes and processing the left-handed and right-handed coordinate systems.

10. A computer storage medium, characterized in that, The storage medium stores a computer program, which, when executed, performs the real-time flight data-driven variable-scale unmanned swarm 3D visualization method as described in claim 8 or 9.

Images