Data-driven scene dynamic interaction method based on virtual comprehensive display system
By using a data-driven dynamic interaction method in a virtual integrated display system, the problems of data latency and poor synchronization in virtual reality technology are solved, enabling real-time synchronization and efficient interaction between virtual scenes and real experimental objects, thus improving the real-time performance and stability of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-03-20
AI Technical Summary
Existing virtual reality technology suffers from problems such as data latency, poor synchronization, and inconsistent model response in the fusion and interaction of multi-source information in aircraft systems, making it difficult to achieve real-time interaction and immersive operation.
A data-driven dynamic scene interaction method based on a virtual integrated display system is adopted. Through the virtual integrated display system scene platform, data receiving and parsing module and data-driven model module, the scene is constructed using the 3D modeling software Unity3D. Combined with lighting and camera resources, the data interface and model mapping are realized. A dual-thread mechanism and cache queue management are used for data parsing and model driving to ensure real-time data synchronization.
The system enables dynamic updates and real-time interaction of models in the virtual integrated display system, ensuring the consistency of time and state between the virtual scene and the real experimental object, reducing latency and jitter, and improving the real-time performance and stability of the system.
Smart Images
Figure CN121706512A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of virtual reality and simulation technology, specifically relating to a data-driven dynamic interaction method for scenes based on a virtual integrated display system. Background Technology
[0002] With the development of virtual reality and 3D visualization technologies, aircraft systems are increasingly reliant on virtual simulation environments during research and development and verification. Traditional virtual display methods typically only present experimental results in the form of static models or videos, lacking real-time interactivity and immersion, and are unable to meet the demands of modern experiments for data visualization, scene interaction, and multi-source information integration.
[0003] When it comes to the fusion and interaction of multi-source information in aircraft systems, existing methods still have obvious shortcomings: (1) In terms of real-time data-driven, due to the lack of efficient data parsing and scene synchronization mechanisms, data delay, poor synchronization and inconsistent model response are easy to occur; (2) In terms of comprehensive interaction of complex tasks, existing technologies are mostly limited to the demonstration level and cannot achieve immersive interactive operation at the task level. Summary of the Invention
[0004] The purpose of this invention is to provide a data-driven dynamic interaction method for virtual integrated display systems, which solves the problems of data latency and poor synchronization and inconsistent model response in existing virtual integrated display systems, and realizes interactive operation of virtual integrated display systems.
[0005] The technical solution adopted in this invention is a data-driven scene dynamic interaction method based on a virtual integrated display system, wherein the virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data driving model module; The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0006] The invention is further characterized in that: Step 1 is as follows: Step 1.1: The target model for the virtual integrated display system scene platform design includes aircraft, ground vehicles, and environmental element models; therefore, the 3D modeling software Unity3D is used to model the aircraft, ground vehicles, and environmental elements, generating model files containing geometry, materials, and textures; Step 1.2: Based on Step 1.1, import the aircraft, ground vehicle, and environmental element models into Unity3D, and combine them with lighting, camera, and particle system resources to build a realistic test environment; add lighting, camera, skybox, and particle system resources, and configure first-person / third-person switching, free roaming, and multi-view observation functions in Unity3D; Step 1.3: After the design in Step 1.2 is completed, data interfaces, including serial communication interfaces and network communication interfaces, are reserved in the virtual integrated display system to adapt to the application scenarios of real-time experiments; after completing this step, the construction of the virtual integrated display system scenario platform is complete.
[0007] In step 1.3, the network communication interface is TCP or UDP.
[0008] Step 2 is as follows: Step 2.1: First, establish a connection between the virtual integrated display system scene platform and the Mission Planer platform via serial port; during initialization, the virtual integrated display system data receiving and parsing module needs to configure the serial port parameters: baud rate, data bits, stop bits, and parity method; Step 2.2: After the virtual integrated display system scene platform establishes a connection with the external Mission Planner platform in Step 2.1, the virtual integrated display system scene platform starts receiving data based on a dual-thread mechanism; the read thread is responsible for reading the raw byte stream from the serial port buffer and writing it into the cache queue, while the main thread is responsible for reading data frame by frame and driving the scene. The read thread and the main thread work together to process the data reception and parsing process. Step 2.3: First, the received raw data is preprocessed, including: removing garbled characters, checking the validity of the JSON format, and removing redundant symbols and illegal characters; then, an efficient JSON parsing library is used to convert the data into C# objects or predefined structures; then, the virtual integrated display system performs integrity checks and validity verification on key fields. In step 2.2, the collaborative processing flow between the read thread and the main thread in data reception and parsing is as follows: a growable buffer is designed in the read thread, and data integrity checks are added; the parsing stage will only begin when a frame of data meets the detection requirements; if an abnormal frame is detected, it will be logged and discarded to avoid polluting the rendering thread; the cache queue adopts a thread-safe ConcurrentQueue structure to ensure the atomicity of enqueue and dequeue operations; in a high-frequency data stream environment, a "backpressure mechanism" is set: when the queue length exceeds the threshold M, some old data is discarded, and only the latest K data are retained, thereby ensuring that the final performance of the virtual integrated display system always follows the latest state.
[0009] In step 2.3: the key fields include latitude and longitude, speed, heading angle, pitch angle, and roll angle; the virtual integrated display system performs integrity checks and legality verification on the key fields as follows: if the latitude and longitude exceed the range of [-180°, 180°] or [-90°, 90°], it is considered an abnormal value; if the speed is negative or exceeds the upper limit of the equipment specifications, anomaly handling is triggered; if the heading angle and roll angle exceed the range of [-180°, 180°], they are considered abnormal values; if the pitch angle exceeds the range of [-90°, 90°], it is considered an abnormal value; if parsing fails or a field is abnormal, a log is recorded and the data is discarded.
[0010] In step 2.1: the verification method is odd parity, even parity, or no parity.
[0011] Step 3 specifically involves: Step 3.1: After completing the data parsing in Step S2, the data classification and matching stage begins. The virtual integrated display system scene platform distinguishes data from different sources based on the message ID: Lead aircraft data: including latitude and longitude, altitude, heading, speed, and attitude angle; Wingman data: the same as lead aircraft data, but identified as wingman; Ground vehicle data: position, speed, and direction; Test equipment data: theodolite, GPS receiver; Environmental data: temperature, wind speed, humidity, air pressure, and altitude. The classified data is mapped to the corresponding model in the virtual integrated display system scene platform. The lead aircraft and wingman data are ultimately mapped to "Aircraft Class," ground vehicle data to "Vehicle Class," test equipment data to "Test Class," and environmental data to "Environment Class." Step 3.2: Update the state of the corresponding model in the virtual scene based on the classified data; for the lead aircraft and wingman, update the position of the lead aircraft model in three-dimensional space using latitude, longitude and altitude data, and update the attitude using pitch angle, roll angle and yaw angle to ensure that the lead aircraft's performance in the virtual scene is synchronized with the real flight state; for ground vehicles, convert geographical coordinates into scene coordinates through coordinate transformation, and use speed data to drive the vehicle along a set path or physical engine trajectory; for environmental information data, the virtual integrated display system dynamically adjusts the environmental information display parameters in the virtual scene based on temperature, wind speed and humidity data.
[0012] The beneficial effects of this invention are: This invention achieves dynamic updates and real-time interaction of models in a virtual integrated display system through a unified process of data reception, parsing, classification, and model control, ensuring consistency between the virtual integrated display system and the real experimental object in terms of time and state. Compared with existing technologies, this invention effectively solves the problems of data latency in virtual integrated display systems, as well as the resulting poor synchronization and inconsistent model responses, providing an efficient and scalable solution for aircraft simulation verification and virtual experimental interaction. Attached Figure Description
[0013] Figure 1 This is a rendering of the virtual integrated display system scene platform in step 1 of a specific embodiment of the present invention; Figure 2 This is a schematic diagram of the collaborative processing flow between the read thread and the main thread in step 2 of a specific embodiment of the present invention; Figure 3 This is a flowchart illustrating the data classification and matching relationship in step 3 of a specific embodiment of the present invention. Detailed Implementation
[0014] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0015] This invention provides a data-driven dynamic scene interaction method based on a virtual integrated display system. The complete implementation of the virtual integrated display system can be divided into three parts: a virtual integrated display system scene platform construction, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. These three parts constitute the virtual integrated display system of this invention and ensure that the virtual integrated display system has the ability to drive dynamic scene interaction based on data. The following three steps will be described in detail: Step 1: The goal of this step is to complete the design and construction of the virtual integrated display system scene platform. A virtual 3D scene is built based on Unity3D, and the 3D model of the experimental object is imported to complete the rendering and configuration of the basic environment.
[0016] Step 1.1: The target model for the virtual integrated display system scene platform design includes aircraft (lead aircraft, wingman aircraft), ground vehicles (experimental vehicles), and environmental elements (terrain, weather, wind field) models; accordingly, the 3D modeling software Unity3D is used to model the aircraft, ground vehicles, and environmental elements, generating model files containing geometry, materials, and textures; Step 1.2: Based on Step 1.1, import the aircraft, ground vehicle, and environmental element models into Unity3D, and combine them with lighting, camera, and particle system resources to build a realistic test environment; add lighting, camera, skybox, and particle system resources, and configure first-person / third-person switching, free roaming, and multi-view observation functions in Unity3D; Step 1.3: After completing the design in Step 1.2, reserve data interfaces in the virtual integrated display system, including serial communication interfaces and network communication interfaces (TCP / UDP), to adapt to the application scenarios of real-time experiments; after completing this step, the construction of the virtual integrated display system scenario platform is complete. Figure 1 The diagram shown is an example of a virtual integrated display system scene platform according to the present invention.
[0017] Step 2: Based on the virtual integrated display system scene platform built in Step 1, complete the design and implementation of the virtual integrated display system data receiving and parsing module; the data receiving and parsing module establishes a connection between the virtual integrated display system scene platform and the external Mission Planner platform through a communication interface, receives data transmitted from the Mission Planner platform in real time, the data is encapsulated based on the communication protocol and input into the virtual integrated display system scene platform in the form of a byte stream, performs format parsing and integrity checks on the received raw data, converts it into a unified data structure, and performs validity verification on key fields.
[0018] Step 2.1: In the design of the data receiving and parsing module of the virtual integrated display system, the virtual integrated display system scene platform and the Mission Planner platform are first connected via serial port. During initialization, the virtual integrated display system data receiving and parsing module needs to be configured with serial port parameters: baud rate (e.g., 57600bps, 115200bps), data bits (8 bits), stop bits (1 or 2 bits), and parity method (odd parity, even parity, or no parity). These parameters can be adjusted according to the data transmission and reception situation after the virtual integrated display system scene platform is connected to the external Mission Planner platform.
[0019] Step 2.2: After the virtual integrated display system scene platform establishes a connection with the external Mission Planner platform in Step 2.1, the virtual integrated display system scene platform starts receiving data based on a dual-thread mechanism; the read thread is responsible for reading the raw byte stream from the serial port buffer and writing it into the cache queue, while the main thread (Unity Update) is responsible for reading data frame by frame and driving the scene. The read thread and the main thread work together to process the data reception and parsing process. The collaborative processing flow between the read thread and the main thread in data reception and parsing is as follows: Figure 2 As shown, specifically: to prevent data incompleteness caused by data packet segmentation, this invention designs a growable buffer in the read thread and adds data integrity detection; only when a frame of data meets the detection requirements will it enter the parsing stage; if an abnormal frame is detected, it is logged and discarded to avoid polluting the rendering thread; the cache queue adopts a thread-safe ConcurrentQueue structure to ensure the atomicity of enqueue and dequeue operations; in a high-frequency data stream environment, a "backpressure mechanism" is set: when the queue length exceeds the threshold M, some old data is discarded, and only the latest K data are retained, thereby ensuring that the final virtual integrated display system always follows the latest state.
[0020] Step 2.3: Based on Step 2.2, during the parsing phase, the received raw data is first preprocessed, including: removing garbled characters, checking the validity of the JSON format, and removing redundant symbols and illegal characters. Then, an efficient JSON parsing library is used to convert the data into C# objects or predefined structures. Next, the virtual integrated display system performs integrity checks and validity verification on key fields, including latitude and longitude, speed, heading angle, pitch angle, and roll angle. Specifically, the virtual integrated display system performs integrity checks and validity verification on key fields as follows: if the latitude and longitude exceed the range of [-180°, 180°] or [-90°, 90°], it is considered an anomaly; if the speed is negative or exceeds the equipment specification limit, exception handling is triggered; if the heading angle and roll angle exceed the range of [-180°, 180°], they are considered anomalies; if the pitch angle exceeds the range of [-90°, 90°], it is considered an anomaly; if parsing fails or a field is abnormal, a log is recorded and the data is discarded.
[0021] Step 3: Design of the data-driven model module for the virtual integrated display system, such as... Figure 3As shown, the virtual integrated display system model (the model consists of the aircraft, ground vehicle, and environmental element models imported into the virtual integrated display system in step S1) is matched and driven with the parsed data. Based on the identifiers contained in the parsed data, device data from different sources is distinguished and mapped to the corresponding models in the virtual integrated display system to achieve unified management of multi-source data. Based on the classified device data, the position, attitude, velocity, and environmental parameters of the corresponding models in the final virtual integrated display system scene platform are updated in real time, ensuring that the performance of the virtual model remains synchronized with the real experimental object.
[0022] Step 3.1: After completing the data parsing in Step S2, the data classification and matching stage begins. The virtual integrated display system scene platform distinguishes data from different sources based on the message ID (MsgId): Lead aircraft data: including latitude and longitude, altitude, heading, speed, and attitude angle; Wingman data: the same as lead aircraft data, but identified as wingman; Ground vehicle data: position, speed, and direction; Test equipment data: theodolite, GPS receiver; Environmental data: temperature, wind speed, humidity, air pressure, and altitude. The classified data is mapped to the corresponding model in the virtual integrated display system scene platform. Lead aircraft and wingman data are ultimately mapped to "Aircraft Class", ground vehicle data to "Vehicle Class", test equipment data to "Test Class", and environmental data to "Environment Class". This method ensures the consistency and scalability of data from different devices in the subsequent driving process.
[0023] Step 3.2: Update the state of the corresponding model in the virtual scene based on the classified data; for the lead aircraft and wingman, update the position of the lead aircraft model in three-dimensional space using latitude, longitude and altitude data, and update the attitude using pitch angle, roll angle and yaw angle to ensure that the lead aircraft's performance in the virtual scene is synchronized with the real flight state; for ground vehicles, convert geographical coordinates into scene coordinates through coordinate transformation, and use speed data to drive the vehicle along a set path or physical engine trajectory; for environmental information data, the virtual integrated display system dynamically adjusts the environmental information display parameters in the virtual scene based on temperature, wind speed and humidity data; during the control process, not only are the static parameters of the model updated, but the physical engine is also called to simulate the combined effects of speed, acceleration, drag and environmental factors; during the climb, the aircraft will dynamically correct its trajectory by combining acceleration and air resistance to make the performance more consistent with real physical laws.
[0024] Through the above three steps, the design of the virtual integrated display system and the implementation of the data-driven scene dynamic interaction method of the virtual integrated display system were completed. The problems of data delay, poor synchronization and inconsistent model response in the existing virtual integrated display system were solved, and the interactive operation of the virtual integrated display system was realized.
[0025] Example 1 Low latency verification of virtual integrated display system To verify the real-time response capability of this invention in the data-driven virtual model process, this embodiment tested the end-to-end latency of the virtual integrated display system. The experiment used an external platform to continuously transmit data containing latitude, longitude, altitude, attitude angle, and velocity at a frequency of 50Hz. This invention operates according to the architecture described in the specification, where data reception and parsing employ a dual-thread mode. After parsing, the data is written to a thread-safe queue, and older data is automatically discarded when the queue reaches a preset threshold to ensure priority for the latest state. The model update process is driven by the rendering thread according to timestamps. In contrast, a traditional single-threaded serial parsing method was selected, where data drives the model in the order of arrival, without queue management or a discarding strategy.
[0026] During the experiment, the same data stream was simultaneously input to both the virtual integrated display system of this invention and the control system. The transmission time and rendering time of each data frame were recorded to calculate the end-to-end latency. In a 10-minute test, the results showed that the average latency of the traditional solution was approximately 310ms, with a maximum latency of up to 850ms when data was piling up, exhibiting a large latency variation and significant jitter. In contrast, the average latency of this invention was approximately 118ms, with a maximum latency not exceeding 210ms, showing stable latency distribution and significantly reduced jitter. Furthermore, the P99 latency of this invention remained around 175ms, a substantial reduction compared to the traditional solution. The experimental results are shown in the table below.
[0027] Table 1 Delay verification results
[0028] The above tests demonstrate that this invention effectively reduces end-to-end latency and jitter, significantly improving real-time performance and stability when processing continuous data streams and driving virtual models. The experimental results fully prove that this invention solves the problems of data latency and the resulting poor synchronization and inconsistent model responses in virtual integrated display systems.
[0029] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.
[0030] Feasibility analysis: The method of this invention enables real-time interaction between external device data and the virtual integrated display system based on the Mavlink2 protocol. The Mavlink2 protocol is a lightweight and efficient data transmission protocol capable of reliable transmission of multi-source data in low-bandwidth environments, featuring a stable CRC check mechanism and an extensible message format. Therefore, the accuracy of data reception and parsing is guaranteed at the communication level.
[0031] At the implementation level of the virtual integrated display system, this invention separates external platform data reception and rendering through a dual-thread mechanism and cache queue management, avoiding rendering thread blocking issues. Furthermore, through timestamp delay monitoring and backpressure mechanisms, it ensures low latency and stability even under high-frequency input. JSON parsing and task-based verification methods ensure the integrity of the data format and the rationality of the values. Finally, through classification and unified data structure mapping, it masks differences between different devices, enabling the virtual scene to be compatible with multi-source data.
[0032] At the model-driven level, this invention uses Unity3D's physics engine to simulate and calculate position, velocity, and posture, and combines this with environmental parameters to drive changes in the virtual scene, thereby ensuring the realism and naturalness of the virtual model's motion trajectory and environmental representation. The above mechanism theoretically guarantees a high degree of consistency between the data-driven virtual scene and the external experimental object in time and state.
[0033] Example 2 A data-driven scene dynamic interaction method based on a virtual integrated display system. The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0034] Example 3 A data-driven scene dynamic interaction method based on a virtual integrated display system. The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 1 is as follows: Step 1.1: The target model for the virtual integrated display system scene platform design includes aircraft, ground vehicles, and environmental element models; therefore, the 3D modeling software Unity3D is used to model the aircraft, ground vehicles, and environmental elements, generating model files containing geometry, materials, and textures; Step 1.2: Based on Step 1.1, import the aircraft, ground vehicle, and environmental element models into Unity3D, and combine them with lighting, camera, and particle system resources to build a realistic test environment; add lighting, camera, skybox, and particle system resources, and configure first-person / third-person switching, free roaming, and multi-view observation functions in Unity3D; Step 1.3: After the design in Step 1.2 is completed, data interfaces, including serial communication interfaces and network communication interfaces, are reserved in the virtual integrated display system to adapt to the application scenarios of real-time experiments; after completing this step, the construction of the virtual integrated display system scenario platform is complete.
[0035] Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0036] Example 4 A data-driven scene dynamic interaction method based on a virtual integrated display system. The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 1 is as follows: Step 1.1: The target model for the virtual integrated display system scene platform design includes aircraft, ground vehicles, and environmental element models; therefore, the 3D modeling software Unity3D is used to model the aircraft, ground vehicles, and environmental elements, generating model files containing geometry, materials, and textures; Step 1.2: Based on Step 1.1, import the aircraft, ground vehicle, and environmental element models into Unity3D, and combine them with lighting, camera, and particle system resources to build a realistic test environment; add lighting, camera, skybox, and particle system resources, and configure first-person / third-person switching, free roaming, and multi-view observation functions in Unity3D; Step 1.3: After the design in Step 1.2 is completed, data interfaces, including serial communication interfaces and network communication interfaces, are reserved in the virtual integrated display system to adapt to the application scenarios of real-time experiments; after completing this step, the construction of the virtual integrated display system scenario platform is complete.
[0037] In step 1.3, the network communication interface is TCP or UDP.
[0038] Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0039] Example 5 A data-driven scene dynamic interaction method based on a virtual integrated display system. The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 2 is as follows: Step 2.1: First, establish a connection between the virtual integrated display system scene platform and the Mission Planer platform via serial port; during initialization, the virtual integrated display system data receiving and parsing module needs to configure the serial port parameters: baud rate, data bits, stop bits, and parity method; Step 2.2: After the virtual integrated display system scene platform establishes a connection with the external Mission Planner platform in Step 2.1, the virtual integrated display system scene platform starts receiving data based on a dual-thread mechanism; the read thread is responsible for reading the raw byte stream from the serial port buffer and writing it into the cache queue, while the main thread is responsible for reading data frame by frame and driving the scene. The read thread and the main thread work together to process the data reception and parsing process. Step 2.3: First, the received raw data is preprocessed, including: removing garbled characters, checking the validity of the JSON format, and removing redundant symbols and illegal characters; then, an efficient JSON parsing library is used to convert the data into C# objects or predefined structures; then, the virtual integrated display system performs integrity checks and validity verification on key fields. Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0040] Example 6 A data-driven scene dynamic interaction method based on a virtual integrated display system. The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data-driven model module. The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
[0041] Step 3 specifically involves: Step 3.1: After completing the data parsing in Step S2, the data classification and matching stage begins. The virtual integrated display system scene platform distinguishes data from different sources based on the message ID: Lead aircraft data: including latitude and longitude, altitude, heading, speed, and attitude angle; Wingman data: the same as lead aircraft data, but identified as wingman; Ground vehicle data: position, speed, and direction; Test equipment data: theodolite, GPS receiver; Environmental data: temperature, wind speed, humidity, air pressure, and altitude. The classified data is mapped to the corresponding model in the virtual integrated display system scene platform. The lead aircraft and wingman data are ultimately mapped to "Aircraft Class," ground vehicle data to "Vehicle Class," test equipment data to "Test Class," and environmental data to "Environment Class." Step 3.2: Update the state of the corresponding model in the virtual scene based on the classified data; for the lead aircraft and wingman, update the position of the lead aircraft model in three-dimensional space using latitude, longitude and altitude data, and update the attitude using pitch angle, roll angle and yaw angle to ensure that the lead aircraft's performance in the virtual scene is synchronized with the real flight state; for ground vehicles, convert geographical coordinates into scene coordinates through coordinate transformation, and use speed data to drive the vehicle along a set path or physical engine trajectory; for environmental information data, the virtual integrated display system dynamically adjusts the environmental information display parameters in the virtual scene based on temperature, wind speed and humidity data.
Claims
1. A data-driven dynamic scene interaction method based on a virtual integrated display system, characterized in that, The virtual integrated display system includes: a virtual integrated display system scene platform, a virtual integrated display system data receiving and parsing module, and a virtual integrated display system data driving model module; The method includes the following steps: Step 1: Building the virtual integrated display system scene platform; Step 2: Complete the design and implementation of the data receiving and parsing module of the virtual integrated display system; Step 3: Design the data-driven model module of the virtual integrated display system, match and drive the virtual integrated display system model with the parsed data, distinguish device data from different sources based on the identifiers contained in the parsed data, and map them to the corresponding model in the virtual integrated display system.
2. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 1, characterized in that, Step 1 is as follows: Step 1.1: The target model for the virtual integrated display system scene platform design includes aircraft, ground vehicles, and environmental element models; therefore, the 3D modeling software Unity3D is used to model the aircraft, ground vehicles, and environmental elements, generating model files containing geometry, materials, and textures; Step 1.2: Based on Step 1.1, import the aircraft, ground vehicle, and environmental element models into Unity3D, and combine them with lighting, camera, and particle system resources to build a realistic test environment; add lighting, camera, skybox, and particle system resources, and configure first-person / third-person switching, free roaming, and multi-view observation functions in Unity3D; Step 1.3: After the design in Step 1.2 is completed, data interfaces, including serial communication interfaces and network communication interfaces, are reserved in the virtual integrated display system to adapt to the application scenarios of real-time experiments; after completing this step, the construction of the virtual integrated display system scenario platform is complete.
3. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 2, characterized in that, In step 1.3, the network communication interface is TCP or UDP.
4. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 2, characterized in that, Step 2 is as follows: Step 2.1: First, establish a connection between the virtual integrated display system scene platform and the Mission Planer platform via serial port; during initialization, the virtual integrated display system data receiving and parsing module needs to configure the serial port parameters: baud rate, data bits, stop bits, and parity method; Step 2.2: After the virtual integrated display system scene platform establishes a connection with the external Mission Planner platform in Step 2.1, the virtual integrated display system scene platform starts receiving data based on a dual-thread mechanism; the read thread is responsible for reading the raw byte stream from the serial port buffer and writing it into the cache queue, while the main thread is responsible for reading data frame by frame and driving the scene. The read thread and the main thread work together to process the data reception and parsing process. Step 2.3: First, the received raw data is preprocessed, including: removing garbled characters, checking the validity of the JSON format, and removing redundant symbols and illegal characters; then, an efficient JSON parsing library is used to convert the data into C# objects or predefined structures; then, the virtual integrated display system performs integrity checks and validity verification on key fields.
5. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 4, characterized in that, In step 2.2, the collaborative processing flow between the read thread and the main thread in data reception and parsing is as follows: a growable buffer is designed in the read thread, and data integrity checks are added; the parsing stage will only begin when a frame of data meets the detection requirements; if an abnormal frame is detected, it will be logged and discarded to avoid polluting the rendering thread; the cache queue adopts a thread-safe ConcurrentQueue structure to ensure the atomicity of enqueue and dequeue operations; in a high-frequency data stream environment, a "backpressure mechanism" is set: when the queue length exceeds the threshold M, some old data is discarded, and only the latest K data are retained, thereby ensuring that the final performance of the virtual integrated display system always follows the latest state.
6. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 4, characterized in that, In step 2.3: the key fields include latitude and longitude, speed, heading angle, pitch angle, and roll angle; the virtual integrated display system performs integrity checks and legality verification on the key fields as follows: if the latitude and longitude exceed the range of [-180°, 180°] or [-90°, 90°], it is considered an abnormal value; if the speed is negative or exceeds the upper limit of the equipment specifications, anomaly handling is triggered; if the heading angle and roll angle exceed the range of [-180°, 180°], they are considered abnormal values; if the pitch angle exceeds the range of [-90°, 90°], it is considered an abnormal value; if parsing fails or a field is abnormal, a log is recorded and the data is discarded.
7. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 4, characterized in that, In step 2.1: the verification method is odd parity, even parity, or no parity.
8. The data-driven scene dynamic interaction method based on a virtual integrated display system according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1: After completing the data parsing in Step S2, the data classification and matching stage begins. The virtual integrated display system scene platform distinguishes data from different sources based on the message ID: Lead aircraft data: including latitude and longitude, altitude, heading, speed, and attitude angle; Wingman data: the same as lead aircraft data, but identified as wingman; Ground vehicle data: position, speed, and direction; Test equipment data: theodolite, GPS receiver; Environmental data: temperature, wind speed, humidity, air pressure, and altitude. The classified data is mapped to the corresponding model in the virtual integrated display system scene platform. The lead aircraft and wingman data are ultimately mapped to "Aircraft Class", ground vehicle data to "Vehicle Class", test equipment data to "Test Class", and environmental data to "Environment Class". Step 3.2: Update the state of the corresponding model in the virtual scene based on the classified data; for the lead aircraft and wingman, update the position of the lead aircraft model in three-dimensional space using latitude, longitude and altitude data, and update the attitude using pitch angle, roll angle and yaw angle to ensure that the lead aircraft's performance in the virtual scene is synchronized with the real flight state; for ground vehicles, convert geographical coordinates into scene coordinates through coordinate transformation, and use speed data to drive the vehicle along a set path or physical engine trajectory; for environmental information data, the virtual integrated display system dynamically adjusts the environmental information display parameters in the virtual scene based on temperature, wind speed and humidity data.