Low-altitude operation electronic sand table based on real-time digital twinning

Abstract

The invention relates to the technical field of space flight and aviation, in particular to a low-altitude operation electronic sand table based on real-time digital twinning, which is composed of a main control computer, an interactive control touch screen, a 3D projector, a projection curtain, 3D glasses and an image sensing camera. PBR rendering is achieved through Unity 3D and HDRP pipelines, high-precision and low-polygon model dynamic switching is supported, polarization type 3D projection and laser touch interaction are combined, first-person / global view angle switching is provided, user immersion and operation intuition are enhanced, a user can interact with a sand table, low-altitude environment data are collected in real time through a sensor network, and the user experience is improved. And low-delay transmission is implemented to ensure that the updating frequency per second of the digital twin model is greater than or equal to 10Hz, which is obviously superior to that of a static or low-frequency updating mode of a traditional electronic sand table, so that the requirement of real-time simulation is met, and the digital twin modeling module interacts with a physical entity and supports seamless joint of a third-party system, so that the sand table can implement cross-domain collaborative management.

Description

Technical Field

[0001] The present invention relates to the field of aerospace technology, and in particular to a low-altitude operation electronic sandbox based on real-time digital twins. Background Art

[0002] As a virtual simulation system built on digital technology, the Low-Altitude Operations Electronic Sandbox's core function is to provide high-precision simulation and dynamic display of operational scenarios within low-altitude airspace (covering specific altitude ranges such as urban low-altitude and mountainous areas). By integrating 3D modeling, geographic information systems (GIS), and data visualization technologies, the platform transforms complex geographical environments (such as topography and building distribution), infrastructure networks (such as roads and bridges), and aircraft trajectories into interactive digital models, providing intuitive and dynamic scenario-based support tools for low-altitude airspace planning and decision-making, operational supervision, and personnel training.

[0003] The electronic sandbox in traditional low-altitude management systems relies on static GIS maps or offline models, with a long data update cycle, and is unable to cope with sudden weather changes or the dynamic obstacle avoidance needs of aircraft. In addition, existing digital sandboxes are mostly used for display, lack real-time data synchronization capabilities, and are difficult to support dynamic decision-making. Most existing systems rely on fixed operating terminals and lack functions such as multi-perspective switching and touch interaction. The user experience is single, and the models generated by traditional 3D modeling software (such as 3ds or Max) are rich in details but have a high computational load, making it difficult to meet real-time simulation needs. Summary of the Invention

[0004] The purpose of the present invention is to provide a low-altitude running electronic sandbox based on real-time digital twins, which has the advantages of increasing user immersion and intuitive operation.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solutions: a low-altitude electronic sandbox based on real-time digital twins, which consists of a main control computer, an interactive control touch screen, a 3D projector, a projection screen, 3D glasses and an image sensing camera. The main control computer and the interactive control touch screen are bidirectionally connected in telecommunication, and the main control computer and the 3D projector are unidirectionally connected in telecommunication. The main control computer has a built-in digital twin modeling module. The sandbox also includes a real-time data acquisition and processing module arranged around the low-altitude area, and the real-time data acquisition and processing module is electrically connected to the main control computer.

[0006] Preferably, the digital twin modeling module is used to construct a virtual model of the low-altitude operating environment, and the virtual model includes aircraft, obstacles, and weather conditions. The digital twin modeling module realizes data synchronization between the virtual model and the physical entity through an API interface or middleware.

[0007] Preferably, the real-time data acquisition and processing module is composed of a wind speed sensor, a temperature sensor, a GPS positioning module and a laser radar.

[0008] Preferably, the parameter package acquired by the real-time data acquisition and processing module includes wind speed, temperature, and aircraft position.

[0009] Preferably, the mutual control touch screen has a built-in visual interactive interface, and the main control computer is also provided with a decision support module.

[0010] Preferably, the three-dimensional modeling software used in the digital twin modeling module includes Blender or Maya, and the virtual model in the digital twin modeling module includes terrain, buildings, aircraft, obstacles, and weather conditions.

[0011] Preferably, an API interface is used in the digital twin modeling module.

[0012] Preferably, the visual interactive interface uses WebGL or Unity3D to develop a three-dimensional visual interface, and the three-dimensional rendering engine in the visual interactive interface uses Unity3D's HDRP pipeline, and the PBR material system is based on the GGX microsurface model, and its BRDF equation is: ; Where: F is the Fresnel term, G is the geometric shading function, and D is the normal distribution function. The viewing angle switching is achieved through quaternion interpolation to achieve smooth transition. The projection system uses polarization 3D technology, and the laser touch screen uses the triangulation principle. The laser plane equation is: ax+by+cz+d=0; Where: x, y, and z are three-dimensional coordinates, and a, b, and c are the three side lengths of the triangle formed by the projection light and the projection screen during the projection process.

[0013] Preferably, the wind speed sensor adopts a three-cup structure design.

[0014] Preferably, the main control computer is further provided with a decision support module.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention realizes PBR rendering through Unity3D and HDRP pipeline, supports dynamic switching between high-precision and low-polygon models, combines polarized 3D projection and laser touch interaction, provides first-person / global perspective switching, enhances user immersion and intuitive operation, allows users to interact with the sandbox, collects low-altitude environmental data in real time through the sensor network, and implements low-latency transmission to ensure that the digital twin model is updated at a frequency of ≥10Hz per second, which is significantly better than the static or low-frequency update mode of traditional electronic sandboxes, thereby meeting the needs of real-time simulation. The digital twin modeling module supports seamless docking with third-party systems by interacting with physical entities, so that the sandbox can implement cross-domain collaborative management. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a system principle block diagram of the present invention; Figure 2 It is a simple schematic diagram of the present invention when in use. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] The low-altitude operation electronic sandbox based on real-time digital twins consists of a main control computer, an interactive control touch screen, a 3D projector, a projection screen, 3D glasses, and an image sensor camera. A bidirectional telecommunication connection is established between the main control computer and the interactive control touch screen, while a unidirectional telecommunication connection is established between the main control computer and the 3D projector. The main control computer has a built-in digital twin modeling module that constructs a virtual model of the low-altitude operation environment, including aircraft, obstacles, and weather conditions, and synchronizes the status data of physical entities in real time. Three-dimensional modeling software is used to construct a virtual model of the low-altitude operation environment, including terrain, buildings, and aircraft. Data synchronization between the virtual model and physical entities is achieved through an API interface or middleware to ensure the real-time status of the model. A real-time data acquisition and processing module is arranged around the low-altitude area and is electrically connected to the main control computer. The sensor network simulates the physical parameters of the low-altitude operation environment in real time, including wind speed, temperature, and aircraft position, and transmits them to the digital twin model for updating. The sensor network is deployed to collect the physical parameters of the low-altitude operation environment in real time. The data processing center cleans, integrates, and analyzes the collected data to update the digital twin model.

[0019] The interactive control touch screen has a built-in visual interactive interface, providing a three-dimensional visual interface to display the real-time status of the low-altitude operating environment and support user interactive operations (such as aircraft path planning, obstacle warning, etc.). The three-dimensional visual interface is developed using WebGL, Unity3D and other technologies to display the real-time status of the low-altitude operating environment and support users to perform interactive operations such as aircraft path planning and perspective switching through a mouse, keyboard or touch device. The main control computer also has a decision support module. The decision support module provides operation strategy optimization suggestions based on the analysis results of the digital twin model to assist decision makers in formulating low-altitude operation plans. Based on the analysis results of the digital twin model, a machine learning algorithm is used to predict the future trend of the low-altitude operating environment and provide operation strategy optimization suggestions, such as aircraft obstacle avoidance path planning and adaptive adjustment of weather conditions.

[0020] Working Principle: The main control computer has a built-in digital twin modeling module, which constructs a virtual model of the low-altitude operating environment based on 3D modeling software such as Blender or Maya. The virtual model of the low-altitude operating environment includes terrain, buildings, aircraft, obstacles, weather conditions, etc. The bottom layer of the digital twin modeling module uses Blender or Maya to achieve parameterized generation of geometric models. In view of the dynamic characteristics of the low-altitude environment, an adaptive mesh subdivision algorithm is introduced. Terrain modeling uses layered texture mapping based on the Diamond-Square algorithm, and buildings use CSG (Constructive Solid Geometry) and B-rep (Boundary Representation) hybrid modeling technology. The aircraft dynamics model integrates X-Plane's six-degree-of-freedom motion equations: ; Where: V is the airspeed, γ is the climb angle, and χ is the heading angle. For model optimization, a level-of-detail algorithm based on QEM (Quadric-Error-Metrics) is used to achieve a smooth transition between high-precision and low-polygon models by minimizing the quadratic error function. This ensures that while maintaining an update frequency of ≥10 Hz, the scene complexity is reduced to 30% to 50% of the original model. The digital twin modeling module uses an API interface to implement data interaction between the virtual model and the physical entity through RESTful-API or WebSocket. At the same time, the middleware of the digital twin modeling module uses MQTT or OPC-UA protocol to ensure low-latency and high-reliability data transmission. The digital twin modeling module updates at a frequency of ≥10Hz per second to ensure the consistency of the virtual model and the physical entity state. The data acquisition and processing module is composed of wind speed sensors, temperature sensors, GPS positioning modules, lidars, etc. deployed in low-altitude areas to collect physical parameters of the low-altitude operating environment in real time. Data transmission uses LoRa or Zigbee protocol to achieve low-power and long-distance data transmission. The data acquisition network is composed of distributed sensor nodes, among which the wind speed sensor The sensor adopts a three-cup structure design. The output signal is processed by a Butterworth filter and modulated by the chip. Spread spectrum technology is used to achieve long-distance transmission distance. The temperature sensor integrates DS18B20 digital output. The GPS module uses RTK differential positioning technology with a planar accuracy of ±1cm. The lidar uses Velodyne-VLP-16. Point cloud data is used for ground segmentation and obstacle clustering through the PCL library. WebGL or Unity3D is used to develop a 3D visualization interface, supporting dynamic switching between low-polygon and high-precision models. PBR (physically based rendering) technology is used to enhance the realism of lighting and materials. The 3D rendering engine uses Unity3D's HDRP pipeline. The PBR material system is based on the GGX microsurface model, and its BRDF equation is: ; Where: F is the Fresnel term, G is the geometric shading function, and D is the normal distribution function. View angle switching is achieved through quaternion interpolation to achieve smooth transition. The projection system uses polarization 3D technology with a brightness of 8000 lumens. The laser touch screen uses the triangulation principle. The laser plane equation is: ax+by+cz+d=0; Where: x, y, and z are three-dimensional coordinates, a, b, and c are the lengths of the three sides of the triangle formed by the projection light and the projection screen during the projection process; Through 3D projection, it provides first-person perspective, third-person perspective and global perspective, and predicts weather changes in the next 5 to 10 minutes based on the LSTM network and displays them on the projection screen. Viewers can operate on the sand table with their fingers. It adopts a working principle similar to that of laser touch screen devices, mainly based on the coordinated use of laser transmitters and signal receivers (cameras). The laser transmitter is installed above the screen (projection screen) and emits lasers to form an invisible light film on the projected 3D sand table that is 1 to 2 mm higher than the surface and about 1 mm thick. When a finger or any opaque object touches the screen surface, the light is reflected to the image sensing camera set above the screen. The main control computer then accurately calculates the photoelectric position to obtain the coordinates of the mouse, thereby realizing the touch function. At the same time, it is projected onto the screen on the wall through the projector. To achieve image display, after the laser touch screen device is installed, the signal receiver and projector are aligned with the center of the screen to achieve interaction between people and the sandbox. Therefore, the invention realizes PBR rendering through Unity3D and HDRP pipeline, supports dynamic switching between high-precision and low-polygon models, and combines polarized 3D projection and laser touch interaction to provide first-person / global perspective switching, enhance user immersion and operation intuitiveness, and allow users to interact with the sandbox. Low-altitude environmental data is collected in real time through the sensor network, and low-latency transmission is implemented to ensure that the digital twin model is updated at a frequency of ≥10Hz per second, which is significantly better than the static or low-frequency update mode of traditional electronic sandboxes, thereby meeting the needs of real-time simulation. The digital twin modeling module supports seamless docking of third-party systems by interacting with physical entities so that the sandbox can implement cross-domain collaborative management.

[0021] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. Low-altitude operation electronic sandbox based on real-time digital twin, characterized by: The sandbox consists of a main control computer, an interactive control touch screen, a 3D projector, a projection screen, 3D glasses and an image sensing camera. The main control computer and the interactive control touch screen have a two-way telecommunication connection, and the main control computer and the 3D projector have a one-way telecommunication connection. The main control computer has a built-in digital twin modeling module. The sandbox also includes a real-time data acquisition and processing module arranged around the low-altitude area, and the real-time data acquisition and processing module is electrically connected to the main control computer.

2. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized by: The digital twin modeling module is used to build a virtual model of the low-altitude operating environment, and the virtual model includes aircraft, obstacles, and weather conditions. The digital twin modeling module realizes data synchronization between the virtual model and the physical entity through an API interface or middleware.

3. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized in that: The real-time data acquisition and processing module is composed of a wind speed sensor, a temperature sensor, a GPS positioning module and a laser radar.

4. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized in that: The parameter packages acquired by the real-time data acquisition and processing module include wind speed, temperature, and aircraft position.

5. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized in that: The mutual control touch screen has a built-in visual interactive interface, and the main control computer is also provided with a decision support module.

6. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized by: The three-dimensional modeling software used in the digital twin modeling module includes Blender or Maya, and the virtual model in the digital twin modeling module includes terrain, buildings, aircraft, obstacles, and weather conditions.

7. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized by: The digital twin modeling module adopts an API interface.

8. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 5 is characterized by: The visual interactive interface uses WebGL or Unity3D to develop a three-dimensional visual interface, and the three-dimensional rendering engine in the visual interactive interface uses Unity3D's HDRP pipeline. The PBR material system is based on the GGX microsurface model, and its BRDF equation is: ； Where: F is the Fresnel term, G is the geometric shading function, and D is the normal distribution function. The viewing angle switching is achieved through quaternion interpolation to achieve smooth transition. The projection system uses polarization 3D technology, and the laser touch screen uses the triangulation principle. The laser plane equation is: ax+by+cz+d=0; Where: x, y, and z are three-dimensional coordinates, and a, b, and c are the three side lengths of the triangle formed by the projection light and the projection screen during the projection process.

9. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 3 is characterized by: The wind speed sensor adopts a three-cup structure design.

10. The low-altitude operation electronic sandbox based on real-time digital twin according to claim 1 is characterized in that: The main control computer is also provided with a decision support module.

Images