Streamline visualization method based on GPU and immersion interaction

Abstract

The invention discloses a streamline visualization method based on GPU and immersion interaction. The method comprises the following steps: acquiring wind field data; sequentially performing surface extraction, format conversion and model processing and color remapping on the wind field data to obtain a rendered real scene; carrying out interactive design on the wind field data, wherein the interactive design comprises the steps of determining the motion and the starting point of a streamline and creating an anchor point for positioning; generating streamline data according to speed information in the wind field data and the interactive design; for the generated streamline data, displaying a dense streamline by adopting an animation streamline based on a GPU (Graphics Processing Unit) to obtain a dynamic rendering result; and combining the rendering real scene and the dynamic rendering result to realize streamline visualization. According to the invention, VR is used as basic equipment, so that an observer can feel the movement in the flow field personally on the scene, and dynamic display is designed according to hardware, so that the observer can clearly grasp the dynamic characteristics of the flow field.

Description

Technical Field

[0001] The present invention relates to the technical field of computational fluid dynamics, and more specifically, to a streamline visualization method based on GPU and immersive interaction. Background Art

[0002] Fluid dynamics simulation plays a cornerstone role in various engineering fields, from aerospace and automotive engineering to environmental and biomedical sciences. Computational fluid dynamics (CFD) simulation involves numerically solving the equations of motion for fluids to predict and analyze the behavior of fluids under different conditions, thereby facilitating the design, optimization, and analysis of various systems and processes. However, the success of CFD simulations depends on the ability to interpret and extract meaningful insights from the large amount of simulation data generated, which is particularly important for mesoscale urban simulations with high accuracy. Visualizing flow fields is of great research significance as a link between raw numerical data and human understanding. Streamlines, as an important method for field visualization, represent the trajectories of fluid particles in a flow field. It provides an intuitive way to visualize fluid motion and identify key flow features such as separation zones, vortices, and recirculation zones. In recent years, increasing interest has focused on exploring interactive and immersive visualization techniques to enhance the understanding and analysis of fluid dynamic phenomena. VR (virtual reality) technology, including head-mounted displays (HMDs) and motion tracking systems, provides a compelling platform to create immersive and interactive visualizations by providing users with a sense of presence in a simulated environment. By leveraging the computational power of modern GPUs and the capabilities of VR hardware, researchers are beginning to explore new ways to visualize and interact with complex flow data in real time.

[0003] The field of scientific visualization has always faced major challenges, and virtual reality (VR) is a proven and widely studied solution. The immersive perspective provided by VR is a powerful tool that can display 3D models and allow users to interact with scientific data in unprecedented ways. For example, VR has made achievements in visualizing various organs for disease diagnosis, highlighting complex structures and optimizing designs, observing molecular dynamics, and non-physical descriptions such as magnetic fields and velocity fields. The Visualization Toolkit (VTK) is a versatile tool for 3D computer graphics, image processing, and visualization. VTK provides a wide range of tools and libraries for creating immersive and information-rich visualizations. Scientific computing often uses the formats developed by VTK, making VTK-based scientific visualization mainstream. Visualizing streamlines plays a vital role in flow field analysis, and existing research has mainly focused on optimizing seed point selection and enhancing rendering and coloring techniques. However, automatic seed selection is time-consuming and laborious when dealing with large-scale 3D flow field data. Therefore, interactively drawing streamlines in user-defined regions of interest becomes essential, while also achieving the goal of interactively exploring the flow field. At the same time, the use of animated streamlines as a tool to represent flow motion has been widely adopted. The motion perception and visualization behind animated streamlines have been widely studied and have been shown to be beneficial to human perception in many studies. The methods used in the animated streamline visualization software ParaView have many problems, which directly lead to their unusability in VR. The current research background of large-scale 3D flow field visualization, dense streamline representation, and real-time performance in VR environments has mostly not been mentioned and solved in previous work.

[0004] In summary, the existing streamline visualization technology has the following main defects: there is a bottleneck in the number of visualized streamlines; the scene application of visualized streamlines has great limitations; and it is difficult for existing visualization software to understand the overall picture and details of the flow field. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a streamline visualization method based on GPU and immersive interaction. The method comprises the following steps:

[0006] Acquiring wind field data, wherein the wind field data reflects speed information and pressure information of the internal wind field;

[0007] Performing surface extraction, format conversion and model processing, and color remapping on the wind field data in sequence to obtain a rendered real scene;

[0008] Performing interactive design for the wind farm data, wherein the interactive design includes determining the movement and starting point of the streamline and creating anchor points for positioning;

[0009] Generate streamline data according to the velocity information in the wind field data and the interactive design;

[0010] For the generated streamline data, GPU-based animation streamlines are used to display dense streamlines and obtain dynamic rendering results;

[0011] The rendered real scene and the dynamic rendering result are combined to realize streamline visualization.

[0012] Compared with the prior art, the advantage of the present invention is that the proposed streamline visualization method based on GPU and immersive interaction can achieve dynamic display of hundreds of millions of streamline micro-elements, and systematically solve the problem of complex flow field observation. Through immersive and interactive flow field exploration, dynamic streamlines, and real-life maps, the flow field, especially the urban wind field, can be effectively observed in detail, which can effectively help urban and building designers find problems and improve solutions.

[0013] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0015] Figure 1 is a schematic diagram of two case scales observed according to an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of global and local immersive perspectives of a flow field according to an embodiment of the present invention

[0017] Figure 3 is a schematic diagram of roaming and interactive streamline generation according to an embodiment of the present invention;

[0018] Figure 4 is a flow chart of a streamline visualization method based on GPU and immersive interaction according to an embodiment of the present invention;

[0019] Figure 5 is a schematic diagram of the overall process of a streamline visualization method based on GPU and immersive interaction according to an embodiment of the present invention;

[0020] Figure 6 It is a schematic diagram of single-frame rendering time for batch processing of different degrees compared with the scale and integration method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0022] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0023] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0024] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0025] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0026] In order to have a clearer understanding of the technical problems that the present invention is committed to solving, two examples are first used to illustrate the method of the present invention. The first case is based on the simulation results of the Shenzhen Greater Bay Area using RANS (Reynolds Averaged Navier-Stokes). The main challenge of this scenario is the vast area (8.8×8.4km^2), which leads to a large amount of data and a heavy computational burden. The second visualization task is based on the LES (large eddy simulation) results of a 1×1.0km^2 downtown area. In this simulation, a three-dimensional incompressible Navier-Stokes solver using the scalable domain decomposition method is introduced to simulate the complex airflow around urban communities. In order to effectively capture turbulence, the mesh is encrypted near the buildings to improve the accuracy of the simulation. See Figure 1 As shown, the visualization of the present invention accurately displays this densification feature, clearly showing the complex details near the building. And the visualization of the present invention covers a full range of landscape and architectural features, including the refined mesh around the building. This visualization shows a clear positive correlation between the degree of mesh densification and wireframe density.

[0027] VR technology has unique advantages in visualizing the complexity of flow field structures, and can seamlessly switch between large-scale overviews and complex local details. In the simulation of the present invention, the resolution of the grid is one meter, and the complex geometry of the computational domain leads to the emergence of many small vortices. Using thick stream tubes to represent these subtle features will not capture their true nature, so the use of thin streamlines is necessary. Given the wide variety of vortices in the flow field, with varying sizes, densities, and complexities, a large number of streamlines are required for a comprehensive display. Figure 2 The streamline visualization of the flow in the Daya Bay area is shown, with a large number of small eddies visible. By combining the power of VR technology with dense streamlines, it can provide users with an immersive and comprehensive visualization experience, allowing them to delve into the complex dynamics of the flow field and gain insights that may otherwise be overlooked. This integrated approach not only enhances the understanding of flow structures, but also opens up new avenues for innovative research and practical applications in various fields from engineering to environmental science.

[0028] In summary, in order to visualize large-scale flow fields, the present invention uses VR interaction and immersive experience to solve exploratory problems; in order to reveal the local features of fine grids, thin lines and high resolution are used for display; in order to handle complex streamline structures, animated streamlines with transparency are used. Combining all these features and challenges, the present invention introduces GPU-based animated streamlines to display dense streamlines to fully address the challenges of large-scale flow field visualization.

[0029] In order to achieve effective interactive design, special attention should be paid to determining the movement and starting point of the streamlines. Given the large scale of the flow field, it is essential to quickly navigate to the vicinity of the building of interest. On the contrary, due to the small starting point of the streamline, the speed of the view movement should be slow to ensure precise control and positioning. Therefore, it is crucial to conveniently adjust the speed of the view movement. On the interactive platform of the present invention, after the scene is loaded, the user is prompted to fine-tune the speed in the horizontal and vertical directions. After the position is adjusted in place, the creation of anchor points for precise positioning determines the location of streamline generation. For example, two different seed point methods are used. The first is the point source mode, which places a point at a specific point and generates a streamline passing through the point. This interactive mode is suitable for quickly testing the flow field at the current position and providing instant feedback. The other is the line source mode, which determines the two vertices of the line segment and evenly distributes points on this line segment to calculate the streamlines passing through these points. The process of placing the line segment is carried out in two steps, corresponding to the two vertices respectively. Subsequently, the operator can move freely, the speed can be adjusted in advance or according to real-time needs, navigate the entire map through controller button mapping, and enter the interior of the streamline to observe the details of the flow. As Figure 3 Schematic diagram of the entire process of roaming and interactive streamline generation.

[0030] Specifically, combined Figure 4 and Figure 5 As shown, the provided streamline visualization method based on GPU and immersive interaction includes the following steps:

[0031] Step S1, obtaining wind field data, where the wind field data includes various physical information of the internal wind field.

[0032] Wind field data usually refers to a data set used to describe the distribution of wind direction and wind speed in the atmosphere. For example, wind field data consists of a large number of volume grids, containing a variety of physical information such as internal wind field speed and pressure, and large-scale wind field data can be obtained through scientific calculations.

[0033] Step S2, obtaining a rendered real scene after surface extraction, format conversion, model processing, and color remapping of the wind farm data.

[0034] For example, the real-life map (such as Daya Bay area and Shenzhen surface area) obtains the map model through Paraview, where the exported model should contain parameters such as wind field speed and pressure for physical rendering. Then, import it into Blender, and according to the data format, you can choose to adjust multiple physical parameters, such as weighting, normalization, etc., and write vertex data, and export it to the three-dimensional format supported by Unity. Finally, implement static loading of the model in Unity, and perform color remapping according to vertex data. Finally, complete the rendering at the fragment stage to obtain the rendered real scene. This method effectively improves performance and facilitates the dynamic adjustment of color mapping relationships, so as to focus on detailed visualization in areas and ranges of interest.

[0035] It should be noted that ParaView is a program for analyzing and visualizing two-dimensional and three-dimensional data. ParaView users can quickly set up a visualization environment and analyze data using quantitative or qualitative methods. Blender is a three-dimensional graphics and imaging software. The integrated creation suite provides comprehensive 3D creation tools, including modeling, mapping, animation, physics simulation of particles and other systems, rendering, motion tracking, etc. Unity is a tool engine for creating visualizations of various types of interactive content such as three-dimensional games, virtual reality, real-time animation, architectural visualization, etc. Those skilled in the art may also use other types of programs, toolkits or self-developed programs to replace ParaView, Blender, etc.

[0036] Step S3, performing interactive design based on wind farm data.

[0037] In this step, interactive development is performed based on the wind farm data and hardware. For example, hardware support can be provided by the SteamVR development toolkit, and interactive extensions are provided by the VRTK toolkit. It is necessary to make corresponding adjustments to the hardware adaptability when Steam VR is initialized, such as the settings of the VR helmet and the binding of controller buttons. Finally, customized interactive development is performed based on VRTK, such as the free movement of characters in the wind farm, the capture and generation of scene objects, etc.

[0038] It should be noted that Steam VR is a fully functional 360° room-scale virtual reality experience. In one embodiment, the development kit of Steam VR may include a head-mounted display, two single-handheld controllers, a positioning system that can track the display and controller simultaneously in space, etc. VRTK is an open source plug-in based on the Unity engine, specially designed for virtual reality developers. By using VRTK, virtual reality applications can be quickly built, and rich functions and interactivity, such as gesture recognition, physical interaction, and object grabbing, can be provided. Those skilled in the art may also use other software or toolkits with corresponding functions to implement the interactive design function of the present invention.

[0039] Step S4, generating streamline data according to the speed information in the wind field data and the interactive design.

[0040] In one embodiment, the streamline data file is obtained by integrating the velocity of the wind field and the source point set in the interaction. The data file may include the vertices and topological structure of the streamline element. For example, the mesh is operated in Unity, the streamline is reconstructed and batch processing is appropriately used for integration. Finally, the drawing mode is set to line mode.

[0041] Step S5: for the generated streamline data, GPU-based animation streamlines are used to display dense streamlines to obtain dynamic rendering results.

[0042] For example, the time domain and the space domain are encoded and processed by custom shaders. The specific method is to alias the sequence number with the time signal and then map it into a periodic signal, so that each vertex on the line changes periodically with a specific rule, and each two adjacent vertices differ by a fixed phase, so that from the overall point of view, the order of the points on the entire streamline changes step by step over time in a certain periodicity.

[0043] Step S6, combining the rendered real scene and the dynamic rendering results to realize streamline visualization.

[0044] Combining the above-obtained rendering real scene, dynamic rendering results and physical field information, an immersive and interactive streamline visualization effect can be achieved.

[0045] In order to further verify the effect of the present invention, experiments were conducted. Considering that the basic collaboration between VTK and VR has both advantages and disadvantages, the present invention has carried out extensive work to derive and demonstrate these methods, and also explored solutions from collaborative software such as Omniverse under NVIDIA. Finally, the above-mentioned solution designed by itself was adopted and has been developed. It has been verified that the present invention can achieve the above functions, and the specific methods are all photos of real machines. In addition, the release package of the software designed based on the present invention has been tested on multiple computers (including workstation hosts and laptops). It has been verified that the calculation of large-scale streamline integrals occupies a large part of the overall preparation time. The integration time at various scales using different integration methods has been compared and analyzed, such as Figure 6 (a). And the rendering speed is compared using different degrees of batch processing, such as Figure 6 (b) as shown.

[0046] In summary, compared with the prior art, the present invention has the following advantages:

[0047] 1) Systematically solve large-scale flow field streamline exploration

[0048] Visualization of flow field has always been a difficult problem. Streamlines, as an important component of flow field visualization, are studied in depth in the present invention. The present invention uses VR as a basic device, which allows observers to experience the movement inside the flow field, including various turbulences and vortices, and also supports interactively changing streamlines in the flow field.

[0049] 2) Dynamic display of dense streamlines

[0050] For large-scale flow field streamlines, the present invention designs a dynamic display based on hardware (GPU), accurately reflecting the speed in the flow field, allowing observers to clearly grasp the dynamic characteristics of the flow field.

[0051] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0052] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0053] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0054] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0055] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0056] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0057] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0058] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0059] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A streamline visualization method based on GPU and immersive interaction, comprising the following steps: Acquiring wind field data, wherein the wind field data reflects speed information and pressure information of the internal wind field; Performing surface extraction, format conversion, model processing, and color remapping on the wind farm data to obtain a rendered real scene; Performing interactive design for the wind farm data, wherein the interactive design includes determining the movement and starting point of the streamline and creating anchor points for positioning; Generate streamline data according to the velocity information in the wind field data and the interactive design; For the generated streamline data, GPU-based animation streamlines are used to display dense streamlines and obtain dynamic rendering results; The rendered real scene and the dynamic rendering result are combined to realize streamline visualization.

2. The method according to claim 1, characterized in that: The interactive design includes: The Steam VR development kit is used as hardware support, and the VRTK toolkit is used to provide interactive expansion; Set up hardware compatibility when initializing Steam VR, including setting bindings for head-mounted displays and controller buttons; Customized interactive development based on VRTK, including the free movement of characters in the wind field, and the capture and generation of scene objects.

3. The method according to claim 1, characterized in that: In the process of generating the streamline data, the seed point is selected in a point source mode, which places a point at a specific point and generates a streamline passing through the point.

4. The method according to claim 1, characterized in that In the process of generating the streamline data, the seed point is selected in a line source mode, which determines two vertices of a line segment, evenly distributes points on the line segment, and calculates streamlines passing through these points.

5. The method according to claim 1, characterized in that Generating streamlines according to the velocity information in the wind field data and the interactive design includes: Integrate the velocity in the wind field data and the source point set in the interactive design to obtain a streamline data file, wherein the streamline data file includes vertices and a topological structure of streamline micro-elements; Operate the mesh in Unity, reconstruct the streamlines and use batch processing to integrate them; Set the drawing mode to line mode.

6. The method according to claim 1, characterized in that For the generated streamlines, GPU-based animation streamlines are used to display dense streamlines including: The time domain and space domain are encoded and processed through custom shaders, the sequential number is mixed with the time signal, and then mapped into a periodic signal, so that each vertex on the line changes periodically according to a set rule, and each adjacent two vertices differ by a fixed phase.

7. The method according to claim 1, characterized in that ParaView is used to perform surface extraction on the wind field data.

8. The method according to claim 1, characterized in that The format conversion and model processing of the wind field data are implemented by using the three-dimensional graphics and image software Blender.

9. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images