Pipeline medium in-situ simulation method based on mixed reality

Abstract

The invention discloses a pipeline medium flow in-situ simulation method based on a mixed reality technology. The method comprises the following steps: extracting information of an instance pipeline and a flowing medium in the instance pipeline; the virtual model of the pipeline system is imported into Unity; creating a medium flow simulation model; mixed reality development; packaging and deploying the project to mixed reality glasses equipment; and wearing mixed reality glasses equipment on site, and checking an internal medium flow simulation result of the pipeline system instance in situ. According to the method, the virtual simulation animation of medium flowing in the pipeline and the real pipeline physical entity are fused and bound through the mixed reality technology, and visual flowing medium in-situ simulation is carried out in a pipeline system. According to the method, a user does not need to check drawings and sensors, flowing medium information, such as flowing medium types, flowing states and flowing directions, in a real pipeline system is rapidly obtained through mixed reality equipment, the working efficiency of field staff is improved, and improvement and convenience are brought to operation and maintenance of large, medium and small pipeline systems.

Description

Technical Field

[0001] The present invention relates to the fields of civil engineering and intelligent construction, and in particular to an in-situ simulation method for pipeline media based on mixed reality. Background Art

[0002] In fields such as industrial production, energy transportation, and chemical processing, pipeline systems are critical infrastructure for media transmission. Understanding the flow characteristics of media within pipelines is crucial for ensuring production safety and optimizing process flows. However, traditional pipeline flow medium identification methods require consulting existing paper drawings or relying on large amounts of real-time data from sensors such as pressure, temperature, and flow to obtain medium flow information. This makes the task more difficult by failing to intuitively connect the medium flow to the actual pipeline.

[0003] While computational fluid dynamics software can simulate the flow of media within pipelines, its results are disconnected from the actual pipeline scenario, requiring complex imaginative mapping and increasing the cognitive burden. Virtual reality technology, while providing an immersive experience, isolates users from reality, posing safety risks. Augmented reality technology, in complex pipeline systems, lacks the accuracy and interactivity to meet practical needs. Problems include unstable information display and limited interaction methods, hindering the efficient acquisition of pipeline media information.

[0004] To overcome these shortcomings, the present invention introduces mixed reality (MR) and eye tracking technologies. Mixed reality (MR) technology combines virtual information with the real environment, creating a blended virtual and real environment, enabling users to interact naturally with virtual objects in the real world. This technology not only enhances the user's perception of virtual information but also provides a more intuitive and immersive experience. Eye tracking technology enables natural interaction between the user and the device by monitoring the user's gaze direction and gaze point in real time. Users can select or manipulate virtual objects simply by looking at them, without manual input, greatly improving the efficiency and convenience of interaction. Summary of the Invention

[0005] In view of the above problems, the present invention aims to provide a method for in-situ simulation of pipeline medium flow based on mixed reality technology, characterized in that the method comprises the following steps: Step 1: Extract the piping system BIM model and its internal flow medium information in Autodesk Revit, export it to FBX format, perform lightweight processing on the model, and import the processed piping system model into Unity; Step 2: Create a medium model using 3ds Max, mesh the medium model using the UV editor, and then import the medium model into Unity. Step 3. In Unity, assign a material with a flow texture to the medium model and apply it to the corresponding pipe model. Add a script to achieve the flow effect of the medium. Simulate the flow of different media in the pipe by adjusting the texture, scrolling direction, and speed. Step 4: Configure the mixed reality environment in Unity to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system. Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model, ensuring that the button remains visible to the user. Step 5: Open the saved project using Visual Studio, compile and publish it as an APPX installation package, deploy the application to the mixed reality glasses device and debug the usage environment, align the virtual model of the pipeline system with the physical entity, and create a spatial anchor to lock the model; In step 6, the user wears mixed reality glasses on site and interacts with buttons bound to the real pipeline to view the simulation results of the medium flow inside the pipeline in situ, including the medium type, flow direction, and flow speed in the pipeline.

[0006] Further, in step 1, perform the following steps in sequence: Step 1.1: Extract the piping system instance BIM model in Autodesk Revit and export it to FBX format; Step 1.2: Lightweight the pipeline system model to reduce the number of triangle meshes and reduce rendering complexity. Step 1.3: In Unity, import the optimized virtual piping system model. Further, in step 2, perform the following steps in sequence: Step 2.1, in 3ds Max software, create a medium model according to the medium type; Step 2.2: In 3ds Max, use the UV Editor to perform UV mapping on the media model. Divide the UV grid according to the media model so that you can add textures and materials to the media model in Unity. Step 2.3: Export the media model in 3ds Max in FBX format and import it into Unity. Further, in step 3, perform the following steps in sequence: Step 3.1: In Unity, assign corresponding textures and materials to the medium model based on different medium information, and apply them to the corresponding pipeline model; In step 3.2, write C# code to define a class named "TextureMovement" that inherits from "MonoBehaviour"; Step 3.3: Define an "Axis" enumeration in this class to specify the medium flow axis, which contains three values: X, Y, and Z. Define variables "moveAxis" and "moveSpeed" to control the medium flow axis and flow speed respectively. In step 3.4, implement the "Update()" method in the class. Use the "switch" statement to set the texture scroll direction based on the value of "moveAxis". Then assign the texture offset value calculated by the script back to the "mainTextureOffset" property of the material to update the texture's display position, thereby achieving the fluid effect of the medium. Step 3.5, simulate the flow of different media in the pipeline by adjusting the texture, rolling direction and speed; Further, in step 4, perform the following steps in sequence: Step 4.1: Configure the mixed reality environment in Unity and import Microsoft's Mixed Reality Toolkit (MRTK). Step 4.2: Use the MRTK toolkit to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system; Step 4.3: Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model. In step 4.4, ensure that the button remains visible to the user by setting the button's Canvas component's Render Mode to Screen Space - Camera and adjusting its Sorting Layer and Order in Layer properties. Further, in step 4, perform the following steps in sequence: Step 5.1, use Visual Studio to open the saved project, compile the project into a format suitable for mixed reality glasses devices, and publish it as an APPX installation package; Step 5.2: Deploy the application to the mixed reality glasses device via wireless connection and debug the usage environment; Step 5.3: Using the interactive function described in step 4.2 and the spatial mapping function of the mixed reality glasses device, align the virtual model of the pipeline system with the physical entity, and create a spatial anchor point locking model to achieve binding between the virtual model of the pipeline system and the physical entity; Furthermore, in step 6, after wearing the mixed reality glasses device, the user can view the simulation results of the medium flow inside the pipeline in situ by interacting with the buttons bound to the real pipeline, including the medium type, flow direction, and flow speed in the pipeline.

[0007] The advantages and positive effects of the present invention are: (1) Optimize the interactivity and efficiency of the identification of flowing media in pipeline systems. The present invention significantly improves the interactivity and efficiency of the identification of flowing media in pipeline systems by combining mixed reality technology with eye tracking technology. With the help of mixed reality devices, users can intuitively view the flow of media in the pipeline. The introduction of eye tracking technology allows users to achieve efficient interaction through natural line of sight operations without the need for tedious manual input. This not only greatly improves the efficiency and accuracy of information acquisition, but also reduces dependence on professional knowledge, reduces the risk of operational errors, and ensures the safety of operations. In addition, the present invention is applicable to pipeline systems of various sizes and has wide practicality.

[0008] (2) Innovation in the application of mixed reality technology in civil engineering. This invention has achieved significant technological innovation in the field of civil engineering by applying mixed reality technology to the identification of flowing media in pipeline systems. This application provides engineers with a new and intuitive identification method that can display the flow state of the medium in the pipeline in real time. This not only helps engineers obtain information quickly and accurately and make scientific decisions, but also optimizes the operation and maintenance strategy of the pipeline system. The application of mixed reality technology has improved work efficiency, reduced training costs and operational difficulty, brought a more natural and immersive interactive experience to the field of civil engineering, and promoted technological innovation and development in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other aspects and advantages of the present invention will become clearer and more easily understood through the detailed description made in conjunction with the following drawings, which are only illustrative and do not limit the present invention, wherein: Figure 1 This is a flow chart of the in-situ simulation method of pipeline media based on mixed reality.

[0010] Figure 2 This is an example diagram of the C# script configuration for the medium flow simulation model in Unity.

[0011] Figure 3 This is an example diagram of the C# script configuration for showing and hiding the medium flow simulation model in Unity.

[0012] Figure 4 This is an example diagram of pipeline medium flow simulation in Unity.

[0013] Figure 5 This is an example diagram of in-situ simulation of flowing media in a pipeline system in Unity.

[0014] Figure 6 This is an example diagram of in-situ simulation of pipeline media based on mixed reality glasses. DETAILED DESCRIPTION

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] like Figure 1 As shown, the present invention aims to provide a pipeline medium flow in-situ simulation method based on mixed reality technology, characterized in that the method includes the following steps: Step 1: Extract the piping system BIM model and its internal flow medium information in Autodesk Revit, export it to FBX format, perform lightweight processing on the model, and import the processed piping system model into Unity; Step 2: Create a medium model using 3ds Max, mesh the medium model using the UV editor, and then import the medium model into Unity. Step 3. In Unity, assign a material with a flow texture to the medium model and apply it to the corresponding pipe model. Add a script to achieve the flow effect of the medium. Simulate the flow of different media in the pipe by adjusting the texture, scrolling direction, and speed. Step 4: Configure the mixed reality environment in Unity to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system. Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model, ensuring that the button remains visible to the user. Step 5: Open the saved project using Visual Studio, compile and publish it as an APPX installation package, deploy the application to the mixed reality glasses device and debug the usage environment, align the virtual model of the pipeline system with the physical entity, and create a spatial anchor to lock the model; In step 6, the user wears mixed reality glasses on site and interacts with buttons bound to the real pipeline to view the simulation results of the medium flow inside the pipeline in situ, including the medium type, flow direction, and flow speed in the pipeline.

[0017] Further, in step 1, perform the following steps in sequence: Step 1.1: Extract the piping system instance BIM model in Autodesk Revit and export it to FBX format; Step 1.2: Lightweight the pipeline system model to reduce the number of triangle meshes and reduce rendering complexity. Step 1.3: In Unity, import the optimized virtual piping system model. Further, in step 2, perform the following steps in sequence: Step 2.1, in 3ds Max software, create a medium model according to the medium type; Step 2.2: In 3ds Max, use the UV Editor to perform UV mapping on the media model. Divide the UV grid according to the media model so that you can add textures and materials to the media model in Unity. Step 2.3: Export the media model in 3ds Max in FBX format and import it into Unity. Further, in step 3, perform the following steps in sequence: Step 3.1: In Unity, assign corresponding textures and materials to the medium model based on different medium information, and apply them to the corresponding pipeline model; In step 3.2, write C# code to define a class named "TextureMovement" that inherits from "MonoBehaviour"; Step 3.3: Define an "Axis" enumeration in this class to specify the medium flow axis, which contains three values: X, Y, and Z. Define variables "moveAxis" and "moveSpeed" to control the medium flow axis and flow speed respectively. In step 3.4, implement the "Update()" method in the class. Use the "switch" statement to set the texture scroll direction based on the value of "moveAxis". Then assign the texture offset value calculated by the script back to the "mainTextureOffset" property of the material to update the texture's display position, thereby achieving the fluid effect of the medium. Step 3.5, simulate the flow of different media in the pipeline by adjusting the texture, rolling direction and speed; Further, in step 4, perform the following steps in sequence: Step 4.1: Configure the mixed reality environment in Unity and import Microsoft's Mixed Reality Toolkit (MRTK). Step 4.2: Use the MRTK toolkit to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system; Step 4.3: Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model. In step 4.4, ensure that the button remains visible to the user by setting the button's Canvas component's Render Mode to Screen Space - Camera and adjusting its Sorting Layer and Order in Layer properties. Further, in step 4, perform the following steps in sequence: Step 5.1, use Visual Studio to open the saved project, compile the project into a format suitable for mixed reality glasses devices, and publish it as an APPX installation package; Step 5.2: Deploy the application to the mixed reality glasses device via wireless connection and debug the usage environment; Step 5.3: Using the interactive function described in step 4.2 and the spatial mapping function of the mixed reality glasses device, align the virtual model of the pipeline system with the physical entity, and create a spatial anchor point locking model to achieve binding between the virtual model of the pipeline system and the physical entity; Furthermore, in step 6, after wearing the mixed reality glasses device, the user can view the simulation results of the medium flow inside the pipeline in situ by interacting with the buttons bound to the real pipeline, including the medium type, flow direction, and flow speed in the pipeline.

[0018] This invention provides a method for in-situ simulation of pipeline medium flow based on mixed reality technology. This method combines mixed reality with eye tracking technology, significantly improving the interactivity and efficiency of identifying flowing media within pipeline systems. Using mixed reality devices, users can intuitively view the flow of media within pipelines. The introduction of eye tracking technology enables efficient interaction through natural eye contact, eliminating the need for cumbersome manual input. This not only greatly improves the efficiency and accuracy of information acquisition, but also reduces reliance on professional knowledge, reduces the risk of operational errors, and ensures operational safety. Furthermore, this invention achieves significant technological innovation in the field of civil engineering by applying mixed reality technology to the identification of flowing media within pipeline systems. This application provides engineers with a new, intuitive identification method that can display the flow status of media within pipelines in real time. This not only helps engineers quickly and accurately obtain information and make informed decisions, but also optimizes pipeline system operation and maintenance strategies. The application of mixed reality technology improves work efficiency, reduces training costs and operational difficulty, and brings a more natural and immersive interactive experience to the civil engineering field, promoting technological innovation and development in related fields.

Claims

1. A method for in-situ simulation of pipeline medium flow based on mixed reality technology, characterized in that: The method comprises the following steps: Step 1: Extract the piping system BIM model and its internal flow medium information in Autodesk Revit, export it to FBX format, perform lightweight processing on the model, and import the processed piping system model into Unity; Step 2: Create a medium model using 3ds Max, mesh the medium model using the UV editor, and then import the medium model into Unity. Step 3. In Unity, assign a material with a flow texture to the medium model and apply it to the corresponding pipe model. Define a class named "TextureMovement" that inherits from "MonoBehaviour". Define an "Axis" enumeration in this class to specify the medium flow axis, which contains three values: X, Y, and Z. Define variables "moveAxis" and "moveSpeed" to control the medium flow axis and flow speed, respectively. Implement the "Update()" method in the class and use the "switch" statement to set the texture scrolling direction according to the value of "moveAxis". Then assign the texture offset value calculated by the script back to the "mainTextureOffset" property of the material to update the texture display position, thereby achieving the medium flow effect. By adjusting the texture, scrolling direction, and speed, the flow of different media in the pipe can be simulated. Step 4: Configure a mixed reality environment in Unity and import Microsoft's Mixed Reality Toolkit (MRTK). Use the MRTK to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system. Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model. Set the Render Mode of the button's Canvas component to Screen Space - Camera and adjust its Sorting Layer and Order in Layer properties to ensure that the button remains visible to the user. Step 5: Use Visual Studio to open the saved project, compile the project into a format suitable for the mixed reality glasses device, and publish it as an APPX installation package. Deploy the application to the mixed reality glasses device via a wireless connection and debug the usage environment. Utilize the spatial mapping function of the mixed reality glasses device and the interactive function described in step 2 to align the virtual model of the pipeline system with the physical entity, create a spatial anchor point to lock the model, and achieve binding between the virtual model of the pipeline system and the physical entity. In step 6, the user wears mixed reality glasses on site and interacts with buttons bound to the real pipeline to view the simulation results of the medium flow inside the pipeline in situ, including the medium type, flow direction, and flow speed in the pipeline.

2. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 1, perform the following steps in sequence: Step 1.1: Extract the piping system instance BIM model in Autodesk Revit and export it to FBX format; Step 1.2: Lightweight the pipeline system model to reduce the number of triangle meshes and reduce rendering complexity. Step 1.3: In Unity, import the optimized virtual piping system model.

3. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 2, perform the following steps in sequence: Step 2.1, in 3ds Max software, create a medium model according to the medium type; Step 2.2: In 3ds Max, use the UV Editor to perform UV mapping on the media model. Divide the UV grid according to the media model so that you can add textures and materials to the media model in Unity. In step 2.3, export the media model in 3ds Max software in FBX format and import it into Unity.

4. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 3, perform the following steps in order: Step 3.1: In Unity, assign corresponding textures and materials to the medium model based on different medium information, and apply them to the corresponding pipeline model; In step 3.2, write C# code to define a class named "TextureMovement" that inherits from "MonoBehaviour"; Step 3.3: Define an "Axis" enumeration in this class to specify the medium flow axis, which contains three values: X, Y, and Z. Define variables "moveAxis" and "moveSpeed" to control the medium flow axis and flow speed, respectively. In step 3.4, implement the "Update()" method in the class. Use the "switch" statement to set the texture scroll direction based on the value of "moveAxis". Then, assign the texture offset value calculated by the script back to the "mainTextureOffset" property of the material to update the texture's display position, thereby achieving a fluid effect. In step 3.5, the flow of different media in the pipe is simulated by adjusting the texture, scrolling direction, and speed.

5. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 4, perform the following steps in order: Step 4.1: Configure the mixed reality environment in Unity and import Microsoft's Mixed Reality Toolkit (MRTK). Step 4.2: Use the MRTK toolkit to implement interactive functions such as grabbing, moving, rotating, and scaling the virtual model of the pipeline system; Step 4.3: Create a UI button and add a script to control the visibility of a pipeline medium flow simulation model. In step 4.4, ensure that the button remains visible to the user by setting the button's Canvas component's Render Mode to Screen Space - Camera and adjusting its Sorting Layer and Order in Layer properties.

6. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 4, perform the following steps in order: Step 5.1, use Visual Studio to open the saved project, compile the project into a format suitable for mixed reality glasses devices, and publish it as an APPX installation package; Step 5.2: Deploy the application to the mixed reality glasses device via wireless connection and debug the usage environment; In step 5.3, through the interactive function described in step 4.2 and combined with the spatial mapping function of the mixed reality glasses device, the virtual model of the pipeline system is aligned with the physical entity, and a spatial anchor point locking model is created to achieve the binding of the virtual model of the pipeline system with the physical entity.

7. The in-situ simulation method for pipeline medium flow based on mixed reality technology according to claim 1, characterized in that: In step 6, after wearing the mixed reality glasses, the user interacts with the buttons bound to the real pipeline to view the simulation results of the medium flow inside the pipeline in situ, including the medium type, flow direction, and flow speed in the pipeline.

Images