ROV simulation driving equipment based on mixed reality
By integrating MR wearable devices, virtual reality backend, and control system, the mixed reality ROV driving simulator solves the problems of existing ROV operation training equipment being expensive and lacking realism, and achieves an efficient and low-cost immersive training experience.
Patent Information
- Application Number
- CN202520068503.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-13
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2035-01-13
AI Technical Summary
Existing ROV operation training methods suffer from problems such as expensive equipment, large footprint, high maintenance costs, and poor simulation of driving realism, failing to provide an immersive experience that matches actual operation.
It adopts a mixed reality-based ROV driving simulator, integrating components such as MR wearable devices, virtual reality backend, SCADA host, simulation host, control module and programmable logic controller. It achieves seamless integration of virtual scene and real environment through network interface and industrial communication protocol, and provides intuitive operation guidance and support functions.
It enhances the realism and immersion of ROV operation training, improves operational precision and decision-making capabilities, supports multi-person collaboration and teaching needs, and reduces equipment costs and floor space.
Smart Images
Figure CN223842505U_ABST
Abstract
Description
Technical Field
[0001] This utility model relates to the field of driving simulation technology, and in particular to a ROV driving simulation device based on mixed reality. Background Technology
[0002] The development of marine resources is crucial to the future of humanity, especially given the gradual depletion of terrestrial resources. Exploring and utilizing marine resources has become a key task in the 21st century. Remotely operated underwater vehicles (ROVs) play an indispensable role in this process, capable of performing complex tasks in extreme environments, such as seabed exploration, pipeline laying, and maintenance. However, operating ROVs requires a high level of expertise and experience. Improper operation can not only lead to mission failure but also damage expensive equipment and threaten the safety of seabed facilities and construction personnel.
[0003] Existing ROV operation training methods have significant limitations:
[0004] While some imported simulators can realistically recreate the operating environment of ROVs, these devices typically rely on a large amount of supporting hardware, are expensive, occupy a large area, and have high maintenance costs, making them difficult to widely adopt. In contrast, domestically developed general-purpose simulators, although smaller in size, fall short in terms of the realism of simulated driving and cannot provide an immersive experience that matches actual operation. Furthermore, these simulators often lack complete control modules, resulting in discrepancies between simulated and real-world data. Utility Model Content
[0005] The main objective of this invention is to provide a mixed reality-based ROV driving simulation device, which aims to solve the technical problem that existing ROV operation training methods have significant limitations.
[0006] In order to achieve the above-mentioned utility model objectives, the first aspect of this utility model proposes a ROV simulation driving device based on mixed reality, including a visual system and a control system integrated on the control panel.
[0007] The visual system includes an MR wearable device, a virtual reality backend, a SCADA host, a simulation host, and a live broadcast screen. The MR wearable device is connected to the virtual reality backend via a network interface, the MR wearable device is connected to the simulation host via a network interface, the MR wearable device is connected to the SCADA host via a network interface, and the live broadcast screen is connected to the simulation host.
[0008] The control system includes a control module, a programmable logic controller (PLC), an auxiliary HMI, a control panel, an operating joystick, and operating buttons. The auxiliary HMI is connected to the PLC via an industrial communication protocol. The operating signals of the operating joystick and operating buttons are transmitted to the PLC through the control module. The PLC is connected to both the simulation host and the SCADA host.
[0009] Furthermore, the control panel is a dual-operator control panel, with each driver's seat equipped with two control joysticks, an MR headset, a control panel, an operation button, and a seat;
[0010] The operating joystick and operating buttons are all embedded in the operating console, and the SCADA host, simulation host, control module and programmable logic controller are all installed in the internal space of the operating console.
[0011] Furthermore, the programmable logic controller (PLC) in the control module and the control module exchange data via an industry-standard communication protocol.
[0012] Furthermore, the control module in the control module and the programmable logic controller includes multiple control circuit boards, I / O interfaces, and couplers.
[0013] Furthermore, the network interface can be either a wired or wireless connection.
[0014] Furthermore, the electrical components integrated into the control panel consist of buttons, push rods, and knobs, and these components are connected to the control module via hardwired connections or dedicated interfaces.
[0015] Furthermore, the auxiliary HMI is embedded in the surface of the control panel and located in the middle of the control panel.
[0016] Furthermore, the simulation host is responsible for simulating the dynamic behavior of the ROV equipment and information about the surrounding environment, and synchronizing the information to the MR wearable device.
[0017] Furthermore, the SCADA host is used to collect and process data from the simulation host, and display simulation information and monitoring data on the MR wearable device in the form of a virtual screen.
[0018] Furthermore, the live streaming screen is deployed independently of the control panel, allowing third-party observers to watch the navigator's simulated operation process in real time.
[0019] Beneficial effects:
[0020] 1. This invention integrates MR wearable devices, a virtual reality backend, and a simulation host, enabling seamless fusion of virtual scenes with the real world and providing users with a near-realistic ROV operation experience. Users can not only see virtual information superimposed on the real environment but also feel the dynamically changing underwater environment, enhancing the realism and immersion of training. The simulation host accurately simulates the ROV's movements and changes in its surrounding environment, synchronizing this information to the MR wearable device in real time. The SCADA host collects and processes data from the simulation host, displaying it graphically to the user, ensuring that operators can obtain accurate operating parameters and environmental information in a timely manner, thereby improving operational accuracy and decision-making capabilities.
[0021] 2. This utility model utilizes a programmable logic controller (PLC) and an auxiliary HMI in its control system, providing intuitive operation guidance and support functions. Input devices such as joysticks and buttons are connected to the PLC via the control module, ensuring accurate signal transmission and fast response. The dual-operator design allows two drivers to perform simulated operation simultaneously, supporting multi-person collaboration or teaching needs, further improving the flexibility and efficiency of training. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the structure of a mixed reality-based ROV driving simulator according to an embodiment of the present invention.
[0023] Figure 2 This is a side view schematic diagram of a mixed reality-based ROV driving simulator according to an embodiment of the present invention;
[0024] Figure 3 This is a schematic diagram of a mixed reality-based ROV simulation driving equipment system according to an embodiment of the present invention.
[0025] in:
[0026] 1. Control panel; 2. MR head-mounted device; 3. Control joystick; 4. Auxiliary HMI; 5. Control panel; 6. Operation buttons; 7. Seat; 8. Live broadcast screen; 9. SCADA host; 10. Simulation host; 11. Control module and programmable logic controller.
[0027] The realization of the purpose, functional features and advantages of this utility model will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0028] It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
[0029] In the description of this utility model, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience of describing this utility model and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this utility model. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this utility model, "a plurality of" means two or more, unless otherwise explicitly and specifically defined.
[0030] In the description of this utility model, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this utility model according to the specific circumstances.
[0031] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.
[0032] Reference Figures 1-3 An embodiment of this utility model provides a ROV simulation driving device based on mixed reality, including a visual system and a control system integrated on the control panel 1;
[0033] The visual system includes an MR wearable device, a virtual reality backend, a SCADA host 9, a simulation host 10, and a live broadcast screen 8. The MR wearable device is connected to the virtual reality backend via a network interface, the MR wearable device is connected to the simulation host 10 via a network interface, the MR wearable device is connected to the SCADA host 9 via a network interface, and the live broadcast screen 8 is connected to the simulation host 10.
[0034] The control system includes a control module, a programmable logic controller 11, an auxiliary HMI 4, a control panel 5, an operating joystick 3, and operating buttons 6. The auxiliary HMI 4 is connected to the programmable logic controller via an industrial communication protocol. The operating signals of the operating joystick 3 and the operating buttons 6 are transmitted to the programmable logic controller through the control module. The programmable logic controller is connected to the simulation host 10 and the SCADA host 9, respectively.
[0035] In this embodiment, the MR wearable device uses an MR head-mounted display, allowing users to see virtual information superimposed on the real world. The virtual reality backend is responsible for generating virtual scenes and communicating with the MR wearable device, ensuring that the virtual content seen by the user is updated in real time. The SCADA host 9 is a monitoring and data acquisition system used to collect, process, and display data from the simulation host 10, presenting it to the user in a graphical manner. The simulation host 10 simulates the behavior of the ROV and its surrounding environment, synchronizing this information to the MR wearable device, allowing users to experience a realistic operating feel. The live streaming screen 8 is deployed independently for third-party observers to watch the navigator's simulated operation process in real time; it is typically placed near the control panel 1 so that observers can clearly see the simulation.
[0036] The control module includes multiple control circuit boards, I / O interfaces, couplers, and other components. It is responsible for receiving signals from input devices such as the joystick 3 and operation buttons 6, converting them into instructions, and sending them to the programmable logic controller (PLC). The PLC exchanges data with the control module through an industry-standard communication protocol and is also connected to the simulation host 10 and the SCADA host 9 to coordinate the operation of the entire system.
[0037] The auxiliary HMI4, a human-machine interface, is embedded in the center of the surface of the control panel 1, providing additional operation guidance and support functions.
[0038] Optionally, the control panel 1 is a dual-operation-position driver's console, with each driver's position equipped with two operating joysticks 3, an MR head-mounted device 2, a control panel 5, an operating button 6, and a seat 7; the operating joysticks 3 and the operating button 6 are all embedded in the control panel 1, and the SCADA host 9, the simulation host 10, the control module, and the programmable logic controller 11 are all installed in the internal space of the control panel 1.
[0039] It should be noted that, to accommodate multi-person collaboration or teaching needs, the control panel 1 is designed with two operating positions. Each driver's seat is equipped with the necessary operating devices, including two joysticks 3, an MR headset 2, a control panel 5, an operating button 6, and a seat 7. All key components are embedded in the control panel 1, ensuring a clean appearance and ease of use. Furthermore, the SCADA host 9, simulation host 10, control module, and programmable logic controller 11 are all housed inside the control panel 1 to save space and protect the hardware from external influences.
[0040] In one embodiment, the control module and the programmable logic controller 11 exchange data with the control module via an industry-standard communication protocol. The control module includes multiple control circuit boards, I / O interfaces, and couplers. The data exchange between the control module and the programmable logic controller follows industry-standard communication protocols, ensuring efficient and reliable information transmission between them. This design simplifies system integration and improves system compatibility and scalability.
[0041] Optionally, the network interface can be a wired or wireless connection.
[0042] To meet the needs of different application scenarios, the network interface supports both wired and wireless connection methods. This not only increases the flexibility of the device but also makes installation more convenient. For applications requiring a stable connection, a wired connection can be selected; while for applications requiring greater mobility, a wireless connection can be used to reduce physical constraints.
[0043] In one embodiment, the electrical components integrated into the control panel 5 consist of buttons, push rods, and knobs, and these electrical components are connected to the control module via hardwired connections or dedicated interfaces.
[0044] It should be noted that control panel 5 integrates various electrical components, such as buttons, levers, and knobs, which are directly connected to the control module or via dedicated interfaces, facilitating manual operation by the user. The electrical components on control panel 5 are connected to the control module via hard-wired connections or dedicated interfaces, ensuring accurate signal transmission and fast response speed.
[0045] The auxiliary HMI4 is embedded in the surface of the control panel 1 and is located in the middle of the control panel 1.
[0046] It should be noted that the auxiliary HMI4 is located in the center of the control panel 1, ensuring that both drivers can easily access and view relevant information. Its position neither obstructs primary operations nor affects the driver's line of sight, thus enhancing the user experience.
[0047] The simulation host 10 is responsible for simulating the dynamic behavior of the ROV equipment and information about its surrounding environment, and synchronizing the information to the MR wearable device. The SCADA host 9 is used to collect and process data from the simulation host 10, and display simulation information and monitoring data on the MR wearable device in the form of a virtual screen.
[0048] The simulation host 10 is responsible for accurately simulating the ROV's movements and changes in the surrounding environment, and feeding this information back to the MR wearable device in a timely manner, enabling users to train under almost realistic conditions. The SCADA host 9 displays simulation information and monitoring data on the MR wearable device in the form of a virtual screen, helping users better understand and master the various parameters during operation.
[0049] The live streaming screen 8 is deployed independently of the control panel 1 and is used by third-party observers to watch the navigator's simulated operation process in real time.
[0050] The presence of live screen 8 allows non-operators to observe the navigator's simulated operation process in real time. It can be positioned appropriately to ensure that all participants can clearly see the content on the screen, thereby promoting communication and learning.
[0051] Description: This device provides users with an immersive operating environment by combining virtual information with the real world, allowing users to experience a feeling close to actual operation. Details are as follows:
[0052] The MR wearable device serves as the user's primary visual interface, allowing them to see virtual information superimposed on the real environment. The virtual reality backend generates virtual scenes and communicates with the MR wearable device, ensuring that the virtual content seen by the user is updated in real time. The SCADA host 9 is responsible for monitoring and data acquisition, processing data from the simulation host 10, and displaying it to the user graphically. The simulation host 10 simulates the behavior of the ROV and its surrounding environment, synchronizing this information to the MR wearable device, allowing the user to experience a realistic operational feel. A separate live streaming screen 8 is deployed for third-party observers to watch the navigator's simulated operation process in real time.
[0053] Users operate the system via input devices such as joystick 3 and operation buttons 6, and these signals are transmitted to the control module. The control module converts the received operation signals into instructions and sends them to the programmable logic controller (PLC). The PLC exchanges data with the control module through an industry-standard communication protocol and is connected to the simulation host 10 and the SCADA host 9 to coordinate the operation of the entire system. An auxiliary HMI4 (human-machine interface) provides additional operation guidance and support functions and is located in the center of the control panel 1 for easy access by both operators.
[0054] In the above process, all components communicate with each other through a network interface. The network interface supports both wired and wireless connections, increasing the application flexibility of the device. For example, the simulation host 10 not only sends simulation information to the MR wearable device, but also transmits data to the SCADA host 9, which then displays this data on the MR wearable device for the user to view.
[0055] To accommodate the needs of multi-person collaboration or teaching, the driving simulator is designed with two operating positions. Each operating position is equipped with the necessary operating devices, such as two joysticks 3, an MR headset 2, a control panel 5, an operating button 6, and a seat 7, ensuring that two drivers can perform simulated operations simultaneously.
[0056] The above description is only a preferred embodiment of the present utility model and does not limit the patent scope of the present utility model. Any equivalent structural or procedural transformations made based on the content of the present utility model specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present utility model.
Claims
1. A mixed reality-based ROV driving simulation device, characterized in that, This includes the visual and control systems integrated into the control panel; The visual system includes an MR wearable device, a virtual reality backend, a SCADA host, a simulation host, and a live broadcast screen. The MR wearable device is connected to the virtual reality backend via a network interface, the MR wearable device is connected to the simulation host via a network interface, the MR wearable device is connected to the SCADA host via a network interface, and the live broadcast screen is connected to the simulation host. The control system includes a control module, a programmable logic controller (PLC), an auxiliary HMI, a control panel, an operating joystick, and operating buttons. The auxiliary HMI is connected to the PLC via an industrial communication protocol. The operating signals of the operating joystick and operating buttons are transmitted to the PLC through the control module. The PLC is connected to both the simulation host and the SCADA host.
2. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The control panel is a dual-operator driver's console, with each driver's seat equipped with two control joysticks, an MR headset, a control panel, an operation button, and a seat; The operating joystick and operating buttons are all embedded in the operating console, and the SCADA host, simulation host, control module and programmable logic controller are all installed in the internal space of the operating console.
3. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The control module and the programmable logic controller (PLC) exchange data with the control module through an industry-standard communication protocol.
4. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The control module and the programmable logic controller include multiple control circuit boards, I / O interfaces, and couplers.
5. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The network interface can be either a wired or wireless connection.
6. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The control panel integrates electrical components such as buttons, push rods, and knobs, and these components are connected to the control module via hard wiring or dedicated interfaces.
7. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The auxiliary HMI is embedded in the surface of the control panel and located in the middle of the control panel.
8. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The simulation host is responsible for simulating the dynamic behavior of the ROV equipment and information about the surrounding environment, and synchronizing the information to the MR wearable device.
9. The ROV simulation driving device based on mixed reality according to claim 1, characterized in that, The SCADA host is used to collect and process data from the simulation host, and display simulation information and monitoring data on the MR wearable device in the form of a virtual screen.
10. A mixed reality-based ROV driving simulation device according to claim 1, characterized in that, The live streaming screen is deployed independently of the control panel and is used by third-party observers to watch the navigator's simulated operation process in real time.