Integrated ship display and control console carrying VR glasses

Abstract

The invention relates to the technical field of ship display and control, in particular to an integrated ship display and control console carrying VR glasses, and the integrated ship display and control console is used for improving the combat command and control capability of ship operators. The integrated ship display and control console comprises a display unit, a control unit, a display and control computer, a power supply unit and an external sensor unit. The display provides a first display area, and the VR glasses provide a second display area. Ship data output by the external sensor unit is transmitted to the first display area through the display control computer, information such as a 360-degree ship scene created by digital twinning is transmitted to the second display area through the display control computer, and therefore more display picture selections are provided for ship operators. In addition, the ship operator can control and operate the image of the display area in combination with the control unit, so that the limitation that monitoring and operation are carried out by depending on a flat panel display traditionally is eliminated, and the high-intensity and fast-paced battle requirements of modern sea battle can be better met.

Description

Technical Field

[0001] The present invention relates to the field of ship display and control technology, and in particular to an integrated ship display and control console equipped with VR glasses, which is used to enhance the combat command and control capabilities of ship operators. Background Art

[0002] As the center of battlefield command and control and human-computer interaction, the display and operation of the ship's display and control console, as well as the sending of accurate instructions through it, are crucial. In modern ship operation scenarios, the display interface in the ship's display and control console needs to present more graphics, and the operators need to have more image and text information, and the requirements for the fineness of the picture and the display quality are getting higher and higher. Even before and after the ship sets off, the ship's personnel will make corresponding preparations for the navigation mission, such as: arranging the deployment of ship personnel, mission simulation, demonstration of the ship's dynamic operation status, and simulation of the launch equipment performance on the display and control console, etc. These all need to be provided through three-dimensional display to provide multiple perspectives for the ship's person in charge to watch and evaluate. Therefore, whether the ship's display and control console can provide more flexible and efficient display processing technology is the future development trend of the ship's display and control industry. Figure 1 This is a schematic diagram of the application scenario of the traditional ship display and control console. This display method uses traditional keyboards, touch balls and other devices to achieve human-computer interaction under two-hand operation. This method cannot meet the requirements of modern warfare for efficient and multi-screen control. In addition, the display of traditional ship display and control consoles generally relies on integrated flat-panel display arrays. When facing the high-intensity and fast-paced challenges of modern naval warfare, this design method gradually reveals limitations such as insufficient information visualization, limited operation interaction, and low space utilization efficiency.

[0003] With the continuous development of technology, digital twin technology has been favored by the public. Digital twin is a technology that makes full use of physical models, sensor updates, operation history and other data, integrates multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, completes mapping in virtual space, and thus reflects the full life cycle of the corresponding physical equipment. Summary of the invention

[0004] In view of this, the present invention discloses an integrated ship display and control console equipped with VR glasses. The integrated ship display and control console displays ship data output by an external sensor unit through a first display area and a second display area, which can provide ship operators with more display screen options, and combine with a control unit to control the image in the display area, thereby getting rid of the limitations of traditional monitoring and operation relying on flat-panel displays, and better meeting the high-intensity and fast-paced combat requirements of modern naval warfare.

[0005] The integrated ship display and control console includes: a display unit, a control unit, a display and control computer, a power supply unit and an external sensor unit;

[0006] The ship data output from the external sensor unit is transmitted to the display unit through the display control computer; the display unit includes a display and VR glasses located on the ship display control console, the display provides a first display area, and the VR glasses provide a second display area; the display and the VR glasses are electrically connected to the display control computer;

[0007] The power supply unit is used to provide power to the display and control computer, the control unit and the display unit;

[0008] The control unit controls the ship by controlling the image in the first display area or the second display area.

[0009] Preferably, the first display area is used to display information such as ship navigation;

[0010] The second display area is used to display a 360° digital twin scene of ship control including the ship display and control console;

[0011] The display and control computer is also used to produce the 360° ship control digital twin scene;

[0012] The second display area also displays a navigation menu, ship status, and operation prompt images generated by the display and control computer.

[0013] Preferably, the VR glasses are also provided with the interactive control unit;

[0014] The interactive control unit includes an inertial measurement module, a head tracking module, a gesture recognition module, a speech recognition module and a high-precision tracking solution module;

[0015] The inertial measurement module and the head tracking module are used to track and measure the head position, and transmit the acquired data to the high-precision tracking solution module for solution and then send it to the database of the display and control computer for analyzing and collecting the head data of the operator wearing VR glasses to control the ship;

[0016] The gesture recognition module is used to capture the hand movements of the ship operator and transmit the acquired data to the database of the display and control computer after the high-precision tracking and solving module solves the data;

[0017] The speech recognition module is used to receive the speech information of the operator, and transmit the acquired language data to the database of the display and control computer after being solved by the high-precision tracking and solving module;

[0018] The display control computer generates the operation prompt image using the information collected and analyzed in the database, and transmits it to the second display area of ​​the VR glasses.

[0019] Preferably, the VR glasses are also provided with a camera unit, a sensor unit, an external interface unit and a data processing unit;

[0020] The camera unit is used to collect information about people around the VR glasses, and map the collected information to the 360° ship control digital twin scene in the second display area;

[0021] The sensor unit is used to collect motion data of the VR glasses wearer during use, and transmit the acquired motion data to the high-precision tracking and solving module, and finally update the operation prompt image in the second display area in real time through the data processing unit.

[0022] Preferably, the operation prompt image displays a video of a human character's action that serves as a guide, prompting the operator to correctly perform the ship's operation actions under the sensor data and task instructions transmitted by the corresponding sensor network.

[0023] Preferably, the real-time virtual image displayed on the display on the ship display console in the 360° ship control digital twin scene includes the ship navigation information displayed in the first display area;

[0024] The virtual image also includes target strike data information of equipment related to the weapon system.

[0025] Preferably, the control unit includes a keyboard, a mouse and a joystick.

[0026] Beneficial effects:

[0027] (i) The display on the integrated ship display and control console of the present invention and the display area in the VR glasses can display the ship data output by the external sensor unit, and can provide the ship operator with more display screen options. In combination with the control unit, the image in the display area can be controlled and operated. The integrated ship display and control console gets rid of the limitation of traditional monitoring and operation relying on flat-panel displays, and better meets the high-intensity and fast-paced combat requirements of modern naval warfare.

[0028] (ii) The display area in the VR glasses of the present invention serves as the second display area, which can not only display the 360° digital twin scene of ship control including the ship display and control console, but also generate navigation menus, ship status and operation prompt images through the display and control computer; the operator monitors the ship navigation information in the first display area in real time through the 360° digital twin scene of ship control in the VR glasses, and at the same time obtains and controls the navigation menus, ship status and operation prompt images through the control unit, so as to realize fast and efficient monitoring and operation.

[0029] (III) The VR glasses of the present invention are also provided with an interactive control unit, which includes an inertial measurement module, a head tracking module, a gesture recognition module, a voice recognition module and a high-precision tracking and solving module. The inertial measurement module, the head tracking module, the gesture recognition module and the voice recognition module transmit the acquired data to the high-precision tracking and solving module for solving, and then send it to the database of the display and control computer. The display and control computer generates an operation prompt image based on the information collected and analyzed in the database, and transmits it to the second display area of ​​the VR glasses. The operation prompt image displays a video of the action of the character with a guiding prompt function, prompting the operator to correctly perform the ship operation action under the sensor data and task instructions transmitted by the corresponding sensor network.

[0030] (iv) The VR glasses of the present invention are also provided with a sensor unit for collecting motion data of the wearer of the VR glasses during use, transmitting the acquired motion data to a high-precision tracking and solving module, and finally updating the operation prompt image in the second display area in real time through the data processing unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of an application scenario of a traditional display console in the background technology of the present invention;

[0032] Figure 2 Construct a flow chart for the 360° ship control digital twin scenario of the present invention;

[0033] Figure 3 This is a schematic diagram of an application scenario of the integrated ship display and control console equipped with VR glasses of the present invention;

[0034] Figure 4 This is a schematic diagram of the components of the integrated ship display and control console equipped with VR glasses of the present invention; DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 2-4As shown, the present invention provides an integrated ship display and control console equipped with VR glasses, which includes a display unit, a control unit, a display and control computer, a power supply unit and an external sensor unit. The external sensor unit includes a navigation device, a weapon system, and a sensor network; the navigation device is used to provide the display and control computer with navigation information of the ship; the weapon system is used to provide the display and control computer with target strike data of the weapon system-related equipment; the sensor network is a network that transmits the sensor data on the ship that can be presented by the display unit, that is, a network system for transmitting sensor data to the display and control computer; the data output from the navigation device, the weapon system and the sensor network are collectively referred to as ship data, and the ship data is transmitted to the display unit through the database of the display and control computer.

[0037] The display unit includes a display and VR glasses located on the display and control console of the ship; the display and VR glasses are electrically connected to the display and control computer. Specifically, two interface connection methods, USB and HDMI, are used between the display and the display and control computer. The HDMI interface is responsible for transmitting high-definition video signals to ensure that the image displayed on the display is clear and delicate; the USB interface is used to transmit control signals and other data to achieve precise control of the display by the display and control computer; the VR glasses are connected to the display and control computer using a USB interface. This connection method enables the VR glasses to receive three-dimensional images and video data transmitted by the display and control computer in real time, providing operators with a realistic virtual reality experience. At the same time, the USB interface is also used to transmit control signals to achieve precise control of the VR glasses by the display and control computer, such as adjusting the viewing angle, switching scenes, etc. The display and control computer is the core processing unit of the ship's display and control console. Its internal architecture is based on the standard CPCI Express bus. It is responsible for producing and transmitting the content displayed on the display and VR glasses, including the construction of a 360° digital twin scene for ship control and sensor data transmitted by the sensor network (including: ship navigation information issued by navigation equipment, target strike data of weapon system-related equipment). The 360° digital twin scene for ship control completely restores the control room environment where the ship's display and control console is located in a 1:1 true manner, including the ship display console and displays placed on the display console that are the same in appearance and quantity as the real scene, and the display can display sensor data transmitted from the sensor network.

[0038] The power supply unit is used to provide power for the display and control computer, the control unit and the display unit; the control unit includes a keyboard, a mouse and a joystick. The control unit is connected to the display and control computer through a USB interface to realize data interaction and command transmission. This connection method has the characteristics of fast transmission speed and good stability, so that the control unit can control the ship by manipulating the display area image in the display unit. The display area includes a first display area provided on the display and a second display area provided on the VR glasses; wherein the first display area is used to display ship navigation information; the second display area is not only used to display the 360° ship control digital twin scene including the ship display and control console and the sensor data transmitted by the sensor network, but also includes the navigation menu and operation prompt image generated by the display and control computer, so that the operator can monitor the 360° ship control digital twin scene presented in the second display area and the ship navigation information in the first display area of ​​the display in the digital twin environment through VR glasses and the control unit. The virtual image displayed on the ship console in the 360° ship control digital twin scene can not only be synchronized with the ship navigation information displayed in the first display area, but also can selectively display other ship data through the control unit, such as ship navigation information sent by the navigation equipment, target strike data of weapon system-related equipment, etc.

[0039] like Figure 2 As shown in the figure, the construction process of the 360° ship control digital twin scene includes the following steps:

[0040] S1, collect relevant information about the control room environment where the ship's display and control console is located, including the indoor environment of the control room, the control console, and the display on the control console;

[0041] S2, using Unity3D engine to create a three-dimensional virtual ship control room environment model where the display console is located, to increase the authenticity of the scene as much as possible;

[0042] S3, build a 3D model of the integrated ship display console and the display placed above it (output .fbx format file), and import the model into the Unity3D engine, and ensure that the model visually accurately reflects its actual appearance and structure;

[0043] S4, designing a data interface for receiving sensor data transmitted by the sensor network and displaying it in the second display area;

[0044] S5, design the human-computer interaction function, write the interaction logic in Unity3D, so that the user can view the control room environment where the ship display console is located through the second display area on the VR glasses, and control the content of the second display area through the control unit, and view the information displayed in the first display area through the second display area (so that the operator can not only see the virtual 360° ship control digital twin scene through the VR glasses, but also can quickly view the display information of the display in the virtual scene that is synchronized with the reality in real time without taking off the VR glasses, thereby improving the efficiency of ship mission execution);

[0045] S6, the navigation menu and operation prompt image generated by the display control computer are transmitted to the second display area through the display control computer, and the content in the second display area is efficiently manipulated through the display control unit. (The information displayed by the virtual display in the second display area includes the ship navigation information sent by the navigation device and the target strike data information of the weapon system related equipment; the second display area can also display the navigation menu and the operation prompt image).

[0046] As for VR glasses, they are equipped with an interactive control unit including an inertial measurement module, a head tracking module, a gesture recognition module, a voice recognition module and a high-precision tracking and solving module. The inertial measurement module and the head tracking module are used to track and measure the head position, and transmit the acquired data to the database of the display and control computer after the high-precision tracking and solving module is solved, so as to analyze and collect the head data of the operator when wearing VR glasses to control the ship; the gesture recognition module is used to capture the hand movements of the ship operator, and transmit the acquired data to the database of the display and control computer after the high-precision tracking and solving module is solved; the voice recognition module is used to receive the voice information of the operator, and transmit the acquired language data to the database of the display and control computer after the high-precision tracking and solving module is solved. The display and control computer generates an operation prompt image based on the information collected and analyzed in the database, and transmits the generated operation prompt image to the second display area of ​​the VR glasses. The operation prompt image displays a video of the action of the character with a guiding prompt function, prompting the operator to correctly perform the ship operation action under the sensor data and task instructions transmitted by the corresponding sensor network.

[0047] In addition, the VR glasses are also provided with a camera unit, a sensor unit, an external interface unit and a data processing unit; the camera unit is used to collect information about people around the VR glasses, and map the collected information to the 360° ship control digital twin scene in the second display area, increasing the authenticity of the 360° ship control digital twin scene. The sensor unit is used to collect the motion data of the wearer of the VR glasses during use, and transmit the acquired motion data to the high-precision tracking and solving module, and finally update the operation prompt image in the second display area in real time through the data processing unit, thereby improving the control efficiency of the ship operator. The external interface on the VR glasses includes a data interface and a power interface. The data interface receives content generated by the display and control computer (ship navigation information issued by the navigation device, target strike data of weapon system-related equipment, navigation menu and operation prompt image).

[0048] As an example, the sensor unit includes a gyroscope and an accelerometer.

[0049] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An integrated ship display and control console equipped with VR glasses, characterized in that: It includes a display unit, a control unit, a display and control computer, a power supply unit and an external sensor unit; The ship data output from the external sensor unit is transmitted to the display unit through the display control computer; the display unit includes a display and VR glasses located on the ship display control console, the display provides a first display area, and the VR glasses provide a second display area; the display and the VR glasses are electrically connected to the display control computer; The power supply unit is used to provide power to the display and control computer, the control unit and the display unit; The control unit controls the ship by controlling the image in the first display area or the second display area.

2. The integrated ship display and control console equipped with VR glasses according to claim 1, characterized in that: The first display area is used to display ship navigation information; The second display area is used to display a 360° digital twin scene of ship control including the ship display and control console; The display and control computer is also used to generate the 360° ship control digital twin scene; The second display area also displays a navigation menu, ship status, and operation prompt images generated by the display and control computer.

3. The integrated ship display and control console equipped with VR glasses according to claim 2, characterized in that: The VR glasses are also provided with the interactive control unit. The interactive control unit includes an inertial measurement module, a head tracking module, a gesture recognition module, a voice recognition module and a high-precision tracking solution module. The inertial measurement module and the head tracking module are used to track and measure the head position, and transmit the acquired data to the high-precision tracking solution module for solution and then send it to the database of the display and control computer for analyzing and collecting the head data of the operator wearing VR glasses to control the ship; The gesture recognition module is used to capture the hand movements of the ship operator and transmit the acquired data to the database of the display and control computer after the high-precision tracking and solving module solves the data; The speech recognition module is used to receive the speech information of the operator, and transmit the acquired language data to the database of the display and control computer after being solved by the high-precision tracking and solving module; The display control computer generates the operation prompt image using the information collected and analyzed in the database, and transmits it to the second display area of ​​the VR glasses.

4. The integrated ship display and control console equipped with VR glasses according to claim 3, characterized in that: The VR glasses are also provided with a camera unit, a sensor unit, an external interface unit and a data processing unit; The camera unit is used to collect information about people around the VR glasses, and map the collected information to the 360° ship control digital twin scene in the second display area; The sensor unit is used to collect motion data of the VR glasses wearer during use, and transmit the acquired motion data to the high-precision tracking and solving module, and finally update the operation prompt image in the second display area in real time through the data processing unit.

5. The integrated ship display and control console equipped with VR glasses according to claim 2, characterized in that: The operation prompt image displays a video of a character's action with a guiding prompting function, prompting the operator to correctly perform the ship operation action under the sensor data and task instructions transmitted by the corresponding sensor network.

6. An integrated ship display and control console equipped with VR glasses according to any one of claims 2 to 5, characterized in that: The real-time virtual image displayed on the display on the ship display console in the 360° ship control digital twin scene includes the ship navigation information displayed in the first display area; The virtual image also includes target strike data information of equipment related to the weapon system.

7. An integrated ship display and control console equipped with VR glasses according to any one of claims 2 to 5, characterized in that: The control unit includes a keyboard, a mouse and a joystick.

Images