Marine augmented reality system and application method thereof

Abstract

The invention relates to the technical field of integrated ship bridge systems (IBS), in particular to a marine augmented reality system and an application method thereof. Through reasonable arrangement of the display control circuit, the mounting bracket, the camera device, the combined regulator, the optical coupling mirror, the high-brightness display and the reflective projection, and in combination with an information fusion algorithm and a target tracking algorithm, an effect of overlapping display of virtual information and an external target can be realized, so that a ship command operator pays attention to external environment information, and meanwhile, the ship command operator can control the external environment. The ship digital information with enhanced information is obtained, so that the perception ability of ship commanding and operating personnel to the real environment and the information acquisition efficiency are effectively improved; according to the invention, the installation space requirement of the navigation guarantee equipment for the cab centralized control console is effectively reduced, the few-person design of the cab centralized control console is facilitated, and meanwhile, a comfortable and efficient navigation guarantee equipment centralized control interface can be provided for ship command and operation personnel, so that the ship navigation safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated ship bridge systems (IBS), and in particular to a ship-based augmented reality system and an application method thereof. Background Art

[0002] The integrated bridge system is an integrated system that integrates multiple functions such as navigation, driving control, centralized information display, alarm monitoring, communication, and navigation management. It is used to improve the safety of ship navigation and efficient management. The system originated in the 1970s and has become the core equipment of ship automation and has been widely used. The important feature of the integrated bridge system is information integration. It uses computer technology, information processing technology, automation technology, human-machine engineering, etc. to centrally display navigation, propulsion, electronic charts, radar, video, safety alarms and other information through display and control equipment to improve the efficiency and reliability of operators' information acquisition. As an important part of the integrated bridge system, the display and control equipment is the interactive window between the ship operator and the ship system. The rationality and effect of its interface design directly affect the efficiency of operators' information acquisition, thereby affecting navigation safety.

[0003] The display and control equipment of the existing integrated bridge system is usually a screen display method such as a computer or an all-in-one machine. This method has the following disadvantages: 1) The operator needs to match the screen display target with the external actual scene target, which reduces the efficiency of information acquisition; 2) The operator needs to pay attention to the information displayed on the screen and the external actual environment, which can easily cause operator fatigue and increase navigation risks. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a ship-based augmented reality system and an application method thereof, which can enable the commanding and operating personnel to pay attention to the external environment information while obtaining information-enhanced digital information of the ship, thereby effectively improving the commanding and operating personnel's perception of the real environment and the efficiency of information acquisition, and ensuring the safety of ship navigation.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] 1. A marine augmented reality system

[0007] The present invention provides a marine augmented reality system, comprising a display control circuit, a mounting bracket, a camera device, a combined regulator, an optical coupling mirror, a high-brightness display and a reflective projection device;

[0008] The display control circuit is used to obtain the navigation virtual information of the navigation target and process and fuse the information; the camera device is used to obtain external environment information; the reflective projection device is used to reflect and amplify the digital information displayed by the highlight display; the mounting bracket is used to install the augmented reality system to the corresponding position of the ship's control console seat; the combined adjuster is used to connect the optical coupling mirror and the mounting bracket, and adjust the angle of the optical coupling mirror according to the observer's eye position.

[0009] Furthermore, the display and control circuit exchanges information with the ship's integrated navigation system through a network to obtain the direction, distance, speed, heading, and navigation time information of the navigation target.

[0010] Furthermore, the display control circuit performs information fusion and spatial coordinate transformation on the virtual information of the navigation target, generates digital information superimposed with the external environment information, and outputs the digital information in HDMI format to the high-brightness display for display.

[0011] Furthermore, the optical coupling mirror is processed through optical correction and reflected light analysis, and a light-transmitting anti-reflection film is applied on the observer side to reflect the light emitted by the reflective projection device and focus it into the observer's eyes, so as to increase the sensory distance of the digital information for the observer.

[0012] Furthermore, the reflective projection device is composed of a reflective, magnifying, and reflective mirror surface, and is used to reflect, magnify, and project the digital information displayed by the highlight display onto the optical coupling mirror.

[0013] Furthermore, the high brightness display comprises: a liquid crystal box assembly, a backlight assembly, a circuit assembly and a structural assembly;

[0014] The high-brightness display is specifically an LED display, and its backlight adopts a pure green high-light-efficiency LED lamp for displaying digital information.

[0015] Furthermore, the mounting bracket is formed by one-step processing of the whole material.

[0016] 2. Application method of a marine augmented reality system

[0017] Based on the same inventive concept, the present invention also provides an application method of the above-mentioned marine augmented reality system, which specifically comprises the following steps:

[0018] S1, exchanging information with a ship integrated navigation system through a network to obtain virtual navigation information of a navigation target, wherein the virtual navigation information includes the position, distance, speed, heading, and navigation time information of the navigation target;

[0019] S2, fusing the navigation virtual information of the navigation target with the external environment information captured by the camera device through an information fusion algorithm;

[0020] S3, obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror through a target tracking algorithm, so as to realize the superimposed display of the virtual information and the external environment information.

[0021] Among them, the target tracking algorithm includes: obtaining the azimuth, distance, and speed information of the external target relative to the observer, converting the target coordinate information into a virtual coordinate system through spatial transformation, thereby obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror.

[0022] Compared with the prior art, the present invention has the following main advantages:

[0023] 1. The present invention can realize the effect of superimposed display of virtual information and external targets through reasonable arrangement of display control circuit, mounting bracket, camera device, combined regulator, optical coupling mirror, high brightness display and reflective projection, combined with information fusion algorithm and target tracking algorithm, so that the commanding and operating personnel can obtain information-enhanced digital information of the ship while paying attention to the external environment information, thereby effectively improving the commanding and operating personnel's perception ability of the real environment and information acquisition efficiency;

[0024] 2. The present invention not only effectively reduces the installation space requirement of the navigation support equipment for the centralized control console in the cab, facilitating the less-manned design of the centralized control console in the cab, but also can provide a comfortable and efficient centralized control interface for the navigation support equipment for the commanding personnel, thereby ensuring the safety of the ship's navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the composition of a marine augmented reality system in an embodiment of the present invention;

[0026] Figure 2 It is a schematic diagram of the principle of a marine augmented reality system in an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of optical imaging of a marine augmented reality system in an embodiment of the present invention;

[0028] Figure 4 Schematic diagram of the composition of a highlight display in an embodiment of the present invention;

[0029] Figure 5 It is a structural schematic diagram of a mounting bracket in an embodiment of the present invention;

[0030] Figure 6 Schematic diagram of the structure of the combined regulator in an embodiment of the present invention.

[0031] In the figure: 11-display control circuit, 12-mounting bracket, 13-camera device, 14-combination regulator, 15-optical coupling mirror, 16-high brightness display, 17-reflection projection device. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0033] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0034] Embodiment 1: This embodiment provides a marine augmented reality system, such as Figure 1 As shown, it mainly includes a display control circuit 11, a mounting bracket 12, a camera device 13, a combined regulator 14, an optical coupling mirror 15, a high-brightness display 16 and a reflective projection device 17;

[0035] Furthermore, the display control circuit 11 is used for information acquisition, processing and display of the system; the display control circuit is implemented by a computer, which can obtain navigation radar, AIS, video and other information through network communication for processing, fusion and spatial coordinate transformation, generate digital information superimposed with external targets, and output video information in HDMI format to send to the high-brightness display for display.

[0036] Furthermore, the mounting bracket 12 and the combined adjuster 14 are mainly used for fixing and installing the equipment, wherein the combined adjuster 14 can adjust the angle of the optical coupling mirror according to the position of the observer's eyes.

[0037] Furthermore, the camera device 13 is used to capture external target information and is combined with a target tracking algorithm.

[0038] like Figures 2-3 As shown, the optical coupling mirror 15 is made of glass, and a light-transmitting anti-reflection film is applied on the observer side. The device is processed after optical correction and reflected light analysis, and can converge the reflected projected light into the observer's eyes in a manner similar to parallel lines, so that the electronic information sensory distance obtained by the observer is infinite;

[0039] The optical coupling mirror is a device for enhancing transmittance and reflection, which can reflect the light projected by reflection and transmit external target information, so that the observer can see the external information and reflect the projected digital information.

[0040] Furthermore, the reflective projection device 17 is composed of a reflective and magnifying reflective mirror surface, which is used to reflect, magnify and project the image of the highlight display onto the optical coupling mirror;

[0041] The reflective projection device can reflect and amplify the highlighted information, thereby increasing the sensory distance between the digital image and the observer, and can also amplify the picture of the highlighted display and project it onto the optical coupling mirror.

[0042] like Figure 4 As shown, the high-brightness display is mainly composed of four components, namely: liquid crystal box component, backlight component, circuit component and structural component; among them: the liquid crystal box component mainly has functions such as graphic display, liquid crystal panel protection, and low-temperature heating; the backlight component provides uniform and high-brightness backlight for the liquid crystal screen. The main functions of the circuit component are backlight driving and adjustment, external communication, low-temperature heating control, etc. Structural component: mainly composed of panel, backlight frame, back cover, anti-slip screws, etc., used to support, install and protect other components.

[0043] The highlight display adopts an LED display, and its backlight adopts a pure green high-efficiency LED light to display digital information.

[0044] The high-brightness display uses a single green light source for display, thereby increasing the intensity of light reflected by the optical coupling mirror, making it easier for observers to see digital information in a bright environment.

[0045] like Figure 5 As shown, the mounting bracket is formed by one-step processing of the whole material, and the three external mounting holes on the upper part of the mounting bracket are installed on the seat of the driving console;

[0046] like Figure 6 As shown, the combined regulator 14 is used to connect the optical coupling mirror 15 and the mounting bracket 12 , and can adjust the position and angle of the optical coupling mirror 15 relative to the mounting bracket 12 .

[0047] Embodiment 2: Based on the same inventive concept, this embodiment also provides an application method of the above-mentioned marine augmented reality system, which specifically includes the following steps:

[0048] S1, exchanging information with a ship integrated navigation system through a network to obtain virtual navigation information of a navigation target, wherein the virtual navigation information includes the position, distance, speed, heading, and navigation time information of the navigation target;

[0049] S2, fusing the navigation virtual information of the navigation target with the external environment information captured by the camera device through an information fusion algorithm;

[0050] S3, obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror through a target tracking algorithm, so as to realize the superimposed display of the virtual information and the external environment information.

[0051] The above-mentioned target tracking algorithm integrates navigation radar information, AIS information, and video equipment information. It obtains the azimuth, distance, speed and other information of the external target relative to the observer, and then converts the target coordinate information into a virtual coordinate system through spatial transformation, thereby obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror, so as to realize the superimposed display of virtual information and external target information.

[0052] Furthermore, all parts involved in this application that are not described in detail are the same as the prior art or are implemented using the prior art.

[0053] In summary:

[0054] 1. The present invention can realize the effect of superimposed display of virtual information and external targets through reasonable arrangement of display control circuit, mounting bracket, camera device, combined regulator, optical coupling mirror, high brightness display and reflective projection, combined with information fusion algorithm and target tracking algorithm, so that the commanding and operating personnel can obtain information-enhanced digital information of the ship while paying attention to the external environment information, thereby effectively improving the commanding and operating personnel's perception ability of the real environment and information acquisition efficiency;

[0055] 2. The present invention not only effectively reduces the installation space requirement of the navigation support equipment for the centralized control console in the cab, facilitating the less-manned design of the centralized control console in the cab, but also can provide a comfortable and efficient centralized control interface for the navigation support equipment for the commanding personnel, thereby ensuring the safety of the ship's navigation.

[0056] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A marine augmented reality system, characterized in that: It includes display control circuit, mounting bracket, camera device, combined regulator, optical coupling mirror, high brightness display and reflection projection device; The display control circuit is used to obtain the navigation virtual information of the navigation target and process and fuse the information; the camera device is used to obtain external environment information; the reflective projection device is used to reflect and amplify the digital information displayed by the highlight display; the mounting bracket is used to install the augmented reality system to the corresponding position of the ship's control console seat; the combined adjuster is used to connect the optical coupling mirror and the mounting bracket, and adjust the angle of the optical coupling mirror according to the observer's eye position.

2. A marine augmented reality system according to claim 1, characterized in that: The display and control circuit exchanges information with the ship integrated navigation system through the network to obtain the position, distance, speed, course and navigation time information of the navigation target.

3. A marine augmented reality system according to claim 2, characterized in that: The display control circuit performs information fusion and spatial coordinate transformation on the virtual information of the navigation target, generates digital information superimposed with the external environment information, and outputs the digital information in HDMI format to the high-brightness display for display.

4. The marine augmented reality system according to claim 1, characterized in that: The optical coupling mirror is processed through optical correction and reflected light analysis, and a light-transmitting anti-reflection film is applied on the observer side to reflect the light emitted by the reflective projection device and focus it into the observer's eyes, so as to increase the sensory distance of the digital information for the observer.

5. The marine augmented reality system according to claim 1, characterized in that: The reflective projection device is composed of reflective, magnifying and reflective mirrors, and is used to reflect, magnify and project the digital information displayed by the highlight display onto the optical coupling mirror.

6. The marine augmented reality system according to claim 1, characterized in that: The high brightness display comprises: a liquid crystal box assembly, a backlight assembly, a circuit assembly and a structural assembly; The high-brightness display is specifically an LED display, and its backlight adopts a pure green high-light-efficiency LED lamp for displaying digital information.

7. The marine augmented reality system according to claim 1, characterized in that: The mounting bracket is formed by one-step processing of the whole material.

8. An application method of the marine augmented reality system according to any one of claims 1 to 7, characterized in that: The steps include: S1, exchanging information with a ship integrated navigation system through a network to obtain virtual navigation information of a navigation target, wherein the virtual navigation information includes the position, distance, speed, heading, and navigation time information of the navigation target; S2, fusing the navigation virtual information of the navigation target with the external environment information captured by the camera device through an information fusion algorithm; S3, obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror through a target tracking algorithm, so as to realize the superimposed display of the virtual information and the external environment information.

9. The application method according to claim 8, characterized in that: The target tracking algorithm includes: obtaining the azimuth, distance, and speed information of the external target relative to the observer, converting the target coordinate information into a virtual coordinate system through spatial transformation, thereby obtaining the display position of the navigation virtual information of the navigation target on the optical coupling mirror.

Images