Ship auxiliary system based on augmented reality technology

Abstract

The invention discloses a ship auxiliary system based on augmented reality technology, and relates to the crossing field of ship technology and augmented reality technology, and the system comprises AR display equipment which is used for collecting image information, sensing head posture and motion information, and carrying out the fusion display of virtual information and a real scene; the data processing equipment is used for receiving and processing data acquired by the AR display equipment and ship operation data and operating an image rendering algorithm and a system function program; the communication equipment is used for realizing data transmission and communication among the AR display equipment, the data processing equipment and other equipment on the ship; and the software system comprises a sailing auxiliary module, a maintenance auxiliary module and a management auxiliary module, integrates various information in ship sailing, maintenance and management processes through an AR technology, presents the information to sailors and managers in a visual and visual manner, and improves the safety and efficiency of ship sailing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ship technology and augmented reality technology, and particularly relates to a ship auxiliary system based on augmented reality technology. BACKGROUND

[0002] In the field of ships, there are many limitations in the traditional ship navigation, maintenance and management. In terms of navigation, the crew mainly relies on the instrument panel, radar display screen and other equipment on the ship to obtain the ship navigation state and surrounding environment information. These information are scattered and displayed, and the crew needs to frequently switch the view to check, which is not only low in efficiency, but also easy to cause judgment error due to untimely information integration, and there is a safety hazard. In terms of ship maintenance, when the ship equipment fails, the maintenance personnel often need to troubleshoot and repair according to the maintenance manual and their own experience. For complex fault conditions, the maintenance is difficult and time-consuming, and the technical level of the maintenance personnel is uneven, which may lead to incomplete maintenance and affect the normal operation of the ship. In terms of ship management, the management personnel cannot intuitively understand the real-time running state of the internal structure and equipment of the ship, and lack comprehensive and accurate data support for the maintenance and upgrading decision of the ship, which is not conducive to the optimization of ship performance and cost control. With the continuous development of augmented reality technology, it has shown great application potential in many fields. Applying AR technology to the field of ships is expected to solve the problems existing in the traditional way and bring a new solution to the navigation, maintenance and management of ships. However, the current AR technology is not perfect for the application of AR technology to ships and the formation of a complete auxiliary system, and we propose a ship auxiliary system based on augmented reality technology. SUMMARY

[0003] An object of the present application is to solve at least the above problems and to provide at least the advantages described later.

[0004] Another object of the present application is to provide a ship auxiliary system based on augmented reality technology, which integrates various information in the process of ship navigation, maintenance and management through AR technology, and presents it to the crew and management personnel in an intuitive and visual way, improves the safety and efficiency of ship navigation, improves the accuracy and timeliness of ship maintenance, and provides comprehensive data support and decision basis for ship management.

[0005] To achieve the above objects and some other objects, the present application adopts the following technical solutions: A ship auxiliary system based on augmented reality technology, comprising an AR display device for collecting image information, sensing head posture and motion information, and displaying virtual information and real scene fusion; a data processing device for receiving and processing the data collected by the AR display device and the ship running data, running image rendering algorithm and system function program; a communication device for realizing data transmission and communication between the AR display device, the data processing device and other devices on the ship; a software system including a navigation assistance module, a maintenance assistance module and a management assistance module.

[0006] Preferably, the AR display device is equipped with a high-definition display screen for displaying the fused information, a high-resolution camera for capturing images of the environment around the ship and internal equipment, a gyroscope and an accelerometer for sensing head posture and motion information.

[0007] Preferably, the data processing device includes a high-performance computer for processing data and running programs, and a data storage unit for storing ship navigation data, equipment parameters, maintenance manuals and virtual model information.

[0008] Preferably, the communication device includes a wireless network module supporting Bluetooth and Wi-Fi wireless communication protocols, and a wired network interface for connecting to the ship's local area network.

[0009] Preferably, the navigation assistance module receives navigation data from the ship's navigation system and environmental images captured by the AR display device, and displays the ship's real-time navigation state, path, and surrounding obstacles and other ship information on the AR display device in the form of virtual markers and arrow indicators through image recognition and spatial positioning technology.

[0010] Preferably, the maintenance assistance module pre-stores three-dimensional virtual models of ship equipment, maintenance manuals and fault codes, and when a device fails, it identifies the fault through AR display device scanning and provides troubleshooting steps, maintenance operation guidance and tool lists in the form of animation demonstration, text explanation and voice prompt.

[0011] Preferably, the maintenance assistance module is also used to simulate the occurrence of equipment failure and record the time, content and replacement parts information of each maintenance, forming an equipment maintenance file.

[0012] Preferably, the management assistance module allows management personnel to view the three-dimensional virtual model of the overall structure and internal equipment layout of the ship through the AR display device, and displays the equipment running state and performance parameters in real time, supporting remote monitoring and management of equipment.

[0013] Preferably, the management assistance module generates maintenance plans and upgrade scheme suggestions based on the service life and operating condition of the equipment, and simulates the impact of different upgrade schemes on the performance of the ship.

[0014] Preferably, the navigation assistance module, the maintenance assistance module and the management assistance module interact with other systems of the ship through the communication device.

[0015] The present application at least includes the following advantages: 1. Compared with the existing ship navigation assistance system, the ship assistance system based on augmented reality technology can intuitively display the ship navigation information and the surrounding environment through AR technology, so that the crew can quickly and accurately obtain information and make timely decisions, thereby reducing the risk of accidents caused by delayed or incorrect information processing, optimizing the navigation path and improving the navigation efficiency.

[0016] 2. Compared with the existing ship navigation assistance system, the ship assistance system based on augmented reality technology can reduce the difficulty of maintenance, improve the accuracy and timeliness of maintenance, reduce the downtime of the ship, prolong the service life of the equipment and reduce the maintenance cost through the visual maintenance guidance and fault simulation training provided by the maintenance assistance module.

[0017] 3. Compared with the existing ship navigation assistance system, the ship assistance system based on augmented reality technology can help the management personnel to comprehensively and intuitively understand the ship structure and equipment operation, realize remote monitoring and scientific decision-making, optimize the ship maintenance and upgrading scheme, and improve the intelligent level of ship management and overall operation efficiency through the management assistance module.

[0018] Other advantages, objects and features of the present application will be partly embodied in the following description, and partly understood by those skilled in the art through research and practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The system schematic diagram of the present application. DETAILED DESCRIPTION

[0020] The present application will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can implement it after referring to the present description.

[0021] As Figure 1 shown, a ship assistance system based on augmented reality technology includes an AR display device for collecting image information, sensing head posture and motion information, and displaying virtual information and real scene; a data processing device for receiving and processing data collected by the AR display device and ship operation data, running image rendering algorithm and system function program; a communication device for realizing data transmission and communication between the AR display device, the data processing device and other devices on the ship; a software system including a navigation assistance module, a maintenance assistance module and a management assistance module.

[0022] In the above scheme, through the cooperation of multiple devices and modules, ship operation related data is converted into intuitive visual information to assist crew members and management personnel in their work. Such design makes the information acquisition and processing of each link of ship operation more efficient and intelligent, and significantly improves the overall operation level of the ship.

[0023] In a preferred scheme, the AR display device is equipped with a high-definition display screen, a high-resolution camera, a gyroscope and an accelerometer. The high-definition display screen is used to display the fused information. The high-resolution camera is used to collect images of the environment around the ship and internal equipment. The gyroscope and the accelerometer are used to perceive head posture and motion information.

[0024] In the above scheme, after these information is transmitted to the data processing device, it is integrated with the ship operation data, processed by image rendering algorithm, and a picture of virtual information fused with real scene is generated, which is displayed to the user through the high-definition display screen. The crew members or management personnel can obtain high-definition and accurate environment and ship state information at any time, quickly master the situation around and inside the ship, without the need to frequently check different devices, greatly improving the information acquisition efficiency, reducing the operation errors caused by information lag or inaccuracy, and improving the safety and accuracy of ship operation.

[0025] In a preferred scheme, the data processing device includes a high-performance computer and a data storage unit. The high-performance computer is used to process data and run programs. The data storage unit is used to store ship navigation data, equipment parameters, maintenance manuals and virtual model information.

[0026] In the above scheme, the high-performance computer receives images, sensor data from the AR display device and operation data of other monitoring devices of the ship, analyzes, calculates and processes the data according to the program algorithm of the ship auxiliary system, and calls related data in the data storage unit. The data storage unit is responsible for storing and managing various important data in the ship operation process, and provides data support for the high-performance computer.

[0027] In a preferred scheme, the communication device includes a wireless network module and a wired network interface. The wireless network module supports Bluetooth and Wi-Fi wireless communication protocols. The wired network interface is used to connect the ship local area network.

[0028] In the above scheme, the wireless network module realizes wireless data transmission between the AR display device and the data processing device through Bluetooth or Wi-Fi protocol, which facilitates the crew members to use the device freely in each area of the ship. The wired network interface connects the data processing device with the navigation system and monitoring system in the ship local area network through stable wired connection, realizing high-speed and stable transmission of a large amount of data.

[0029] In a preferred embodiment, the navigation assistance module receives navigation data from the ship navigation system and environmental images collected by the AR display device, and through image recognition and spatial positioning technology, superimposes and displays the ship's real-time navigation status, path, surrounding obstacles and other ship information on the AR display device in the form of virtual marks and arrow indicators.

[0030] In the above scheme, the navigation assistance module receives real-time data such as the navigation path, speed, and heading from the ship's navigation system, as well as images of the surrounding environment collected by the AR display device. It uses image recognition technology to identify obstacles, other ships and other targets in the environment, and uses spatial positioning technology to determine their position and distance and direction relative to the ship. Combined with the navigation data, it generates intuitive visual information on the AR display device through an image rendering algorithm.

[0031] In a preferred embodiment, the maintenance assistance module pre-stores a three-dimensional virtual model of the ship equipment, a maintenance manual, and a fault code. When the equipment fails, the AR display device scans and identifies the fault, and provides troubleshooting steps, maintenance operation instructions, and a tool list in the form of animation demonstrations, text descriptions, and voice prompts.

[0032] In this solution, when a ship's equipment malfunctions, the crew uses an AR display to scan the equipment. Image recognition technology identifies the equipment model and fault location. The maintenance assistance module then retrieves the corresponding equipment's 3D virtual model, maintenance manual, fault code, and other information from the database. It then provides detailed maintenance guidance through animated simulations of the repair process, text annotations of key operating points, and voice explanations of the steps. This benefit reduces maintenance personnel's reliance on specialized knowledge and experience. Even inexperienced personnel can quickly and accurately troubleshoot and repair faults based on the system's guidance, shortening maintenance time.

[0033] In a preferred embodiment, the maintenance auxiliary module is further used to simulate the process of equipment failure and record the time, content and replacement parts of each maintenance to form an equipment maintenance file.

[0034] In the above scheme, the maintenance assistance module uses a simulation algorithm to dynamically simulate the equipment failure process based on the equipment fault code and operating data. At the same time, after each maintenance is completed, the maintenance-related information is automatically recorded and stored in the data storage unit to form an equipment maintenance file.

[0035] In a preferred embodiment, the management assistance module enables management personnel to view a three-dimensional virtual model of the overall structure of the ship and the internal equipment layout through an AR display device, and displays the equipment operating status and performance parameters in real time, supporting remote monitoring and management of equipment.

[0036] In the above scheme, the management auxiliary module calls the three-dimensional virtual model data of the ship and the equipment operation state parameters from the data storage unit, presents an intuitive three-dimensional model and real-time data on the AR display device through an image rendering algorithm, and the management personnel can send remote operation instructions to the equipment through gesture, voice and other interaction methods. The instructions are transmitted to the corresponding equipment for execution through the communication device.

[0037] In a preferred scheme, the management auxiliary module generates maintenance plans and upgrade scheme suggestions according to the service life and operating conditions of the equipment, and simulates the influence of different upgrade schemes on the performance of the ship.

[0038] In the above scheme, the management auxiliary module generates scientific and reasonable maintenance plans by analyzing the service life, operating time, failure frequency and other data of the equipment, combined with the preset maintenance rules and algorithms; and based on the equipment parameters and the overall performance model of the ship, simulates the changes in performance such as power, fuel consumption and stability of the ship under different upgrade schemes.

[0039] In a preferred scheme, the navigation auxiliary module, the maintenance auxiliary module and the management auxiliary module interact with other systems of the ship through the communication device.

[0040] In the above scheme, each module transmits and receives data with the ship navigation system, monitoring system, control system and other systems through the wireless network module and wired network interface of the communication device, sends the data generated by itself to the related systems, and at the same time acquires the operating data of other systems, so as to realize information sharing and collaborative work.

[0041] Embodiment 1: System application in ship navigation scenario: during the process of the ship sailing from port A to port B, the crew wears AR glasses to start the ship auxiliary system, the high-resolution camera built-in the AR glasses collects the environmental image information of the sea surface, sky and other ships around the ship at a frame rate of 30 frames / second, the gyroscope and accelerometer sense the head posture and motion information of the crew at a frequency of 100Hz, and transmit these data to the data processing device in the ship control room through Bluetooth wireless communication protocol.

[0042] The high-performance computer in the data processing device receives real-time navigation data from the ship navigation system, including navigation path coordinates, current ship speed, and heading, as well as image and sensor data transmitted by the AR glasses. The high-performance computer processes the collected images using image recognition algorithms to identify a cargo ship traveling in the same direction 5 nautical miles ahead, and determines its relative position and speed through spatial positioning technology. Then, combined with electronic chart data, the system uses AR image rendering algorithms to mark the current navigation path of the ship with a yellow arrow on the high-definition display screen of the AR glasses, highlights the identified cargo ship with a red border, and labels information such as distance and speed next to the cargo ship. When the ship approaches the designated shallow danger area, the system displays a red flashing warning icon on the display screen and prompts "approaching shallow danger area ahead, please adjust course" through voice prompts, while displaying the recommended evasive course on the display screen to help the crew safely and efficiently navigate.

[0043] Example 2: System application in the scenario of ship equipment fault maintenance: when the ship's diesel engine fails, causing abnormal power output, the maintenance crew wears an AR helmet to inspect the engine. The high-resolution camera of the AR helmet captures the engine appearance image and transmits it to the data processing device, which quickly identifies the engine model through image recognition technology. The system retrieves the three-dimensional virtual model of the engine, the maintenance manual, and the relevant fault code information from the data storage unit.

[0044] On the display screen of the AR helmet, the internal structure of the engine is gradually displayed in animation, with the suspected fault part highlighted in red. Detailed troubleshooting steps are provided to the maintenance crew in the form of text and voice prompts: "first check if the fuel filter is clogged, use a spanner to unscrew the fixing bolt when disassembling the filter". During the maintenance process, the system simulates the normal operation and fault state of the fuel injection system in real time to help the maintenance crew understand the fault principle. After maintenance is completed, the maintenance assistance module automatically records the maintenance time, maintenance content, and replacement parts information, forming a maintenance file for the engine and storing it in the data storage unit for future query and management.

[0045] Example 3: System application in the scenario of ship management: the ship management personnel wear AR glasses to connect to the ship auxiliary system in the office. The system retrieves the three-dimensional virtual model of the ship from the data storage unit and presents the overall structure and internal equipment layout of the ship on the AR glasses display screen. The management personnel can view the equipment situation in each area of the ship, such as the cargo stacking state in the cargo hold and the equipment operating parameters in the engine room, by rotating their heads and making hand gestures.

[0046] When the maintenance management of the ballast water system of the ship is needed, the system displays the running state and running time parameter of the ballast water pump in real time. The management auxiliary module generates a maintenance plan suggestion according to the service life and running condition data of the ballast water system equipment: "it is suggested that the ballast water pump be overhauled and the sealing element and bearing be replaced during the next port entry". Meanwhile, aiming at the performance improvement requirement of the ship, the system simulates different upgrading schemes, such as the energy consumption reduction after replacing a new type of high-efficiency ballast water pump, the change of the stability of the ship, and the like, and compares and displays the effects before and after the upgrading in three-dimensional animation, so as to provide comprehensive and intuitive data support for the management personnel to make the ship maintenance and upgrading decisions.

[0047] Although embodiments of the application have been disclosed in connection with the above specification and drawings, it should be understood that they are not intended to limit the application to the particular form disclosed, but rather, the aim is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the application as defined by the appended claims and their equivalents.

Claims

1. A ship auxiliary system based on augmented reality technology, characterized in that: include: AR display devices are used to collect image information, perceive head posture and motion information, and integrate virtual information with real scenes for display; Data processing equipment, used to receive and process data collected by AR display equipment and ship operation data, and run image rendering algorithms and system function programs; Communication equipment, used to realize data transmission and communication between the AR display device, data processing equipment and other equipment on the ship; The software system includes a navigation assistance module, a maintenance assistance module and a management assistance module.

2. A ship auxiliary system based on augmented reality technology as claimed in claim 1, wherein: The AR display device is equipped with a high-definition display screen, a high-resolution camera, a gyroscope and an accelerometer. The high-definition display screen is used to display the fused information, the high-resolution camera is used to collect images of the ship's surrounding environment and internal equipment, and the gyroscope and accelerometer are used to sense head posture and movement information.

3. A ship auxiliary system based on augmented reality technology as claimed in claim 1, wherein: The data processing equipment includes a high-performance computer and a data storage unit. The high-performance computer is used to process data and run programs, and the data storage unit is used to store ship navigation data, equipment parameters, maintenance manuals and virtual model information.

4. A ship auxiliary system based on augmented reality technology as claimed in claim 1, wherein: The communication device includes a wireless network module and a wired network interface. The wireless network module supports Bluetooth and Wi-Fi wireless communication protocols, and the wired network interface is used to connect to the ship's local area network.

5. The ship auxiliary system based on augmented reality technology according to claim 1, wherein: The navigation assistance module receives navigation data from the ship's navigation system and environmental images collected by the AR display device, and through image recognition and spatial positioning technology, superimposes and displays the ship's real-time navigation status, path, surrounding obstacles and other ship information on the AR display device in the form of virtual marks and arrow indicators.

6. A ship auxiliary system based on augmented reality technology as claimed in claim 1, wherein: The maintenance assistance module pre-stores a three-dimensional virtual model of the ship equipment, maintenance manuals and fault codes. When the equipment fails, the AR display device scans and identifies the fault, and provides troubleshooting steps, maintenance operation instructions and tool lists in the form of animation demonstrations, text descriptions and voice prompts.

7. A ship auxiliary system based on augmented reality technology as claimed in claim 6, wherein: The maintenance auxiliary module is also used to simulate the process of equipment failure and record the time, content and replacement parts of each maintenance to form an equipment maintenance file.

8. The ship auxiliary system based on augmented reality technology according to claim 1, wherein: The management assistance module enables managers to view a three-dimensional virtual model of the ship's overall structure and internal equipment layout through AR display devices, and displays the equipment's operating status and performance parameters in real time, supporting remote monitoring and management of equipment.

9. The ship auxiliary system based on augmented reality technology according to claim 1, wherein: The management auxiliary module generates maintenance plans and upgrade proposals based on the equipment's age and operating status, and simulates the impact of different upgrade proposals on ship performance.

10. The ship auxiliary system based on augmented reality technology according to claim 1, wherein: The navigation assistance module, maintenance assistance module and management assistance module exchange information with other systems of the ship through the communication device.

Images