A digital twin remote operation and maintenance system for ship management

Abstract

This invention provides a digital twin remote operation and maintenance system for ship management, relating to the field of intelligent ship management and remote operation and maintenance technology. Its features include: a ship-side system; a shore-based system; a secure communication network; and a network security protection system. The advantages of this invention are: by integrating digital twin and augmented reality technologies, it solves the problems of high communication costs and unintuitive guidance in traditional ship operation and maintenance, enabling shore-based experts to provide hands-on, remote, virtual-real integrated guidance to ships at sea. The system adopts a dual-link redundant architecture of maritime satellite and 5G, as well as an offline edge computing mode, ensuring communication continuity and uninterrupted operation and maintenance services under complex sea conditions. Simultaneously, it integrates multi-layered security protection mechanisms such as national cryptographic algorithm encryption, blockchain data verification, and multi-factor authentication, effectively ensuring the security and tamper-proof capability of data and models throughout the entire chain.

Description

Technical Field

[0001] This invention relates to the field of intelligent ship management and remote operation and maintenance technology, and in particular to a digital twin remote operation and maintenance system for ship management. Background Technology

[0002] With the rapid development of IoT, big data, and artificial intelligence technologies, digital twin technology, as a key means to achieve deep integration of the physical and digital worlds, has gradually shown enormous application potential in fields such as intelligent manufacturing and smart cities. By constructing a virtual model highly consistent with the physical ship, digital twin technology can achieve real-time mapping and monitoring of ship status, equipment operation, and navigation environment, providing a new technological path for remote ship maintenance.

[0003] Currently, existing remote ship maintenance systems still face many serious challenges in practical applications. On the one hand, the system's network security protection capabilities are insufficient, facing severe network security challenges. The existing system architecture often focuses on data collection and model mapping, while neglecting the construction of multi-layered network security protection mechanisms. This makes the system vulnerable to network threats such as data theft, malicious tampering, denial-of-service attacks, or exploitation of vulnerabilities in the virtual model itself. On the other hand, the efficiency of shore-based experts in providing remote support to ships at sea is low, making it difficult to provide intuitive, real-time, and high-precision fault handling guidance. Existing remote guidance relies on voice or static graphics and text, making it difficult for ship personnel to accurately describe the details of the fault. Shore-based experts cannot overlay their operational intentions onto the ship's physical space in real time for synchronous guidance, resulting in high communication costs, a high risk of errors, and a significant increase in fault response and maintenance time.

[0004] Therefore, there is an urgent need for a digital twin system that can effectively resist cyberattacks and enable shore-based experts to provide intuitive and efficient remote operation and maintenance guidance for ships at sea. Summary of the Invention

[0005] The purpose of this invention is to provide a digital twin remote operation and maintenance system for ship management, which solves the technical problems of weak network security protection capabilities and low efficiency of remote guidance from shore-based experts in existing ship digital twin remote operation and maintenance systems.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0007] A digital twin remote operation and maintenance system for ship management, characterized in that it includes:

[0008] The ship-side system, deployed on the target vessel, is used to collect multi-dimensional data about the vessel, perform local computation and rendering of the digital twin model, and provide on-site interactive display.

[0009] The shore-based terminal system, deployed in the remote operation and maintenance center, is used to receive and process ship-side data, build a high-precision digital twin virtual entity, and generate remote operation and maintenance instructions.

[0010] The secure communication network adopts a dual-link redundancy architecture to connect the ship-side system and the shore-based system, enabling bidirectional encrypted data transmission and dynamic link switching.

[0011] The network security protection system is integrated into the ship's end, shore-based end, and communication links, constructing a multi-layered security protection mechanism to ensure the security of data, equipment, and digital twin models.

[0012] As an improvement, the ship-side system includes: a multi-source data acquisition module, which is connected to the interfaces of various ship sensors and control systems to collect ship status data, equipment operation data and navigation environment data in real time;

[0013] The edge computing module has a built-in lightweight digital twin engine, which is used to complete data preprocessing, feature extraction, and local updates of the twin model locally, and supports automatic switching to offline operation and maintenance mode when the network is abnormal.

[0014] The augmented reality interaction module is used to receive visual guidance information from the shore-based terminal and overlay it onto the on-site video footage of the ship to achieve a virtual-real fusion display and interactive feedback;

[0015] The data integrity verification module uses a blockchain hash chain or Merkle tree structure to store and verify the collected data, preventing data tampering.

[0016] As an improvement, the shore-based system includes: a high-precision digital twin construction module, which constructs and drives a full-parameter digital twin model that operates synchronously with the target ship in real time based on data uploaded from the ship. The model integrates geometric model, mechanistic model and data-driven model.

[0017] The remote operation and maintenance decision module is used to analyze the operating status of the twin model, complete fault diagnosis, maintenance plan generation, and maintenance process simulation.

[0018] The augmented reality instruction generation module generates visual annotation information based on the verified maintenance plan, and supports manual annotation by shore-based experts, which is then simultaneously sent to the ship.

[0019] The situation synchronization module uses a differential synchronization algorithm to maintain consistency in the state and time synchronization between the twin models of the ship and the shore.

[0020] The remote verification module periodically sends data integrity challenges to the ship to verify the authenticity and integrity of historical data.

[0021] As an improvement, the secure communication network adopts a dual-link redundancy architecture of maritime satellite communication and 5G communication, and sets up a link dynamic switching unit. The unit automatically selects the primary link based on real-time communication quality, link cost and security policy, and maintains the continuity of encrypted sessions and data without loss during the switching process.

[0022] As an improvement, the network security protection system includes a network security proxy module and a network security protection module, specifically including:

[0023] The device identity authentication unit performs multi-factor authentication for users accessing ship-side devices and platforms based on digital certificates or pre-shared keys;

[0024] The data encryption transmission unit uses national cryptographic algorithms to encrypt the collected data and command data end-to-end to ensure transmission security.

[0025] Access control unit, based on role or attribute access control policies, restricts system resource access permissions for different users and modules;

[0026] The intrusion detection and prevention unit monitors network traffic and system call behavior in real time, identifies and blocks malicious attacks, and triggers alarm mechanisms.

[0027] The model integrity verification unit periodically performs hash verification on key data and logic of the digital twin model to prevent the model from being maliciously tampered with.

[0028] As an improvement, the augmented reality interaction module includes:

[0029] The video acquisition unit collects real-time video streams of the ship's on-site equipment and environment;

[0030] The spatial positioning unit acquires the pose information of the augmented reality display device and uses SLAM technology to achieve spatial registration between the visual information and the physical device.

[0031] The annotation fusion unit performs 3D spatial registration and perspective projection fusion on the received annotation information to generate an augmented reality image with virtual and real overlays.

[0032] The interactive feedback unit collects the operator's voice, gestures, or eye focus information and transmits it back to the shore-based system.

[0033] As an improvement, the remote operation and maintenance decision module includes:

[0034] The fault diagnosis unit compares the real-time output of the twin model with the health baseline model to identify abnormal parameters and locate faulty equipment or subsystems.

[0035] The maintenance plan generation unit calls the maintenance knowledge base based on the fault diagnosis results to generate a maintenance plan that includes maintenance steps, tool list and spare parts information.

[0036] The simulation unit simulates the execution of maintenance plans in a shore-based twin model, verifies the effectiveness of the plans, and predicts the equipment status after maintenance.

[0037] As an improvement, the augmented reality instruction generation module generates visual annotation information, including 3D arrow guidance, equipment highlight boxes, virtual button press prompts, safety warning areas, and maintenance step animations, which supports remote first-person view display synchronized with the perspective of shore-based experts.

[0038] As an improvement, the offline operation and maintenance mode of the edge computing module can independently complete the twin model status estimation, fault warning and emergency operation guidance generation at the ship end, and complete data synchronization and model reconciliation with the shore-based system after the network is restored.

[0039] A digital twin remote operation and maintenance method for ship management, characterized by comprising:

[0040] The ship-side multi-source data acquisition module collects various types of ship data in real time, and after data integrity verification and encryption, it sends the data to the shore-based system through a secure communication network.

[0041] The shore-based terminal system receives data and drives the high-precision digital twin model to update in real time, while monitoring communication security and model integrity through a network security protection system;

[0042] The remote operation and maintenance decision-making module analyzes the status of the twin model, completes fault diagnosis, maintenance plan generation and simulation, and the augmented reality instruction generation module generates visual guidance information.

[0043] The encrypted visual guidance information is transmitted to the ship via a secure communication network. The augmented reality interactive module integrates it with the on-site video to guide the operators in performing the operations.

[0044] The interactive feedback unit transmits the operator's feedback back to the shore-based terminal. The shore-based experts adjust the guidance information based on the feedback until the fault is resolved. When the network is abnormal, the ship automatically switches to offline operation and maintenance mode. After the network is restored, the data and model are synchronized.

[0045] The beneficial effects of this invention are: it solves the security risks in data transmission and storage.

[0046] End-to-end encryption protection: It integrates device authentication, encrypted data transmission (using national cryptographic algorithms) and access control to effectively resist data theft and malicious attacks.

[0047] Anti-tampering mechanism: The collected data is verified using a blockchain hash chain or Merkle tree structure, and the integrity of the digital twin model is verified using hash verification technology, which ensures the authenticity of historical data and model logic and prevents malicious tampering.

[0048] By combining digital twin and augmented reality technologies, the problems of high cost and error-proneness in traditional voice / text guidance communication are solved:

[0049] Virtual-real integration guidance: Visual annotations (such as 3D arrows, highlighted boxes, and maintenance animations) generated by shore-based experts can be superimposed on the ship's on-site images in real time, achieving synchronous guidance from a first-person perspective and significantly reducing ship-shore communication costs.

[0050] Scheme pre-simulation and verification: The maintenance scheme is simulated and deduced at the shore base using a twin model to verify the effectiveness of the scheme and predict the post-maintenance status, thus avoiding secondary failures or delays caused by scheme errors.

[0051] The system possesses strong adaptability, addressing the complex and easily disrupted communication environment at sea.

[0052] Dual-link redundancy and dynamic switching: It adopts a dual-link architecture of maritime satellite and 5G, which automatically switches according to communication quality and cost, and maintains the continuity of encrypted sessions and data without loss during the switching process.

[0053] Offline operation and maintenance mode: When the network is abnormal, the ship-side edge computing module can run independently to perform status estimation and emergency guidance; after the network is restored, it automatically completes data synchronization and model reconciliation, ensuring the continuity of operation and maintenance services.

[0054] Multi-model fusion: The digital twin model built on shore integrates geometric models, mechanistic models and data-driven models, and combined with differential synchronization algorithms, it achieves a high degree of consistency and real-time synchronization with the physical ship, providing a data foundation for precise operation and maintenance. Attached Figure Description

[0055] Figure 1 This is an overall architecture diagram of a digital twin remote operation and maintenance system for ship management according to the present invention.

[0056] Figure 2 This is an overall flowchart of a digital twin remote operation and maintenance method for ship management according to the present invention. Detailed Implementation

[0057] To make the content of this invention easier to understand, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Identical components are represented by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, while the terms "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0058] like Figures 1 to 2 As shown, a digital twin remote operation and maintenance system for ship management is characterized by comprising: a ship-side system deployed on the target ship for collecting multi-dimensional ship data, performing local calculation and rendering of the digital twin model, and providing on-site interactive display; a shore-based system deployed at the remote operation and maintenance center for receiving and processing ship-side data, constructing a high-precision digital twin virtual entity, and generating remote operation and maintenance instructions; a secure communication network employing a dual-link redundant architecture to connect the ship-side system and the shore-based system, enabling bidirectional encrypted data transmission and dynamic link switching; and a network security protection system integrated into the ship-side, shore-based, and communication links, constructing a multi-layered security protection mechanism to ensure the security of data, equipment, and the digital twin model.

[0059] The ship-side system includes: a multi-source data acquisition module, which connects to various sensors and control systems on the ship to collect ship status data, equipment operation data, and navigation environment data in real time; an edge computing module, which has a built-in lightweight digital twin engine for local data preprocessing, feature extraction, and local updates of the twin model, and supports automatic switching to offline operation and maintenance mode when the network is abnormal; an augmented reality interaction module, which receives visual guidance information from the shore-based terminal and overlays it onto the ship's on-site video footage to achieve virtual-real fusion display and interactive feedback; and a data integrity verification module, which uses a blockchain hash chain or Merkle tree structure to store and verify the collected data to prevent data tampering.

[0060] The shore-based system includes: a high-precision digital twin construction module, which constructs and drives a full-parameter digital twin model that operates synchronously with the target vessel in real time based on data uploaded from the ship; the model integrates geometric, mechanistic, and data-driven models; a remote operation and maintenance decision-making module, used to analyze the operating status of the twin model, complete fault diagnosis, maintenance plan generation, and maintenance process simulation; an augmented reality instruction generation module, which generates visualized annotation information based on verified maintenance plans, supports manual annotation by shore-based experts, and synchronously sends it to the ship; a situation synchronization module, which uses a differential synchronization algorithm to maintain consistency in status and time synchronization between the ship-based and shore-based twin models; and a remote verification module, which periodically initiates data integrity challenges to the ship to verify the authenticity and integrity of historical data.

[0061] The secure communication network adopts a dual-link redundancy architecture of maritime satellite communication and 5G communication, and sets up a link dynamic switching unit. The unit automatically selects the primary link based on real-time communication quality, link cost and security policy, and maintains the continuity of encrypted sessions and data without loss during the switching process.

[0062] The network security protection system includes a network security proxy module and a network security protection module, specifically comprising: a device identity authentication unit, which performs multi-factor authentication for users accessing ship-side devices and platforms based on digital certificates or pre-shared keys; a data encryption transmission unit, which uses national cryptographic algorithms to perform end-to-end encryption on collected data and command data to ensure transmission security; an access control unit, which restricts system resource access permissions for different users and modules based on role or attribute-based access control policies; an intrusion detection and prevention unit, which monitors network traffic and system call behavior in real time, identifies and blocks malicious attacks, and triggers alarm mechanisms; and a model integrity verification unit, which periodically performs hash verification on key data and logic of the digital twin model to prevent the model from being maliciously tampered with.

[0063] The augmented reality interaction module includes: a video acquisition unit, which acquires real-time video streams of on-site equipment and environment on the ship; a spatial positioning unit, which obtains the pose information of the augmented reality display device and realizes spatial registration between the visualized information and the physical device through SLAM technology; an annotation and fusion unit, which performs 3D spatial registration and perspective projection fusion on the received annotation information to generate an augmented reality image with virtual and real superposition; and an interactive feedback unit, which collects the operator's voice, gestures, or gaze focus information and transmits it back to the shore-based system.

[0064] The remote operation and maintenance decision-making module includes: a fault diagnosis unit, which compares the real-time output of the twin model with the health baseline model to identify abnormal parameters and locate faulty equipment or subsystems; a maintenance plan generation unit, which calls the maintenance knowledge base based on the fault diagnosis results to generate a maintenance plan containing maintenance steps, tool lists and spare parts information; and a simulation and deduction unit, which simulates the execution of the maintenance plan in the shore-based twin model to verify the effectiveness of the plan and predict the equipment status after maintenance.

[0065] The augmented reality instruction generation module generates visual annotation information, including 3D arrow guidance, equipment highlight boxes, virtual button press prompts, safety warning areas, and maintenance step animations, supporting remote first-person view display synchronized with the shore-based expert's perspective. The edge computing module's offline operation and maintenance mode can independently complete twin model status estimation, fault warning, and emergency operation guidance generation on the ship. After network recovery, data synchronization and model reconciliation with the shore-based system are completed.

[0066] A digital twin remote operation and maintenance method for ship management is characterized by the following: a multi-source data acquisition module on the ship collects various types of ship data in real time, verifies and encrypts the data integrity, and then sends it to a shore-based system via a secure communication network; the shore-based system receives the data, drives the high-precision digital twin model to update in real time, and monitors communication security and model integrity through a network security protection system; a remote operation and maintenance decision module analyzes the twin model status, completes fault diagnosis, maintenance plan generation and simulation, and an augmented reality instruction generation module generates visual guidance information; the visual guidance information is encrypted and sent to the ship via a secure communication network, and an augmented reality interaction module integrates it with the on-site scene to guide operators to perform operations; an interactive feedback unit transmits the operator's operation feedback back to the shore-based system, and shore-based experts adjust the guidance information based on the feedback until the fault is resolved; in the event of network anomalies, the ship automatically switches to offline operation and maintenance mode, and completes data and model synchronization after the network is restored.

[0067] During implementation, multi-source data acquisition modules deployed on the ship acquire real-time data on ship status, equipment operation, and environment. After encryption and integrity verification, this data is transmitted to the shore-based terminal via a dual-link redundant network of maritime satellite and 5G. The shore-based terminal uses a high-precision digital twin construction module to drive a virtual model synchronized with the physical ship. When a fault occurs, the remote operation and maintenance decision module diagnoses the fault and generates a maintenance plan. After simulation and verification, the augmented reality instruction generation module converts the maintenance instructions into visual annotation information and sends it to the ship. The ship's augmented reality interaction module uses SLAM technology to accurately overlay these virtual instructions onto the on-site equipment screen, enabling shore-based experts to provide "hands-on" remote guidance to the ship's personnel. At the same time, the system constructs a multi-layered network security protection system through encryption using national cryptographic algorithms and blockchain hash verification throughout the entire process. In the event of network anomalies, the ship's edge computing module can automatically switch to offline mode to operate independently, and automatically synchronize data after the network is restored, thus completing a complete closed loop from data perception, remote diagnosis, virtual and real guidance to emergency response.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A digital twin remote operation and maintenance system for ship management, characterized in that, include: The ship-side system, deployed on the target vessel, is used to collect multi-dimensional data about the vessel, perform local computation and rendering of the digital twin model, and provide on-site interactive display. The shore-based terminal system, deployed in the remote operation and maintenance center, is used to receive and process ship-side data, build a high-precision digital twin virtual entity, and generate remote operation and maintenance instructions. The secure communication network adopts a dual-link redundancy architecture to connect the ship-side system and the shore-based system, enabling bidirectional encrypted data transmission and dynamic link switching. The network security protection system is integrated into the ship's end, shore-based end, and communication links, constructing a multi-layered security protection mechanism to ensure the security of data, equipment, and digital twin models.

2. The digital twin remote operation and maintenance system for ship management according to claim 1, characterized in that, The ship-side system includes: a multi-source data acquisition module, which is connected to the interfaces of various sensors and control systems of the ship to collect ship status data, equipment operation data and navigation environment data in real time; The edge computing module has a built-in lightweight digital twin engine, which is used to complete data preprocessing, feature extraction, and local updates of the twin model locally, and supports automatic switching to offline operation and maintenance mode when the network is abnormal. The augmented reality interaction module is used to receive visual guidance information from the shore-based terminal and overlay it onto the on-site video footage of the ship to achieve a virtual-real fusion display and interactive feedback; The data integrity verification module uses a blockchain hash chain or Merkle tree structure to store and verify the collected data, preventing data tampering.

3. The digital twin remote operation and maintenance system for ship management according to claim 1, characterized in that, The shore-based system includes: a high-precision digital twin construction module, which constructs and drives a full-parameter digital twin model that operates synchronously with the target ship in real time based on data uploaded from the ship. The model integrates geometric model, mechanistic model and data-driven model. The remote operation and maintenance decision module is used to analyze the operating status of the twin model, complete fault diagnosis, maintenance plan generation, and maintenance process simulation. The augmented reality instruction generation module generates visual annotation information based on the verified maintenance plan, and supports manual annotation by shore-based experts, which is then simultaneously sent to the ship. The situation synchronization module uses a differential synchronization algorithm to maintain consistency in the state and time synchronization between the twin models of the ship and the shore. The remote verification module periodically sends data integrity challenges to the ship to verify the authenticity and integrity of historical data.

4. A digital twin remote operation and maintenance system for ship management according to claim 1, characterized in that, The secure communication network adopts a dual-link redundancy architecture of maritime satellite communication and 5G communication, and sets up a link dynamic switching unit. The unit automatically selects the primary link based on real-time communication quality, link cost and security policy, and maintains the continuity of encrypted sessions and data without loss during the switching process.

5. A digital twin remote operation and maintenance system for ship management according to claim 1, characterized in that, The network security protection system includes a network security proxy module and a network security protection module, specifically including: The device identity authentication unit performs multi-factor authentication for users accessing ship-side devices and platforms based on digital certificates or pre-shared keys; The data encryption transmission unit uses national cryptographic algorithms to encrypt the collected data and command data end-to-end to ensure transmission security. Access control unit, based on role or attribute access control policies, restricts system resource access permissions for different users and modules; The intrusion detection and prevention unit monitors network traffic and system call behavior in real time, identifies and blocks malicious attacks, and triggers alarm mechanisms. The model integrity verification unit periodically performs hash verification on key data and logic of the digital twin model to prevent the model from being maliciously tampered with.

6. A digital twin remote operation and maintenance system for ship management according to claim 2, characterized in that, The augmented reality interaction module includes: The video acquisition unit collects real-time video streams of the ship's on-site equipment and environment; The spatial positioning unit acquires the pose information of the augmented reality display device and uses SLAM technology to achieve spatial registration between the visual information and the physical device. The annotation fusion unit performs 3D spatial registration and perspective projection fusion on the received annotation information to generate an augmented reality image with virtual and real overlays. The interactive feedback unit collects the operator's voice, gestures, or eye focus information and transmits it back to the shore-based system.

7. A digital twin remote operation and maintenance system for ship management according to claim 3, characterized in that, The remote operation and maintenance decision-making module includes: The fault diagnosis unit compares the real-time output of the twin model with the health baseline model to identify abnormal parameters and locate faulty equipment or subsystems. The maintenance plan generation unit calls the maintenance knowledge base based on the fault diagnosis results to generate a maintenance plan that includes maintenance steps, tool list and spare parts information. The simulation unit simulates the execution of maintenance plans in a shore-based twin model, verifies the effectiveness of the plans, and predicts the equipment status after maintenance.

8. A digital twin remote operation and maintenance system for ship management according to claim 7, characterized in that, The augmented reality instruction generation module generates visual annotation information, including 3D arrow guidance, equipment highlight boxes, virtual button press prompts, safety warning areas, and maintenance step animations, which support remote first-person view display synchronized with the perspective of shore-based experts.

9. A digital twin remote operation and maintenance system for ship management according to claim 2, characterized in that, The offline operation and maintenance mode of the edge computing module can independently complete the twin model status estimation, fault warning and emergency operation guidance generation at the ship end, and complete data synchronization and model reconciliation with the shore-based system after the network is restored.

10. A digital twin remote operation and maintenance method for ship management, characterized in that, include: The ship-side multi-source data acquisition module collects various types of ship data in real time, and after data integrity verification and encryption, it sends the data to the shore-based system through a secure communication network. The shore-based terminal system receives data and drives the high-precision digital twin model to update in real time, while monitoring communication security and model integrity through a network security protection system; The remote operation and maintenance decision-making module analyzes the status of the twin model, completes fault diagnosis, maintenance plan generation and simulation, and the augmented reality instruction generation module generates visual guidance information. The encrypted visual guidance information is transmitted to the ship via a secure communication network. The augmented reality interactive module integrates it with the on-site video to guide the operators in performing the operations. The interactive feedback unit transmits the operator's feedback back to the shore-based terminal. The shore-based experts adjust the guidance information based on the feedback until the fault is resolved. When the network is abnormal, the ship automatically switches to offline operation and maintenance mode. After the network is restored, the data and model are synchronized.

Images