Unity-Qt-ROS-based underwater intelligent digital twin system and implementation method

Abstract

The application provides an underwater intelligent digital twin system based on Unity-Qt-ROS and an implementation method thereof, comprising: a physical entity module: based on an AUV physical prototype and real physical sensors carried thereon, real data of a real world is captured to realize mapping of a physical space to a digital space; a digital space module: a digital mapping body and a digital native body; the digital mapping body is a digital reproduction of the physical entity; the digital native body comprises a digital native scene and a digital native sensor; a human-computer interaction module: a target task is obtained through a graphical user interface, a plurality of subtasks are obtained by decomposing the target task by using a function service module, and each subtask is transmitted to the digital space module and / or the physical entity module for execution; in the execution process, real-time data interaction and state synchronization are realized through a message passing mechanism between the digital space module and the physical entity module, so that real-time interaction from the digital space to the physical entity is completed.

Description

Technical Field

[0001] This invention relates to the field of underwater robot technology, specifically to an underwater intelligent digital twin system and implementation method based on Unity-Qt-ROS, which provides a low-cost, low-risk, and high-efficiency testing and verification platform for intelligent algorithms of autonomous underwater vehicles (AUVs). Background Technology

[0002] With the development of the strategic goal of "entering the ocean, exploring the ocean, and utilizing the ocean," autonomous underwater vehicles (AUVs) play a crucial role. The stability and reliability of the intelligent algorithms deployed on AUVs are key to their successful and safe completion of various operational tasks. Currently, the testing of AUV intelligent algorithms mostly relies on pure numerical simulation. Although numerical simulation brings convenience to the initial testing of algorithms, it is difficult to accurately simulate the complexity and unpredictability of the underwater environment, resulting in the inability to directly apply the tested intelligent algorithms to actual engineering projects. On the other hand, directly testing algorithms in real sea areas is extremely risky, with huge labor and economic costs, and testing under extreme conditions is also very difficult. These problems pose significant challenges to the practical application of intelligent algorithms. Therefore, it is particularly important to develop and implement an efficient and risk-controlled algorithm testing scheme, which is of great significance for improving the overall intelligence of AUVs and ensuring their safety in real-world operations.

[0003] Digital twin technology, one of the top ten strategic technology development trends, offers a new approach to solving the aforementioned challenges due to its characteristics of virtual-real integration and real-time interaction, iterative operation and optimization, and data-driven approach covering all elements, processes, and business aspects. Through highly refined modeling and rendering techniques and real-time virtual-real interaction, digital twin technology ensures close coupling and synchronization between the real world and digital space, thereby achieving high-fidelity digital mapping of physical entities in the real world throughout their entire lifecycle in digital space. Currently, digital twin technology has been researched and applied in multiple fields such as aerospace, urban management, healthcare, CNC machine tools, and transportation systems. In the field of marine engineering, research has already constructed intelligent operation and maintenance systems based on digital twin technology for ship propulsion systems. However, AUV systems span multiple highly specialized disciplines such as computer science, marine engineering, and mechanical engineering. Their operating environment is extremely harsh, and the implementation of intelligent algorithms is extremely complex, resulting in research on AUV digital twins still being in its early stages. Existing technologies lack a dedicated testing system that can seamlessly connect the real ocean world with the digital ocean space, achieve bidirectional mapping and real-time dynamic information interaction, and support high-fidelity digital native ocean environment simulation. This makes it difficult to meet the urgent need for efficient, safe, and low-cost verification of AUV intelligent algorithms.

[0004] Patent document CN118015228A (application number: 202410105707.2) discloses a multi-AUV digital twin system based on mixed reality, including: a virtual environment system, a virtual AUV cluster system, a visual system, a realistic dynamics simulation system, a VR control system, and an AR display system. The virtual environment system is used for simulation experiments, generating multiple AUV sensor data. The virtual AUV cluster system completes a series of underwater exploration simulation experiments, such as ocean exploration, in a master-slave mode. The visual system is responsible for displaying the interactive UI and 3D scene, as well as various sensor data and real-time images of the AUVs. The realistic dynamics simulation system can conduct simulation experiments based on real physics and save experimental data. The VR control system is responsible for organically integrating virtual reality with the AUV cluster to achieve first-person operation. The AR display system is responsible for bringing the virtual AUVs into reality. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide an underwater intelligent digital twin system and its implementation method based on Unity-Qt-ROS.

[0006] An underwater intelligent digital twin system based on Unity-Qt-ROS, provided by the present invention, includes: Physical entity: Based on the AUV physical prototype and its onboard real physical sensors, real data of the real world is captured, and the mapping from physical space to digital space is realized through real-time communication technology; The digital space module includes: a digital mapping body and a digital native body; wherein, the digital mapping body is a digital representation of a physical entity; the digital native body includes a digital native scene and digital native sensors; the digital native scene is used to realize the visualization of the underwater environment; the digital native sensors are used to simulate the real sensor perception process; Human-Computer Interaction Module: The module acquires the target task through a graphical user interface, transmits the target task to the functional service module, uses the functional service module to break down the target task into multiple subtasks, and transmits each subtask to the digital space module and / or physical entity module for execution. During execution, real-time data interaction and state synchronization are achieved through the message passing mechanism between the digital space module and the physical entity module, thereby completing the real-time virtual-real interaction from digital space to physical entity.

[0007] Preferably, the physical entity includes: the AUV physical prototype is equipped with a forward-looking sonar, a side-scan sonar, a global positioning system, an inertial navigation unit, a depth gauge, a Doppler velocimeter, environmental perception and state measurement sensors; Meanwhile, the AUV physical prototype integrates multiple communication methods including WI-FI, radio, fiber optic microcable and acoustic; it is equipped with a high-performance microcomputer, on which the Robot Operating System (ROS) is deployed, which is used to achieve communication through a message passing mechanism.

[0008] Preferably, the digital mapping is a high-fidelity digital reproduction of a physical entity, including its shape, size, material, texture, lighting, and motion behavior; The digital native scene is constructed using 3D Max tools, including underwater terrain and static and dynamic obstacles, and then imported into the Unity3D engine. The built-in PhysX physics engine of Unity3D engine is used for real-time rendering to achieve a visual presentation of the underwater environment. The digital native sensor simulates the real sensor's sensing process by simulating the working mode of real acoustic and optical sensors, and then realizes the effective data output of the digital native sensor in the underwater environment through script compilation. Digital native sensors are integrated into the digital mapping body, and virtual environment information is perceived in real time through the interaction between the digital native sensors and the digital native scene.

[0009] Preferably, the graphical user interface in the human-computer interaction module is developed using the Qt framework; The target task is obtained through a graphical user interface and transmitted to the functional service module via ROS. The functional service module decomposes the target task into multiple subtasks and transmits each subtask to the digital space module and / or physical entity module for execution via ROS. During execution, message passing between the digital space module and the physical entity module is achieved through ROS.

[0010] Preferably, the system further includes: an AUV-Unity 3D communication link and an AUV-Qt communication link; The AUV-Unity 3D communication link is used to map the real-time status of the AUV onto the virtual ocean environment, enabling seamless data integration and instant visual feedback. The AUV-Qt communication link is used to send user commands and receive AUV status, enabling efficient human-machine interaction.

[0011] A method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS, provided by the present invention, includes: Step S1: Capture real-world data based on the AUV physical prototype and its onboard physical sensors, and realize the mapping from physical space to digital space through real-time communication technology; Step S2: Construct a digital mapping body and a digital native body; wherein, the digital mapping body is a digital representation of a physical entity; the digital native body includes a digital native scene and digital native sensors; the digital native scene is used to realize the visualization of the underwater environment; the digital native sensors are used to simulate the real sensor perception process; Step S3: Obtain the target task through the graphical user interface, transmit the target task to the function service module, use the function service module to decompose the target task to obtain multiple sub-tasks, and transmit each sub-task to the digital space module and / or physical entity module for execution. During the execution process, real-time data interaction and state synchronization are realized through the message passing mechanism between the digital space module and the physical entity module, thereby completing the virtual-real real-time interaction from digital space to physical entity.

[0012] Preferably, the AUV physical prototype is equipped with a forward-looking sonar, a side-scan sonar, a global positioning system, an inertial navigation unit, a depth gauge, a Doppler velocimeter, and environmental perception and state measurement sensors. Meanwhile, the AUV physical prototype integrates multiple communication methods including WI-FI, radio, fiber optic microcable and acoustic; it is equipped with a high-performance microcomputer, on which the Robot Operating System (ROS) is deployed, which is used to achieve communication through a message passing mechanism.

[0013] Preferably, the digital mapping is a high-fidelity digital reproduction of a physical entity, including its shape, size, material, texture, lighting, and motion behavior; The digital native scene is constructed using 3D Max tools, including underwater terrain and static and dynamic obstacles, and then imported into the Unity3D engine. The built-in PhysX physics engine of Unity3D engine is used for real-time rendering to achieve a visual presentation of the underwater environment. The digital native sensor simulates the real sensor's sensing process by simulating the working mode of real acoustic and optical sensors, and then realizes the effective data output of the digital native sensor in the underwater environment through script compilation. Digital native sensors are integrated into the digital mapping body, and virtual environment information is perceived in real time through the interaction between the digital native sensors and the digital native scene.

[0014] Preferably, the graphical user interface is developed using the Qt framework; The target task is obtained through a graphical user interface and transmitted to the functional service module via ROS. The functional service module decomposes the target task into multiple subtasks and transmits each subtask to the digital space module and / or physical entity module for execution via ROS. During execution, message passing between the digital space module and the physical entity module is achieved through ROS.

[0015] Preferably, the method further includes: an AUV-Unity 3D communication link and an AUV-Qt communication link; The AUV-Unity 3D communication link is used to map the real-time status of the AUV onto the virtual ocean environment, enabling seamless data integration and instant visual feedback. The AUV-Qt communication link is used to send user commands and receive AUV status, enabling efficient human-machine interaction.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention provides a complete underwater intelligent digital twin system framework, which realizes synchronous operation, bidirectional mapping and real-time interaction between real AUV and digital AUV. It effectively breaks through the technical bottleneck of existing AUV digital twin technology, which can only realize single-dimensional digital mapping, lacks bidirectional real-time interaction capability and cannot build a high-fidelity underwater environment. Relying on multi-dimensional collaborative modeling and virtual-real linkage technology, it builds a low-risk, low-cost and highly reliable dedicated verification platform for AUV intelligent algorithm testing. 2. This invention adopts a loosely coupled, modular design. The overall architecture mainly consists of four parts: physical entities, digital space, functional services, and human-computer interaction. These parts are closely connected, with information flowing between them and cooperating with each other to jointly support the overall operation of the system. 3. This invention addresses the core challenges of integrating multiple technologies such as Unity 3D, Qt, and ROS, including interface incompatibility, asynchronous data transmission, and difficulties in cross-platform adaptation. By customizing standardized interface protocols and building a unified data interaction middleware, it achieves seamless collaborative integration of multiple technologies, improving the system's scalability and maintainability, and reducing overall complexity and development costs. 4. This invention constructs a high-fidelity, flexibly configurable digital-native marine test scenario and digital-native sensors. Addressing the technical challenges of existing technologies, such as distorted scene modeling and insufficient realism in the interaction between virtual sensors and virtual obstacles, it uses 3D Max to construct high-precision 3D models of terrain and dynamic / static obstacles. These models are then imported into the Unity3D engine and rendered in real-time using Unity3D's built-in PhysX physics engine, accurately reproducing underwater light refraction, water flow disturbance, and other environmental characteristics, achieving a high-fidelity visualization of the underwater environment. Furthermore, it employs Unity3D's Raycast ray detection technology combined with C# scripting to simulate the ranging of a multibeam echo sounder. By compiling scripts, it reproduces the effective data output characteristics of the sensor in different underwater environments, completing a high-precision simulation of the digital-native sensor. This effectively improves the scenario coverage and test realism of AUV intelligent algorithm testing, providing a test environment that closely matches actual operational scenarios for algorithm verification. 5. This invention is not only applicable to the testing and optimization of AUV intelligent perception, planning, and control algorithms, but its general framework can also be extended to other fields of underwater unmanned systems, such as structural design and optimization, system prediction and maintenance, etc., showing broad application prospects. Attached Figure Description

[0017] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of an underwater intelligent digital twin system based on Unity-Qt-ROS.

[0018] Figure 2 This is an overall view of the graphical user interface (GUI) of an underwater intelligent digital twin system.

[0019] Figure 3 This is a schematic diagram of an underwater terrain model built using World Machine software.

[0020] Figures 4a to 4b This is a schematic diagram of a partial underwater static obstacle model (rock) built using SolidWorks.

[0021] Figure 5 This is a schematic diagram of a partial underwater dynamic obstacle model (robot) built using SolidWorks.

[0022] Figure 6 This is a schematic diagram of a digital native multibeam echo sounder simulated within Unity.

[0023] Figure 7A schematic diagram of a scaled-down digital AUV 3D model for importing into Unity for visualization rendering and collision detection.

[0024] Figures 8a to 8l This is a schematic diagram of the key functional interfaces of the human-computer interaction interface.

[0025] Figure 9 This is a test scenario for autonomous constant speed cruise and autonomous directional navigation control algorithms conducted in a lake experiment.

[0026] Figure 10 This is a test scenario for implementing autonomous speed, orientation, and depth control algorithms in a towed pool environment.

[0027] Figure 11 This is an experimental scenario for testing underwater obstacle avoidance perception, planning, and control algorithms in a towed pool environment. Detailed Implementation

[0028] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0029] Example 1 The underwater intelligent digital twin system based on Unity-Qt-ROS provided by the present invention, such as Figure 1 As shown, it consists of four parts: physical entities, digital space, functional services, and human-computer interaction. These parts are closely connected, with information flowing between them, cooperating with each other, and jointly supporting the overall operation of the system.

[0030] The physical entity consists of an AUV physical prototype and its onboard real physical sensors, used to capture real-world data and achieve precise mapping from physical space to digital space through real-time communication technology. The AUV physical prototype is equipped with environmental perception and state measurement sensors such as forward-looking sonar, side-scan sonar, Global Positioning System (GPS), Inertial Measurement Unit (IMU), depth gauge, and Doppler velocimeter (DVL), and is fitted with actuators such as channel thrusters, cross rudders, and tail thrusters. Simultaneously, the prototype integrates multiple communication methods including Wi-Fi, radio, fiber optic microcables, and acoustics, and is equipped with a high-performance microcomputer, such as the Nvidia Jetson AGX Xavier, on which a Robot Operating System (ROS) is deployed, providing hardware support and communication assurance for the system.

[0031] The digital space comprises two core components: digital mappings and digital natives. Digital mappings are high-fidelity digital representations of the shape, size, material, texture, lighting, and motion behavior of physical entities. Digital natives include digital native scenes used for testing, such as complex underwater terrain, static and dynamic obstacles, and high-fidelity digital native sensors, such as multi-beam sonar simulators. Digital native scenes are modeled using specialized software, such as World Machine and SolidWorks. Figure 3 As shown, the digital native sensor simulates the perception process of a real sensor by working in a manner similar to that of a real acoustic and optical sensor. By integrating a digital native sensor onto a digital AUV, the sensor interacts with the virtual environment to perceive information about the virtual environment in real time, approximating the perception process of information about the real environment.

[0032] The functional service module is responsible for data management, algorithm deployment, and execution. It has efficient data read and write capabilities for managing physical sensor data, simulated test virtual data, and historical records; and provides distributed interface support, making it easy for users to deploy, execute, and evaluate various intelligent algorithms in real time using different programming languages.

[0033] like Figure 2 As shown, the human-computer interaction component utilizes the Qt framework to develop a graphical user interface (GUI), serving as a bridge between the user and functional services. This interface can retrieve and visualize data in real time and allows operators to input or select specific mission commands, which are then processed by the functional services and sent to the twin AUV for execution. Simultaneously, a highly realistic marine environment visualization sub-window is constructed using the Unity 3D engine, providing an immersive visual experience.

[0034] The underwater intelligent digital twin system based on Unity-Qt-ROS ensures seamless and real-time data transmission between the human-machine interface, the AUV entity, and the digital ocean space by constructing an efficient ROS-Qt and ROS-Unity layered communication mechanism, thereby achieving efficient integration of the virtual and real worlds and real-time human-machine interaction.

[0035] This embodiment reduces the cost, risk, and complexity of AUV intelligent algorithm testing and accelerates its transformation from R&D to practical application. By constructing a high-fidelity digital ocean space and achieving efficient virtual-real integration and real-time human-machine interaction, the system can reflect reality from the virtual and control reality from the virtual, providing a safe and efficient testing platform for intelligent algorithms.

[0036] Example 2 A method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS, provided by the present invention, includes: Step S1: Capture real-world data based on the AUV physical prototype and its onboard physical sensors, and realize the mapping from physical space to digital space through real-time communication technology; Step S2: Construct a digital mapping body and a digital native body; wherein, the digital mapping body is a digital representation of a physical entity; the digital native body includes a digital native scene and digital native sensors; the digital native scene is used to realize the visualization of the underwater environment; the digital native sensors are used to simulate the real sensor perception process; Step S3: Obtain the target task through the graphical user interface, transmit the target task to the function service module, use the function service module to decompose the target task to obtain multiple sub-tasks, and transmit each sub-task to the digital space module and / or physical entity module for execution. During the execution process, real-time data interaction and state synchronization are realized through the message passing mechanism between the digital space module and the physical entity module, thereby completing the virtual-real real-time interaction from digital space to physical entity.

[0037] Specifically, the AUV physical prototype is equipped with a forward-looking sonar, a side-scan sonar, a global positioning system, an inertial navigation unit, a depth gauge, a Doppler velocimeter, and environmental perception and state measurement sensors. Meanwhile, the AUV physical prototype integrates multiple communication methods including WI-FI, radio, fiber optic microcable and acoustic; it is equipped with a high-performance microcomputer, on which the Robot Operating System (ROS) is deployed, which is used to achieve communication through a message passing mechanism.

[0038] Specifically, the digital mapping body is a high-fidelity digital reproduction of a physical entity, including its shape, size, material, texture, lighting, and motion behavior; The digital native scene is constructed using 3D Max tools, including underwater terrain and static and dynamic obstacles, and then imported into the Unity3D engine. The built-in PhysX physics engine of Unity3D engine is used for real-time rendering to achieve a visual presentation of the underwater environment. The digital native sensor simulates the real sensor's sensing process by simulating the working mode of real acoustic and optical sensors, and then realizes the effective data output of the digital native sensor in the underwater environment through script compilation. Digital native sensors are integrated into the digital mapping body, and virtual environment information is perceived in real time through the interaction between the digital native sensors and the digital native scene.

[0039] Specifically, the graphical user interface is developed using the Qt framework; The target task is obtained through a graphical user interface and transmitted to the functional service module via ROS. The functional service module decomposes the target task into multiple subtasks and transmits each subtask to the digital space module and / or physical entity module for execution via ROS. During execution, message passing between the digital space module and the physical entity module is achieved through ROS.

[0040] Specifically, the method further includes: an AUV-Unity 3D communication link and an AUV-Qt communication link; The AUV-Unity 3D communication link is used to map the real-time status of the AUV onto the virtual ocean environment, enabling seamless data integration and instant visual feedback. The AUV-Qt communication link is used to send user commands and receive AUV status, enabling efficient human-machine interaction.

[0041] This embodiment integrates Unity 3D's high-fidelity rendering, Qt's user-friendly human-computer interaction, and ROS's distributed robot framework to build a virtual-real fusion, real-time interactive AUV algorithm R&D and verification platform. This platform can effectively reduce the risks and costs of algorithm verification in complex scenarios, improve the efficiency of algorithm testing and verification, and provide strong technical support for underwater intelligent algorithms to move from theory to engineering application.

[0042] Example 3 Example 3 is a preferred example of Example 1. An underwater intelligent digital twin system based on Unity-Qt-ROS, provided by the present invention, includes: A Linux operating system environment was configured in the AUV physical prototype, and the ROS operating system was deployed on it to ensure the development, operation, and maintenance of the AUV embedded system. ROS, through its hardware abstraction layer mechanism, provides a unified interface for different hardware platforms, such as PCs and ARM processors, facilitating code portability. This is particularly important for the cross-platform deployment of algorithms on AUV embedded systems and PC-based simulation systems. Simultaneously, ROS's distributed architecture decomposes complex robot tasks into multiple independent nodes, achieving communication between nodes through an efficient message passing mechanism. This simplifies the code structure and ensures real-time interaction between the virtual and physical worlds. More specifically, the user issues target task instructions to the robot system through the human-machine interface. After acquiring the target task, the robot system, based on ROS's distributed architecture, decomposes the target task into multiple logically independent but functionally collaborative subtasks. Each subtask is deployed as an independent node in ROS, distributed across the physical entity (the robot body) and / or the corresponding computing unit in the digital space. Through ROS's built-in efficient message passing mechanism, real-time data interaction and state synchronization are achieved between nodes, thereby completing the real-time virtual-real interaction process from digital space instruction input to physical entity behavior execution. This architecture not only simplifies the code structure of complex tasks but also ensures the collaborative operation of each sub-task node in a distributed environment, guaranteeing the real-time performance and reliability of the robot system.

[0043] In addition, ROS has a large ecosystem that integrates a large number of tool libraries for robot perception, decision-making, and motion control, which greatly improves the efficiency and flexibility of system development.

[0044] The digital space layer is constructed based on the Unity 3D engine. Unity 3D uses C# as its primary programming language, supporting flexible cross-platform development (such as Windows, Linux, Mac OS, etc.) and strong scalability. In particular, Unity 3D can communicate with ROS in real time via TCP / IP protocol, providing the necessary conditions for high-fidelity virtual ocean environments to interact with real AUVs. First, a high-fidelity digital ocean environment is created. The seabed terrain model is generated using World Machine software. Its advanced Berlin and basic noise generators, along with filters such as smoothing and blurring, work together to output the geometric collision model and appearance texture model of the terrain, such as... Figure 3 As shown. The underwater obstacle model was built using SolidWorks and includes both static and dynamic obstacles, such as... Figure 4a , Figures 4b to 5As shown. Simultaneously, a simulated digital native sensor, such as a multibeam echo sounder simulated using Unity's Physics.Raycast method, calculates the distance information returned by the corresponding beam by detecting whether it intersects with a collision model in the environment. This information is then used as input to an environmental perception algorithm to complete the underwater environmental perception task, such as... Figure 6 As shown. Finally, a 3D model of the real AUV at scale is created, as shown... Figure 7 As shown, import it into Unity 3D for visualization rendering and collision detection.

[0045] like Figures 8a to 8l As shown, the human-computer interaction interface is developed using the Qt technology framework, including the system settings interface, such as... Figure 8a As shown; the message listening interface, as Figure 8b As shown; Mission Management Interface, as Figures 8c to 8g As shown, where, Figure 8c This is the mission management interface in manual mode; Figures 8d to 8g This refers to the mission management interface in automatic mode; the message storage interface, such as... Figure 8h As shown; visual interface, such as Figure 8i As shown; the security management interface, such as Figure 8j As shown; the log printing interface, as... Figure 8k As shown; the operation data management interface, such as Figure 8l As shown. Its applications are built on C++, support cross-platform compilation and execution, and possess rich user interface components and customization capabilities, as well as an efficient signal and slot mechanism. More importantly, Qt applications can be seamlessly integrated into the ROS ecosystem, running as independent nodes and exchanging information efficiently with other ROS nodes through topic subscription and publish mechanisms. Furthermore, Qt applications can also act as clients or servers to handle service requests, listening, and responses. By combining Qt's signal and slot mechanism with ROS communication mechanisms, responsive and feature-rich human-computer interaction applications can be built, providing users with a more user-friendly experience.

[0046] The system also includes a complete closed-loop system for virtual-real interaction and human-computer interaction, constructed through two major information flow links: AUV-Unity 3D and AUV-Qt. The AUV-Unity 3D communication link ensures that the real-time status of the AUV is accurately mapped to the virtual ocean environment, achieving seamless data integration and instant visual feedback. The AUV-Qt communication link focuses on sending user commands and receiving AUV status information, enabling efficient human-computer interaction. This strategy embodies a loosely coupled and modular design philosophy at the system development level, optimizing the development process and facilitating subsequent system maintenance and upgrades.

[0047] like Figures 9 to 11As shown, to verify the effectiveness of this system, a series of digital twin experiments were conducted in lakes and towed pools. Experiment 1: Using Wi-Fi communication technology, an autonomous constant-speed cruise and autonomous directional navigation control algorithm test experiment was conducted on an open lake surface. This demonstrated that the system achieves precise control of the real world from the digital world, highlighting the system's algorithm testing capability of "reflecting reality with the virtual and controlling reality with the virtual." More specifically, during the virtual-real interaction process, the digital world, based on the real-time position, speed, and other information fed back by the real AUV, as well as the virtual environment data perceived by the virtual sensors, invokes the built-in perception, planning, and control algorithms to calculate the control commands for the rudder and thrusters. These commands are then sent to the real AUV through the ROS communication mechanism, thereby driving its movement.

[0048] Experiment 2: Using fiber optic communication in a towed pool, more complex autonomous speed, orientation, and depth navigation tests were conducted. This not only verified the stability and adaptability of the system under different communication conditions, but also more intuitively demonstrated that the system achieved two-way interaction between the digital and physical worlds, and further verified the system's algorithm testing capability of "controlling the real world with the virtual".

[0049] Experiment 3: Using fiber optic communication in a towed pool, the perception, planning and control algorithms for underwater obstacle avoidance were tested. In the experiment, the AUV not only completed the waypoint tracking task in the digital native scene under virtual-real interaction, but also successfully avoided digital native obstacles in the digital native scene, demonstrating the system's powerful ability to test advanced algorithms in complex task scenarios.

[0050] This embodiment verifies through a series of digital twin experiments on lakes and towed pools that it can reliably achieve the function of "reflecting reality with the virtual and controlling reality with the virtual". That is, the digital AUV can accurately map the dynamic behavior of the real AUV in complex underwater environments and control the real AUV in real time to complete predetermined tasks such as obstacle avoidance in complex digital native marine test scenarios. This significantly reduces the risk and economic cost of algorithm verification in complex scenarios and improves the efficiency of test verification.

[0051] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0052] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An underwater intelligent digital twin system based on Unity-Qt-ROS, characterized in that, include: Physical entity: Based on the AUV physical prototype and its onboard real physical sensors, real data of the real world is captured, and the mapping from physical space to digital space is realized through real-time communication technology; The digital space module includes: a digital mapping body and a digital native body; wherein, the digital mapping body is a digital representation of a physical entity; the digital native body includes a digital native scene and digital native sensors; the digital native scene is used to realize the visualization of the underwater environment; the digital native sensors are used to simulate the real sensor perception process; Human-Computer Interaction Module: The module acquires the target task through a graphical user interface, transmits the target task to the functional service module, uses the functional service module to break down the target task into multiple subtasks, and transmits each subtask to the digital space module and / or physical entity module for execution. During execution, real-time data interaction and state synchronization are achieved through the message passing mechanism between the digital space module and the physical entity module, thereby completing the real-time virtual-real interaction from digital space to physical entity.

2. The underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 1, characterized in that, The physical entity includes: the AUV physical prototype is equipped with a forward-looking sonar, a side-scan sonar, a global positioning system, an inertial navigation unit, a depth gauge, a Doppler velocimeter, environmental perception and state measurement sensors; Meanwhile, the AUV physical prototype integrates multiple communication methods including WI-FI, radio, fiber optic microcable and acoustic; it is equipped with a high-performance microcomputer, on which the Robot Operating System (ROS) is deployed, which is used to achieve communication through a message passing mechanism.

3. The underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 1, characterized in that, The digital mapping volume is a high-fidelity digital reproduction of a physical entity, including its shape, size, material, texture, lighting, and motion behavior. The digital native scene is constructed using 3D Max tools, including underwater terrain and static and dynamic obstacles, and then imported into the Unity3D engine. The built-in PhysX physics engine of Unity3D engine is used for real-time rendering to achieve a visual presentation of the underwater environment. The digital native sensor simulates the real sensor's sensing process by simulating the working mode of real acoustic and optical sensors, and then realizes the effective data output of the digital native sensor in the underwater environment through script compilation. Digital native sensors are integrated into the digital mapping body, and virtual environment information is perceived in real time through the interaction between the digital native sensors and the digital native scene.

4. The underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 1, characterized in that, The graphical user interface in the human-computer interaction module is developed using the Qt framework; The target task is obtained through a graphical user interface and transmitted to the functional service module via ROS. The functional service module decomposes the target task into multiple subtasks and transmits each subtask to the digital space module and / or physical entity module for execution via ROS. During execution, message passing between the digital space module and the physical entity module is achieved through ROS.

5. The underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 1, characterized in that, The system also includes: an AUV-Unity 3D communication link and an AUV-Qt communication link; The AUV-Unity 3D communication link is used to map the real-time status of the AUV onto the virtual ocean environment, enabling seamless data integration and instant visual feedback. The AUV-Qt communication link is used to send user commands and receive AUV status, enabling efficient human-computer interaction.

6. A method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS, characterized in that, include: Step S1: Capture real-world data based on the AUV physical prototype and its onboard physical sensors, and realize the mapping from physical space to digital space through real-time communication technology; Step S2: Construct a digital mapping body and a digital native body; wherein, the digital mapping body is a digital representation of a physical entity; the digital native body includes a digital native scene and digital native sensors; the digital native scene is used to realize the visualization of the underwater environment; the digital native sensors are used to simulate the real sensor perception process; Step S3: Obtain the target task through the graphical user interface, transmit the target task to the function service module, use the function service module to decompose the target task to obtain multiple sub-tasks, and transmit each sub-task to the digital space module and / or physical entity module for execution. During the execution process, real-time data interaction and state synchronization are realized through the message passing mechanism between the digital space module and the physical entity module, thereby completing the virtual-real real-time interaction from digital space to physical entity.

7. The method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 6, characterized in that, The physical prototype of the AUV is equipped with a forward-looking sonar, a side-scan sonar, a global positioning system, an inertial navigation unit, a depth gauge, a Doppler velocimeter, and environmental perception and state measurement sensors. Meanwhile, the AUV physical prototype integrates multiple communication methods including WI-FI, radio, fiber optic microcable and acoustic; it is equipped with a high-performance microcomputer, on which the Robot Operating System (ROS) is deployed, which is used to achieve communication through a message passing mechanism.

8. The method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 6, characterized in that, The digital mapping volume is a high-fidelity digital reproduction of a physical entity, including its shape, size, material, texture, lighting, and motion behavior. The digital native scene is constructed using 3D Max tools, including underwater terrain and static and dynamic obstacles, and then imported into the Unity3D engine. The built-in PhysX physics engine of Unity3D engine is used for real-time rendering to achieve a visual presentation of the underwater environment. The digital native sensor simulates the real sensor's sensing process by simulating the working mode of real acoustic and optical sensors, and then realizes the effective data output of the digital native sensor in the underwater environment through script compilation. Digital native sensors are integrated into the digital mapping body, and virtual environment information is perceived in real time through the interaction between the digital native sensors and the digital native scene.

9. The method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 6, characterized in that, The graphical user interface was developed using the Qt framework; The target task is obtained through a graphical user interface and transmitted to the functional service module via ROS. The functional service module decomposes the target task into multiple subtasks and transmits each subtask to the digital space module and / or physical entity module for execution via ROS. During execution, message passing between the digital space module and the physical entity module is achieved through ROS.

10. The method for implementing an underwater intelligent digital twin system based on Unity-Qt-ROS according to claim 6, characterized in that, The method also includes: an AUV-Unity 3D communication link and an AUV-Qt communication link; The AUV-Unity 3D communication link is used to map the real-time status of the AUV onto the virtual ocean environment, enabling seamless data integration and instant visual feedback. The AUV-Qt communication link is used to send user commands and receive AUV status, enabling efficient human-computer interaction.

Images