Marine Information Experiment Teaching Platform Based on Master-Slave Remote Operating System

Abstract

This invention belongs to the technical field of underwater acoustic test platforms, specifically relating to a marine information experimental teaching platform based on a master-slave teleoperation system. The invention includes a master device and a slave device. The master device carries a master-slave teleoperation system, within which a virtual model of the slave device is established. Experimenters operate the virtual model of the slave device through the master-slave teleoperation system. The system converts the actions of the virtual model into control commands and transmits them to the slave device. Upon receiving the control commands, the slave device adjusts the position and orientation of the transducer and hydrophone. Experimenters send electrical signals to the transducer through the system, which converts these signals into underwater acoustic signals that propagate in the experimental pool. The hydrophone receives these signals and feeds them back to the master device, enabling remote teleoperation of the experimental process by the experimenters.

Description

Technical Field

[0001] This invention belongs to the technical field of underwater acoustic test platforms, specifically relating to a marine information experimental teaching platform based on a master-slave teleoperation system. Background Technology

[0002] Marine information is carried by various physical fields, including sound, light, electricity, and magnetism. The fundamental concept of marine information engineering is the perception, acquisition, transmission, processing, and integrated application of marine information, enabling observation, detection, and monitoring of the sea, both at sea and for the benefit of the sea. Experimental teaching platforms are crucial for theoretical research and engineering practice. However, experimental teaching in marine information engineering faces several challenges. The complexity of the marine environment, the multi-source nature of information, the sheer volume of data, the intelligence required for processing, and the diversity of systems make conducting marine information-related experiments extremely difficult. Furthermore, the high cost of related equipment hinders large-scale undergraduate experiments. Applying remote human-computer collaboration systems and technologies to experimental teaching can help achieve deep integration of information technology and education, promote resource sharing, improve educational quality and efficiency, and foster a new education system adapted to the digital age.

[0003] In 2006, Zhou Siyue of Shanghai University designed and implemented a robotic dexterous hand telemanipulation platform based on virtual reality technology. This platform uses a data glove to capture the operator's hand movements, employs the OpenGL development library to provide users with basic primitive drawing commands, and utilizes position sensors, angle sensors, and force feedback technology to control and manipulate the dexterous hand. However, the data glove has limited stability and flexibility, and is easily affected by minute hand movements.

[0004] In 2011, Wang Biao and Zeng Qingjun of Jiangsu University of Science and Technology designed a small underwater information communication network and conducted an innovative design experiment on microwave passive networks to achieve underwater-to-surface communication. The network uses buoy gateway nodes to enable wireless access between underwater information and land-based systems. However, the underwater acoustic signal propagation suffers from significant loss and delay, and the system's stability in complex underwater environments needs improvement.

[0005] In 2019, Liu Shengxing of Xiamen University constructed an experimental teaching platform for underwater acoustic communication based on a PC and a sound card. This platform simplifies hardware configuration and implements communication processes such as modulation / demodulation and channel coding / decoding through software, reducing the abstractness of communication theory and stimulating students' practical and innovative abilities.

[0006] In 2021, Liu Ni from Jiangsu University of Science and Technology used Unity3D and MATLAB co-simulation to build a virtual underwater robotic arm teleoperation simulation platform, which verified the angle tracking and force feedback functions of the master and slave robotic arms in the virtual environment. However, the simulation analysis of this virtual force feedback is difficult to completely simulate the real underwater environment.

[0007] In 2021, Zhan Hong, Liang Congyuan, and others from South China University of Technology proposed a robot teleoperation experimental platform based on mixed reality technology. This platform detects the operator's hand movements through Leap Motion and displays a fused real and virtual scene through a mixed reality display device (such as Microsoft HoloLens). However, the system has limited application scenarios, mainly designed for terrestrial environments, and lacks adaptability for underwater operation.

[0008] In 2023, Yang Junhao, Wang Bingyan, and others from the China Nuclear Power Research and Design Institute studied a robot control system for underwater cleaning tasks, employing the UR5 underwater serial robot. A workspace mapping algorithm was used to synchronize the actions of the master and slave devices, thereby improving the efficiency and accuracy of the cleaning task. However, this workspace mapping algorithm has a long response time and limited coverage.

[0009] In 2023, Zhang Yanan of Zhejiang University researched a multi-modal human-computer interaction teleoperation technology based on binocular vision and force feedback for ultrasonic scanning of underwater components. This technology improves the accuracy of 3D reconstruction and the fidelity of remote force perception reproduction, while reducing the operational difficulty for operators. However, the system is highly complex, requiring significant maintenance and training costs, which is not conducive to basic teaching. Summary of the Invention

[0010] The purpose of this invention is to provide a marine information experimental teaching platform based on a master-slave teleoperation system. This platform integrates human-computer collaboration technology to enable remote teleoperation of the experimental process. Through the local intelligence of the underwater remote execution system and the on-site digital model, it provides students and teachers with a semi-physical experimental scenario that combines virtual and real elements. Simultaneously, the on-site experimental operation device provides students and teachers with a sense of remote presence. The platform is characterized by its close alignment with engineering applications and high flexibility. It can also improve sensor systems, enhance operational accuracy, and provide support for scientific research.

[0011] The marine information experimental teaching platform based on the master-slave teleoperation system includes master-end equipment and slave-end equipment. The slave-end equipment includes an experimental pool, a longitudinal guide rail installed on the experimental pool, a transducer-end moving beam and a hydrophone-end moving beam installed on the longitudinal guide rail, and a transducer and a hydrophone set in the experimental pool. The transducer is installed below the transducer-end moving beam via a first connecting rod, and a first indexing plate is provided on the first connecting rod. The top surface of the hydrophone-end moving beam is provided with a transverse guide rail, and a slider is provided on the transverse guide rail. The hydrophone is installed below the slider via a second connecting rod, and a second indexing plate is provided on the second connecting rod.

[0012] The master device is equipped with a master-slave teleoperation system, in which a virtual model of the slave device is established. The experimenter operates the virtual model of the slave device through the master-slave teleoperation system, which converts the actions of the virtual model into control commands and transmits them to the slave device. Upon receiving the control commands, the slave device controls the rotation angle of the transducer and hydrophone via the first and second indexing dials. It adjusts the longitudinal position of the transducer and hydrophone by moving the transducer-end moving beam and the hydrophone-end moving beam on the longitudinal guide rail, and adjusts the lateral position of the hydrophone by moving the slider on the transverse guide rail on the top surface of the hydrophone-end moving beam. The experimenter sends electrical signals to the transducer through the master-slave teleoperation system. The transducer converts the electrical signals into underwater acoustic signals that propagate in the experimental pool. The hydrophone receives the underwater acoustic signals and feeds them back to the master device, enabling the experimenter to remotely control the experimental process.

[0013] Furthermore, a virtual model of the slave device is established in the master-slave remote operating system, specifically as follows:

[0014] Establish a world coordinate system O with the vertex of one end of the experimental pool as the origin. w xyz; Establish a model of the slave device in the world coordinate system, and set the hydrophone end coordinate system O. A xyz and transducer end coordinate system O B xyz;

[0015] Hydrophone end coordinate system O A xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T A for:

[0016]

[0017] Transducer end coordinate system O B xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T B for:

[0018]

[0019] Hydrophone end coordinate system O A xyz to transducer end coordinate system O B The homogeneous transformation matrix of xyz B T A for:

[0020]

[0021] Where, x A y represents the distance the hydrophone extends along the longitudinal guide rail;A denoted as , where is the distance the hydrophone extends along the moving beam at the hydrophone end; a is the length of the longitudinal guide rail; b is the length of the moving beam at the transducer end; z is... A The underwater depth of the hydrophone; z B θ represents the underwater depth of the transducer. A θ is the rotation angle of the hydrophone. B The angle of rotation of the transducer is denoted as .

[0022] Furthermore, to achieve precise control of the slave device, mapping is performed within the workspace. The poses of the hydrophone and transducer are calculated using a forward kinematics model. The desired poses of each actuator in the slave device are obtained according to the mapping algorithm. Then, the position or angle of each actuator is obtained according to the inverse kinematics algorithm. The actuator includes a transducer end moving crossbeam, a hydrophone end moving crossbeam, a first indexing plate, a second indexing plate, and a slider.

[0023] The workspace mapping algorithm uses world coordinates O w Using the xyz coordinate system as a reference, the pose of the virtual model in the slave device and the master device remains consistent. The movement trajectory of the virtual model in the master device is analyzed in real time, and the real-time data is sent to the slave device to complete the slave device's tracking of the virtual model in the master device. Based on the movement range of the master and slave devices, the mapping coefficients satisfy the following conditions:

[0024]

[0025] Where, k n U represents the mapping scaling factor in the n-direction, where n = x, y, z; n,max U represents the maximum range of motion of the hydrophone or transducer in the n-direction within the slave device. n,min T represents the minimum range of motion of the hydrophone or transducer in the n-direction within the slave device; n,max T represents the maximum range of motion of the hydrophone or transducer in the n-direction within the virtual model of the master device; n,min This represents the minimum range of motion of the hydrophone or transducer in the n-direction within the virtual model of the master device. To ensure that the pose of the virtual model of the master device matches the workspace range of the slave device after mapping, the mapping formulas for the hydrophone end and the transducer end are as follows:

[0026]

[0027] Among them, [x u1 y u1 z u1 θ u1 ] T Indicates the desired position and orientation of the hydrophone in the slave device; [x t1 y t1 zt1 θ t1 ] T This indicates the actual location of the hydrophone in the virtual model of the master device; [x u2 θ u2 ] T Indicates the desired position and orientation of the transducer in the slave device; [x t2 θ t2 ] T This indicates the actual location of the transducer in the virtual model of the master device;

[0028] The actual positions and orientations of the hydrophones and transducers in the virtual model of the master device can be mapped to the desired positions and orientations of the hydrophones and transducers in the slave device. Then, the movement trajectories and rotation angles of each actuator at the hydrophone and transducer end are obtained according to the inverse kinematics formula of the slave device, thereby realizing remote operation control of the master and slave devices.

[0029] Furthermore, the master-slave remote operating system includes a network communication connection and monitoring module, a VE human-computer interaction control module, an AR human-computer interaction control module, a view switching module, and a data display and analysis module;

[0030] The network communication connection and monitoring module is used to set the connection addresses of the master device and the slave device, and to monitor the network communication connection status and network signal strength.

[0031] The VE human-computer interaction control module is used to construct a virtual model of the slave device and to allow experimenters to perform operations on the virtual model of the slave device.

[0032] The AR human-computer interaction control module is used to provide an augmented reality environment. Experimenters can operate the master device in the augmented reality environment and restore the sense of presence of the underwater environment while controlling the slave device in real time.

[0033] The view switching module is used by experimenters to observe the virtual model of the slave device from different angles, including front view, side view and top view.

[0034] The data display and analysis module is used to display the underwater acoustic signals received by the hydrophone in the slave device in real time, and also to convert the operation of the virtual model by the experimenter into control commands and transmit them to the slave device.

[0035] Furthermore, to account for the delay in the underwater propagation of the analog signal, a buffer is set in the master-slave remote operating system, and this buffer is configured to store N command records, which are used to temporarily store the instructions generated by the master device. During the control process, the master device generates control instructions according to a preset time interval ΔT. The control instructions are not directly transmitted to the slave device, but are stored at the end of the buffer. At the same time, the earliest control instruction is extracted from the top of the buffer and transmitted to the slave device through the network, so that the slave device executes the corresponding control instruction. Then, the control instruction is deleted from the buffer. The residence time of each control instruction in the buffer is recorded, which is the delay T of the analog signal during underwater propagation, T = N × ΔT.

[0036] Furthermore, the delay process of the analog signal during underwater propagation is specifically as follows:

[0037] 1) Initialization; the master-slave remote operating system reads the preset time interval ΔT to determine the frequency of data acquisition and processing; calculates the delay mid-range boundary value to evaluate subsequent data delay; the master-slave remote operating system initializes the record sequence using an empty string;

[0038] 2) Fixed delay process: Start the timer to ensure that the time interval for instruction checking meets the requirements; extract the control instruction from the top layer of the buffer and determine whether the current control instruction is a valid command from the upper layer. If the determination is "yes", execute the control instruction and place the new control instruction at the bottom of the buffer to ensure that subsequent control instructions are processed in order, thus implementing the FIFO instruction execution logic; if the determination is "no", do not perform any further operations.

[0039] 3) Delay and Record Update Process: If dynamic updates to the master-slave remote operating system (MARS) are required regarding delay, the latest delay status is first recorded and the delay data is saved to the MARS. Subsequently, the MARS determines whether the delay has changed based on control commands issued from the buffer. If the delay increases, the current delay change value is updated, and various delay indicators, including area delay, global delay, and the accumulated delay value of power equipment, are calculated and accumulated. These data are recorded in the corresponding delay record table for subsequent analysis and optimization. If the delay does not change, the MARS directly returns to wait for the next detection cycle. Through this procedure, the MARS can dynamically track the delay status, ensure the real-time nature of delay information, and provide support for delay optimization.

[0040] The beneficial effects of this invention are as follows:

[0041] This invention integrates human-machine collaboration technology, utilizing a master-slave teleoperation system embedded in the master device to remotely control the experimental process, providing researchers with a semi-physical experimental scenario and a sense of presence that combines virtual and real elements. While retaining the basic functions of a large dedicated water tank, this invention features a miniaturized size, effectively resolving the contradiction between the large amount of teaching time and the limited amount of teaching equipment. Within the limited space of a laboratory, various underwater testing environments are simulated in the water tank testing space using transducers and receivers of different frequency bands. Combined with corresponding parameter testing methods and signal processing techniques, experimental operations for collecting marine information are completed. This invention features high stability and reliability, simple operation, and convenient maintenance. Attached Figure Description

[0042] Figure 1 This is a system block diagram of a marine information experimental teaching platform based on a master-slave teleoperation system.

[0043] Figure 2 This is a control principle diagram.

[0044] Figure 3 For control operation architecture diagram.

[0045] Figure 4 This is a schematic diagram of the slave device.

[0046] Figure 5 This is a diagram of a network communication interface.

[0047] Figure 6 This is a diagram of the VE human-computer interaction interface.

[0048] Figure 7 This is a diagram of an AR human-computer interaction interface.

[0049] Figure 8 Screenshot of the data viewing interface.

[0050] Figure 9 Flowchart for delay initialization.

[0051] Figure 10 This is a flowchart of a fixed-delay program.

[0052] Figure 11 Flowchart for the delay update process.

[0053] Figure 12 A schematic diagram is created for the kinematic coordinate system.

[0054] Figure 13 This is a graph showing the position-time variation of the hydrophone.

[0055] Figure 14 This is a graph showing the speed-time variation of the hydrophone.

[0056] Figure 15 This is a graph showing the acceleration-time variation of a hydrophone.

[0057] Figure 16 This is a graph showing the transducer's position, velocity, and acceleration over time.

[0058] Figure 17 This is a graph showing the rotation angle of the hydrophone and transducer over time.

[0059] Figure 18 This is a diagram illustrating the reverberation time.

[0060] Figure 19 This is a schematic diagram of the spatial average sound pressure level. Detailed Implementation

[0061] The present invention will now be further described with reference to the accompanying drawings.

[0062] To address the challenge of conducting underwater acoustic experiments in complex marine environments in real-world conditions, this invention designs a marine information experimental teaching platform based on a master-slave teleoperation system. This platform simulates different underwater acoustic environments within a laboratory using acoustic transmitting and receiving transducers, enabling parameter testing of acoustic equipment. Employing remote control and intelligent technologies, combined with an underwater execution system and digital models, it provides a semi-physical experimental scenario that blends virtual and real elements. This invention improves teaching efficiency and student participation, meeting the needs of personalized and flexible teaching. Furthermore, this innovative approach deeply integrates information technology with underwater acoustic engineering education, promoting the development of educational informatization and possessing significant potential for widespread application.

[0063] The marine information experimental teaching platform based on a master-slave teleoperation system provided by this invention includes a master device and a slave device; such as Figure 4 As shown, the slave device comprises two robot structures. The transducer end (transmitter end) is a 2-DOF robot with vertical movement and rotation capabilities, including a transducer end moving beam, a first link, and a first indexing plate; the hydrophone end (receiver end) is a 4-DOF robot with three-dimensional Cartesian coordinate movement and rotation capabilities, including a hydrophone end moving beam, a transverse guide rail on the top surface of the hydrophone end moving beam, a slider on the transverse guide rail, a second link, and a second indexing plate.

[0064] The master device is equipped with a master-slave teleoperation system, within which a virtual model of the slave device is established. Experimenters operate the virtual model of the slave device through the master-slave teleoperation system, which converts the actions of the virtual model into control commands and transmits them to the slave device. Upon receiving the control commands, the slave device controls the rotation angle of the transducer and hydrophone via the first and second indexing dials. It adjusts the longitudinal position of the transducer and hydrophone by moving the transducer-end moving beam and the hydrophone-end moving beam on the longitudinal guide rail, and adjusts the lateral position of the hydrophone by moving the slider on the transverse guide rail on the top surface of the hydrophone-end moving beam. Experimenters send electrical signals to the transducer through the master-slave teleoperation system. The transducer converts these signals into underwater acoustic signals that propagate in the experimental pool. The hydrophone receives these signals and feeds them back to the master device, enabling remote teleoperation of the experimental process. This master-slave teleoperation combined with the underwater acoustic equipment provides students and teachers with a semi-physical experimental scenario that blends virtual and real elements, offering a sense of remote presence.

[0065] In a master-slave teleoperation system, the control architecture is crucial to ensuring that the master operator can accurately control the remote robot to perform tasks. This paper will detail the components of this architecture and its working principles. The master-slave teleoperation system consists of a master device, a slave device, a communication module, and a control algorithm. Figure 1 The overall architecture of the system is shown.

[0066] The master device mainly includes various human-machine interface devices, such as computers, tablets, and smartphones. These devices are used by the operator to input control commands and receive feedback information from the slave devices. The master device interacts with the operator through a graphical user interface (GUI), providing functions such as remote control, real-time monitoring, and data analysis.

[0067] The human-machine interface is one of the core components of the main device, and its main functions include:

[0068] 1) Control command input: The operator inputs control commands through a computer or a handheld device.

[0069] 2) Feedback information display: The system displays the real-time data collected from the end device (such as location information, speed information, acoustic information, underwater environment images, etc.) to the operator in a graphical manner.

[0070] 3) Data analysis and processing: Real-time analysis of feedback data from the end to assist operators in decision-making.

[0071] The slave device is the one that performs specific tasks. Its main function is to receive control commands from the master device, execute corresponding actions, and feed back the execution results, as well as information such as underwater acoustics and environmental conditions, to the master device. The slave device is equipped with a high-precision control module responsible for processing the master device's instructions and controlling the actions. The main components of the control module include:

[0072] 1) Position control: Adjust the robot's position according to the master command.

[0073] 2) Speed ​​control: Controlling the robot's movement speed to ensure smooth operation.

[0074] 3) Feedback system: Real-time monitoring of robot status and underwater sound information, and feedback of data such as position, speed, and underwater sound.

[0075] The communication module is responsible for transmitting control commands and feedback data between the master and slave devices. The design of the communication module needs to consider factors such as latency, bandwidth, and reliability. The system typically uses wireless communication technologies (such as Wi-Fi, 4G / 5G, etc.) to achieve remote data transmission. The advantages of wireless communication are its flexibility and wide coverage. To ensure the coordinated operation of the master and slave devices, the communication module implements an efficient data synchronization mechanism, reducing latency and data transmission errors, ensuring that every command from teachers and students is transmitted to the slave devices in a timely and accurate manner.

[0076] The control algorithm is the core of the master-slave teleoperation system, determining its response speed, accuracy, and stability. This invention employs Proportional-Integral-Derivative (PID) control. PID control is a classic feedback control algorithm that minimizes the error between the system output and the desired value by adjusting control parameters. PID control is widely used in master-slave teleoperation systems due to its advantages such as simple implementation and fast response. The control system structure diagram is shown below. Figure 2 As shown.

[0077] This invention allows experimenters to control slave devices remotely by inputting commands, enabling remote manipulation of the slave devices to a specified position and emitting underwater acoustic signals at the transducer end, while the hydrophone end performs data acquisition and transmission. The system mainly consists of the following parts: an operator terminal, a control system for the experimental platform, a robot control cabinet, and slave devices, etc. Figure 3 As shown.

[0078] The experimenters interact with the system through a control terminal. This terminal, equipped with a human-machine interface, runs the control software to control the slave devices and the experimental platform. The experimental platform's control system manages and coordinates the various components on the platform, communicating with the operator's terminal and the underwater acoustic equipment via TCP / IP. The robot control cabinet is the core component controlling the robots; it includes a motion control system and control algorithms / software. It receives commands from the operator's terminal and translates them into actions executed by the two slave robots. The transducer and hydrophone ends in the slave devices are located in different positions, responsible for adjusting their respective postures to meet experimental requirements. The transducer receives underwater acoustic data from the experimental platform's control system and sends underwater acoustic signals to the hydrophone. The hydrophone receives underwater acoustic signals from the transducer and sends underwater acoustic data to the experimental platform's control system.

[0079] The software system includes a network communication connection and monitoring module, a VE (Virtual Environment) human-computer interaction control module, an AR (Augmented Reality) human-computer interaction control module, a view switching module, and a data display and analysis module. These modules together construct a complete underwater teleoperation experimental platform, which can measure, process, and analyze underwater acoustic signals through the control of underwater devices to complete experimental teaching and scientific research.

[0080] The network settings module is used to configure the connection addresses of the remote robot and the master robot.

[0081] Remote robot address: Students need to enter the IP address of the remote robot in the input box to connect to the corresponding device.

[0082] Master robot address: Enter the IP address of the master robot.

[0083] Connection status: The current network connection status is indicated by color (green or red), with green indicating that the network is connected and red indicating that it is not connected or that the connection is incorrect.

[0084] Signal Strength: Displays the current network signal strength, helping teachers and students assess network quality. Low signal strength may require adjusting network settings or checking the network environment. Figure 5 As shown.

[0085] The VE human-machine interaction control module provides 3D graphical display and control functions for the robot, and is the core module for remote operation. This module intuitively allows teachers and students to understand and operate the master device to control the slave device. It includes the following functions:

[0086] Network Connection Test: Enter the IP addresses of the remote and master robots in the network settings module to confirm the network connection status. 3D Control Test: Test the robot's movement and posture changes by inputting different X, Y, Z coordinate values ​​and joint angle values. 3D Model Display: This section displays the robot's 3D model, allowing teachers and students to observe the robot in detail through rotation, zoom, and other operations. Position Adjustment: X-axis, Y-axis, and Z-axis Input Boxes: Users can enter specific coordinate values ​​in the corresponding input boxes to adjust the robot's position. If the entered coordinate commands are all within the workspace, the "Execute" button will be clicked for the command to take effect; otherwise, the "Execute" button cannot be clicked. Angle Adjustment: The angles in the joint angle input boxes can be adjusted individually. Users need to enter angle values ​​(in degrees) in the corresponding input boxes.

[0087] View switching test: Switch between front view, side view and top view to observe the display effect of the robot from different perspectives.

[0088] Camera image test: View the live camera feed, take and save images to confirm image quality and the usability of the shooting function.

[0089] The view switching module allows users to view the slave device from different angles, including front view, side view, and top view.

[0090] Front view image: Displays an image of the robot viewed from the front, allowing users to clearly see the robot's front structure.

[0091] Side view image: Displays an image of the robot viewed from the side, allowing users to understand the details of the robot's side profile.

[0092] Top-down view: Shows an image of the robot viewed from above, which is very useful for examining the robot's overall layout and top details. For example... Figure 6 As shown.

[0093] The AR human-computer interaction control module is a core module combining augmented reality technology with remote control, designed to provide teachers and students with a more intuitive and convenient control experience. Through this module, teachers and students can control the master device in an augmented reality environment, simultaneously controlling the slave device in real time while recreating the immersive feeling of an underwater environment—an immersive experience that VE human-computer interaction unlocking cannot provide. The AR human-computer interaction module includes the following functions:

[0094] ① Augmented Reality Environment Display

[0095] The AR environment displays a 3D model of the robot, allowing teachers and students to view the overlay of the robot model in the real world through PC devices or AR glasses, enhancing the interactive experience.

[0096] ② Posture adjustment

[0097] Position Adjustment: The AR human-computer interaction interface includes X-axis, Y-axis, and Z-axis input boxes. Teachers and students can enter specific coordinate values ​​in the corresponding input boxes to adjust the robot's position. Similar to the VE module, the input coordinate commands must be within the workspace, and the command will only take effect after clicking the "Execute" button.

[0098] Angle adjustment: The AR human-computer interaction interface includes a joint angle input box, allowing users to adjust the angle value of each joint individually (in degrees) and observe the adjustment effect in real time through the AR interface.

[0099] ③View switching

[0100] Users can switch between front, side, and top views in the AR interface to observe the robot's display in augmented reality from different angles. These view switches include:

[0101] Front view image: This shows an image of the robot viewed from the front. Teachers and students can clearly see the front structure of the robot. This angle is often used for fine-tuning and can more intuitively and clearly reflect the perspective information from the end device.

[0102] Side view image: Displays an image of the robot viewed from the side, making it easier for teachers and students to understand the details of the robot's side.

[0103] Top-down view: Displays an image of the robot viewed from above, allowing users to check the robot's overall layout and top details. This angle makes it easier to perform main layout during remote operation.

[0104] ④ Camera image test

[0105] Users can view live camera footage, take and save images within the AR interface, and verify image quality and the availability of the shooting function. For example... Figure 7 As shown.

[0106] The data viewing interface for this experiment is as follows: Figure 8 As shown, this is primarily used to display and analyze acoustic data collected during the experiment. The experiment involves measuring the equivalent free-field radiated acoustic power of a spherical sound source in a reverberation tank, with the aim of analyzing changes in the acoustic signal graphically. Operators can select different analysis dimensions to conduct a more in-depth study of the signal and understand the radiated acoustic power characteristics of the sound source under different environments.

[0107] The propagation of signals underwater differs from that in air. Underwater, absorption and scattering by the water medium cause a delay compared to air propagation, and this delay becomes increasingly pronounced with distance. Therefore, in addition to the drag inherent in underwater teleoperation, the technical challenges of teleoperation, especially underwater teleoperation, should primarily focus on addressing the delay of underwater acoustic signals.

[0108] To investigate the impact of underwater network latency on the slave device system, buffering technology was introduced in a laboratory environment to simulate these latency characteristics, including the simulation of high latency and random variations in network transmission. The specific implementation steps are as follows: A buffer is set up, capable of storing N command records, to temporarily store instructions generated by the master device. During control, the master device generates control commands according to a preset time interval ΔT. This command record is not directly transmitted to the slave device but is stored at the end of the buffer. Simultaneously, the earliest command record is extracted from the top of the buffer and transmitted to the slave device via the network, causing the slave device to execute the corresponding control command. Then, this record is deleted from the buffer. In this way, the residence time of each command record in the buffer is the simulated additional latency time T, T = N × ΔT, thus achieving the simulation and control of network latency effects.

[0109] 1) Initialization process

[0110] This experimental system requires initialization before operation, mainly for setting basic parameters to ensure the system functions as expected. The delay initialization flowchart is as follows: Figure 9 As shown.

[0111] First, the system reads a preset time interval to determine the frequency of data acquisition and processing. Fixed time values ​​are established to provide a baseline for subsequent delay measurement and recording. Furthermore, the system calculates the delay mid-range boundary value (delay mid-range boundary time). This step aims to provide a baseline value for evaluating subsequent data delay. The system initializes the recording sequence with an empty string to ensure a clear initial state for data recording. Once the system's basic parameters are set, the foundation for subsequent recording and transmission processes is laid.

[0112] 2) Fixed Delay Process

[0113] Fixed latency is the buffer management mechanism of this system, responsible for receiving, judging, and managing the execution order of upper-level instructions. First, a timer is started to ensure that the instruction checking interval meets the system's requirements. Then, the system retrieves the initial instruction record from the buffer and determines whether the current instruction is a valid command from the upper layer. If the determination is "yes," the system retrieves the topmost instruction from the buffer for execution, ensuring the instruction priority order. Simultaneously, new instructions are placed at the bottom of the buffer to ensure that subsequent instructions are processed in order, implementing FIFO (First-In, First-Out) instruction execution logic. If the received instruction is not a command, the process returns directly without performing any further operations. Its flowchart is as follows: Figure 10 As shown.

[0114] 3) Delay and record update process

[0115] If dynamic updates to the system's latency are required, the flowchart is as follows. Figure 11 As shown, this system first records the latest latency information and saves the latency data to the system. Then, based on the instructions issued by the PD (Power Device) terminal, the system determines whether the latency has changed. If the latency has increased, the current latency change value is updated, and various latency indicators are calculated and accumulated, including area latency, global latency, and the accumulated latency value of power equipment. This data is recorded in the corresponding latency record table for subsequent analysis and optimization.

[0116] If the latency remains unchanged, the system will return to wait for the next detection cycle. This procedure allows the system to dynamically track latency, ensuring real-time latency information and providing support for system latency optimization.

[0117] The master-slave teleoperation principle is as follows: the master end (PC end) sends control signals to the slave end (hydrophone and transducer robot end), the slave end executes the corresponding operation according to these signals, and transmits feedback (such as position, underwater acoustic signals, etc.) back to the master end, so that the operator can perceive and adjust the operation.

[0118] The underwater master-slave teleoperation system has two robots on the slave end: a transducer-end (transmitter) robot and a hydrophone-end (receiver) robot. The transducer-end robot has two degrees of freedom: a translational degree of freedom in the Z-axis direction and a rotational degree of freedom at its end effector. The hydrophone-end robot has four degrees of freedom, including three Cartesian coordinate translational joints and one tail rotational joint.

[0119] like Figure 12 Establish the origin O of the world coordinate system w ( Figure 12 Establish a fixed coordinate system O at the top right corner of the water tank. w xyz: World coordinate system, established at the hydrophone end; establishes the attached movement coordinate system O.x xyz: Attached to the moving crossbeam at the end of the hydrophone; establish the attachment's moving coordinate system O. y xyz: Attached to the moving crossbeam at the end of the hydrophone; establish the attachment's moving coordinate system O. z xyz: Attached to the hydrophone; establish the attachment rotation coordinate system O A xyz: Attached to the scale of the hydrophone; establish the attachment's moving coordinate system O z2 xyz: Attached to the transducer; establish the attached rotating coordinate system O. B xyz: Attached to the indexing plate of the transducer; World coordinate origin O is set. w The coordinates are (0, 0, 0).

[0120] 1) Kinematic modeling of the transducer end

[0121] set up B T z Coordinate system O z2 xyz to world coordinate system O w The homogeneous transformation matrix of xyz; B T θ Let O be the coordinate system at the transducer end. B xyz to coordinate system O z2 The homogeneous transformation matrix of xyz;

[0122]

[0123] The above equation yields the transducer end coordinate system O. B xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T B .

[0124]

[0125] Where a is the length of the longitudinal guide rail; b is the length of the transducer end moving crossbeam; and z B θ is the underwater depth of the transducer. B The angle of rotation of the transducer is denoted as .

[0126] 2) Kinematic modeling of the hydrophone end:

[0127] set up A T x Coordinate system O x xyz to world coordinate system O w The homogeneous transformation matrix of xyz; A T y Coordinate system O y xyz to coordinate system O x The homogeneous transformation matrix of xyz; A Tz Coordinate system O z xyz to coordinate system O y The homogeneous transformation matrix of xyz; A T θ Let O be the coordinate system at the hydrophone end. A xyz to coordinate system O z The homogeneous transformation matrix of xyz is shown in the following equation:

[0128]

[0129] Finally, the coordinate system O at the hydrophone end is obtained. A xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T A for:

[0130]

[0131] Where x A y is the distance the hydrophone extends along the longitudinal guide rail. A z is the distance the hydrophone extends along the moving crossbeam. A θ represents the underwater depth of the hydrophone. A This represents the rotation angle of the hydrophone.

[0132] 3) Hydrophone end-transducer end modeling

[0133] Hydrophone end coordinate system O A xyz to transducer end coordinate system O B The homogeneous transformation matrix of xyz B T A for:

[0134] B T A = B T W · W T A

[0135]

[0136] To achieve effective control of a master-slave remote operating system, a virtual environment needs to be established in the computer. This virtual environment realistically reflects the actual operating environment through mathematical modeling and physical simulation. The main components of the virtual environment are as follows.

[0137] 3D Model: Three-dimensional models of the hydrophone end and transducer end robots.

[0138] Physics Engine: The physical laws governing the motion and interaction of robots.

[0139] Control interface: Used to receive control signals from experimenters or slave devices and output corresponding feedback.

[0140] The physics engine in the virtual environment uses a physics-based model to simulate the robot's motion characteristics in different environments, including gravity, underwater friction, and collisions.

[0141] The user interface is an important component of the virtual environment, primarily used to display the robot's status and receive input from the experimenter. The user interface design should consider the following points.

[0142] Real-time performance: Ensure that the robot status and the operation of the experimenter are displayed on the interface in real time.

[0143] Intuitive: The interface is simple and clear, enabling experimenters to quickly get started.

[0144] Interactivity: Supports experimenters to interact with the system through various input devices (such as keyboard, mouse, gamepad, etc.).

[0145] The user interface displays information including the current location and attitude of the slave device, as well as underwater acoustics and environmental data. Simultaneously, the interface allows researchers to make real-time adjustments and configurations to adapt to different task requirements.

[0146] To achieve precise control of the slave devices (hydrophone segment robot and transducer segment robot), mapping is performed within the workspace. This requires calculating the end effector pose of the devices using a forward kinematics model, then obtaining the desired pose of the end effector mechanism of the slave devices using a mapping algorithm, followed by obtaining the position or angle of each joint using an inverse kinematics algorithm, and finally sending the joint position or angle information to the slave devices via the controller. [8] The workspace mapping algorithm also uses world coordinates O. w Using the xyz coordinate system as a reference can largely ensure that the end-effector poses of the slave device and the master virtual device remain consistent. Its biggest difference from joint space mapping is its real-time capability; that is, it analyzes the movement trajectory of the master virtual device in real time and sends the real-time data to the slave device to complete the slave device's tracking of the master virtual device. Based on the range of motion of the master and slave devices, the mapping coefficients should be within the following range:

[0147]

[0148] In the formula k n The mapping scaling factor in the n direction (n = x, y, z); U n,max U represents the maximum range of motion of the slave device in the n-direction; n,min T represents the minimum range of motion of the slave device in the n-direction; n,maxThe maximum range of motion of the main virtual device in the n-direction; T n,min This represents the minimum range of motion of the master virtual device in the n-direction. Combined with the position and angle of the robotic arm's movement, it ensures that the pose of the master virtual device, after mapping, matches the workspace range of the slave device. [9] Finally, the mapping formulas for the hydrophone end and the transducer end are obtained as follows:

[0149]

[0150] Among them, [x u1 y u1 z u1 θ u1 ] T The desired position and orientation of the hydrophone in the slave device are represented in Cartesian space coordinates; [x t1 y t1 z t1 θ t1 ] T The actual position of the hydrophone in the virtual model of the master device is represented in Cartesian space coordinates; [x u2 θ u2 ] T This represents the desired position and orientation of the transducer in the slave device, expressed in Cartesian coordinates; [x t2 θ t2 ] T This indicates the actual position of the transducer in the virtual model of the master device, expressed in Cartesian space coordinates;

[0151] The actual position and attitude of the end of the master virtual device can be mapped to the desired position and attitude of the end of the slave device. Then, the movement trajectory and joint rotation angle of the robot at the hydrophone end and the transducer end can be obtained according to the inverse kinematics formula of the slave device, thereby realizing remote operation control of the master and slave devices.

[0152] Example 1:

[0153] The robot trajectory was generated using MATLAB, and the motion performance of the hydrophone-end robot and the transducer-end robot was simulated. By defining the initial and target end positions, as well as several intermediate point positions, time parameters were generated, and trajectory calculations were performed, including rotation angles, velocities, and accelerations during the motion.

[0154] The position-time curve at the hydrophone end is as follows: Figure 13 As shown (where the X-axis represents movement along the longitudinal guide rail, the Y-axis represents movement along the direction of the moving crossbeam, and the Z-axis represents the depth of immersion in water).

[0155] The velocity-time curve at the hydrophone end is as follows: Figure 14 As shown. The "acceleration-time" curve at the hydrophone end is as follows. Figure 15 As shown.

[0156] The transducer end "position-time", "velocity-time" and "acceleration-time" curves are as follows: Figure 16 As shown. The "rotation angle-time" curves at the hydrophone end and the transducer end are as follows. Figure 17 As shown.

[0157] For the experimental teaching case of water tank reverberation field measurement in our center, a physical simulation was conducted for verification. This experiment measures the equivalent free-field radiated sound power of a spherical sound source in a reverberant water tank. The experimental steps are as follows.

[0158] I. Reverberation Time Measurement

[0159] 1) Students log into the experimental operating system remotely one by one in order. Only one student or one group of students can be in operation at any given time. A measurement system is set up, consisting of one auxiliary robot and one hydrophone. The hydrophone is operated remotely via a master-slave controller robot in a virtual environment. The master controller robot controls the slave device 1 to place it in the required pose (position and orientation) within the reverberation tank and distribute it appropriately within the tank.

[0160] 2) Use a spherical sound source as the excitation source, and connect it to the corresponding output terminal of the power amplifier using a dedicated drive cable. Similarly, using a master-slave remote operation mode, the master controller robot operates the virtual environment, controlling slave device 2 to place it in the required pose (position and attitude) within the reverberation tank.

[0161] 3) After remotely checking the connections of each device to confirm that they are correct, perform a self-test of all device functions and set the gain switch of the power amplifier to the lowest setting. Test the computer, robot, power amplifier and data acquisition unit in sequence.

[0162] 4) Once all experimental equipment is ready, generate a single-frequency sine wave using a signal source to create a diffusion field within the reverberation tank. Then, cut off the sound source and measure its attenuation curve to calculate the reverberation time. The reverberation time measurement requires six iterations.

[0163] II. Measuring the average sound pressure level

[0164] 1) Similarly, a measurement system is set up, with the sound source to be measured placed in one or more typical installation positions in the reverberation tank relative to the boundary surface in a normal installation manner, with the sound source placed at least 0.2m away from any tank surface.

[0165] 2) Remotely turn on the data acquisition device, set the sampling frequency to 10 times the signal frequency, and select 5 seconds for the acquisition time.

[0166] 3) Remotely turn on the power amplifier switch, click the measurement start button, and in the reverberation tank, let the hydrophone perform spatial averaging to obtain the required data.

[0167] According to the center frequencies of 1 / 3 octave: 8kHz, 10kHz, 12.5kHz.

[0168] III. Experimental Data Processing

[0169] 1) Reverberation time data processing

[0170] Use software to write a data processing program to calculate the reverberation time according to the definition of reverberation time.

[0171] 2) Spatial average sound pressure level data processing

[0172] A data processing program is written using software to perform Fourier transform on the read time-domain data, read the sound pressure level at the test frequency, and then calculate the radiated sound power.

[0173] Observations and feedback from three trial simulated teaching cycles revealed that students' understanding of relevant theoretical knowledge improved, and they showed great interest in understanding interdisciplinary concepts such as teleoperated robots, local artificial intelligence, and supervised control through experiments. This greatly stimulated the learning enthusiasm of the participating students, and the experimental results were very satisfactory. The teleoperation system design of this invention allows students to truly delve into the marine environment for scientific experiments. The data sources are authentic, and the experience of the experimenters is becoming increasingly closer to being on-site. All of these are of great significance for cultivating innovative and high-quality talents.

[0174] This invention provides a marine information experimental teaching platform based on a master-slave teleoperation system. While retaining the basic functions of a large dedicated water tank, its compact size effectively resolves the contradiction between the large amount of teaching time and the limited amount of teaching equipment. Within the limited space of a laboratory, various underwater testing environments are simulated in the water tank testing space using transmitting and receiving transducers of different frequency bands. Combined with corresponding parameter testing methods and signal processing techniques, experimental operations for marine information acquisition are completed. This invention features high stability and reliability, simple operation, and convenient maintenance, making it particularly suitable for experimental teaching.

[0175] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A marine information experimental teaching platform based on a master-slave teleoperation system, characterized in that: It includes a master device and a slave device; the slave device includes an experimental water tank, a longitudinal guide rail installed on the experimental water tank, a transducer end moving crossbeam and a hydrophone end moving crossbeam installed on the longitudinal guide rail, and a transducer and a hydrophone disposed in the experimental water tank; the transducer is installed below the transducer end moving crossbeam via a first connecting rod, and a first indexing plate is provided on the first connecting rod; the top surface of the hydrophone end moving crossbeam is provided with a transverse guide rail, and a slider is provided on the transverse guide rail; the hydrophone is installed below the slider via a second connecting rod, and a second indexing plate is provided on the second connecting rod; The master device is equipped with a master-slave teleoperation system, in which a virtual model of the slave device is established. The experimenter operates the virtual model of the slave device through the master-slave teleoperation system, which converts the actions of the virtual model into control commands and transmits them to the slave device. Upon receiving the control commands, the slave device controls the rotation angle of the transducer and hydrophone via the first and second indexing dials. It adjusts the longitudinal position of the transducer and hydrophone by moving the transducer-end moving beam and the hydrophone-end moving beam on the longitudinal guide rail, and adjusts the lateral position of the hydrophone by moving the slider on the transverse guide rail on the top surface of the hydrophone-end moving beam. The experimenter sends electrical signals to the transducer through the master-slave teleoperation system. The transducer converts the electrical signals into underwater acoustic signals that propagate in the experimental pool. The hydrophone receives the underwater acoustic signals and feeds them back to the master device, enabling the experimenter to remotely control the experimental process.

2. The marine information experimental teaching platform based on a master-slave teleoperation system according to claim 1, characterized in that: In the master-slave remote operating system, a virtual model of the slave device is established, specifically as follows: Establish a world coordinate system O with the vertex of one end of the experimental pool as the origin. w xyz; Establish a model of the slave device in the world coordinate system, and set the hydrophone end coordinate system O. A xyz and transducer end coordinate system O B xyz; Hydrophone end coordinate system O A xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T A for: Transducer end coordinate system O B xyz to world coordinate system O w The homogeneous transformation matrix of xyz W T B for: Hydrophone end coordinate system O A xyz to transducer end coordinate system O B The homogeneous transformation matrix of xyz B T A for: Where, x A y represents the distance the hydrophone extends along the longitudinal guide rail; A denoted as , where is the distance the hydrophone extends along the moving beam at the hydrophone end; a is the length of the longitudinal guide rail; b is the length of the moving beam at the transducer end; z is... A The underwater depth of the hydrophone; z B θ represents the underwater depth of the transducer. A θ is the rotation angle of the hydrophone. B The angle of rotation of the transducer is denoted as .

3. The marine information experimental teaching platform based on a master-slave teleoperation system according to claim 2, characterized in that: To achieve precise control of the slave device, mapping is performed within the workspace. The poses of the hydrophone and transducer are calculated using a forward kinematics model. The desired poses of each actuator in the slave device are obtained according to the mapping algorithm. Then, the position or angle of each actuator is obtained according to the inverse kinematics algorithm. The actuator includes a transducer end moving crossbeam, a hydrophone end moving crossbeam, a first indexing plate, a second indexing plate, and a slider. The workspace mapping algorithm uses world coordinates O w Using the xyz coordinate system as a reference, the pose of the virtual model in the slave device and the master device remains consistent. The movement trajectory of the virtual model in the master device is analyzed in real time, and the real-time data is sent to the slave device to complete the slave device's tracking of the virtual model in the master device. Based on the movement range of the master and slave devices, the mapping coefficients satisfy the following conditions: Where, k n U represents the mapping scaling factor in the n-direction, where n = x, y, z; n,max U represents the maximum range of motion of the hydrophone or transducer in the n-direction within the slave device. n,min T represents the minimum range of motion of the hydrophone or transducer in the n-direction within the slave device; n,max T represents the maximum range of motion of the hydrophone or transducer in the n-direction within the virtual model of the master device; n,min This represents the minimum range of motion of the hydrophone or transducer in the n-direction within the virtual model of the master device. To ensure that the pose of the virtual model of the master device matches the workspace range of the slave device after mapping, the mapping formulas for the hydrophone end and the transducer end are as follows: Among them, [x u1 y u1 z u1 θ u1 ] T Indicates the desired position and orientation of the hydrophone in the slave device; [x t1 y t1 z t1 θ t1 ] T This indicates the actual location of the hydrophone in the virtual model of the master device; [x u2 θ u2 ] T Indicates the desired position and orientation of the transducer in the slave device; [x t2 θ t2 ] T This indicates the actual location of the transducer in the virtual model of the master device; The actual positions and orientations of the hydrophones and transducers in the virtual model of the master device can be mapped to the desired positions and orientations of the hydrophones and transducers in the slave device. Then, the movement trajectories and rotation angles of each actuator at the hydrophone and transducer end are obtained according to the inverse kinematics formula of the slave device, thereby realizing remote operation control of the master and slave devices.

4. The marine information experimental teaching platform based on a master-slave teleoperation system according to claim 1, characterized in that: The master-slave remote operating system includes a network communication connection and monitoring module, a VE human-computer interaction control module, an AR human-computer interaction control module, a view switching module, and a data display and analysis module; The network communication connection and monitoring module is used to set the connection addresses of the master device and the slave device, and to monitor the network communication connection status and network signal strength. The VE human-computer interaction control module is used to construct a virtual model of the slave device and to allow experimenters to perform operations on the virtual model of the slave device. The AR human-computer interaction control module is used to provide an augmented reality environment. Experimenters can operate the master device in the augmented reality environment and restore the sense of presence of the underwater environment while controlling the slave device in real time. The view switching module is used by experimenters to observe the virtual model of the slave device from different angles, including front view, side view and top view. The data display and analysis module is used to display the underwater acoustic signals received by the hydrophone in the slave device in real time, and also to convert the operation of the virtual model by the experimenter into control commands and transmit them to the slave device.

5. The marine information experimental teaching platform based on a master-slave teleoperation system according to claim 1, characterized in that: To simulate the delay of signals propagating underwater, a buffer is set in the master-slave remote operating system, and this buffer is configured to store N command records, which are used to temporarily store the instructions generated by the master device. During the control process, the master device generates control commands according to a preset time interval ΔT. The control commands are not directly transmitted to the slave device, but are stored at the end of the buffer. At the same time, the earliest control command is extracted from the top of the buffer and transmitted to the slave device through the network, so that the slave device executes the corresponding control command. Then, the control command is deleted from the buffer. The residence time of each control command in the buffer is recorded, which is the delay T of the analog signal during underwater propagation, T = N × ΔT.

6. The marine information experimental teaching platform based on a master-slave teleoperation system according to claim 5, characterized in that: The delay process of the analog signal propagating underwater is specifically as follows: 1) Initialization; the master-slave remote operating system reads the preset time interval ΔT to determine the frequency of data acquisition and processing; calculates the delay mid-range boundary value to evaluate subsequent data delay; the master-slave remote operating system initializes the record sequence using an empty string; 2) Fixed delay process; Start a timer to ensure that the instruction check interval meets the requirements; extract control instructions from the top layer of the buffer and determine whether the current control instruction is a valid command from the upper layer. If the determination is "yes", execute the control instruction and place the new control instruction at the bottom of the buffer to ensure that subsequent control instructions are processed in order, thus implementing the FIFO instruction execution logic; if the determination is "no", do not perform any further operations. 3) Delay and record update process; if dynamic updates to the master-slave remote operating system are required in terms of delay, the latest delay information is first recorded and the delay data is saved to the master-slave remote operating system; then, the master-slave remote operating system determines whether the delay has changed based on the control instructions issued from the buffer. If the latency increases, the current latency change value is updated, and various latency indicators, including area latency, global latency, and the cumulative latency value of power equipment, are calculated and accumulated. These data are recorded in the corresponding latency record table for subsequent analysis and optimization. If the latency does not change, the master-slave remote operating system returns directly to wait for the next detection cycle. Through this procedure, the master-slave remote operating system can dynamically track the latency situation, ensure the real-time nature of latency information, and provide support for latency optimization.

Images