Marine information experiment teaching platform based on master-slave teleoperation system

Abstract

The invention belongs to the technical field of underwater acoustic test platforms, and particularly relates to a marine information experiment teaching platform based on a master-slave teleoperation system. The system comprises a master-end device and a slave-end device, a master-slave teleoperation system is carried in the master-end device, and a virtual model of the slave-end device is established in the master-slave teleoperation system; an experimenter operates the virtual model of the slave-end device through the master-slave teleoperation system, and the master-slave teleoperation system converts the action of the virtual model of the slave-end device into a control instruction and transmits the control instruction to the slave-end device; after receiving the control instruction, the slave end equipment adjusts the pose state of the transducer and the hydrophone; an experimenter sends an electric signal to the transducer through the master-slave teleoperation system, the transducer converts the electric signal into an underwater acoustic signal to be spread in the experimental pool, the hydrophone feeds back the underwater acoustic signal to the master-end equipment after receiving the underwater acoustic signal, and remote teleoperation processing of the experimenter on the experimental process is achieved.

Description

Technical Field

[0001] The invention belongs to the technical field of underwater acoustic test platforms, and in particular relates to an ocean information experimental teaching platform based on a master-slave teleoperation system. Background Art

[0002] The carriers of ocean information include various physical fields such as sound, light, electricity, and magnetism. The basic connotation of ocean information engineering is the perception, acquisition, transmission, processing, and integrated application of ocean information, so as to realize the observation, detection, and monitoring of the sea, in the sea, and for the sea. The experimental teaching platform is an important carrier of theoretical research and engineering practice. There are still some problems in the experimental teaching of ocean information engineering. Due to the complexity of the marine environment, the multi-source of information, the hugeness of data, the intelligence of processing, and the diversity of systems, it is very difficult to carry out experiments related to ocean information. At the same time, it is difficult to carry out undergraduate experiments on a large scale because the relevant equipment is very expensive. Applying remote human-computer collaboration systems and technologies to experimental teaching will help to achieve the deep integration of information technology and education and teaching in the field of education, promote the sharing of educational resources, improve the quality and efficiency of education, and form a new education system that adapts to the digital age.

[0003] In 2006, Zhou Siyue of Shanghai University designed and implemented a robot dexterous hand remote operation platform based on virtual reality technology. The platform uses data gloves to capture the movement of the operator's hand, uses the basic primitive drawing commands provided by the OpenGL development library for users, and realizes the control and operation of the dexterous hand through position sensors, angle sensors and force feedback technology. However, the stability and flexibility of data gloves are limited and are easily affected by small movements of the hand.

[0004] In 2011, Wang Biao and Zeng Qingjun from Jiangsu University of Science and Technology designed a small underwater information communication network and opened a microwave passive network innovative design experiment for underwater to surface communication, realizing the wireless access function of underwater information and land systems through buoy gateway nodes. The propagation loss and delay of the underwater acoustic signal are large, and the stability of the system in complex underwater environments needs to be improved.

[0005] In 2019, Liu Shengxing from Xiamen University built an underwater acoustic communication experimental teaching platform based on PC and sound card. The platform simplifies the hardware configuration and implements communication processes such as modulation / demodulation and channel coding / decoding through software, which reduces the abstractness of communication theory and stimulates students' practical and innovative abilities.

[0006] In 2021, Liu Ni from Jiangsu University of Science and Technology used Unity3D and MATLAB joint simulation to build an underwater robotic arm remote operation simulation platform in a virtual environment, verifying the angle tracking and force feedback functions of the master and slave robotic arms in the virtual environment, but the simulation analysis of the virtual force feedback is difficult to fully simulate the real underwater environment.

[0007] In 2021, Zhan Hong, Liang Congyuan and others from South China University of Technology proposed a robot teleoperation experimental platform based on mixed reality technology. The platform detects the operator's hand movements through Leap Motion and displays the fusion of reality and virtual scenes through a mixed reality display device (such as Microsoft HoloLens). The system has limited application scenarios and is mainly designed for land environments, lacking adaptability to underwater operations.

[0008] In 2023, Yang Junhao, Wang Bingyan and others from the China Nuclear Power Research and Design Institute studied a robot control system for underwater cleaning tasks, using the UR5 underwater tandem robot. Through the workspace mapping algorithm, the action synchronization of the master and slave devices is achieved to improve the efficiency and accuracy of the cleaning task. However, the workspace mapping algorithm has a long response time and limited coverage.

[0009] In 2023, Zhang Yanan from Zhejiang University studied a multi-modal human-machine interactive remote operation technology based on binocular vision and force feedback for ultrasonic scanning of underwater components. This technology improves the accuracy of three-dimensional reconstruction and the fidelity of remote force reproduction, reducing the difficulty of operation for operators. The overall complexity of the system is high, requiring high maintenance costs and operation training costs, which is not conducive to basic teaching. Summary of the invention

[0010] The purpose of the present invention is to provide a marine information experiment teaching platform based on a master-slave teleoperation system, integrate human-machine collaboration technology, realize remote teleoperation processing of the experimental process, provide students and teachers with a virtual-real semi-physical experimental scene through the local intelligence and on-site digital model of the underwater remote execution system, and provide students and teachers with a remote telepresence through the experimental on-site operation device. The platform has the characteristics of being close to engineering and highly flexible, and can also improve the sensor system and improve the operation accuracy to provide support for scientific research.

[0011] The marine information experimental teaching platform based on the master-slave teleoperation system comprises a master-end device and a slave-end device; the slave-end device comprises an experimental water pool, a longitudinal guide rail installed on the experimental water pool, a transducer-end moving beam and a hydrophone-end moving beam installed on the longitudinal guide rail, and a transducer and a hydrophone arranged in the experimental water pool; the transducer is installed below the transducer-end moving beam through a first connecting rod, and a first indexing plate is arranged on the first connecting rod; a transverse guide rail is arranged on the top surface of the hydrophone-end moving beam, and a slider is arranged on the transverse guide rail; the hydrophone is installed below the slider through a second connecting rod, and a second indexing plate is arranged on the second connecting rod;

[0012] The master-end device is equipped with a master-slave teleoperation system, in which a virtual model of the slave-end device is established; the experimenter operates the virtual model of the slave-end device through the master-slave teleoperation system, and the master-slave teleoperation system converts the actions of the virtual model of the slave-end device into control instructions and transmits them to the slave-end device; after receiving the control instructions, the slave-end device controls the rotation angle of the transducer and the hydrophone through the first dividing plate and the second dividing plate, adjusts the longitudinal position of the transducer and the hydrophone through the movement of the transducer-end moving beam and the hydrophone-end moving beam on the longitudinal guide rail, and adjusts the lateral position of the hydrophone through the movement of the slider on the lateral guide rail on the top surface of the hydrophone-end moving beam; the experimenter sends an electrical signal to the transducer through the master-slave teleoperation system, the transducer converts the electrical signal into an acoustic signal and propagates it in the experimental water pool, and the hydrophone receives the acoustic signal and feeds it back to the master-end device, so that the experimenter can remotely operate the experimental process.

[0013] Furthermore, a virtual model of the slave device is established in the master-slave teleoperation system, specifically:

[0014] Take the top corner of one end of the experimental pool as the origin and establish the world coordinate system O w xyz; Establish the model of the slave device in the world coordinate system and set the hydrophone end coordinate system O A xyz and transducer end coordinate system O B xyz;

[0015] Hydrophone end coordinate system O A xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T A for:

[0016]

[0017] Transducer end coordinate system O B xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T B for:

[0018]

[0019] Hydrophone end coordinate system O A xyz to transducer end coordinate system O B Homogeneous transformation matrix of xyz B T A for:

[0020]

[0021] Among them, x A y is the distance that the hydrophone extends along the longitudinal rail;A is the distance that the hydrophone extends along the moving beam at the hydrophone end; a is the length of the longitudinal guide rail; b is the length of the moving beam at the transducer end; z A is the underwater depth of the hydrophone; z B is the underwater depth of the transducer; θ A is the rotation angle of the hydrophone; θ B is the rotation angle of the transducer.

[0022] Furthermore, in order to achieve fine control of the slave device, mapping is performed in the workspace, the positions and postures of the hydrophone and the transducer are calculated by the kinematic forward solution model, the desired positions and postures of each actuator in the slave device are obtained according to the mapping algorithm, and then the position or angle of each actuator is obtained according to the kinematic inverse solution algorithm; the actuator includes a transducer end moving beam, a hydrophone end moving beam, a first indexing plate, a second indexing plate and a slider;

[0023] The workspace mapping algorithm uses the world coordinate O w xyz is the reference coordinate system to ensure that the position and posture of the virtual model in the slave device and the master device are consistent, analyze the movement trajectory of the virtual model in the master device in real time, and send the real-time data to the slave device to complete the tracking of the virtual model in the master device by the slave device; according to the motion range of the master and slave ends, the mapping coefficients meet the following conditions:

[0024]

[0025] Among them, k n Indicates the mapping scale factor in the n direction, n = x, y, z; U n,max It represents the maximum range of motion of the hydrophone or transducer in the slave device in the n direction; U n,min T represents the minimum motion range of the hydrophone or transducer in the slave device in the n direction; n,max T represents the maximum motion range of the hydrophone or transducer in the virtual model of the master device in the n direction; n,min Indicates the minimum motion range of the hydrophone or transducer in the virtual model of the master device in the n direction; To ensure that the posture of the virtual model of the master device matches the working space range of the slave device after mapping, the mapping formulas of the hydrophone and transducer are:

[0026]

[0027] Among them, [x u1 y u1 z u1 θ u1 ] T represents the expected position and attitude of the hydrophone in the slave device; [x t1 y t1 zt1 θ t1 ] T represents the actual position of the hydrophone in the virtual model of the master device; [x u2 θ u2 ] T represents the expected position and attitude of the transducer in the slave device; [x t2 θ t2 ] T Indicates the actual position of the transducer in the virtual model of the master device;

[0028] The actual position and posture of the hydrophone and transducer in the virtual model of the master-end device can be used to map the expected position and posture of the hydrophone and transducer in the slave-end device. Then, the movement trajectory and rotation angle of each actuator on the hydrophone and transducer end can be obtained according to the inverse kinematics formula of the slave-end device, thereby realizing remote operation control of the master-slave device.

[0029] Furthermore, the master-slave teleoperation system includes a network communication connection and monitoring module, a VE human-computer interaction control module, an AR human-computer interaction control module, a view switching module, and a data display and analysis module;

[0030] The network communication connection and monitoring module is used to set the connection addresses of the master device and the slave device, and monitor the network communication connection status and network signal strength;

[0031] The VE human-computer interaction control module is used to construct a virtual model of the slave device and is used by the experimenter to perform operations on the virtual model of the slave device;

[0032] The AR human-computer interaction control module is used to provide an augmented reality environment, in which the experimenter can control the master device and restore the sense of presence of the underwater environment while controlling the slave device in real time;

[0033] The view switching module is used for the experimenter to observe the virtual model of the slave device from different angles, including front view, side view and top view;

[0034] The data display and analysis module is used to display the water acoustic signal received by the hydrophone in the slave device in real time, and is also used to convert the operation of the experimenter on the virtual model into control instructions and transmit them to the slave device.

[0035] Furthermore, in order to simulate the delay of the signal in the process of underwater propagation, a buffer is set in the master-slave teleoperation system, and the buffer is set to be able to store N command records, which is used to temporarily store the instructions generated by the master device; in the control process, the master device generates a control instruction according to a preset time interval ΔT, and the control instruction is not directly transmitted to the slave device, but is stored at the end of the buffer; at the same time, the earliest control instruction is extracted from the top of the buffer, and it is transmitted to the slave device through the network, so that the slave device executes the corresponding control instruction, and then the control instruction is deleted from the buffer; the residence time of each control instruction in the buffer is recorded, which is the delay T of the simulated signal in the process of underwater propagation, T=N×ΔT.

[0036] Furthermore, the delay process of the analog signal during underwater propagation is specifically as follows:

[0037] 1) Initialization; the master-slave teleoperation system reads a preset time interval ΔT to determine the frequency of data acquisition and processing; calculates the delay mid-zone boundary value for evaluating subsequent data delay conditions; the master-slave teleoperation system uses an empty string to initialize the record sequence;

[0038] 2) Fixed delay process: start the timer to ensure that the time interval of the instruction check meets the requirements; extract the control instruction from the top layer of the buffer, and determine whether the current control instruction is a valid command from the upper layer. If the judgment is "yes", execute the control instruction, and place the new control instruction at the bottom of the buffer to ensure that the subsequent control instructions are processed in order to implement the FIFO instruction execution logic; if the judgment is "no", do not perform any further operations;

[0039] 3) Delay and record update process; if the delay of the master-slave teleoperation system needs to be dynamically updated, the latest delay situation is first recorded and the delay data is saved in the master-slave teleoperation system; then, the master-slave teleoperation system determines whether the delay has changed according to the control instructions sent from the buffer; if the delay increases, the current delay change value is updated, and various delay indicators are calculated and accumulated, including zone delay, global delay and delay accumulation value of power equipment. These data will be recorded in the corresponding delay record table for subsequent analysis and optimization; if the delay does not change, the master-slave teleoperation system directly returns to wait for the next detection cycle; through this procedure, the master-slave teleoperation system can dynamically track the delay situation, ensure the real-time nature of the delay information, and provide support for delay optimization.

[0040] The beneficial effects of the present invention are:

[0041] The present invention integrates human-machine collaboration technology, and realizes remote teleoperation processing of the experimental process through the master-slave teleoperation system carried by the master-end device, providing the experimenters with a semi-physical experimental scene and a sense of presence that combines virtuality and reality. On the basis of having the basic functions of a large special water pool, the present invention has a miniaturized appearance, which effectively solves the contradiction between the large amount of teaching hours and the small amount of teaching equipment. In the limited space of the laboratory, various underwater test environments are simulated in the water tank test space through transmitting transducers and receiving transducers of different frequency bands, and combined with corresponding parameter testing methods and signal processing means, the experimental operation of collecting ocean information is completed. The present invention has the characteristics of high stability and reliability, simple operation and use, and convenient later maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is the system block diagram of the marine information experimental teaching platform based on the master-slave teleoperation system.

[0043] Figure 2 This is the control schematic diagram.

[0044] Figure 3 This is a diagram of the control operation architecture.

[0045] Figure 4 This is a schematic diagram of the structure of the slave device.

[0046] Figure 5 This is a diagram of the network communication interface.

[0047] Figure 6 This is the VE human-computer interaction interface diagram.

[0048] Figure 7 This is a diagram of the AR human-computer interaction interface.

[0049] Figure 8 This is a diagram of the data viewing interface.

[0050] Fig. 9 This is the delay initialization flow chart.

[0051] Fig.10 This is the flow chart of the fixed delay program.

[0052] Fig.11 Update the process flow chart for time delay.

[0053] Fig.12 Create a schematic diagram for the kinematic coordinate system.

[0054] Fig.13 This is the hydrophone position-time variation curve.

[0055] Fig.14 This is the hydrophone speed-time variation curve.

[0056] Fig.15 This is the hydrophone acceleration-time variation curve.

[0057] Fig.16 It is the graph of transducer position, velocity and acceleration-time variation.

[0058] Fig.17 This is a curve diagram of the rotation angle of the hydrophone and transducer over time.

[0059] Fig.18 This is a diagram of reverberation time.

[0060] Fig.19 Schematic diagram of the spatial average sound pressure level. DETAILED DESCRIPTION

[0061] The present invention is further described below in conjunction with the accompanying drawings.

[0062] In order to solve the problem that underwater acoustic experimental teaching in complex marine environments is difficult to conduct in real sea areas, the present invention designs a marine information experimental teaching platform based on a master-slave remote operation system. The platform simulates different underwater sound field environments in the laboratory through acoustic transmitting and receiving transducers to achieve parameter testing of acoustic equipment. Remote control and intelligent technology are used, combined with underwater execution systems and digital models, to provide a semi-physical experimental scene that combines virtual and real. The present invention can improve teaching efficiency and student participation, and meet the needs of personalized and flexible teaching. The present invention innovatively integrates information technology with underwater acoustic engineering education, promotes the development of educational informatization, and has good promotion and application value.

[0063] The marine information experimental teaching platform based on the master-slave remote operation system provided by the present invention includes a master-end device and a slave-end device; Figure 4 As shown, the slave end device includes two robot structures. The transducer end (transmitter end) is a 2-DOF robot with vertical movement and rotation functions, including a transducer end moving beam, a first connecting rod and a first indexing plate; the hydrophone end (receiving end) is a 4-DOF robot with three-dimensional rectangular coordinate movement and rotation functions, including a hydrophone end moving beam, a transverse guide rail set on the top surface of the hydrophone end moving beam, a slider set on the transverse guide rail, a second connecting rod and a second indexing plate.

[0064] The master-end device is equipped with a master-slave teleoperation system, in which a virtual model of the slave-end device is established; the experimenter operates the virtual model of the slave-end device through the master-slave teleoperation system, and the master-slave teleoperation system converts the actions of the virtual model of the slave-end device into control instructions and transmits them to the slave-end device; after receiving the control instructions, the slave-end device controls the rotation angle of the transducer and the hydrophone through the first indexing plate and the second indexing plate, adjusts the longitudinal position of the transducer and the hydrophone through the movement of the transducer end moving beam and the hydrophone end moving beam on the longitudinal guide rail, and adjusts the transverse position of the hydrophone through the movement of the slider on the transverse guide rail on the top surface of the hydrophone end moving beam; the experimenter sends an electrical signal to the transducer through the master-slave teleoperation system, and the transducer converts the electrical signal into an underwater acoustic signal to propagate in the experimental water pool. After receiving the underwater acoustic signal, the hydrophone feeds back to the master-end device, so that the experimenter can remotely operate the experimental process. The master-slave teleoperation combined with the underwater acoustic equipment provides students and teachers with a virtual-real semi-physical experimental scene and a remote sense of presence.

[0065] In the master-slave teleoperation system, the control architecture is the key to ensure that the master operator can accurately control the remote robot to perform tasks. The components of the architecture and its working principle will be introduced in detail. The master-slave teleoperation system consists of a master device, a slave device, a communication module, and a control algorithm. Figure 1 The overall architecture of the system is shown.

[0066] The master device mainly includes various human-machine interface devices, such as computers, tablet computers, smart phones, etc. These devices are used for operators to input control commands and receive feedback information from slave devices. The master device interacts with the operator through a graphical user interface (GUI) to provide remote control, real-time monitoring, data analysis and other functions.

[0067] The human-machine interface is one of the core components of the host device. Its main functions include:

[0068] 1) Control command input: The operator inputs control commands through devices such as computers or handles.

[0069] 2) Feedback information display: The system displays the real-time data collected from the terminal device (such as position information, speed information, acoustic information, underwater environment images, etc.) to the operator in a graphical manner.

[0070] 3) Data analysis and processing: Real-time analysis of feedback data from the end to assist operators in making decisions.

[0071] The slave device is a device that performs specific tasks. Its main function is to receive control commands sent by the master, perform corresponding actions, and feed back the execution results and water sound, environment and other information to the master. The slave device is equipped with a high-precision control module, which is responsible for processing the master's instructions and performing action control. The main components of the control module include:

[0072] 1) Position control: adjust the position of the robot according to the master command.

[0073] 2) Speed ​​control: Control the robot's movement speed to ensure smooth operation.

[0074] 3) Feedback system: monitor the robot status and water sound information in real time, and provide feedback on position, speed, water sound and other data.

[0075] The communication module is responsible for transmitting control instructions and feedback data between the master device and the slave device. The design of the communication module needs to consider factors such as latency, bandwidth, and reliability. The system usually uses wireless communication technology (such as Wi-Fi, 4G / 5G, etc.) to achieve remote data transmission. The advantages of wireless communication are flexibility and wide coverage. In order to ensure the collaborative work of the master and slave, the communication module implements an efficient data synchronization mechanism, reduces latency and data transmission errors, and ensures that every instruction from teachers and students can be transmitted to the slave device in a timely and accurate manner.

[0076] The control algorithm is the core of the master-slave teleoperation system, which determines the response speed, accuracy and stability of the system. The control algorithm of the present invention adopts proportional-integral-differential (PID) control. PID control is a classic feedback control algorithm that minimizes the error between the system output and the expected value by adjusting the control parameters. PID control is widely used in master-slave teleoperation systems and has the advantages of simple implementation and fast response. The control system structure block diagram is shown in the figure. Figure 2 shown.

[0077] The present invention supports the experimenter to control the slave device by inputting commands, realizes remote control of the slave device to a specified position, and transmits hydroacoustic signals at the transducer end, and collects and transmits data at the hydrophone end. The system mainly consists of the following parts: the experimenter operation end, the control system for the experimental platform, the robot control cabinet, the slave device, etc. Figure 3 shown.

[0078] The experimenter interacts with the system through the operation end. The operation end is equipped with a human-machine interface for running the control software to control the slave device and the experimental platform. The control system of the experimental platform is responsible for managing and coordinating the various components on the experimental platform, communicating data with the experimenter's operation end and communicating with the hydroacoustic equipment through the TCP / IP protocol. The robot control cabinet is the core component of the robot control. It includes a motion control system and a control algorithm / control software. It is responsible for receiving instructions from the experimenter's operation end and converting them into actions performed by the two slave robots. The transducer end and the hydrophone end of the slave device are located in different positions and are responsible for adjusting each other's posture to meet the experimental requirements. The transducer is used to receive hydroacoustic data from the control system of the experimental platform and send hydroacoustic signals to the hydrophone. The hydrophone is used to receive hydroacoustic signals from the transducer and send hydroacoustic data to the control system of the experimental platform.

[0079] The software system includes network communication connection and monitoring module, VE (Virtual Environment, VE) human-computer interaction control module, AR (Augmented Reality, AR) human-computer interaction control module, view switching module and data display and analysis module. These modules together build a complete underwater remote operation experimental platform, which can complete experimental teaching and scientific research work by controlling underwater devices and then measuring, processing and analyzing underwater acoustic signals.

[0080] The network setting module is used to set the connection address between the remote robot and the host robot.

[0081] Remote robot address: Students need to enter the IP address of the remote robot in the input box to connect to the corresponding device.

[0082] Master robot address: Enter the IP address of the master robot.

[0083] Connection status: Indicates the current network connection status through color (green or red), green means connected, red means not connected or connection error.

[0084] Signal Strength: Displays the current network signal strength to help teachers and students judge the network quality. When the signal strength is low, you may need to adjust the network settings or check the network environment. Figure 5 shown.

[0085] The VE human-machine interaction control module provides the robot's three-dimensional graphic display and control functions, and is the core module of remote control. This module allows teachers and students to intuitively understand and control the master device to control the slave device. It includes the following functions:

[0086] Network connection test: Enter the IP addresses of the remote and master robots in the network setting module to confirm the network connection status. Three-dimensional control test: Test the movement and posture changes of the robot by entering different X, Y, Z coordinate values ​​and joint angle values. Three-dimensional model display: This part displays the three-dimensional model of the robot. Teachers and students can observe the robot in detail through operations such as rotation and scaling. Position adjustment: X-axis, Y-axis, and Z-axis input boxes: Users can enter specific coordinate values ​​in the corresponding input boxes to adjust the position of the robot. If the entered coordinate instructions are all within the workspace, you can click the "Execute" button for the instructions to take effect. If the input is wrong, you cannot click the "Execute" button. Angle adjustment: The angle of the joint angle input box can be adjusted separately. The user needs to enter the angle value (in degrees) in the corresponding input box.

[0087] View switching test: Switch between the front view, side view and top view to observe the display effects of the robot at different viewing angles.

[0088] Camera Image Test: View live camera feeds, capture and save pictures, and verify image quality and usability of capture functions.

[0089] The view switching module allows the user to observe the slave device from different angles, including front view, side view and top view.

[0090] Front view image: Displays the image of the robot viewed from the front, and the user can clearly see the front structure of the robot.

[0091] Side view image: Displays the image of the robot from the side, which helps users understand the side details of the robot.

[0092] Top View Image: Displays an image of the robot as viewed from above, which is useful for checking the overall layout and top details of the robot. Figure 6 shown.

[0093] The AR human-computer interaction control module is the core module that combines augmented reality technology with remote control, and is designed to provide teachers and students with a more intuitive and convenient control experience. Through this module, teachers and students can control the master device in an augmented reality environment, while controlling the slave device in real time, restoring the sense of presence in the underwater environment, which is something that VE human-computer interaction unlocking cannot provide. The AR human-computer interaction module includes the following functions:

[0094] ① Augmented reality environment display

[0095] The three-dimensional model of the robot is displayed in the AR environment. Teachers and students can view the superimposed display of the robot model in the real environment through PC devices or AR glasses to enhance the interactive experience.

[0096] ② Posture adjustment

[0097] Position adjustment: The AR human-computer interaction interface includes X-axis, Y-axis, and Z-axis input boxes: teachers and students can enter specific coordinate values ​​in the corresponding input boxes to adjust the position of the robot. Similar to the VE module, the input coordinate instructions need to be in the workspace, and the "Execute" button must be clicked for the instructions to take effect.

[0098] Angle adjustment: The AR human-computer interaction interface includes a joint angle input box, where users can adjust the angle value of each joint individually (in degrees) and observe the adjustment effect in real time through the AR interface.

[0099] ③View switching

[0100] Users can switch between the front view, side view, and top view in the AR interface to observe the robot's display effects in augmented reality from different angles. These view switches include:

[0101] Front view image: displays the image of the robot from the front. Teachers and students can clearly see the front structure of the robot. This angle is often used for fine adjustment and can reflect the perspective information from the terminal device more intuitively and clearly.

[0102] Side view image: Displays the image of the robot from the side, which helps teachers and students understand the side details of the robot.

[0103] Top view image: Displays the image of the robot as viewed from above, allowing users to check the overall layout and top details of the robot. This angle makes it easier to carry out the main layout during teleoperation.

[0104] ④Camera image test

[0105] Viewing the live camera feed, users can take and save pictures in the AR interface to confirm image quality and the availability of the capture function. Figure 7 shown.

[0106] The data viewing interface of this experiment is as follows Figure 8 As shown in the figure, it is mainly used to display and analyze the acoustic data collected during the experiment. The experiment measures the equivalent free-field radiated sound power of a spherical sound source in a reverberant water tank, with the purpose of analyzing the changes in the acoustic signal in a graphical way. The operator can conduct a more in-depth study of the signal by selecting different analysis dimensions and understand the radiated sound power characteristics of the sound source in different environments.

[0107] The process of signal propagation underwater is different from that in the air. When propagating underwater, the signal will be absorbed and scattered by the water medium, resulting in a certain delay compared to propagation in the air. This delay will become more and more obvious as the distance increases. Therefore, the technical difficulties of remote control and underwater remote control, in addition to the resistance of underwater remote control, should also focus on the delay of underwater acoustic signals.

[0108] In order to explore the impact of underwater network delay on the slave device system in this experiment, these delay characteristics are simulated by introducing buffering technology in the laboratory environment, which also includes the simulation of high delay and random changes in network transmission. The specific implementation steps are as follows: set a buffer, and set the buffer to store N command records, which are used to temporarily store the instructions generated by the master device; during the control process, the master device generates a control instruction according to the preset time interval ΔT. This command record is not directly transmitted to the slave device, but is stored at the end of the buffer. At the same time, the earliest command record is extracted from the top of the buffer and transmitted to the slave device through the network, so that the slave device executes the corresponding control instruction, and then the record is deleted from the buffer. In this way, the residence time of each command record in the buffer is the simulated additional delay time T, T = N × ΔT, thereby realizing the simulation and control of the network delay effect.

[0109] 1) Initialization process

[0110] This experimental system needs to be initialized before operation, mainly to set basic parameters to ensure that the system can work as expected. The delay initialization flow chart is as follows Fig. 9 shown.

[0111] First, the system reads the preset time interval to determine the frequency of data collection and processing. The fixed time value provides a benchmark for subsequent delay measurement and recording. In addition, the delay mid-zone boundary value (delay mid-zone boundary time) of this system is calculated. The purpose of this step is to provide a benchmark value for evaluating subsequent data delays. The system uses an empty string to initialize the record sequence to ensure that the initial state of the data record is clear. After the basic parameters of the system are set, the foundation is laid for the subsequent recording and transmission process.

[0112] 2) Fixed delay process

[0113] Fixed delay is the buffer management mechanism of this system, which is responsible for receiving, judging and managing the execution order of upper-level instructions. First, start the timer to ensure that the time interval for instruction checking meets the requirements of this system. Then the system extracts the initial instruction record from the buffer and determines whether the current instruction is a valid command from the upper layer. If the judgment is "yes", the system will extract the top-level instruction from the buffer for execution to ensure the priority of the instructions. At the same time, the new instruction is placed at the bottom of the buffer to ensure that subsequent instructions are processed in sequence, realizing the FIFO (first in, first out) instruction execution logic. If the instruction transmitted is not a command instruction, the process returns directly without performing any further operations. The program flow chart is as follows Fig.10 shown.

[0114] 3) Delay and record update process

[0115] If the system needs to be dynamically updated in terms of delay. Fig.11 As shown in the figure, the system first records the latest delay and saves the delay data to the system. Subsequently, the system will determine whether the delay has changed according to the instructions issued by the PD. If the delay increases, the current delay change value is updated, and various delay indicators are calculated and accumulated, including the zone delay, global delay, and the accumulated delay value of the power equipment. These data will be recorded in the corresponding delay record table for subsequent analysis and optimization.

[0116] If the delay does not change, the system directly returns to wait for the next detection cycle. Through this procedure, the system can dynamically track the delay situation, ensure the real-time nature of the delay information, and provide support for the system's delay optimization.

[0117] The principle of master-slave teleoperation is: the master end (PC end) sends control signals to the slave end (hydrophone and transducer end robot), the slave end performs corresponding operations according to these signals, and transmits feedback (such as position, hydroacoustic signal, etc.) back to the master end, so that the operator can perceive and adjust the operation.

[0118] In the underwater master-slave teleoperation system, there are two robots at the slave end, namely the transducer end (transmitter end) robot and the hydrophone end (receiver end) robot. The transducer end robot has two degrees of freedom, namely the freedom of movement in the Z-axis direction and the degree of freedom of rotation at the end. The hydrophone end robot has four degrees of freedom, including three rectangular coordinates of the mobile joint and a tail end of the rotation joint.

[0119] like Fig.12 Establish the origin of the world coordinate system O w ( Fig.12 The top corner of the water tank in the upper right corner of the middle), establish a fixed coordinate system O w xyz: world coordinate system, established at the hydrophone end; establish the attached mobile coordinate system Ox xyz: attached to the moving beam at the end of the hydrophone; establish the attached moving coordinate system O y xyz: attached to the moving beam at the end of the hydrophone; establish the attached moving coordinate system O z xyz: attached to the hydrophone; establish the attached rotating coordinate system O A xyz: attached to the indexing plate of the hydrophone; establish the attached moving coordinate system O z2 xyz: attached to the transducer; establish the attached rotating coordinate system O B xyz: attached to the indexing plate of the transducer; set the world coordinate origin O w The coordinates are (0, 0, 0).

[0120] 1) Kinematic modeling of the transducer end

[0121] set up B T z is the coordinate system O z2 xyz to world coordinate system O w The homogeneous transformation matrix of xyz; B T θ is the transducer end coordinate system O B xyz to coordinate system O z2 The homogeneous transformation matrix of xyz;

[0122]

[0123] Through the above formula, we can get the transducer end coordinate system O B xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T B .

[0124]

[0125] Where a is the length of the longitudinal guide rail; b is the length of the moving beam at the transducer end, z B is the underwater depth of the transducer, θ B is the rotation angle of the transducer.

[0126] 2) Kinematic modeling of hydrophone:

[0127] set up A T x is the coordinate system O x xyz to world coordinate system O w The homogeneous transformation matrix of xyz; A T y is the coordinate system O y xyz to coordinate system O x The homogeneous transformation matrix of xyz; A Tz is the coordinate system O z xyz to coordinate system O y The homogeneous transformation matrix of xyz; A T θ is the hydrophone end coordinate system O A xyz to coordinate system O z The homogeneous transformation matrix of xyz is shown as follows:

[0128]

[0129] Finally, the hydrophone end coordinate system O is obtained A xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T A for:

[0130]

[0131] where x A is the distance that the hydrophone extends along the longitudinal rail, y A is the distance that the hydrophone extends along the moving beam, z A is the underwater depth of the hydrophone, θ A is the rotation angle of the hydrophone.

[0132] 3) Hydrophone-Transducer Modeling

[0133] Hydrophone end coordinate system O A xyz to transducer end coordinate system O B Homogeneous transformation matrix of xyz B T A for:

[0134] B T A = B T W · W T A

[0135]

[0136] In order to achieve effective control of the master-slave teleoperation system, it is necessary to establish a virtual environment in the computer. This virtual environment truly reflects the actual operating environment through mathematical modeling and physical simulation. The main components of the virtual environment are as follows.

[0137] 3D model: 3D model of the hydrophone end and transducer end robot.

[0138] Physics engine: used to simulate the physical laws of robot movement and interaction.

[0139] Control interface: used to receive control signals from experimenters or slave devices and output corresponding feedback.

[0140] The physics engine in the virtual environment uses a physics-based model that can simulate the robot's motion characteristics in different environments, including gravity, underwater friction and collisions.

[0141] The user interface is an important part of the virtual environment, which is mainly used to display the robot status and receive the operation input of the experimenter. The user interface design considers the following points.

[0142] Real-time: Ensure real-time display of robot status and experimenter operations on the interface.

[0143] Intuitive: The interface is simple and clear, allowing experimenters to quickly get started.

[0144] Interactivity: Supports experimenters to interact with the system through various input devices (such as keyboard, mouse, handle, etc.).

[0145] The information displayed in the user interface includes the current position information and posture information of the slave device, as well as water sound, environment and other information. At the same time, the interface allows experimenters to make real-time adjustments and configurations to adapt to different task requirements.

[0146] In order to achieve fine control of the slave devices (hydrophone segment robot and transducer end robot), mapping is performed in the workspace. The end position of the device needs to be calculated by the kinematic forward solution model, and then the desired position of the end actuator of the slave device is obtained according to the mapping algorithm. Then, the position or angle of each joint is obtained according to the kinematic inverse solution algorithm. Finally, the position or angle information of the joint is sent to the slave device through the controller. [8] The workspace mapping algorithm also uses the world coordinate O w xyz is the reference coordinate system, which can largely ensure that the terminal posture of the slave device and the master virtual device are consistent. The biggest difference between it and joint space mapping is real-time, that is, the movement trajectory of the master virtual device is analyzed in real time, and the real-time data is sent to the slave device to complete the tracking of the master virtual device by the slave device. According to the motion range of the master and slave ends, the mapping coefficient should be within the following rules:

[0147]

[0148] Where k n is the mapping scale factor in the n direction (n = x, y, z); U n,max is the maximum motion range of the slave device in the n direction; U n,min is the minimum motion range of the slave device in the n direction; T n,maxT is the maximum motion range of the master virtual device in the n direction; n,min The minimum motion range of the master virtual device in the n direction. Combined with the position and angle of the robot arm movement, ensure that the posture of the master virtual device matches the working space range of the slave device after mapping. [9] , and finally the mapping formulas at the hydrophone end and the transducer end are obtained as follows:

[0149]

[0150] Among them, [x u1 y u1 z u1 θ u1 ] T represents the desired position and attitude of the hydrophone in the slave device, expressed in Cartesian space coordinates; [x t1 y t1 z t1 θ t1 ] T represents the actual position of the hydrophone in the virtual model of the master device, expressed in Cartesian space coordinates; [x u2 θ u2 ] T represents the desired position and attitude of the transducer in the slave device, expressed in Cartesian space coordinates; [x t2 θ t2 ] T represents the actual position of the transducer in the virtual model of the master device, expressed in Cartesian space coordinates;

[0151] The actual position and posture of the master virtual device terminal can be used to map the expected position and posture of the slave device terminal. Then, the movement trajectory and joint rotation angle of the hydrophone and transducer robots can be obtained according to the inverse kinematics formula of the slave device, thereby realizing remote operation control of the master and slave devices.

[0152] Embodiment 1:

[0153] The robot trajectory was generated by MATLAB, and the motion performance of the hydrophone-side robot and the transducer-side robot was simulated. By defining the initial and target end positions, as well as several intermediate point positions, the time parameters were generated and the trajectory was calculated as well as the rotation angle, velocity, and acceleration during the motion.

[0154] The "position-time" curve at the hydrophone end is as follows: Fig.13 As shown (wherein the X-axis represents movement along the longitudinal guide rail, the Y-axis represents movement along the moving beam direction, and the Z-axis represents the depth of entry into the water).

[0155] The "speed-time" curve at the hydrophone end is as follows: Fig.14 The “acceleration-time” curve at the hydrophone end is shown in Fig.15 shown.

[0156] The "position-time", "velocity-time" and "acceleration-time" curves at the transducer end are as follows: Fig.16 The “rotation angle-time” curves of the hydrophone end and the transducer end are shown in Fig.17 shown.

[0157] For the experimental teaching case of our center - water tank reverberation field measurement, a physical simulation verification was carried out. This experiment measures the equivalent free-field radiated sound power of a spherical sound source in a reverberation tank. The following are the measurement experimental steps.

[0158] 1. Reverberation time measurement

[0159] 1) Students log in to the experimental operating system one by one in order, and only one or a group of students are in operation at the same time. Build a measurement system, an auxiliary robot, and a hydrophone for collecting data; the hydrophone is remotely operated by the master controller robot in the virtual environment, and controls the slave device No. 1 to place it in the required position (position and posture) in the reverberation tank, and distribute it reasonably in the reverberation tank.

[0160] 2) Use a spherical sound source as the excitation source, and use a dedicated drive cable to connect it to the corresponding output terminal of the power amplifier. Similarly, in the master-slave remote operation mode, the master controller robot operates the virtual environment and controls the slave device No. 2 to place it in the required position (position and posture) in the reverberation tank.

[0161] 3) After remotely checking the connections of each device and confirming that they are correct, self-check the functions of all devices and set the gain switch of the amplifier to the lowest level, and test the computer, robot, amplifier, collector, etc. in turn.

[0162] 4) After all experimental devices are ready, use the signal source to generate a single-frequency sine wave to generate a diffusion field in the reverberation tank, then cut off the sound source and measure its attenuation curve to calculate the reverberation time. The reverberation time measurement requires 6 times.

[0163] 2. Measuring the average sound pressure level

[0164] 1) Similarly, a measurement system is constructed. The sound source to be measured is placed in a normal installation manner in one or more typical installation positions relative to the boundary surface in the reverberation water tank. The sound source is placed in the water tank at least 0.2 m away from any tank surface.

[0165] 2) Remotely open the collector, set the sampling frequency to 10 times the signal frequency, and select the collection time to 5 seconds.

[0166] 3) Remotely turn on the power amplifier switch, click the measurement start button, and let the hydrophone perform spatial averaging in the reverberation tank to obtain the required data.

[0167] According to 1 / 3 octave center frequency: 8kHz, 10kHz, 12.5kHz.

[0168] 3. Experimental Data Processing

[0169] 1) Reverberation time data processing

[0170] Use software to write a data processing program to calculate the reverberation time according to the definition of reverberation time.

[0171] 2) Spatial average sound pressure level data processing

[0172] A data processing program is written using software to perform Fourier transform on the read time domain data, read the sound pressure level at the test frequency, and then calculate the radiated sound power.

[0173] Through observation and feedback from three trial simulation teaching cycles, it was found that students' ability to understand relevant theoretical knowledge has been improved, and they are very interested in understanding cross-disciplinary concepts such as teleoperated robots, local artificial intelligence, and supervisory control in experiments, which greatly stimulated the learning enthusiasm of students participating in the test, and the experimental results are very satisfactory. The teleoperation system design of the present invention allows students to truly go deep into the marine environment to conduct scientific experiments. The data source is real, and the experimenter's feeling is getting closer and closer to the scene, which is of great significance to the cultivation of innovative and high-quality talents.

[0174] The ocean information experimental teaching platform based on the master-slave remote control system provided by the present invention has the basic functions of a large dedicated water tank, and the external dimensions are miniaturized, which effectively solves the contradiction between the large teaching hours and the small amount of teaching equipment. In the limited space of the laboratory, various underwater test environments are simulated in the water tank test space through transmitting transducers and receiving transducers of different frequency bands, and the experimental operation of ocean information collection is completed in combination with corresponding parameter testing methods and signal processing means. The present invention has the characteristics of high stability and reliability, simple operation and use, and convenient later maintenance, and is particularly suitable for experimental teaching.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The marine information experimental teaching platform based on the master-slave teleoperation system is characterized by: It comprises a master-end device and a slave-end device; the slave-end device comprises an experimental water pool, a longitudinal guide rail installed on the experimental water pool, a transducer-end moving beam and a hydrophone-end moving beam installed on the longitudinal guide rail, and a transducer and a hydrophone arranged in the experimental water pool; the transducer is installed below the transducer-end moving beam through a first connecting rod, and a first indexing plate is arranged on the first connecting rod; a transverse guide rail is arranged on the top surface of the hydrophone-end moving beam, and a slider is arranged on the transverse guide rail; the hydrophone is installed below the slider through a second connecting rod, and a second indexing plate is arranged on the second connecting rod; The master-end device is equipped with a master-slave teleoperation system, in which a virtual model of the slave-end device is established; the experimenter operates the virtual model of the slave-end device through the master-slave teleoperation system, and the master-slave teleoperation system converts the actions of the virtual model of the slave-end device into control instructions and transmits them to the slave-end device; after receiving the control instructions, the slave-end device controls the rotation angle of the transducer and the hydrophone through the first dividing plate and the second dividing plate, adjusts the longitudinal position of the transducer and the hydrophone through the movement of the transducer-end moving beam and the hydrophone-end moving beam on the longitudinal guide rail, and adjusts the lateral position of the hydrophone through the movement of the slider on the lateral guide rail on the top surface of the hydrophone-end moving beam; the experimenter sends an electrical signal to the transducer through the master-slave teleoperation system, the transducer converts the electrical signal into an acoustic signal and propagates it in the experimental water pool, and the hydrophone receives the acoustic signal and feeds it back to the master-end device, so that the experimenter can remotely operate the experimental process.

2. The marine information experimental teaching platform based on the master-slave teleoperation system according to claim 1 is characterized by: A virtual model of the slave device is established in the master-slave teleoperation system, specifically: Take the top corner of one end of the experimental pool as the origin and establish the world coordinate system O w xyz; Establish the model of the slave device in the world coordinate system and set the hydrophone end coordinate system O A xyz and transducer end coordinate system O B xyz; Hydrophone end coordinate system O A xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T A for: Transducer end coordinate system O B xyz to world coordinate system O w Homogeneous transformation matrix of xyz W T B for: Hydrophone end coordinate system O A xyz to transducer end coordinate system O B Homogeneous transformation matrix of xyz B T A for: Among them, x A y is the distance that the hydrophone extends along the longitudinal guide rail; A is the distance that the hydrophone extends along the moving beam at the hydrophone end; a is the length of the longitudinal guide rail; b is the length of the moving beam at the transducer end; z A is the underwater depth of the hydrophone; z B is the underwater depth of the transducer; θ A is the rotation angle of the hydrophone; θ B is the rotation angle of the transducer.

3. The marine information experimental teaching platform based on the master-slave teleoperation system according to claim 2 is characterized by: In order to achieve fine control of the slave device, mapping is performed in the workspace, the position and posture of the hydrophone and the transducer are calculated by the kinematic forward solution model, the expected position and posture of each actuator in the slave device are obtained according to the mapping algorithm, and then the position or angle of each actuator is obtained according to the kinematic inverse solution algorithm; the actuator includes a moving beam at the transducer end, a moving beam at the hydrophone end, a first indexing plate, a second indexing plate and a slider; The workspace mapping algorithm uses the world coordinate O w xyz is the reference coordinate system to ensure that the position and posture of the virtual model in the slave device and the master device are consistent, analyze the movement trajectory of the virtual model in the master device in real time, and send the real-time data to the slave device to complete the tracking of the virtual model in the master device by the slave device; according to the motion range of the master and slave ends, the mapping coefficients meet the following conditions: Among them, k n Indicates the mapping scale factor in the n direction, n = x, y, z; U n,max It represents the maximum range of motion of the hydrophone or transducer in the slave device in the n direction; U n,min T represents the minimum motion range of the hydrophone or transducer in the slave device in the n direction; n,max T represents the maximum motion range of the hydrophone or transducer in the virtual model of the master device in the n direction; n,min Indicates the minimum motion range of the hydrophone or transducer in the virtual model of the master device in the n direction; To ensure that the posture of the virtual model of the master device matches the working space range of the slave device after mapping, the mapping formulas of the hydrophone and transducer are: Among them, [x u1 y u1 z u1 θ u1 ] T represents the expected position and attitude of the hydrophone in the slave device; [x t1 y t1 z t1 θ t1 ] T represents the actual position of the hydrophone in the virtual model of the master device; [x u2 θ u2 ] T represents the expected position and attitude of the transducer in the slave device; [x t2 θ t2 ] T Indicates the actual position of the transducer in the virtual model of the master device; The actual position and posture of the hydrophone and transducer in the virtual model of the master-end device can be used to map the expected position and posture of the hydrophone and transducer in the slave-end device. Then, the movement trajectory and rotation angle of each actuator on the hydrophone and transducer end can be obtained according to the inverse kinematics formula of the slave-end device, thereby realizing remote operation control of the master-slave device.

4. The marine information experimental teaching platform based on the master-slave teleoperation system according to claim 1 is characterized by: The master-slave teleoperation system includes a network communication connection and monitoring module, a VE human-computer interaction control module, an AR human-computer interaction control module, a view switching module, and a data display and analysis module; The network communication connection and monitoring module is used to set the connection addresses of the master device and the slave device, and monitor the network communication connection status and network signal strength; The VE human-computer interaction control module is used to construct a virtual model of the slave device and is used by the experimenter to perform operations on the virtual model of the slave device; The AR human-computer interaction control module is used to provide an augmented reality environment, in which the experimenter can control the master device and restore the sense of presence of the underwater environment while controlling the slave device in real time; The view switching module is used for the experimenter to observe the virtual model of the slave device from different angles, including front view, side view and top view; The data display and analysis module is used to display the water acoustic signal received by the hydrophone in the slave device in real time, and is also used to convert the operation of the experimenter on the virtual model into control instructions and transmit them to the slave device.

5. The marine information experimental teaching platform based on the master-slave teleoperation system according to claim 1 is characterized by: In order to simulate the delay of the signal in the process of underwater propagation, a buffer is set in the master-slave teleoperation system, and the buffer is set to be able to store N command records, which is used to temporarily store the instructions generated by the master device; During the control process, the master device generates control instructions according to the preset time interval ΔT. The control instructions are not directly transmitted to the slave device, but are stored at the end of the buffer. At the same time, the earliest control instruction is extracted from the top of the buffer and transmitted to the slave device through the network, so that the slave device executes the corresponding control instruction, and then the control instruction is deleted from the buffer. The residence time of each control instruction in the buffer is recorded, which is the delay T of the analog signal in the process of underwater propagation, T = N × ΔT.

6. The marine information experimental teaching platform based on the master-slave teleoperation system according to claim 5 is characterized by: The delay process of the analog signal during underwater propagation is specifically as follows: 1) Initialization; the master-slave teleoperation system reads a preset time interval ΔT to determine the frequency of data acquisition and processing; calculates the delay mid-zone boundary value for evaluating subsequent data delay conditions; the master-slave teleoperation system uses an empty string to initialize the record sequence; 2) Fixed delay process; Start the timer to ensure that the time interval for instruction checking meets the requirements; extract the control instruction from the top layer of the buffer and determine whether the current control instruction is a valid command from the upper layer. If the judgment is "yes", execute the control instruction and place the new control instruction at the bottom of the buffer to ensure that subsequent control instructions are processed in order to implement the FIFO instruction execution logic; if the judgment is "no", do not perform any further operations; 3) Delay and record update process: If the delay of the master-slave teleoperation system needs to be dynamically updated, the latest delay situation is first recorded and the delay data is saved in the master-slave teleoperation system; then, the master-slave teleoperation system determines whether the delay has changed according to the control instructions sent from the buffer; If the delay increases, the current delay change value is updated, and various delay indicators are calculated and accumulated, including zone delay, global delay and accumulated delay value of power equipment. These data will be recorded in the corresponding delay record table for subsequent analysis and optimization; if the delay does not change, the master-slave teleoperation system directly returns to wait for the next detection cycle; through this program, the master-slave teleoperation system can dynamically track the delay situation, ensure the real-time nature of the delay information, and provide support for delay optimization.

Images