An underwater positioning system for underwater robots based on human-machine interaction

Abstract

The application provides an underwater positioning system for an underwater robot based on human-computer interaction. The method comprises the following steps: in response to a positioning request of a user, a multi-dimensional sensor network of the underwater robot is constructed; multi-dimensional monitoring data of a target space is dynamically collected along a moving track of the underwater robot by using the multi-dimensional sensor network; a multi-branch strategy network is used to pre-process the multi-dimensional monitoring data to obtain multi-dimensional data to be processed; a sparse feature map is generated by using ORB-SLAM3, and a local dense map is generated by using a point cloud registration ICP algorithm; a sparse feature map, a local dense map and a pose feature in the third data to be processed are fused by using EKF to obtain a local scene map; a segmented bundle adjustment algorithm is used to segmentally optimize the local scene map according to a motion mode, and positioning information of the underwater robot is output to the user in real time based on the local scene map. The positioning accuracy of the underwater robot can be improved, and the positioning efficiency of the underwater robot can be improved.

Description

Technical Field

[0001] The embodiments of this application relate to the field of data processing, and more specifically, the embodiments of this application relate to an underwater robot positioning system based on human-computer interaction. Background Technology

[0002] Underwater robots are intelligent devices that can autonomously or remotely perform specific tasks in underwater environments. Their core technologies integrate multiple disciplines such as mechanical engineering, electronic communication, artificial intelligence, and marine science.

[0003] Taking swimming pools as an example, as a highly dynamic environment with limited space, the unique spatial scale and activity characteristics of swimming pools place more stringent requirements on the positioning system of underwater robots.

[0004] Swimming pools typically have limited space, with common pools ranging from 20-50 meters in length and 10-25 meters in width. Underwater robots must perform precise operations within this limited space, such as edge cleaning, obstacle detection, or fixed-point inspection. This confined space characteristic places extremely high demands on positioning accuracy. Underwater robots must maintain accurate positional awareness in environments close to pool walls, near other equipment such as underwater lights and drains, or alongside other dynamic objects (such as swimmers). Otherwise, positioning errors may lead to mission failure or collisions and other safety incidents. Summary of the Invention

[0005] In this context, the embodiments of this application aim to provide an underwater robot positioning system based on human-computer interaction, which can improve the positioning accuracy and efficiency of underwater robots.

[0006] In a first aspect of the embodiments of this application, an underwater robot positioning method based on human-computer interaction is provided, comprising:

[0007] In response to the user's positioning request, a multi-dimensional sensor network for the underwater robot is constructed; the multi-dimensional sensor network is used to dynamically collect multi-dimensional monitoring data of the target space along the movement trajectory of the underwater robot; wherein, the multi-dimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor.

[0008] A multi-branch strategy network is used to preprocess the multidimensional monitoring data to obtain multidimensional data to be processed; the multidimensional data to be processed includes at least: first data to be processed based on the spatial visual information, second data to be processed based on the sonar echo signal, and third data to be processed based on the high-frequency motion data.

[0009] The first data to be processed is used to generate a sparse feature map using ORB-SLAM3, and the second data to be processed is used to generate a locally dense map using the point cloud registration ICP algorithm. The sparse feature map, the empty locally dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot.

[0010] The segmented bundle adjustment algorithm (Segmented BA) is used to segment and optimize the local scene map according to the motion mode. Based on the optimized local scene map, the positioning information of the underwater robot is output to the user in real time.

[0011] In a second aspect of the embodiments of this application, an underwater robot positioning system based on human-computer interaction is provided, comprising:

[0012] A construction module is used to construct a multi-dimensional sensor network for an underwater robot in response to a user's positioning request; the multi-dimensional sensor network is used to dynamically collect multi-dimensional monitoring data of the target space along the movement trajectory of the underwater robot; wherein, the multi-dimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor.

[0013] The preprocessing module is used to preprocess the multidimensional monitoring data using a multi-branch strategy network to obtain multidimensional data to be processed; the multidimensional data to be processed includes at least: first data to be processed based on the spatial visual information, second data to be processed based on the sonar echo signal, and third data to be processed based on the high-frequency motion data.

[0014] The prediction module is used to generate a sparse feature map from the first data to be processed using ORB-SLAM3, and to generate a locally dense map from the second data to be processed using the point cloud registration ICP algorithm; the sparse feature map, the locally dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot.

[0015] The output module is used to perform segmented optimization of the local scene map according to the motion mode using the segmented bundle adjustment algorithm (Segmented BA), and output the underwater robot's positioning information to the user in real time based on the optimized local scene map.

[0016] In a third aspect of the embodiments of this application, a terminal device is provided, the terminal device comprising: at least one processor, a memory, and an input / output unit; wherein the memory is used to store a computer program, and the processor is used to invoke the computer program stored in the memory to execute the underwater robot underwater positioning method based on human-computer interaction as described in the first aspect.

[0017] In a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, which includes instructions that, when executed on a computer, cause the computer to perform the underwater robot positioning method based on human-computer interaction as described in the first aspect.

[0018] In a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the underwater robot underwater positioning method based on human-computer interaction as described in the first aspect.

[0019] According to an embodiment of this application, an underwater robot positioning system based on human-computer interaction first responds to a user's positioning request by constructing a multi-dimensional sensor network for the underwater robot. The multi-dimensional sensor network then dynamically collects multi-dimensional monitoring data of the target space along the underwater robot's movement trajectory. This multi-dimensional monitoring data includes at least: sonar echo signals collected by a high-frequency sonar module targeting dynamic objects, pool walls, and / or pool bottoms; spatial visual information collected by a visual sensor; and high-frequency motion data collected by an inertial sensor. Next, a multi-branch strategy network is used to preprocess the multi-dimensional monitoring data to obtain multi-dimensional data to be processed. This multi-dimensional data to be processed includes at least: first data to be processed based on the spatial visual information; second data to be processed based on the sonar echo signals; and third data to be processed based on the high-frequency motion data. Then, the first data to be processed is used to generate a sparse feature map using ORB-SLAM3, and the second data to be processed is used to generate a locally dense map using a point cloud registration algorithm. Then, the sparse feature map, the empty local dense map, and the pose features in the third data to be processed are fused using extended Kalman filtering to obtain a local scene map containing the underwater robot's pose state. Finally, a piecewise bundle adjustment algorithm is used to optimize the local scene map in segments according to the motion mode, and the underwater robot's positioning information is output to the user in real time based on the optimized local scene map.

[0020] The embodiments described in this application enable the construction of a multi-dimensional sensor network for underwater robots, achieving dynamic acquisition of multi-dimensional monitoring data of the target space. This is combined with data preprocessing via a multi-branch strategy network, and the generation and fusion of maps using various algorithms. Finally, a piecewise bundle adjustment algorithm is used to optimize the map and output positioning information. This embodiment effectively overcomes the interference of complex underwater environments on positioning, organically integrating multi-source information such as sonar, vision, and inertial sensors. It avoids the limitations of single sensors, significantly improving the real-time performance and accuracy of positioning information. Simultaneously, the optimized algorithm reduces the computational load of data processing, enabling underwater robots to quickly and accurately determine their pose in complex and changing underwater environments. This provides reliable positioning support for underwater exploration and operations, improving the positioning accuracy and efficiency of underwater robots. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating an underwater robot positioning method based on human-computer interaction, as shown in this application.

[0022] Figure 2 This application presents a schematic diagram of the structure of an underwater robot positioning system based on human-computer interaction.

[0023] Figure 3 The present invention provides a schematic diagram of the structure of a medium according to an embodiment of the present application. Detailed Implementation

[0024] The following is for reference. Figure 1 , Figure 1 This is a flowchart illustrating an underwater robot positioning method based on human-computer interaction, provided as an embodiment of this application. It should be noted that the implementation methods of this application can be applied to any applicable underwater operation system usage scenario and / or maintenance scenario.

[0025] The applicant discovered that traditional positioning technologies perform poorly in short-range scenarios. In shallow water or short-baseline scenarios of swimming pools, sound waves are more easily reflected by the water surface and pool bottom, creating a large amount of pseudorange interference beyond the direct signal. For example, when a robot is close to the pool wall in shallow water, sound waves may be reflected by the pool wall before being captured by the receiving array, resulting in a significant increase in ranging error, exceeding the acceptable accuracy range for short-range operations. Furthermore, traditional visual positioning (such as SLAM) relies on identifiable features in the environment (such as lane lines, pool wall textures, decorative patterns, etc.) for pose estimation. In actual swimming pools, older pools may lack clear lane lines, and the pool walls may be worn, stained, or covered with algae due to long-term use, resulting in blurred textures. Some pools use smooth ceramic tiles or glass with high surface reflectivity, easily producing light spots or reflections, which interfere with feature extraction. In these cases, the accuracy of visual positioning will be significantly reduced due to feature loss or quality degradation, making it difficult to meet the requirements of short-range operations.

[0026] To address at least one of the aforementioned technical problems, this application provides an underwater robot positioning system based on human-computer interaction. Specifically, the underwater robot positioning system based on human-computer interaction first constructs a multi-dimensional sensor network for the underwater robot in response to a user's positioning request. Then, the multi-dimensional sensor network dynamically collects multi-dimensional monitoring data of the target space along the underwater robot's movement trajectory. This multi-dimensional monitoring data includes at least: sonar echo signals collected by a high-frequency sonar module targeting dynamic objects, pool walls, and / or pool bottoms; spatial visual information collected by a visual sensor; and high-frequency motion data collected by an inertial sensor. Next, a multi-branch strategy network is used to preprocess the multi-dimensional monitoring data to obtain multi-dimensional data to be processed. This multi-dimensional data to be processed includes at least: first data to be processed based on the spatial visual information; second data to be processed based on the sonar echo signals; and third data to be processed based on the high-frequency motion data. The first data to be processed is used to generate a sparse feature map using ORB-SLAM3, and the second data to be processed is used to generate a locally dense map using a point cloud registration algorithm. Then, the sparse feature map, the empty local dense map, and the pose features in the third data to be processed are fused using extended Kalman filtering to obtain a local scene map containing the underwater robot's pose state. Finally, a piecewise bundle adjustment algorithm is used to optimize the local scene map in segments according to the motion mode, and the underwater robot's positioning information is output to the user in real time based on the optimized local scene map.

[0027] Figure 1 The flowchart of an embodiment of this application provides an underwater robot positioning method based on human-computer interaction, including:

[0028] Step S101: In response to the user's positioning request, construct a multi-dimensional sensor network for the underwater robot.

[0029] In this embodiment, a multi-dimensional sensor network integrates various physical sensing modules to construct a data acquisition system covering multi-dimensional environmental features. Its core principle lies in utilizing the complementary physical characteristics of different sensors to achieve omnidirectional perception of the target space. For example, a high-frequency sonar module emits sound waves and receives echoes to analyze target distance, shape, and motion state, and is particularly adept at obstacle detection in low-visibility environments. Visual sensors capture spatial texture and geometric structures through optical imaging, providing semantically rich environmental information. Inertial sensors record the robot's acceleration and angular velocity through high-frequency sampling, reflecting changes in motion state in real time. The network nodes of the multi-dimensional sensor network adopt a distributed architecture, achieving dynamic deployment through a self-organizing topology. Data from each sensor is fused after time synchronization and spatial calibration to form a unified environmental model. This multi-dimensional sensor network, through multi-source information collaboration, overcomes the perception limitations of a single sensor, providing highly robust data support for complex underwater environments.

[0030] Step S102: Using the multi-dimensional sensor network, multi-dimensional monitoring data of the target space is dynamically collected along the movement trajectory of the underwater robot.

[0031] In this embodiment of the application, the multidimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor.

[0032] For example, multidimensional monitoring data is generated collaboratively by multimodal sensors, covering environmental geometry, motion state, and dynamic target features. Sonar echo signals are analyzed using time-frequency analysis to extract target reflection intensity and distance information, constructing a high-resolution point cloud. Visual information is extracted and matched using image features to generate spatial topology and semantic labels. Motion data is used to calculate pose changes through integration and filtering algorithms, providing dynamic motion constraints. In the above example, the three types of data complement each other in the spatiotemporal dimensions. Specifically, sonar compensates for the limitations of vision in turbid waters, vision provides fine-grained features that are difficult for sonar to capture, and inertial data eliminates the impact of motion blur on perception. During data fusion, a dynamic weight allocation strategy (such as increasing the weight of inertial data during intense motion) optimizes the confidence of multi-source information, ultimately forming a joint representation that includes environmental state and robot pose.

[0033] Thus, the deep integration of multi-dimensional sensor networks and monitoring data significantly improves the environmental adaptability and reliability of underwater robot positioning systems. Through collaborative perception of sonar, vision, and inertial data, the system maintains stable environmental modeling capabilities in complex scenarios such as dynamic occlusion and low light, effectively reducing the false matching rate. The multi-modal data fusion algorithm, through joint optimization at the feature level and decision level, enhances the correlation analysis between local features and global semantics, enabling positioning results to possess both geometric accuracy and semantic interpretability. Furthermore, networked deployment and dynamic weighting mechanisms give the system real-time adaptive capabilities, allowing it to quickly adjust perception strategies during high-speed movement or sudden environmental changes, ensuring positioning continuity and robustness. Through multi-dimensional information complementarity and intelligent fusion, a high-precision, interference-resistant positioning solution is provided for underwater robots, particularly suitable for dynamic scenarios such as pool cleaning and underwater engineering surveys, significantly improving operational safety and efficiency.

[0034] Step S103: Use a multi-branch strategy network to preprocess the multidimensional monitoring data to obtain multidimensional data to be processed.

[0035] In this embodiment of the application, the multidimensional data to be processed includes at least: first data to be processed based on the spatial visual information, second data to be processed based on the sonar echo signal, and third data to be processed based on the high-frequency motion data.

[0036] As an optional embodiment, in step S103, the preprocessing of the multidimensional monitoring data using a multi-branch strategy network to obtain multidimensional data to be processed includes:

[0037] In the first branch, the Brown-Conrady model is used to map the distorted pixel coordinates in the spatial visual information back to ideal coordinates through polynomial fitting of the distortion coefficients, thus achieving distortion removal. In the first branch, the distortion-removed spatial visual information is then used to perform dynamic target detection through a lightweight YOLOv8 network to obtain the first data to be processed. The first data to be processed includes: distortion-removed spatial visual information and a dynamic mask used to calibrate the position of the dynamic target. In the second branch, a time-division strategy network is used to perform multipath suppression on the sonar echo signal to remove multipath artifacts, thus obtaining the second data to be processed. In the third branch, quaternion integral compensation is used to process motion distortion in the high-frequency motion data, and a Kalman filter-based sensor noise reduction model is used to remove pose offsets in the high-frequency motion data, thus obtaining the third data to be processed.

[0038] Specifically, the first branch performs geometric correction on the raw images acquired by the visual sensor using the Brown-Conrady distortion model. It establishes a mapping relationship between distorted pixels and ideal pixels using polynomial fitting of distortion coefficients, eliminating image distortion caused by fisheye or barrel distortion. The distortion-free image is then used for dynamic object detection via a lightweight YOLOv8 network. The YOLOv8 network employs depthwise separable convolution and channel pruning techniques to reduce computational load while maintaining detection accuracy. It identifies dynamic obstacles in the image in real time, such as swimmers and floating objects, and generates dynamic masks to mark their spatial locations. This process is accelerated by hardware (e.g., GPU parallel computing) to achieve millisecond-level processing, ensuring real-time perception capabilities in dynamic environments and providing high-confidence target location information for subsequent obstacle avoidance decisions.

[0039] To address the multipath effect in sonar echo signals (i.e., artifact interference caused by the multiple superposition of sound waves after reflection from the pool wall), a time-division strategy network analyzes the time delay and intensity characteristics of the echo using a time-domain signal separation algorithm, combined with multi-channel adaptive filtering technology to suppress environmental noise and reverberation. The network employs a Long Short-Time Memory (LSTM) structure to capture the temporal correlation of the sonar signal, dynamically adjusting filtering parameters to distinguish between target echoes and multipath interference, ultimately outputting a denoised high-fidelity sonar image. This method effectively reduces misjudgments of complex geometric structures (such as pool corners and underwater pipes), improves the sonar's ability to distinguish between static and dynamic targets, and significantly enhances the clarity of target outlines, especially in low signal-to-noise ratio scenarios.

[0040] The motion processing branch compensates for motion distortion in high-frequency motion data from inertial sensors using a quaternion integration algorithm, leveraging the rotational invariance of quaternions to eliminate pose jumps caused by high-frequency vibrations and sampling noise. Subsequently, a Kalman filter-based sensor denoising model jointly optimizes accelerometer and gyroscope data, correcting pose estimation errors through state prediction and measurement update iterations, suppressing high-frequency noise and random drift. The Kalman filter's adaptive covariance matrix adjustment mechanism dynamically allocates sensor weights based on the intensity of motion (e.g., enhancing gyroscope weights during high-speed rotation), ensuring the smoothness and robustness of pose calculation. This process provides a high-precision time-synchronized pose reference for visual and sonar data, reducing spatiotemporal misalignment issues during multi-sensor fusion.

[0041] Thus, the multi-branch strategy network, through modular design and algorithmic collaboration, achieves efficient preprocessing of multi-dimensional monitoring data. The distortion correction and dynamic detection mechanisms of the vision branch (i.e., the first branch) significantly improve image quality and target recognition efficiency, providing accurate semantic information for environmental modeling; the multipath suppression algorithm of the sonar branch (i.e., the second branch) enhances the reliability of target detection in complex geometric environments and solves the imaging blurring problem of traditional sonar in enclosed waters; the integral compensation and filtering optimization of the motion branch (i.e., the third branch) effectively suppress high-frequency motion noise, ensuring the continuity and stability of pose data. The deep fusion of these three branches lays a high-quality data foundation for subsequent multi-source data fusion and map construction, enabling high-precision positioning and obstacle avoidance capabilities even in extreme scenarios such as dynamic occlusion and low visibility, providing key technical support for the autonomous navigation of underwater robots.

[0042] In step S103 above, in the first branch, the Brown-Conrady model is used to map the distorted pixel coordinates in the spatial visual information back to ideal coordinates by fitting the distortion coefficients with a polynomial, thereby achieving distortion removal. In the first branch, the distortion-removed spatial visual information is used for dynamic target detection through a lightweight YOLOv8 network to obtain the first data to be processed.

[0043] In this embodiment of the application, the first data to be processed includes: distortion-free spatial visual information and a dynamic mask for calibrating the location of a dynamic target.

[0044] Specifically, the Brown-Conrady model establishes a mapping relationship between distorted pixels and ideal pixels by fitting lens distortion characteristics using a polynomial. The core principle of the Brown-Conrady model lies in decomposing image distortion into radial distortion and tangential distortion. Radial distortion is caused by light bending due to uneven lens curvature, while tangential distortion originates from non-orthogonal projection caused by lens assembly deviations.

[0045] Understandably, the Brown-Conrady model quantifies the degree of distortion using polynomial coefficients and iteratively approximates the distorted coordinates using Taylor expansion. During distortion correction, the distorted pixel coordinates are used as input, and the coordinate values ​​are gradually corrected through iterative formulas, ultimately mapping back to the ideal coordinate system without distortion. This process eliminates edge deformation through nonlinear optimization, for example, correcting the curved edges of barrel distortion to straight lines while preserving high fidelity in the central region.

[0046] The distortion-corrected visual information restores the true spatial relationships of the scene by eliminating geometric distortions. For example, the curved lines of the pool wall, originally caused by wide-angle distortion, are corrected to straight lines, enhancing the coherence of the environmental semantics. This process achieves dynamic object detection through a lightweight YOLOv8 network. The network uses depthwise separable convolutions to reduce computation, identifies moving objects (such as swimmers or floating objects) in the image in real time, and generates dynamic masks to mark their spatial locations. The dynamic masks label the target regions using a binarization matrix; for example, the pixel location of the swimmer is set to a high confidence value, while other areas are set to low confidence values, providing spatial constraints for subsequent obstacle avoidance algorithms.

[0047] Thus, the polynomial fitting mechanism of the Brown-Conrady model effectively balances distortion correction accuracy and computational efficiency, making it particularly suitable for underwater cameras with large field of view, and solving the problem of excessive stretching in edge regions of traditional fisheye models. The dynamic mask, generated by a lightweight target detection network, achieves real-time localization and region masking of dynamic targets, reducing the interference of environmental noise on SLAM mapping. For example, in a pool cleaning scenario, the dynamic mask can filter out temporary deformations caused by swimmer movement, making the sparse map generated by ORB-SLAM3 more focused on static pool wall features. The synergistic optimization of both enhances the robustness of visual SLAM in dynamic environments, providing a high-quality spatial benchmark for subsequent multi-sensor fusion.

[0048] In step S103 above, in the second branch, a time-division strategy network is used to perform multipath suppression on the sonar echo signal to remove multipath artifacts in the sonar echo signal, thereby obtaining the second data to be processed, including:

[0049] Based on the number of people entering the target area, the density of active objects in the target area is predicted for the current time period. If the density of active objects is greater than a set threshold, the current time period is considered a peak period. If the density of active objects is not greater than the set threshold, the current time period is considered an off-peak period. For peak periods, a graph convolutional network (GCN) based on wavelet transform is used to separate the direct signal and interference signal from the sonar echo signal, and the direct signal is used as the second data to be processed. For off-peak periods, a physical constraint-based generative adversarial network (PC-GAN) is used to remove noise signals contained in the sonar echo signal through a generator with a U-Net architecture. A physical prior model incorporating sonar propagation and a GAN discriminator are used to predict the authenticity of the generator's output signal to obtain the second data to be processed.

[0050] Specifically, the system employs an adaptive strategy to switch multipath suppression modes based on the varying density of moving objects at different times. During peak periods (such as densely populated swimming pool areas), sonar echo signals generate dense interference due to multipath reflections. In this case, a hybrid network (GCN) based on wavelet transform and graph convolution achieves interference separation through multi-scale time-frequency decomposition and spatial relationship modeling. After wavelet transform decomposes the signal into multiple frequency subbands, the low-frequency subband (LL) uses GCN to model the geometric topology of the sonar array (such as the sound wave propagation paths and geometric constraints between sensor nodes) and constructs an adjacency matrix to capture the spatial consistency features of the direct signal. The high-frequency subband (HH) utilizes 1D-CNN to extract transient pulse features (such as random spikes in multipath interference), and finally separates the direct signal from the interference through feature fusion and a classifier. During off-peak hours (such as open water areas with low pedestrian traffic), a physically constrained generative adversarial network (PC-GAN) combined with the U-Net architecture is used. The generator removes noise based on the multi-scale feature extraction capability of U-Net, while the discriminator introduces physical priors of sonar propagation (such as sound speed profile and multipath delay model) to constrain the authenticity of the generated signal. The noise suppression effect is optimized through adversarial training.

[0051] Understandably, the adaptive strategy balances suppression accuracy and computational efficiency in complex environments by dynamically switching processing modes. During peak hours, the GCN hybrid network utilizes the multi-scale characteristics of the wavelet domain to separate direct and interfering signals. The GCN's modeling of the sonar array's geometry effectively suppresses false targets caused by multipath reflections, maintaining stable identification of the direct beam even at pool corners. During off-peak hours, the PC-GAN generates the process through physical prior constraints, avoiding the dependence of traditional denoising methods on noise distribution. It accurately filters out random noise (such as water flow disturbances) in low-density environments while preserving detailed features of the sonar image (such as pool wall texture). The synergistic mechanism of both enables the system to maintain high-fidelity sonar imaging even under dynamic changes in pedestrian flow, providing a reliable signal source for underwater robot obstacle avoidance and target detection, significantly reducing false positive and false negative rates.

[0052] For example, during peak hours, the above steps employ a graph convolutional network (GCN) based on wavelet transform to separate the direct signal and interference signal from the sonar echo signal, and use the direct signal as the second data to be processed, including:

[0053] The sonar echo signal is decomposed into multi-frequency sub-bands using wavelet transform; the sonar matrix corresponding to the low-frequency sub-band is modeled using GCN, and the geometric topological relationship in the sonar matrix is ​​extracted to obtain low-frequency spatial features, which are used to capture the spatial consistency of the direct signal; transient features in the high-frequency sub-band are extracted using 1D-CNN to obtain high-frequency time-frequency features, which are used to capture pulse fluctuations caused by multipath random interference; the low-frequency spatial features and high-frequency time-frequency features are fused and spliced, and a classifier is used to perform binary classification on the spliced ​​features to separate the direct signal and the interference signal, and the direct signal is used as the second data to be processed.

[0054] In the above scheme, efficient separation of direct signals and interference signals in sonar echo signals is achieved through a collaborative mechanism of wavelet transform and graph convolution. The core lies in utilizing the multi-scale time-frequency decomposition capability of wavelet transform to decompose the sonar signal into low-frequency (LL), mid-frequency (LH), and high-frequency (HH) sub-bands, corresponding to the spatial structure characteristics and transient interference characteristics of the signal, respectively. The low-frequency sub-band (LL) models the geometric topology of the sonar array using a graph convolutional network (GCN). For example, sensor nodes are constructed as a graph structure, with edge weights between nodes determined by the geometric characteristics of the sound wave propagation path (such as propagation time and angle). The GCN aggregates neighborhood node information through a message passing mechanism, capturing the spatial consistency characteristics of the direct signal in complex underwater acoustic environments. The high-frequency sub-band (HH) uses a one-dimensional convolutional neural network (1D-CNN) to extract transient pulse features, such as random spike signals generated by multipath interference, capturing high-frequency time-frequency abrupt change patterns through local receptive fields. Ultimately, low-frequency spatial features and high-frequency time-frequency features are combined through feature stitching and fully connected layer classification to achieve accurate binary classification of direct signals and interference signals, providing a clean signal source for subsequent sonar imaging.

[0055] Thus, the aforementioned hybrid network effectively separates the physical features of different frequency components in sonar signals through the multi-scale decomposition characteristics of wavelet transform, avoiding the problem of low-frequency interference and high-frequency noise confusion caused by the fixed receptive field in traditional convolutional networks. GCN's ability to model the geometric topology of the sonar array significantly enhances the spatial consistency recognition of direct signals under complex underwater acoustic propagation paths, maintaining stable tracking of the direct beam even in multipath reflection scenarios. 1D-CNN's extraction of high-frequency transient features dynamically suppresses the random pulse characteristics of multipath interference, reducing the false detection rate through local feature enhancement and noise filtering mechanisms. The feature fusion strategy combines low-frequency spatial global constraints with high-frequency time-frequency local details to form complementary feature representations, improving the classifier's robustness to complex interference scenarios. In the dynamic environment of shallow water, this scheme significantly reduces the generation of pseudo-point clouds through multi-scale feature decoupling in the wavelet domain and spatial modeling using graph convolution. Simultaneously, it achieves real-time processing through lightweight network design, providing an efficient and reliable signal processing solution for underwater robot sonar obstacle avoidance and target recognition.

[0056] For example, in the above steps, during off-peak hours (such as in a swimming pool with low foot traffic), sonar echo signals are dominated by noise due to reduced environmental reflection paths. At this time, a physically constrained generative adversarial network (PC-GAN) achieves noise suppression and signal restoration through adversarial training between the generator and discriminator, combined with the physical laws of sonar propagation. Furthermore, the generator adopts a U-Net architecture, where the encoder extracts the time-frequency features of the sonar signal through multi-layer convolution, and the decoder gradually reconstructs a clean signal through transposed convolution. Skip connections fuse low-level details and high-level semantic information from the encoder, preserving key temporal features of the sonar waveform (such as the peak value of the direct wave). The discriminator, based on a PatchGAN structure, judges the authenticity of the signal through local receptive fields. For example, it identifies abnormal fluctuations in the generated signal that do not conform to the laws of sound wave propagation. Physical constraints dynamically verify the generated signal by introducing sonar propagation models (such as multipath delay and attenuation characteristics). For example, it forces the proportion of direct wave energy to conform to the geometric path calculation results, suppressing random noise from non-physical paths. Taking a static pool scene as an example, the original sonar signal may contain clutter reflected from the water surface and multipath interference from the pool bottom. The generator gradually removes noise through adversarial training, and the discriminator verifies the signal coherence through a local discrimination module. Finally, a high-fidelity sonar image is output for localization.

[0057] In the examples above, PC-GAN significantly improves the signal-to-noise ratio and positioning accuracy of sonar signals during off-peak hours through a synergistic mechanism of physical laws and data-driven approaches. The generator's U-Net architecture accurately separates noise from valid signals through residual learning, preserving weak target reflection features (such as pool edge structures) in low-noise environments, while the discriminator's patch-level discrimination capability effectively suppresses global noise interference. The introduction of physical constraints enhances the model's adaptability to sonar propagation characteristics; for example, by constraining the direct wave energy distribution through an attenuation model, it avoids edge artifacts caused by neglecting physical laws in traditional filtering methods. For instance, in a static pool scene, the positioning error can be reduced to within 3 cm, while adversarial training optimizes the generation quality, improving the signal-to-noise ratio by more than 20 dB. Compared to traditional Kalman filtering or U-Net denoising, PC-GAN significantly reduces the false detection rate while preserving sonar image details (such as pool wall texture), providing underwater robots with highly reliable environmental perception capabilities.

[0058] Understandably, PCGAN (Physically Constrained Generative Adversarial Network) constructs a physically interpretable sonar signal processing framework by fusing the adversarial training mechanism of Generative Adversarial Networks (GANs) with the physical laws of sonar propagation modeling. Its core structure consists of three collaborative parts: a generator, a discriminator, and a physical constraint module. The generator adopts the U-Net encoder-decoder architecture, responsible for reconstructing clean signals from noisy sonar signals. The encoder progressively downsamples the input signal through multiple convolutions (combining 3×3 convolutional kernels with the LeakyReLU activation function), extracting multi-scale time-frequency features. For example, the input sonar time-domain waveform is downsampled four times to generate a low-dimensional feature vector, capturing global propagation characteristics. The decoder progressively upsamples the feature map through transposed convolutions to restore signal resolution. Simultaneously, it uses skip connections to fuse features from each layer of the encoder with corresponding layers of the decoder, preserving local details (such as direct peak values) and global context (such as multipath reflection patterns). The decoder introduces residual blocks, which directly transmit low-frequency noise information through skip connections, allowing the network to focus on learning the difference between noise distribution and clean signals. The discriminator, based on the PatchGAN architecture, employs a multi-scale local discrimination mechanism. It divides the input signal into multiple local regions (e.g., 70×70 pixel blocks), independently calculating the authenticity probability of each region. Local texture and global structural features are extracted through spectral normalization and multi-layer convolution (4×4 convolution kernels), suppressing high-frequency noise interference. The physical constraint module imposes physical constraints on the generated signal by introducing mathematical models of sonar propagation (e.g., ray acoustic equations and attenuation laws). For example, it calculates the theoretical time delay of the direct wave based on the geometric path, forcing the multipath reflection time delay distribution of the generated signal to conform to the actual propagation path. Simultaneously, it constrains the proportion of direct wave energy based on the attenuation characteristics of sound waves, reducing abnormal energy fluctuations caused by multipath interference. The generator and discriminator are dynamically optimized through adversarial training. The generator attempts to generate clean signals that conform to physical laws to deceive the discriminator, while the discriminator identifies noise and abnormal patterns through the local discrimination module. The two alternately optimize until convergence. A weighted fusion of physical constraint losses (e.g., time delay loss and attenuation loss) and adversarial losses guides the generator to generate signals that are both physically consistent and highly realistic during adversarial training. In the collaborative working mechanism, the generator receives noisy sonar signals and reconstructs denoised signals using U-Net. The physical constraint module dynamically adjusts the generation direction to ensure the signal conforms to the propagation model. The discriminator performs local discrimination between the generated and real signals, optimizing the generation quality through patch-level feedback, such as suppressing anomalous waveforms that do not conform to the sound velocity profile. Finally, the generator and discriminator reach Nash equilibrium in an adversarial balance, outputting high-fidelity sonar signals for localization and obstacle avoidance. This design, through the combination of physical law embedding and adversarial training, overcomes the limitations of traditional methods that rely on statistical assumptions, enabling the model to maintain high-precision signal reconstruction in complex underwater acoustic environments and providing robust environmental perception capabilities for underwater robots.

[0059] In step S103 above, in the third branch, quaternion integral compensation is used to process the motion distortion in the high-frequency motion data, and a sensor noise reduction model based on Kalman filtering is used to remove the pose offset in the high-frequency motion data to obtain the third data to be processed.

[0060] For example, in high-frequency motion scenarios (such as rapid turns of underwater robots or sudden stops of drones), raw sensor data is prone to motion distortion and pose shift due to mechanical vibration or high-frequency noise. To address this issue, the third branch employs a collaborative strategy of quaternion integral compensation and Kalman filtering for noise reduction. Quaternion integral compensation suppresses motion distortion by correcting the accumulated error in rotational integration. For instance, when a gyroscope measures angular velocity, traditional integration methods can lead to attitude estimation errors due to sensor zero-bias drift. Quaternion integral compensation utilizes the group theory properties of quaternions (such as closure and singularity) and combines accelerometer or visual aid data to dynamically adjust the integration step size to correct the error. Kalman filtering filters sensor data through a state estimation model (such as linearized equations for position, velocity, and attitude), suppressing high-frequency noise and pose shift. For example, in robot motion, Kalman filtering fuses IMU angular velocity and accelerometer data to predict the pose at the next moment and corrects the predicted value through measurement updates (such as visual odometry or GPS), thereby eliminating pose jitter caused by sensor noise. Alternatively, the original high-frequency motion data is first corrected for rotational error by a quaternion integral compensation module, then input into a Kalman filter model for noise suppression and state optimization, and finally outputs smooth and stable pose information as the third data to be processed.

[0061] Thus, the accuracy and robustness of high-frequency motion data are significantly improved through the synergistic optimization of quaternion integral compensation and Kalman filtering. Quaternion integral compensation effectively suppresses long-term drift in rotational integrals, maintaining the continuity of attitude estimation even when the UAV is rotating at high speed, and avoiding attitude abrupt changes caused by zero bias accumulation in traditional methods. Kalman filtering, through dynamic noise estimation and state prediction, suppresses the influence of high-frequency sensor noise on pose, ensuring stable output of the robot's trajectory even in turbulent water flow. The combination of these two methods enables the system to maintain high-precision motion control in complex dynamic environments (such as multi-obstacle avoidance), reducing misjudgments caused by data distortion or noise.

[0062] Step S104: Use ORB-SLAM3 to generate a sparse feature map from the first data to be processed, and use the point cloud registration ICP algorithm to generate a locally dense map from the second data to be processed.

[0063] In this embodiment of the application, the sparse feature map is used to indicate the global pose reference and position information reference of the underwater robot, and the local dense map is used to supplement the local position details of the area where the underwater robot is located.

[0064] In step S104, the sparse feature map and the locally dense map achieve a balance between accuracy and efficiency in underwater robot navigation through a complementary mechanism.

[0065] The sparse feature map, based on the ORB-SLAM3 visual SLAM framework, extracts ORB feature points (such as FAST corner points and BRIEF descriptors) from images, performs feature matching and triangulation between keyframes, and generates a sparse 3D point cloud map. This sparse feature map retains only keypoints with significant geometric or texture features (such as wall edges and obstacle corner points), and achieves global consistency through Local Bundling Optimization (LocalBA) and Loop Closing. ORB-SLAM3's threaded architecture (tracking, mapping, and loop closure) concentrates computational resources on feature point matching and pose optimization, avoiding processing all image pixels, thus maintaining real-time performance.

[0066] Sparse feature maps provide global localization references for robots through the 3D coordinates of feature points. For example, in complex pipeline networks, feature point matching enables pose continuity across regions. The rotation and scale invariance of ORB features allows for stable matching even in low-light or water flow disturbance environments, avoiding localization drift caused by visual degradation. The data volume of sparse point cloud is only 1% to 5% of that of dense maps, significantly reducing storage and computational load, making it suitable for real-time processing on edge devices (such as underwater robot main control chips).

[0067] Locally dense maps, based on point cloud registration algorithms (such as improved ICP), align high-frequency motion data (such as sonar or lidar point clouds) with the local environment through iterative nearest neighbor (ICP) optimization. Traditional ICP matches point pairs using Euclidean distance, while improved algorithms (such as NDT-ICP and MAD-ICP) introduce KD-trees to accelerate nearest neighbor search and combine plane segmentation (such as PCA analysis) to improve registration efficiency. For example, MAD-ICP downsamples point clouds through voxel grid filtering and reduces false matches using plane segmentation, ultimately generating a high-resolution locally dense surface model.

[0068] Thus, dense maps capture subtle geometric features of the underwater environment (such as pool textures and shipwreck debris), compensating for the lack of information in complex terrain by sparse maps and improving obstacle avoidance accuracy. When the robot makes rapid turns or encounters dynamic obstacles, dense maps quickly update the environmental model through local optimization, such as identifying temporary interference from fishing nets or bubbles.

[0069] By combining IMU data with sonar point clouds, dense maps can correct pose shifts caused by water flow disturbances, such as maintaining stable output of pipe detection paths even in strong ocean currents. ORB-SLAM3's sparse map provides a global pose reference, constraining the initial transformation estimation of ICP and preventing local registration from getting trapped in local optima; while the dense map optimizes local regions of the global map through detail feedback (e.g., updating the 3D shape of the pipe interface through MAD-ICP). The lightweight nature of sparse maps supports processing speeds of tens of frames per second, while the improved ICP algorithms of dense maps (such as NDT preprocessing) reduce registration time from seconds to sub-seconds, meeting the dynamic navigation requirements of underwater robots. In low-visibility waters, sparse maps rely on the robustness of feature points to maintain localization; in structured environments (such as artificial reefs), dense maps enhance obstacle avoidance reliability through high-precision surface models.

[0070] As an optional embodiment, in step S104, the first data to be processed is used to generate a sparse feature map using ORB-SLAM3, including:

[0071] Each frame of the original image in the first data to be processed is converted to the ORB-SLAM3 processing format, and the camera parameters of the vision sensor are calibrated. The undistort function is used to eliminate fisheye or pinhole distortion, resulting in intermediate images. The FAST-9 algorithm is used to detect the corner points of each intermediate image frame. The gradient direction of each corner point is calculated using the gray-scale centroid method to enhance rotation invariance. Based on the calculation results, a 256-bit binary BRIEF descriptor is generated as the robust feature point of each intermediate image frame. When the ghosting error between consecutive intermediate images exceeds a threshold, or the motion amplitude of the underwater robot exceeds a threshold, keyframes are selected from consecutive intermediate images at preset intervals, and the correlation between multiple keyframes is established. The robust feature points of the current frame are clustered into visual words using a bag-of-words model. Feature matching is performed with historical visual words in historical keyframes with correlation stored in a hash table. A preset number of candidate map feature points are selected in descending order of matching degree. The bag-of-words model is pre-trained based on the ORB dictionary. The relative pose between the current frame and historical keyframes is estimated using the EPnP algorithm. Based on epipolar geometric constraints, the three-dimensional coordinates of the selected candidate map feature points are calculated and projected into the global coordinate system to obtain the three-dimensional coordinates of the target map feature points. The sparse feature map is then constructed according to a preset data structure.

[0072] Specifically, in step S104, the core process of ORB-SLAM3 generating sparse feature maps revolves around the typical architecture of visual SLAM, achieving environmental modeling through feature extraction, keyframe management, pose optimization, and 3D reconstruction.

[0073] Specifically, the first input data to be processed (the original image sequence) needs to undergo camera parameter calibration and distortion correction. For example, the fisheye lens on an underwater robot introduces radial distortion. The image is geometrically corrected using the undistort function to eliminate the distortion and obtain an intermediate image. Subsequently, the FAST-9 algorithm is used to detect corner points (such as edges and areas of abrupt texture changes) in the image, and the gradient direction of each corner point is calculated using the gray-scale centroid method to enhance the rotation invariance of feature points. Based on the gradient direction and position information, a 256-bit BRIEF descriptor is generated as a unique identifier for the feature point. For example, in a swimming pool scene, FAST-9 can quickly locate the strongly textured areas of coral edges, and the BRIEF descriptor encodes its local gradient pattern, ensuring consistent feature matching across different viewpoints.

[0074] To balance computational efficiency and map accuracy, the system dynamically selects keyframes: when the ghosting error between consecutive frames (such as inconsistency in feature point matching) exceeds a threshold, or when the robot's movement is large (such as sharp turns or rapid translations), the current frame is selected as the keyframe. Keyframes are associated with historical keyframes using a bag-of-words model. Based on a pre-trained ORB dictionary, the bag-of-words model clusters the feature points of the current frame into visual terms (such as "wall texture," "pool buoy," "circular cross-section of a pipe," etc.) and matches them with the vocabulary of historical keyframes, selecting candidate map feature points with high similarity. For example, when the robot passes through the same pipe twice, the bag-of-words model quickly locates historical keyframes using visual terms such as "circular cross-section of a pipe," accelerating feature matching.

[0075] The EPnP algorithm is used to estimate the relative pose between the current keyframe and historical keyframes. EPnP utilizes the 3D coordinates and projection relationship of feature points to construct a nonlinear optimization problem, minimizing the reprojection error to obtain accurate rotation and translation vectors. Subsequently, candidate points are screened based on epipolar geometric constraints (such as the fundamental or essential matrix) to eliminate mismatched points. For example, under dynamic water flow interference, some feature points may produce incorrect matches due to motion blur; EPnP uses RANSAC iteration to eliminate outliers while retaining robust geometric constraints. Finally, the candidate points are projected onto the global coordinate system to generate a sparse feature map.

[0076] The ORB-SLAM3 sparse mapping mechanism enables efficient and robust environmental perception in underwater environments. ORB feature extraction and bag-of-words model matching significantly reduce computational load, with single-frame processing time controlled within 20ms, meeting real-time requirements. For example, at 1080P resolution, the system can process 30 frames per second, generating a sparse map with a point density only 5% of that of a dense map, while maintaining high key feature retention. The combination of FAST-9 and BRIEF demonstrates strong adaptability to changes in lighting and water turbidity. Through bag-of-words modeling and loop closure detection, the system can correct accumulated errors. For example, when the robot returns to its starting point in a loop, the pose deviation of the sparse map can be controlled within 3cm, superior to the 10cm error of the traditional ICP algorithm. Sparse maps only need to store keyframes and feature point coordinates, reducing memory usage compared to dense maps, making them suitable for long-term deployment on edge devices.

[0077] In pool cleaning robot tasks, sparse feature maps can quickly construct topological information of key structures such as pool walls and steps. When the robot detects water surface reflection interference, it uses a bag-of-words model to match historical pool edge features, suppressing the influence of water surface clutter and ensuring positioning accuracy. Experiments show that this solution maintains a positioning error within 5 cm in dynamic pool scenarios, while consuming only 1 / 3 of the computational resources of dense SLAM.

[0078] As an optional embodiment, in step S104, the second data to be processed is used to generate a locally dense map using the point cloud registration ICP algorithm, including: for each frame of sonar point cloud in the second data to be processed, removing point clouds in the current frame point cloud where the sonar echo intensity is lower than the dynamic noise threshold; calculating the normal direction of each point in the current frame point cloud, used to match similar regions between the current frame point cloud and the historical frame point cloud by the angle between the normals; extracting the local curvature of the current frame point cloud as a texture feature, used to distinguish smooth regions and edge regions in the target space; using KD-Tree to accelerate nearest neighbor search, obtaining point cloud pairs with similar matching features from the current frame point cloud and the historical frame point cloud according to the normal direction and texture features; using the selected point cloud pairs, aligning the current frame point cloud and the historical frame point cloud, and constructing the locally dense map with the aligned point cloud pairs.

[0079] In step S104, the local dense map generation based on the ICP algorithm achieves high-precision environmental modeling through multi-dimensional feature matching and efficient search strategies. Its core process revolves around the preprocessing, feature extraction, and dynamic optimization of sonar point clouds. Specifically, the input second data to be processed (the sonar point cloud sequence) is first subjected to noise filtering. Low-intensity echo points are removed using a dynamic noise threshold, retaining valid surface data. For example, in turbid water, sonar echoes may generate noise due to scattering by suspended particles; the dynamic threshold can be adaptively adjusted according to environmental noise statistics to avoid mistakenly deleting valid points. Subsequently, the normal direction of each point is calculated, and similar regions are identified by analyzing the geometric directionality of the local surface of the point cloud. For example, in a swimming pool scene, the normal direction of the coral surface exhibits regular changes; by calculating the angle between the normals of adjacent points, similar structures in the current frame and historical frames can be quickly matched.

[0080] Optionally, local curvature can be extracted as texture features to distinguish smooth regions (such as the bottom and walls of a pool) from edge regions (such as lane boundaries). Curvature calculation is achieved by analyzing the distribution density and orientation changes of points within the neighborhood of the point cloud. For example, at the edge of a pool, the curvature value will be significantly higher than in flat areas. Based on the normal direction and curvature features, a KD-Tree is used to accelerate nearest neighbor search. Specifically, the historical frame point cloud is constructed as a KD-Tree structure, and candidate matching points for the current frame point are quickly located by recursively partitioning the space, reducing computational complexity. For example, in dynamic water flow, the rapid movement of the robot causes point cloud distortion. The KD-Tree can efficiently filter out historical points that match the geometric features of the current frame point, suppressing erroneous associations caused by motion blur. Finally, by iteratively optimizing the selected matching point pairs, the optimal transformation matrix (such as rotation matrix and translation vector) is calculated to align the current frame and historical frame point clouds, and the matching points are fused into a locally dense map.

[0081] Step S105: The sparse feature map, the empty local dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot.

[0082] For example, when the underwater robot is in motion, the dynamic weight of the locally dense map during EKF fusion is higher than the dynamic weight of the sparse feature map. When the underwater robot is stationary, the dynamic weight of the locally dense map during EKF fusion is lower than the dynamic weight of the sparse feature map.

[0083] In this embodiment, the local scene map containing the underwater robot's pose state is constructed by fusing multi-source sensor data and dynamic environmental features to create a composite map that simultaneously reflects the robot's own motion state and the spatial relationship with its surrounding environment. Its core information includes the robot's real-time pose (position, attitude), the three-dimensional geometric structure of the local environment (such as obstacle distribution and terrain undulations), and dynamic semantic features (such as the outline of a shipwreck and the boundary of a swimming pool). The purpose of this type of map is to provide the robot with a high-precision and timely environmental cognition foundation, enabling it to achieve autonomous navigation, obstacle avoidance decisions, and task planning in complex underwater scenarios.

[0084] Map construction is based on multi-sensor data fusion technology. Extended Kalman Filter (EKF) dynamically weights and fuses the sparse feature map (global pose reference) generated by visual SLAM with the locally dense map (detail supplementation) generated by sonar point cloud registration. For example, in a swimming pool detection task, visual SLAM identifies the pool's geometric outline as static features, while the sonar point cloud captures subtle texture changes on the reef surface. EKF suppresses noise interference through dynamic weight adjustment (e.g., increasing sonar weights when water flow disturbs the surface) and uses the robot's motion model (IMU data) to constrain pose changes, ensuring the spatiotemporal consistency of the map. Furthermore, the map employs a hierarchical storage structure: the upper layer contains sparse feature points of the global pose reference, and the lower layer contains locally dense point clouds and semantic labels (e.g., "pipeline interface," "fishing net area"), enabling efficient information organization and rapid retrieval.

[0085] For example, in a subsea pipeline inspection task, a local scene map is fused in real time with the pipeline orientation (sparse features) identified visually and the corrosion areas of the pipe wall (dense point cloud) detected by sonar scanning. When the robot approaches a pipe bend, visual features are lost due to occlusion. At this time, the high-frequency data from the sonar is dynamically weighted using EKF to supplement the bend curvature information, ensuring continuous pose estimation. Simultaneously, the semantic label "corrosion area" in the map triggers an obstacle avoidance strategy, guiding the robot to detour along a safe path.

[0086] Local scene maps significantly enhance the adaptability of underwater robots to dynamic environments. In static scenarios (such as submarine cable inspection), vision-driven sparse maps provide centimeter-level positioning accuracy, supporting high-precision path tracking. In dynamic scenarios (such as risk areas entangled in fishing nets), dense sonar maps supplement local details, enabling underwater robots to detect obstacles and adjust their course seconds in advance. Furthermore, the map's semantic hierarchical structure supports task-driven decision optimization; for example, in pool salvage missions, areas with metal structures are automatically marked as high-risk zones, and a safe approach path is planned based on historical pose data.

[0087] It is worth noting that the core of step S105, which achieves multi-sensor pose fusion through extended Kalman filtering (EKF), lies in the collaborative optimization of dynamic weight allocation and heterogeneous observation models. Its state vector includes IMU attitude quaternions, position, velocity, and sensor bias, while the observation model fuses pose estimation results from sonar and vision. The sonar constraint (Tsonar) is generated through point cloud registration, reflecting the robot's local motion trajectory; its high translational noise (0.1m) corresponds to sonar ranging error. The visual constraint (Tvis) is extracted by ORB-SLAM3, capturing global pose features, but is limited by feature matching uncertainty (high rotational noise 0.2rad). The dynamic weight switching logic adjusts the fusion ratio according to the environmental state. In dynamic scenes (such as when swimmer occlusion leads to sparse visual features), the sonar weight α is increased to 0.7, utilizing its high-frequency, anti-occlusion characteristics to compensate for visual degradation; in static scenes, the visual weight β is increased to 0.7, relying on its high-precision feature matching to stabilize pose estimation. For example, when a robot traverses a pool area, vision may cause pose drift due to complex texture matching errors. In this case, EKF suppresses visual noise interference and ensures pose continuity by increasing the sonar weight (weight is 0.7). When the robot is stationary in open water, vision can accurately identify the features of the pool bottom, and the weight is reduced to 0.3 to avoid the accumulation of sonar ranging errors.

[0088] Thus, the dynamic weighting mechanism significantly improves the robustness of localization in complex underwater environments. In dynamic scenarios, high-frequency sonar data compensates for the time delay of visual inter-frame intervals. For example, when a robot turns rapidly, sonar can capture instantaneous pose changes, while vision may fail due to motion blur. In this case, weight switching can maintain the continuity of localization. In static scenarios, the geometric constraints of vision (such as plane matching and feature point cloud registration) provide high-precision pose estimation and suppress IMU cumulative drift. For example, in pipe inspection tasks, vision assistance controls the pose deviation to within 2cm, while the pure sonar solution can reach an error of 15cm. In addition, EKF's nonlinear optimization capability integrates complementary information from multiple sensors. Sonar depth information corrects visual scale ambiguity (such as scale drift in monocular SLAM), and visual semantic features (such as the outlines of objects scattered in a pool) constrain the local distortion of sonar point clouds.

[0089] Step S106: The segmented bundle adjustment algorithm (Segmented BA) is used to segment and optimize the local scene map according to the motion mode, and the positioning information of the underwater robot is output to the user in real time based on the optimized local scene map.

[0090] As an optional embodiment, in step S106, a piecewise bundle adjustment algorithm is used to segment and optimize the local scene map according to the motion mode. This includes: based on the underwater robot's motion mode parameters, identifying whether the underwater robot is currently in a straight-line or turning segment. If it is in a straight-line segment, a sparse BA objective function is used to optimize the relative poses of keyframes and preset map feature points in the local scene map. If it is in a turning segment, a robust kernel function of dense BA is used to optimize the relative poses of all frames and all map feature points to complete the segmented optimization of the local scene map.

[0091] Specifically, the core of the segmented bundle adjustment algorithm in step S106 lies in dynamically adjusting the optimization strategy according to the underwater robot's motion mode to balance computational efficiency and pose accuracy. First, by analyzing the rate of change of rotation angle between adjacent frames, the trajectory is divided into straight segments (rotation angle below a threshold) and turning segments (rotation angle above a threshold). For example, when the robot is traveling in a straight line at a constant speed of 0.1 m / s, if the rotation angle between adjacent frames is less than 5°, it is determined to be in a straight segment. When the robot makes a sharp turn (such as when avoiding obstacles, the rotation angle exceeds 15°), it enters a turning segment. Firstly, for the straight segment, only the relative pose of keyframes and preset sparse feature points (such as corner points extracted by ORB-SLAM3) are optimized. For example, when traveling straight on a flat seabed, sparse bundle adjustment (BA) corrects global pose drift by minimizing the reprojection error between keyframes, avoiding redundant computation. Secondly, for the turning segment, a robust kernel function of dense bundle adjustment (such as the Huber kernel) is introduced to optimize the relative pose of all frames and dense feature points (such as locally dense maps generated from sonar point clouds). For example, when a robot turns, visual features may become invalid due to sudden changes in perspective. Dense BA (Based Optimization) captures local geometric details (such as the texture of a swimming pool surface) by fusing high-frequency sonar data, suppressing errors caused by motion blur. Specifically, the weights of the optimization targets are adjusted according to the motion pattern. For straight sections, the weight of dense BA is reduced to 0.3 to decrease computation. For turning sections, the weight of dense BA is increased to 0.7 to enhance the intensity of local optimization.

[0092] In a swimming pool exploration mission, when the underwater robot enters a U-shaped bend from a straight section, visual SLAM causes abrupt pose changes due to mismatches in dense coral textures. After segmented BA detects the bend, it triggers dense BA optimization. Sparse BA uses sparse feature points (such as coral cross-sections at the bend entrance) to provide initial pose constraints. Dense BA fuses 3D points (1000 points per square meter) generated from sonar point clouds on the inner wall of the bend, uses a robust kernel function to remove outliers, and optimizes the pose of all keyframes. This ultimately generates a smooth bend trajectory, avoiding the cumulative errors caused by global optimization in traditional BA.

[0093] Thus, dynamic segmentation optimization significantly improves the robustness of localization in complex underwater scenarios. The computation time of sparse BA in straight sections is only 1 / 5 that of dense BA, reducing the global BA time. Under dense optimization in turning sections, the absolute pose error (APE) decreases, showing improvement compared to the traditional ORB-SLAM3. Robust kernel functions suppress anomalous observations (such as sonar noise caused by water flow disturbances), maintaining pose continuity even in low-visibility waters. The fusion of sparse feature points (such as pipe interfaces) and dense point clouds (such as pipe wall corrosion areas) supports task-driven semantic map construction, assisting obstacle avoidance and path planning.

[0094] For example, in the straight segment, the optimization objective focuses on the consistency between the keyframe pose and sparse feature points, and the optimization objective function is: Where K is the set of keyframes, ξ i Let F be the pose algebra of the i-th frame, and F be the sparse feature point set. ij Indicates the optimization of feature point P in the i-th frame. j The projection error. ρ Huber REG(ξ) is a robust kernel function. i ) represents the pose regularization term, λ pose REG(ξ) is the regularization weight. i It is used to constrain the pose smoothness of adjacent keyframes.

[0095] In the turning segment, the optimization objective is extended to all frames and dense feature points. A robust kernel function is introduced to suppress abnormal observations, and the optimization objective function is: Where D is the set of dense feature points in a locally dense map, e ij Indicates the optimization of feature point P in the i-th frame. j The projection error, ρ Cauchy Used to enhance the optimized feature point P in the i-th frame j The robustness function, Δξ IMU For the IMU pre-integration constraint term, ξ i Let ξ be the pose algebra of the i-th frame. i+1 Let be the pose algebra of the (i+1)th frame.

[0096] Then, in step S106, the underwater robot's positioning information is output to the user in real time based on the optimized local scene map, including: determining whether the currently requested positioning data is short-term or long-term data. If short-term data is requested, a particle filter resampling strategy is used to extract the underwater robot's real-time position coordinates and real-time pose from the optimized local scene map. If long-term data is requested, a factor graph optimization strategy is used to extract the underwater robot's globally corrected position coordinates and weighted average pose from the optimized local scene map.

[0097] In this embodiment, a multi-dimensional sensor network for the underwater robot is constructed to dynamically acquire multi-dimensional monitoring data of the target space. Combined with data preprocessing via a multi-branch strategy network, various algorithms are used to generate and fuse a map. Finally, a segmented bundle adjustment algorithm is used to optimize the map and output positioning information. This implementation effectively overcomes the interference of complex underwater environments on positioning, organically integrating multi-source information such as sonar, vision, and inertial sensors. It avoids the limitations of single sensors, significantly improving the real-time performance and accuracy of positioning information. Simultaneously, the optimized algorithm reduces the computational load of data processing, enabling the underwater robot to quickly and accurately determine its pose in complex and ever-changing underwater environments. This provides reliable positioning assurance for underwater exploration, operations, and other applications, improving the positioning accuracy and efficiency of the underwater robot.

[0098] After introducing the methods of exemplary embodiments of this application, the following references are made. Figure 2 An exemplary embodiment of this application describes an underwater robot positioning system based on human-computer interaction, the device comprising:

[0099] A construction module is used to construct a multi-dimensional sensor network for an underwater robot in response to a user's positioning request; the multi-dimensional sensor network is used to dynamically collect multi-dimensional monitoring data of the target space along the movement trajectory of the underwater robot; wherein, the multi-dimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor.

[0100] The preprocessing module is used to preprocess the multidimensional monitoring data using a multi-branch strategy network to obtain multidimensional data to be processed; the multidimensional data to be processed includes at least: first data to be processed based on the spatial visual information, second data to be processed based on the sonar echo signal, and third data to be processed based on the high-frequency motion data.

[0101] The prediction module is used to generate a sparse feature map from the first data to be processed using ORB-SLAM3, and to generate a locally dense map from the second data to be processed using the point cloud registration ICP algorithm; the sparse feature map, the locally dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot.

[0102] The output module is used to perform segmented optimization of the local scene map according to the motion mode using the segmented bundle adjustment algorithm (Segmented BA), and output the underwater robot's positioning information to the user in real time based on the optimized local scene map.

[0103] The above system can implement the steps described in the above method implementation, and the specific implementation of each step will not be repeated here.

[0104] After introducing the methods and systems of exemplary embodiments of this application, the following describes a terminal device according to an exemplary embodiment of this application. This terminal device is used to construct a multi-dimensional sensor network for an underwater robot in response to a user's positioning request. The multi-dimensional sensor network dynamically collects multi-dimensional monitoring data of the target space along the underwater robot's movement trajectory. The multi-dimensional monitoring data includes at least: sonar echo signals collected by a high-frequency sonar module targeting dynamic objects, pool walls, and / or pool bottoms; spatial visual information collected by a visual sensor; and high-frequency motion data collected by an inertial sensor. A multi-branch strategy network is used to preprocess the multi-dimensional monitoring data to obtain multi-dimensional data to be processed. The multi-dimensional data to be processed includes at least: first data to be processed based on the spatial visual information; second data to be processed based on the sonar echo signals; and third data to be processed based on the high-frequency motion data. The first data to be processed is used to generate a sparse feature map using ORB-SLAM3, and the second data to be processed is used to generate a locally dense map using the point cloud registration ICP algorithm. The sparse feature map, the locally dense map, and the pose features in the third data to be processed are fused using an extended Kalman filter (EKF) to obtain a local scene map containing the underwater robot's pose state. A segmented bundle adjustment (Segmented BA) algorithm is used to optimize the local scene map according to the motion mode, and the underwater robot's positioning information is output to the user in real time based on the optimized local scene map. The terminal device described above can implement the steps described in the above method implementation, and the specific implementation methods of each step will not be repeated here.

[0105] After introducing the methods, systems, and terminal devices of exemplary embodiments of this application, the following references will be made. Figure 3 The computer-readable storage medium of exemplary embodiments of this application will be described, please refer to... Figure 3 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it implements the steps described in the above method embodiments. The specific implementation methods of each step will not be repeated here.

[0106] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical and magnetic storage media, which will not be elaborated here. The above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and are not intended to limit it. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in this application, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An underwater robot positioning method based on human-computer interaction, characterized in that, The method includes: In response to the user's positioning request, a multi-dimensional sensor network for the underwater robot is constructed; the multi-dimensional sensor network is used to dynamically collect multi-dimensional monitoring data of the target space along the movement trajectory of the underwater robot; wherein, the multi-dimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor. A multi-branch strategy network is used to preprocess the multidimensional monitoring data to obtain multidimensional data to be processed. This includes: in the first branch, a Brown-Conrady model is used to map the distorted pixel coordinates in the spatial visual information back to ideal coordinates through polynomial fitting of the distortion coefficients, achieving distortion removal; in the first branch, the distorted spatial visual information is used to perform dynamic target detection through a lightweight YOLOv8 network to obtain first data to be processed; the first data to be processed includes: distorted spatial visual information and a dynamic mask used to calibrate the location of the dynamic target; in the second branch… In the first branch, a time-division strategy network is used to perform multipath suppression on the sonar echo signal to remove multipath artifacts and obtain the second data to be processed. In the third branch, quaternion integral compensation is used to process motion distortion in the high-frequency motion data, and a sensor noise reduction model based on Kalman filtering is used to remove pose offset in the high-frequency motion data to obtain the third data to be processed. The multidimensional data to be processed includes at least: the first data to be processed based on the spatial vision information, the second data to be processed based on the sonar echo signal, and the third data to be processed based on the high-frequency motion data. The first data to be processed is used to generate a sparse feature map using ORB-SLAM3, and the second data to be processed is used to generate a locally dense map using the point cloud registration ICP algorithm. The sparse feature map, the locally dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot. The segmented bundle adjustment algorithm (Segmented BA) is used to segment and optimize the local scene map according to the motion mode. Based on the optimized local scene map, the positioning information of the underwater robot is output to the user in real time.

2. The underwater robot positioning method based on human-computer interaction according to claim 1, characterized in that, In the second branch, a time-division strategy network is used to perform multipath suppression on the sonar echo signal to remove multipath artifacts in the sonar echo signal, thereby obtaining the second data to be processed, including: Based on the number of people entering the target area, predict the density of active objects in the target area during the current time period; if the density of active objects is greater than a set threshold, the current time period is a peak period; if the density of active objects is not greater than the set threshold, the current time period is an off-peak period. During peak hours, a graph convolutional network (GCN) based on wavelet transform is used to separate the direct signal and interference signal from the sonar echo signal, and the direct signal is used as the second data to be processed. During off-peak hours, a physical constraint-based generative adversarial network (PC-GAN) removes noise signals from the sonar echo signals using a generator based on a U-Net architecture. It then uses a physical prior model incorporating sonar propagation and a GAN discriminator to predict the authenticity of the generator's output signal in order to obtain the second set of data to be processed.

3. The underwater robot positioning method based on human-computer interaction according to claim 2, characterized in that, The method employs a graph convolutional network (GCN) based on wavelet transform to separate the direct signal and interference signal from the sonar echo signal, and uses the direct signal as the second data to be processed, including: The sonar echo signal is decomposed into multiple frequency subbands using wavelet transform; By modeling the sonar matrix corresponding to the low-frequency sub-band using GCN, the geometric topological relationships in the sonar matrix are extracted to obtain low-frequency spatial features, which are then used to capture the spatial consistency of the direct signal. Transient features in high-frequency subbands are extracted using 1D-CNN to obtain high-frequency time-frequency features, which are used to capture pulse fluctuations caused by multipath random interference. Low-frequency spatial features and high-frequency time-frequency features are fused and spliced ​​together, and a classifier is used to perform binary classification on the spliced ​​features to separate the direct signal and the interference signal, and the direct signal is used as the second data to be processed.

4. The underwater robot positioning method based on human-computer interaction according to claim 1, characterized in that, The step of generating a sparse feature map from the first data to be processed using ORB-SLAM3 includes: Each frame of the original image in the first data to be processed is converted into the format of ORB-SLAM3 processing, and the camera parameters of the vision sensor are calibrated. The undistort function is used to eliminate fisheye or pinhole distortion to obtain an intermediate image. The FAST-9 algorithm is used to detect the corner points of the intermediate images in each frame. The gray-scale centroid method is used to calculate the gradient direction of each corner point to enhance rotation invariance. Based on the calculation results, a 256-bit binary BRIEF descriptor is generated as a robust feature point of the intermediate images in each frame. When the ghosting error between multiple consecutive intermediate images exceeds the threshold, or the movement amplitude of the underwater robot is greater than the threshold, key frames are selected from multiple consecutive intermediate images at preset intervals, and the correlation between multiple key frames is established. The robust feature points of the current frame are clustered into visual words using a bag-of-words model. These visual words are then matched with historical visual words in related historical keyframes stored in a hash table. A preset number of candidate map feature points are selected in descending order of matching degree. The bag-of-words model is pre-trained based on the ORB dictionary. The relative pose between the current frame and historical keyframes is estimated using the EPnP algorithm. Based on epipolar geometric constraints, the three-dimensional coordinates of the selected candidate map feature points are calculated and projected into the global coordinate system to obtain the three-dimensional coordinates of the target map feature points. The sparse feature map is then constructed according to a preset data structure.

5. The underwater robot positioning method based on human-computer interaction according to claim 1, characterized in that, The step of generating a locally dense map from the second data to be processed using the point cloud registration ICP algorithm includes: For each frame of sonar point cloud in the second data to be processed, remove point clouds in the current frame whose sonar echo intensity is lower than the dynamic noise threshold. Calculate the normal direction of each point in the current frame point cloud, which is used to match similar regions between the current frame point cloud and the historical frame point cloud by the angle between the normals; Extract the local curvature of the point cloud in the current frame as texture features to distinguish smooth regions from edge regions in the target space; KD-Tree is used to accelerate nearest neighbor search, and point cloud pairs with similar matching features are obtained from the current frame point cloud and the historical frame point cloud based on normal direction and texture features; The selected point cloud pairs are used to align the point cloud of the current frame with the point cloud of the historical frame, and the aligned point cloud pairs are used to construct the local dense map.

6. The underwater robot positioning method based on human-computer interaction according to claim 4 or 5, characterized in that, The sparse feature map is used to indicate the global pose reference and position information reference of the underwater robot, and the local dense map is used to supplement the local position details of the area where the underwater robot is located. When the underwater robot is in motion, the dynamic weight of the local dense map is higher than the dynamic weight of the sparse feature map during the EKF fusion process; When the underwater robot is stationary, the dynamic weight of the locally dense map during the EKF fusion process is lower than the dynamic weight of the sparse feature map.

7. The underwater robot positioning method based on human-computer interaction according to claim 1, characterized in that, The segmented bundle adjustment algorithm (Segmented BA) is used to optimize the local scene map by segmenting it according to the motion pattern, including: Based on the motion mode parameters of the underwater robot, the current motion state of the underwater robot can be identified as either a straight section or a turning section; If the scene is in a straight section, a sparse BA objective function is used to optimize the relative poses of keyframes and preset map feature points in the local scene map; if the scene is in a turning section, a robust kernel function of dense BA is used to optimize the relative poses of all frames and all map feature points to complete the segmented optimization of the local scene map.

8. The underwater robot positioning method based on human-computer interaction according to claim 7, characterized in that, The method of outputting the underwater robot's positioning information to the user in real time based on the optimized local scene map includes: Determine whether the location data currently requested is short-term or long-term; If short-term data is requested, a particle filter resampling strategy is used to extract the real-time position coordinates and pose of the underwater robot from the optimized local scene map. If long-term data is requested, a factor graph optimization strategy is adopted to extract the global corrected position coordinates and weighted average pose of the underwater robot from the optimized local scene map.

9. An underwater robot positioning system based on human-computer interaction, characterized in that, The system includes: A construction module is used to construct a multi-dimensional sensor network for an underwater robot in response to a user's positioning request; the multi-dimensional sensor network is used to dynamically collect multi-dimensional monitoring data of the target space along the movement trajectory of the underwater robot; wherein, the multi-dimensional monitoring data includes at least: sonar echo signals collected by the high-frequency sonar module for dynamic objects, pool walls and / or pool bottom, spatial visual information collected by the visual sensor, and high-frequency motion data collected by the inertial sensor. The preprocessing module is used to preprocess the multidimensional monitoring data using a multi-branch strategy network to obtain multidimensional data to be processed; the multidimensional data to be processed includes at least: first data to be processed based on the spatial visual information, second data to be processed based on the sonar echo signal, and third data to be processed based on the high-frequency motion data. Specifically, the preprocessing module, when obtaining multidimensional data to be processed, is used as follows: In the first branch, the Brown-Conrady model is used to map the distorted pixel coordinates in the spatial visual information back to ideal coordinates through polynomial fitting of distortion coefficients, thereby achieving distortion removal processing; In the first branch, the distorted spatial visual information is used to perform dynamic target detection through a lightweight YOLOv8 network to obtain the first data to be processed; The first data to be processed includes: distorted spatial visual information and a dynamic mask used to calibrate the position of the dynamic target; In the second branch, a time-division strategy network is used to perform multipath suppression on the sonar echo signal to remove multipath artifacts in the sonar echo signal, thereby obtaining the second data to be processed; In the third branch, quaternion integral compensation is used to process the motion distortion in the high-frequency motion data, and a sensor noise reduction model based on Kalman filtering is used to remove the pose offset in the high-frequency motion data, thereby obtaining the third data to be processed; The prediction module is used to generate a sparse feature map from the first data to be processed using ORB-SLAM3, and to generate a locally dense map from the second data to be processed using the point cloud registration ICP algorithm; the sparse feature map, the locally dense map and the pose features in the third data to be processed are fused by extended Kalman filter (EKF) to obtain a local scene map containing the pose state of the underwater robot. The output module is used to perform segmented optimization of the local scene map according to the motion mode using the segmented bundle adjustment algorithm (Segmented BA), and output the underwater robot's positioning information to the user in real time based on the optimized local scene map.

Images