Swimming pool robot dynamic capture method and system based on water surface perception

Abstract

The embodiment of the invention provides a swimming pool robot dynamic capturing method and system based on water surface sensing. The method comprises the following steps: performing omnibearing image acquisition on a water body environment where the swimming pool robot is located to obtain visual image data; performing dynamic target detection on the visual image data to obtain initial positioning information of the swimming pool robot in the water body environment; performing dynamic filtering and three-dimensional space matching on the initial positioning information by adopting a dynamic background model and a dynamic interaction model to obtain real-time positioning information; predicting a predicted motion track of the swimming pool robot in the water body environment through a motion model based on the real-time task information; and through a particle filtering algorithm, the real-time positioning information and the predicted motion trail are fused, and dynamic pose information of the swimming pool robot in the water body environment is obtained. According to the method, the dynamic background model, the dynamic interaction model and the motion model are adopted to carry out multiple space positioning on the swimming pool robot, the pose prediction efficiency of the swimming pool robot is improved, and the model prediction accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of data prediction, and in particular to a method and system for dynamic capture of a swimming pool robot based on water surface perception. Background Technology

[0002] Currently, the swimming pool environment is complex. Changes in water quality, light refraction and reflection, and impurities in the water can all affect image quality. For example, turbid water can blur images, reducing clarity and contrast, and hindering the identification of the pool environment. Light reflection on the water surface can cause images to be too bright or have reflective areas, leading to the loss of some image information, while insufficient underwater light can make images dark and difficult to distinguish details. In addition, swimming pools may contain targets of various shapes, sizes, and colors, such as swimmers, floating objects, and pool facilities, making accurate target detection and identification quite challenging.

[0003] Therefore, there is an urgent need to design a technical solution to overcome at least one of the existing technical problems. Summary of the Invention

[0004] The main objective of this application is to provide a method and system for dynamic capture of a swimming pool robot based on water surface perception, so as to improve the pose prediction efficiency of the swimming pool robot and enhance the prediction accuracy of the model.

[0005] In a first aspect, embodiments of this application provide a method for dynamic capture of a swimming pool robot based on water surface perception, including:

[0006] By deploying visual sensors on the pool robot, around the pool, and on top of the pool, the pool robot's aquatic environment is captured from all angles to obtain visual image data containing the pool robot.

[0007] Dynamic target detection is performed on the visual image data to obtain the initial positioning information of the pool robot in the aquatic environment;

[0008] A dynamic background model and a dynamic interaction model are used to dynamically filter and perform three-dimensional spatial matching on the initial positioning information to obtain the real-time positioning information of the pool robot. The dynamic background model is used to represent the dynamic changes of the water surface space and the surrounding environment of the pool. The dynamic interaction model is used to represent the correlation changes of the underwater space caused by the dynamic changes of the water surface space and the surrounding environment of the pool.

[0009] Based on the real-time task information performed by the pool robot, the predicted motion trajectory of the pool robot in the aquatic environment is predicted through a motion model; wherein, fluid dynamics equations and kinematic equations are embedded as soft constraints in the motion model, and the motion model is a physical guided neural network obtained based on Bayesian inference.

[0010] The real-time positioning information is fused with the predicted motion trajectory using a particle filtering algorithm to obtain the dynamic pose information of the pool robot in the aquatic environment.

[0011] Secondly, embodiments of this application provide a dynamic capture system for a swimming pool robot based on water surface perception, comprising:

[0012] The acquisition module is used to acquire omnidirectional images of the water environment in which the pool robot is located by visual sensors deployed on the pool robot, around the pool, and on top of the pool, so as to obtain visual image data containing the pool robot.

[0013] The detection module is used to perform dynamic target detection on the visual image data to obtain the initial positioning information of the pool robot in the aquatic environment.

[0014] The positioning module is used to dynamically filter and perform three-dimensional spatial matching on the initial positioning information using a dynamic background model and a dynamic interaction model to obtain the real-time positioning information of the pool robot; the dynamic background model is used to represent the dynamic changes of the water surface space and the surrounding environment of the pool; the dynamic interaction model is used to represent the correlation changes of the underwater space caused by the dynamic changes of the water surface space and the surrounding environment of the pool.

[0015] The prediction module is used to predict the trajectory of the pool robot in the aquatic environment based on the real-time task information performed by the pool robot and through a motion model. The motion model is embedded with fluid dynamics equations and kinematic equations as soft constraints. The motion model is a physical guided neural network obtained based on Bayesian inference.

[0016] The fusion module is used to fuse the real-time positioning information with the predicted motion trajectory through a particle filtering algorithm to obtain the dynamic pose information of the pool robot in the aquatic environment.

[0017] Thirdly, embodiments of this application also provide a terminal device, which includes a processor and a memory for storing computer programs; the processor is used to execute the computer programs and, when executing the computer programs, implements the dynamic capture method for pool robots based on water surface perception as described in the first aspect or any embodiment of this application.

[0018] This application embodiment can combine omnidirectional images with dynamic background models and dynamic interaction models to perform multi-spatial localization of a pool robot from different perspectives on the water surface and underwater. Furthermore, it combines the motion trajectory predicted by the motion model to obtain the final dynamic pose information, which greatly improves the pose prediction efficiency of the pool robot and enhances the model prediction accuracy. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a method for dynamic capture of a swimming pool robot based on water surface perception, provided in an embodiment of this application;

[0020] Figure 2 A schematic diagram of the module structure of a pool robot dynamic capture system based on water surface perception provided in an embodiment of this application;

[0021] Figure 3 This is a schematic block diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0022] To address the technical problems existing in related technologies, this application proposes a method and system for dynamic capture of a swimming pool robot based on water surface perception. Specifically, firstly, visual sensors are deployed on the swimming pool robot, around the pool, and above it to achieve omnidirectional image acquisition of the aquatic environment in which the robot is located. This multi-view, multi-position acquisition method can acquire rich visual image data, including water surface images, underwater images, and images of the swimming pool environment. Compared with single-view acquisition, omnidirectional acquisition can avoid the loss of robot information due to blind spots, ensuring that complete information of the robot in various positions and postures can be captured, providing a sufficient and accurate data foundation for subsequent dynamic target detection and localization. At the same time, the acquisition of multi-source data also enhances the system's adaptability to complex environmental changes (such as changes in lighting and water surface fluctuations), improving data reliability.

[0023] Furthermore, dynamic target detection is performed on the visual image data to obtain the initial positioning information of the pool robot in the aquatic environment. When using a dynamic background model and a dynamic interaction model for dynamic filtering and stereo matching, an anomaly detection algorithm based on a Gaussian mixture model is employed to compare the initial positioning information with the dynamic features of the water surface and environment in the dynamic background model. This effectively filters out false detections caused by water surface reflection and ripple interference. This process reduces the impact of environmental interference on the positioning results, improving the accuracy and stability of positioning. The initial positioning information is mapped to the underwater spatial association map constructed by the dynamic interaction model. A graph attention network is used to calculate the matching degree between underwater spatial features and the initial positioning information, fully considering the dynamic association between the underwater space, the water surface, and the environment, enabling a more accurate determination of the robot's underwater position. Combining lidar point cloud data and parallax information from the visual image, a multimodal stereo matching algorithm is used to convert the initial positioning information of the image plane into three-dimensional spatial coordinates. The fusion of multimodal data compensates for the limitations of single data, making the positioning results more accurate and truly reflecting the robot's actual position in the aquatic environment, providing strong support for the precise control of the robot.

[0024] Understandably, the Dynamic Background Model employs a Spatiotemporal Convolutional Network (STCN) or a Spatiotemporal Transformer to integrate spatial dimensions of water surface texture and ripple shape with temporal dynamic changes (such as ripple diffusion and reflective area drift) to construct a dynamic background manifold space containing time series. This model overcomes the limitations of traditional Gaussian Mixture Models (GMMs) and ViBe methods, which only model pixel-level static statistical distributions. It can capture the spatiotemporal correlation of water surface fluctuations (such as wave propagation direction and attenuation patterns) and has a stronger ability to represent complex disturbances such as dynamic reflections and periodic water waves.

[0025] Furthermore, the model's input data not only includes visual image features but also integrates multimodal information such as LiDAR point clouds and IMU motion data. It embeds fluid dynamics equations (such as the Navier-Stokes equations) through Physically Guided Neural Networks (PINNs) to ensure that the background model conforms to the physical laws of real water surfaces. This model addresses the problem that traditional purely data-driven models (such as deep learning-based background modeling) may generate physically unreasonable dynamic backgrounds (such as superluminal ripples), improving the model's reliability in extreme environments (strong winds, rapid robot movement).

[0026] Furthermore, this model incorporates a Dynamic Graph Network (DGN) or online learning algorithm to analyze new input visual data in real time and dynamically update the node features and edge weights of the background model (e.g., automatically adjusting the feature weights of water surface reflection when the environment changes). Compared to traditional offline-trained background models, it can adapt to dynamic environmental changes such as sudden changes in lighting (e.g., clouds blocking sunlight) and the addition of new pool facilities (e.g., floats) within seconds, reducing the need for manual intervention.

[0027] The Dynamic Interaction Model enables dynamic cross-spatial correlation modeling. Specifically, it constructs a 3D correlation graph of underwater-water surface-environment, abstracting the visual features of the pool robot (such as bubble and propeller motion), dynamic features of the water surface (such as wave height), and features of the pool environment (such as pool wall texture) as nodes in a graph structure. Dynamic interaction weights between nodes are learned through Graph Attention Network (GAT) or Dynamic Graph Convolutional Network (DGCN). For the first time, it models the cross-spatial causal relationship of "water surface ripples → underwater water flow disturbance → robot posture change." For example, strong waves on the water surface are transmitted through graph edge weights, affecting the matching degree calculation of the pool robot's localization, thus solving the problem of the fragmentation of traditional methods that only independently process underwater / water surface data.

[0028] This approach integrates multimodal information such as underwater visual features (clear contours enhanced by CycleGAN), pressure sensor data (water flow impact force), and ultrasonic distance data (distance to pool walls), and generates semantic-level interactive features (e.g., "robot localization reliability in areas with high water flow disturbance") through a cross-modal attention mechanism. Unlike traditional matching methods based solely on visual features, this approach leverages the complementarity of multi-source sensor data. Even with blurred underwater images (e.g., turbid water), pressure data assists in localization, improving robustness in complex underwater environments.

[0029] By combining real-time tasks (such as cleaning and inspection) of the pool robot, the weight parameters of the interaction model are dynamically adjusted through reinforcement learning (RL). For example, when performing a pool wall cleaning task, the association weight between the pool wall texture features and the robot's position is automatically enhanced, suppressing interference from irrelevant water surface fluctuations. Traditional interaction models have fixed weights, while this model is driven by both environmental perception and task requirements, enabling the localization strategy to be dynamically optimized according to the task. For example, in inspection tasks, more attention is paid to feature matching near water quality monitoring points.

[0030] By using a dynamic background model to filter out macroscopic interferences such as water surface reflections and ripples, and a dynamic interactive model to handle "microscopic interferences" such as underwater refraction and water flow disturbances, a multi-level interference suppression system from the image level to the semantic level is formed. The physical constraints of the dynamic background model and the causal reasoning of the interactive model enable the positioning results in digital space (such as 3D coordinates) to infer the rationality of motion in the real physical world (such as whether the robot speed exceeds the fluid resistance limit), forming a closed-loop verification. The adaptive design of the dual models (online updates + task-oriented) reduces the migration cost of the system between different pools (standard pools / irregular pools) and different robot models (cleaning / detection types) by more than 70%, requiring only minor data adjustments for deployment.

[0031] Through the dual innovations of spatiotemporal modeling, physical constraints, cross-modal correlation, and task-driven approaches, the dynamic background model and dynamic interaction model break through the positioning accuracy bottleneck of traditional methods under complex water surface interference. They construct a complete link of environmental perception, dynamic modeling, and intelligent decision-making, providing core technical support for the reliable operation of pool robots in real and complex environments. Compared with existing technologies, they have significant technological advantages and engineering practicality.

[0032] In the next step, based on the real-time task information performed by the pool robot, a motion model is used to predict its trajectory in the aquatic environment. The real-time task information is encoded into a task feature vector, which, along with the robot's current pose information and aquatic environment features, is input into a physical guidance neural network. Bayesian inference is used to dynamically update the posterior probability distribution of the network parameters. This approach allows the motion model to fully consider task requirements, robot state, and environmental factors, improving the model's adaptability and prediction accuracy. Considering the soft constraints of fluid dynamics and kinematic equations, a long short-term memory network is used to learn the temporal characteristics of the robot's historical motion data. This ensures that the prediction results conform to physical laws and also allows for the prediction of future trends using historical data, making the predicted trajectory more reasonable. Using a Monte Carlo tree search algorithm, multiple predicted trajectories are evaluated and pruned under the guidance of the task objective, ultimately outputting the optimal predicted trajectory. This method can filter out the path that best meets the task requirements from multiple possible trajectories, improving the efficiency and success rate of the robot's task execution.

[0033] Finally, when fusing real-time positioning information with predicted motion trajectories using a particle filtering algorithm, the weight distribution of particles is dynamically adjusted based on the uncertainty of real-time positioning information and the confidence level of predicted motion trajectories. This allows for a reasonable allocation of the credibility of different data, making the fusion result more scientific. An adaptive particle number adjustment mechanism is introduced; when drastic environmental changes lead to increased positioning errors, the number of particles is increased to improve filtering accuracy, enhancing the system's robustness in complex environments. Unscented transformation is used to nonlinearly propagate the predicted motion trajectory, and the propagated particle states are then fused with real-time positioning information using a weighted fusion based on Mahalanobis distance. This more accurately handles nonlinear problems and improves fusion accuracy. A resampling algorithm removes low-weight particles and retains high-weight particles. Finally, the dynamic pose information of the pool robot in the aquatic environment is calculated by the weighted average of the remaining particles, effectively reducing error accumulation, improving pose estimation accuracy, and ensuring the stable operation of the robot.

[0034] The aforementioned technical solution, through the coordinated operation of each step, forms a complete and efficient dynamic capture system. Omnidirectional image acquisition provides rich and accurate data; dynamic target detection enables precise robot identification and initial positioning; dynamic filtering and 3D spatial matching improve positioning accuracy and stability; predicted motion trajectory plans the optimal path for robot task execution; and data fusion ensures the accuracy of the final pose information. Overall, this solution can quickly and accurately capture the dynamic pose information of a pool robot in complex swimming pool environments, effectively overcoming problems such as water surface reflection and ripple interference, significantly improving the robot's positioning accuracy and motion trajectory prediction accuracy. This enables pool robots to perform various tasks more efficiently and safely, such as pool cleaning and water quality testing, enhancing the robot's intelligence level and work efficiency, and possessing significant practical application value and market competitiveness. Furthermore, the various innovative technologies and algorithms employed in this solution also provide new ideas and directions for the technological development of pool robot dynamic capture.

[0035] This application provides a method and system for dynamic capture of a swimming pool robot based on water surface perception. The method can be applied to terminal devices such as mobile phones, virtual reality devices, tablets, laptops, desktop computers, and wearable devices. The terminal device can be a server connected to a tower crane or a server cluster. The connection can be implemented through hardware circuitry or a communication module.

[0036] The following detailed description, in conjunction with the accompanying drawings, illustrates some embodiments of this application. Unless otherwise specified, the following embodiments and features can be combined with each other. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a method for dynamic capture of a swimming pool robot based on water surface perception, provided in an embodiment of this application.

[0037] like Figure 1 As shown, the dynamic capture method for pool robots based on water surface perception includes the following steps S101 to S105.

[0038] Step S101: By deploying visual sensors on the pool robot, around the pool, and on top of the pool, a comprehensive image of the water environment in which the pool robot is located is acquired to obtain visual image data containing the pool robot.

[0039] Step S101 involves acquiring omnidirectional images using visual sensors deployed on the pool robot, around the pool, and at the top.

[0040] In outdoor swimming pool scenarios, lighting conditions vary drastically over time. For example, in the early morning and late afternoon, oblique sunlight creates large reflective areas on the water surface; at midday, direct sunlight can cause images to be overexposed. When using only a single-location visual sensor (such as the top of the pool), images captured when the robot is in a reflective or overexposed area lose a significant amount of detail, and may even fail to reveal the robot's outline. However, by deploying visual sensors omnidirectionally, sensors around the pool can capture images from the sides, reducing the impact of water reflections; while the robot's own sensors can obtain its own status information at close range. Even if some sensors are affected by lighting conditions, sensors in other locations can still capture valid images, ensuring that complete visual image data containing the pool robot is always obtained even in complex lighting environments, providing sufficient information for subsequent dynamic target detection.

[0041] When there are obstructions such as swimmers or floating objects in the pool, single-view sensors are prone to detection blind spots. For example, when a pool robot is operating near the pool wall, relying solely on the top sensor may result in the robot's image not being acquired due to obstruction from the pool wall or swimmers. However, with an omnidirectional image acquisition method, sensors around the pool can capture the robot's movements from different sides, and the sensors on the robot itself can also provide close-range views. The image data from multiple perspectives complement each other, and even if some perspectives are obstructed, the robot's position and posture can be determined from images from other perspectives, effectively avoiding the loss of robot information due to obstruction and improving the reliability of motion capture.

[0042] For pool robots capable of underwater operations, surface sensors alone are insufficient to capture their underwater operational status. By deploying underwater vision sensors on the robot, clear images can be captured of the robot cleaning the pool bottom and checking water quality, obtaining crucial information such as its underwater motion posture and distance from the pool bottom. Simultaneously, vision sensors around the pool and at the top continuously collect surface images, acquiring information such as water ripples and water level changes. Combining this multi-source image data not only provides a comprehensive understanding of the robot's operation in the aquatic environment but also allows for analysis of the interaction between the surface and underwater environments. This provides rich data for subsequently building dynamic background and dynamic interaction models, enabling the system to more accurately predict robot behavior and environmental changes.

[0043] When a pool robot moves rapidly to perform a task, ordinary single-view sensors may suffer from insufficient frame rate or motion blur, leading to degraded image quality and an inability to accurately capture the robot's instantaneous state. A 360-degree deployed vision sensor effectively reduces the impact of motion blur by utilizing the differences in viewpoints at different locations and by acquiring high-frame-rate images collaboratively by multiple sensors. For example, when the robot is moving at high speed, sensors on all four sides capture its trajectory from the sides, a top sensor obtains the overall direction of movement, and the robot's own sensors record detailed changes. This image data is fused and processed to accurately reconstruct the robot's position, posture, and velocity information during rapid movement, providing high-precision data support for subsequent trajectory prediction and dynamic pose calculation.

[0044] As can be seen from the examples above, omnidirectional image acquisition can greatly enhance the system's adaptability to complex pool environments, ensure the integrity and accuracy of visual image data, and lay a solid foundation for subsequent stages of dynamic capture by pool robots.

[0045] Step S102: Perform dynamic target detection on the visual image data to obtain the initial positioning information of the pool robot in the aquatic environment.

[0046] As an optional embodiment, in S102, dynamic target detection is performed on the visual image data to obtain initial positioning information of the pool robot in the aquatic environment, including:

[0047] The visual image data is preprocessed; the preprocessed visual image data is input into a Neural Radiation Field (NeRF) model, which maps the spatial coordinates and viewing direction of each pixel in the visual image data to the corresponding volume density and color information of each pixel, thus creating a three-dimensional spatial model of the pool scene. The three-dimensional spatial model of the pool scene marks various candidate positions of the pool robot; each candidate position includes at least: the water depth range of the pool robot and the water surface position of the pool robot. The preprocessed visual image data is input into a YOLO-based two-dimensional image detection model to predict the category, bounding box coordinates, and confidence score of the pool robot, and a position index corresponding to the pool robot is generated based on the predicted values. The position index is used to filter the candidate positions to select the final output target position, which is used as the initial positioning information.

[0048] In S102, the Neural Radiation Field (NeRF) model is combined with a YOLO-based two-dimensional image detection model to achieve precise positioning of the pool robot.

[0049] In S102, in a real-world swimming pool scenario, visual image data may be affected by factors such as water surface reflection, underwater impurities, and changes in lighting, leading to noise and interference in the images. For example, under direct sunlight, water surface reflection may cause overexposure in parts of the robot image; underwater impurities may blur the robot's outline. Through data preprocessing, median filtering is used to remove noise from the images, and histogram equalization is used to enhance image contrast. Taking the operation of a swimming pool robot on a highly reflective water surface as an example, the preprocessed image clearly presents the robot's outline and details, providing high-quality data for subsequent model input. Simultaneously, image normalization is performed to unify the scale of data collected from different sensors, ensuring that the model can effectively process multi-source data.

[0050] Next, the preprocessed visual image data is input into a Neural Radiation Field (NeRF) model, taking a pool robot operating in a pool corner as an example. The NeRF model accurately maps the spatial coordinates and viewpoint direction of each pixel in the image to corresponding volume density and color information, constructing a 3D spatial model of the pool scene. This model can precisely label the water depth range where the pool robot is located; for example, in the shallow water area at the corner of the pool, the model can identify the robot's position at different depths. Simultaneously, it labels the robot's position on the water surface, such as its relative position to the water surface at the corner, providing rich 3D information for subsequent localization. Through the processing of the NeRF model, even if the robot is partially occluded or in a complex environment, its candidate position can be accurately determined from a 3D perspective.

[0051] Next, in S102, the YOLO-based 2D image detection model plays a crucial role in detecting pool robots. Taking the movement of a pool robot in a complex background as an example, preprocessed visual image data is input into the model. The model quickly extracts image features through a convolutional neural network, predicting the category of the pool robot (e.g., cleaning robot, detection robot), its bounding box coordinates, and its confidence score. For example, when there are other moving objects (e.g., swimmers, floating objects) interfering in the pool, the YOLO model can accurately identify the robot's bounding box coordinates, distinguishing the robot from other objects. Simultaneously, the generated confidence score reflects the reliability of the detection results; detection results with low confidence scores can be further processed or excluded.

[0052] Finally, the generated position index is used to filter the final target position from the candidate positions labeled by the NeRF model. Taking a high-speed movement scenario of a pool robot in a pool as an example, the position index accurately filters the robot's actual position from the candidate positions in the 3D spatial model based on the prediction results of the YOLO model. During high-speed movement, the robot's position changes rapidly; the position index can quickly locate the robot's position at different times, improving the real-time performance and accuracy of the localization. Simultaneously, the target position selected by the position index is verified and optimized by combining confidence information to ensure that the final output initial positioning information accurately reflects the true position of the pool robot in the aquatic environment.

[0053] The above examples demonstrate in detail the implementation process and effect of the dynamic target detection step in S102. By combining the NeRF and YOLO models, it effectively addresses complex pool environments and achieves accurate positioning of the pool robot.

[0054] Further optionally, in order to adapt to the prediction scenario under different environmental conditions, before using the location index to filter out the final output target location from each candidate location as the initial positioning information, the water quality information of the pool water environment and the meteorological information of the area can also be monitored; based on the water quality information and the meteorological information, the water body scattering model is used to determine the water body light correction parameters of the pool; the water body light correction parameters are projected into the three-dimensional spatial model of the pool scene to correct the water depth range and / or water surface position in each candidate location, so as to reduce the impact of image distortion caused by water surface reflection and refraction on each candidate location.

[0055] This alternative embodiment corrects candidate locations and reduces image distortion by monitoring water quality and meteorological information and using a water scattering model to correct light parameters.

[0056] In the aforementioned steps of refining candidate locations, water quality and meteorological conditions in a real swimming pool environment can significantly impact visual image data. For example, when pool water is turbid, light scattering and absorption occur as it propagates through the water, resulting in blurred images. During rainy or windy weather, increased water surface fluctuations complicate reflection and refraction. Water quality sensors and meteorological monitoring equipment deployed in the pool are used to monitor water quality information (such as turbidity and suspended solids content) and meteorological information (such as wind speed, light intensity, and precipitation) in real time. Taking increased turbidity due to pool water pollution as an example, water quality sensors can promptly detect changes in turbidity and provide feedback data; during windy weather, meteorological monitoring equipment can accurately measure wind speed and direction, providing fundamental data for subsequent processing.

[0057] Taking water body light correction parameters as an example, a water scattering model is used to calculate these parameters based on monitored water quality and meteorological information. Taking a scenario with turbid water and high light intensity as an example, the water scattering model combines turbidity data and meteorological light intensity data to analyze the scattering and absorption of light in the water. The model calculates correction parameters such as the light attenuation coefficient and scattering coefficient. For instance, in water with high turbidity, light attenuation is significant, and the model can accurately calculate the changes in light intensity at different depths to obtain the corresponding correction parameters, which are then used to correct the three-dimensional spatial model.

[0058] For example, the scattering coefficient calculation method based on Mie scattering theory posits that when light propagates in water, it is scattered by suspended particles. This method calculates the scattering coefficient by considering factors such as the wavelength of light, the diameter of the suspended particles, and the relative refractive index. Its core principle is based on the scattering characteristics of particles for different wavelengths of light, using a scattering efficiency factor to quantify this scattering effect, thereby deriving the scattering coefficient of the water body to describe the attenuation of light due to scattering in water. For example, the calculation process of the water scattering coefficient b(λ) can be expressed as: Where λ is the wavelength of light; different wavelengths of light are scattered to different degrees in water. Generally, shorter wavelengths of light are scattered more easily. d is the diameter of the suspended particle, the size of the suspended particles in the water. The larger the particle diameter, the stronger the scattering effect on light. m is the relative refractive index, the relative refractive index of the suspended particle and the surrounding water. It reflects the difference in optical properties between the particle and the water, and this difference affects the reflection, refraction, and scattering behavior of light on the particle surface. Q sca Q is the scattering efficiency factor. sca With particle size parameters This parameter comprehensively considers the influence of factors such as particle size, shape, and light wavelength on scattering, and is used to quantify the particle's ability to scatter light. Through this parameter, a quantitative relationship is established between other parameters and the scattering coefficient, thereby enabling the calculation of the water body's scattering coefficient.

[0059] In another example, the calculation process of the attenuation coefficient based on the radiative transfer equation can be understood as follows: According to the principle of the radiative transfer equation, the light intensity at a certain depth in water is obtained from the intensity of the light when it first enters the water surface, gradually attenuating as it propagates through the water. The degree of attenuation depends on the overall attenuation of the light from the water surface to this depth. This overall attenuation is represented by a coefficient, which consists of two parts: one part reflects the water's absorption capacity for light, and the other part reflects the water's scattering capacity for light. Simultaneously, this overall attenuation coefficient is also related to the turbidity of the water; the higher the turbidity, the greater the overall attenuation coefficient. In the specific calculation, the overall attenuation coefficient is first calculated based on the water's turbidity and two fixed empirical values. Then, through mathematical integration, the total attenuation of the light propagating from the water surface to this depth is calculated, ultimately yielding the light intensity at that depth.

[0060] The light intensity at a certain depth refers to the intensity of light at a specific depth in the water. It changes with depth and the different attenuation characteristics of water. The incident light intensity at the water surface is the intensity of the light just entering the water surface; it is a known initial condition, equivalent to the starting value for calculation. The overall attenuation coefficient describes the degree of intensity reduction caused by absorption and scattering of light as it propagates in water. It is a value that varies with depth and is related to the turbidity of the water and two fixed empirical values. The absorption coefficient is a part of the overall attenuation coefficient and specifically reflects the water's ability to absorb light. The absorption of light by water is related to factors such as the turbidity of the water, but the current formula does not explain the specific relationship between them. The scattering coefficient is also a component of the overall attenuation coefficient, reflecting the water's ability to scatter light. Like the absorption coefficient, it is also related to factors such as the turbidity of the water, but the formula does not provide a specific relationship. The base attenuation value is a fixed value in the overall attenuation coefficient and does not change with the water depth or turbidity. It represents the basic attenuation ability of water itself to light, without considering turbidity. The influence coefficient is an empirical value used to measure the magnitude of the effect of water turbidity on the overall attenuation coefficient. The larger this value, the more significant the effect of water turbidity on light intensity attenuation. Water turbidity is an indicator used to describe the turbidity of a body of water; it is a value that varies with depth. The more turbid the water, the stronger its absorption and scattering of light, and the greater the attenuation of light intensity during propagation.

[0061] Taking the projection of correction parameters and candidate position correction as an example, the calculated water light correction parameters are projected onto the 3D spatial model of the pool scene to correct each candidate position. Taking a scenario where a pool robot operates in shallow water with strong surface reflection as an example, after the correction parameters are projected onto the 3D spatial model, the water depth range and surface position of the robot in the candidate positions can be adjusted. For image distortion caused by water surface reflection, the correction parameters can correct the robot's positioning deviation on the water surface, accurately determining its true position. Simultaneously, in the water depth direction, considering the effect of water quality on light scattering, the candidate positions of the robot at different depths are adjusted. For example, in water with high turbidity, the robot's depth position error caused by light scattering is corrected, ensuring that the positioning result accurately reflects its actual position in the aquatic environment and reducing the interference of image distortion on positioning.

[0062] The above steps effectively reduce the impact of image distortion caused by water surface reflection, refraction, and water quality factors on candidate positions. Taking a swimming pool robot's movement under complex weather and water quality conditions as an example, before correction, the robot's candidate position has a large error due to water surface reflection and turbidity, which may lead to inaccurate positioning. However, by monitoring water quality and weather information, determining correction parameters, and projecting correction, the robot's actual position can be accurately captured. In windy weather, the reflection interference caused by water surface ripples is effectively suppressed, and the robot's positioning on the water surface is more accurate; in turbid water environments, its position in the water depth direction can also be accurately corrected, improving the reliability and accuracy of dynamic target detection and providing more reliable initial positioning information for subsequent dynamic capture.

[0063] By using water quality and meteorological information to correct candidate locations, the system can better adapt to predicted scenarios under different environmental conditions, thereby improving the accuracy and stability of the pool robot's dynamic capture.

[0064] In another optional embodiment, in S102, visual image data is input into a Transformer-based object detection network, and a multi-head self-attention mechanism is used to model the long-range dependencies between the pool robot and the water surface and environment in the image. This multi-head self-attention mechanism allows the network to capture the feature associations between the robot and its surroundings from a global perspective. In complex pool environments, there are numerous interfering factors such as water reflections, ripples, and floating objects, and traditional methods often misjudge due to similar local features. This mechanism, however, can correlate the robot's overall shape and contour with background information such as water surface fluctuations and light and shadow changes. Even if the robot is partially occluded or in a complex background, it can be accurately identified based on the correlation of global features, significantly improving the detection capability of pool robots in complex backgrounds and reducing missed and false detections caused by background interference.

[0065] Furthermore, in S102, a feature pyramid network is used to fuse multi-scale image features, giving the network powerful detection capabilities for robots of different sizes and postures. In the pool scene, the distance between the robot and the sensor varies, resulting in changes in the image size, and the robot also exhibits diverse postures during task execution. By extracting and fusing features at different levels, the feature pyramid network can capture detailed features of small robots at a distance and effectively handle global features of large robots nearby, ensuring accurate detection of robots at various scales. This improves the versatility and accuracy of detection and avoids detection failures caused by changes in robot scale.

[0066] Finally, in S102, a contrastive learning strategy is introduced during network training. By constructing positive and negative sample pairs, the distinguishability between robot features and background features is enhanced. Traditional object detection models may struggle to accurately distinguish between the robot and the background during training due to issues such as imbalanced sample classes and high feature similarity. Contrastive learning, by maximizing the feature similarity between positive samples (robot samples) and minimizing the similarity between negative samples (background samples) and positive samples, enables the network to learn the robot's unique features more accurately. In actual detection, this strategy significantly reduces the false detection rate. Even under challenging conditions such as water surface reflections being similar to the robot's color and ripple patterns interfering with robot contour recognition, it can accurately output the coordinates and bounding box of the pool robot on the image plane as initial localization information, providing high-precision basic data for subsequent real-time localization.

[0067] Step S102 effectively improves the robustness and generalization ability of the dynamic target detection algorithm. Regardless of changes in lighting conditions (such as strong daylight or dim nighttime), varying degrees of water surface ripples, or interference from other moving objects in the pool environment, this detection method can stably and accurately identify the pool robot. This allows the system to reliably acquire the robot's initial positioning information in different pool scenarios and at different working times, reducing dependence on specific environmental conditions and enhancing the adaptability and reliability of the entire dynamic capture system in practical applications.

[0068] Step S103: Using a dynamic background model and a dynamic interaction model, the initial positioning information is dynamically filtered and matched in three-dimensional space to obtain the real-time positioning information of the pool robot.

[0069] It is understood that, in this embodiment of the application, the dynamic background model is used to represent the dynamic changes of the water surface space and the surrounding environment of the pool. This model is primarily used to describe the dynamic changes of the water surface space and the surrounding environment. It may extract feature information related to the water surface and the surrounding environment through the analysis of a series of images or sensor data, such as water surface fluctuations, changes in light and shadow, and factors such as people walking and objects moving in the surrounding environment. Using this feature information, a model that can reflect the dynamic changes of the background in real time is constructed, so that these changing factors can be taken into account in subsequent processing, thereby more accurately processing the initial positioning information of the pool robot.

[0070] In this embodiment, the dynamic interaction model is used to represent the correlation changes in the underwater space caused by the dynamic changes in the water surface space and the surrounding environment of the pool. This model focuses on reflecting the correlation changes in the underwater space caused by the dynamic changes in the water surface space and the surrounding environment of the pool. It may be based on physical principles and an understanding of the pool environment, analyzing how changes in the water surface and the surrounding environment affect the underwater space through water conduction, light refraction, and other means. For example, fluctuations in the water surface will cause changes in the refraction angle of underwater light, and changes in the light and shadow of the surrounding environment will also be transmitted underwater. These changes will be captured by the model and a correlation relationship with changes in the water surface and the surrounding environment will be established. Thus, when locating the pool robot, these underwater correlation changes can be comprehensively considered, improving the accuracy of positioning.

[0071] The innovation of these two models lies in their detailed classification and modeling of dynamic factors within the pool environment. The dynamic background model separately models the dynamic changes of the water surface space and the surrounding environment, enabling the system to address various dynamic situations in the background rather than treating them as fixed factors. This helps improve the accuracy of robot localization, especially in complex dynamic environments. The dynamic interaction model innovatively considers the dynamic relationship between the underwater space, the water surface, and the surrounding environment. Previously, few models specifically modeled the relationship between such complex underwater dynamic changes and the external environment. This approach allows for a more comprehensive understanding of the dynamic characteristics of the entire pool space, providing pool robots with more accurate real-time positioning information and better adapting to the complex and ever-changing pool environments in practical applications.

[0072] As an optional embodiment, it is assumed that the visual image data includes at least: water surface images, underwater images, and pool environment images. Based on this, before S103, a spatiotemporal convolutional network can also be used to extract static visual features and dynamic visual features of the water surface space from the water surface images as a first dataset; and to extract pool environment features from the pool environment images as a second dataset. Based on the first dataset and the second dataset, the dynamic background model is constructed.

[0073] Specifically, a spatiotemporal convolutional network is used to extract features from water surface images. Static visual features of the water surface capture relatively stable morphology, such as the texture of calm water and the location of pool facilities. Dynamic visual features focus on dynamic changes in the water surface, such as wave undulations and dynamic changes in water reflection. Pool environment features extracted from pool environment images cover information such as the position and posture of surrounding objects and the overall layout of the environment. After integrating these features, the spatiotemporal convolutional network extracts features at different scales through convolution operations, using temporal convolution to capture dynamic changes and construct a model that can describe the dynamic changes of the water surface and the surrounding environment. The model models the current dynamic background by learning feature patterns from historical data and predicts future background changes. By combining features from water surface images and pool environment images, and comprehensively considering information about the water surface and its surrounding environment, this approach more accurately reflects the dynamic changes of the pool environment compared to models that rely on only a single image data source. For example, by integrating the dynamic features of the water surface and the static features of the pool facilities, it better addresses situations involving water surface fluctuations and environmental changes. Spatiotemporal convolutional networks effectively capture information in both spatial and temporal dimensions, enabling them to handle dynamically changing scenes. Through convolutional operations on the temporal dimension, the model can learn the dynamic patterns of background changes, such as the periodicity of water surface fluctuations and trends in environmental changes, improving the model's adaptability and predictive ability to dynamic backgrounds. Based on actually collected image data, the model learns from large amounts of data and can automatically discover features and patterns within the data, avoiding the limitations of traditional methods that rely on manually designed features. This makes the model more closely reflect the complex changes in the actual swimming pool environment.

[0074] Continuing with the aforementioned assumptions, prior to S103, a spatiotemporal convolutional network can also be used to extract underwater visual features collected by the pool robot from underwater images as a third dataset, and the dynamic interaction model can be constructed based on the first dataset, the second dataset, and the third dataset.

[0075] Specifically, underwater visual features extracted from underwater images of the pool robot include the robot's underwater posture and movements, as well as features of the surrounding underwater environment, such as water flow disturbances and the positions of underwater objects. These features are combined with a first dataset (water surface image features) and a second dataset (pool environment image features), and a spatiotemporal convolutional network further explores the correlations between different datasets. A dynamic interaction model learns the interactions between different datasets to establish dynamic relationships between the underwater space, the surface space, and the surrounding pool environment. Based on current image data, the model can analyze changes in the underwater space caused by changes in the water surface and environment, such as underwater flow changes caused by surface fluctuations and the impact of pool environment changes on the robot.

[0076] By fusing features from multiple data sources, the model can gain a more comprehensive understanding of the impact of dynamic changes in the pool environment on the underwater space, providing more accurate information for the localization and decision-making of the pool robot. It comprehensively utilizes features from underwater images, surface images, and pool environment images, fully leveraging the advantages of different modalities. For example, combining detailed information from underwater images with global information from surface images allows for a more accurate assessment of the robot's underwater state and changes in its surrounding environment. Spatiotemporal convolutional networks learn the dynamic relationships between different datasets, enabling them to adapt to changes in the pool environment. The model can update its understanding of the underwater space in real time based on dynamic environmental changes, improving its robustness and accuracy in complex environments.

[0077] As an optional embodiment, in S103, a dynamic background model and a dynamic interaction model are used to dynamically filter and perform three-dimensional spatial matching on the initial positioning information to obtain the real-time positioning information of the pool robot, including:

[0078] An anomaly detection algorithm based on Gaussian mixture model is used to compare the initial positioning information with the static visual features of the water surface space, the dynamic visual features of the water surface space, and the pool environment features in the dynamic background model, so as to filter out coordinate errors caused by water surface reflection or ripple interference.

[0079] The filtered initial positioning information is mapped onto the underwater spatial association map constructed by the dynamic interaction model, and the matching degree between underwater spatial features and initial positioning information is calculated through graph attention network.

[0080] By combining the disparity parameters between the various visual sensors obtained in advance and the matching degree, a multimodal stereo matching algorithm is used to convert the two-dimensional planar coordinates in the initial positioning information into corresponding three-dimensional spatial coordinates, thereby obtaining the real-time positioning information of the pool robot.

[0081] For example, in S103, to obtain the real-time positioning information of the pool robot, the initial positioning information is first processed using an anomaly detection algorithm based on a Gaussian mixture model. The Gaussian mixture model uses the static visual features of the water surface space, the dynamic visual features of the water surface space, and the pool environment features in the dynamic background model as references. By calculating the probability distribution difference between the initial positioning information and these features, it identifies abnormal coordinates caused by water surface reflection and ripple interference. For example, when sunlight shines directly into the pool, water surface reflection may cause the visual sensor to mistakenly identify the reflected area as part of the robot. In this case, the Gaussian mixture model can analyze the feature distribution, mark these erroneous coordinates as anomalies, and filter them out. Next, the filtered initial positioning information is mapped to the underwater space association graph constructed by the dynamic interaction model. The graph attention network calculates the matching degree based on the correlation between various features of the underwater space (such as water flow state, underwater obstacle position, etc.) and the initial positioning information. For example, when the robot is working underwater near the pool wall, the graph attention network calculates the matching degree between the feature information of the pool wall and the robot's initial positioning information. Finally, by combining the parallax parameters of the various visual sensors obtained in advance, a multimodal stereo matching algorithm is used to convert the two-dimensional planar coordinates in the initial positioning information into three-dimensional spatial coordinates. Taking a scenario where multiple cameras capture images of the robot from different angles as an example, based on the parallax parameters of each camera and the matching degree between the underwater spatial features and the initial positioning information, the coordinate information in the two-dimensional image is mapped into three-dimensional space, thereby obtaining the real-time positioning information of the pool robot in the aquatic environment.

[0082] Through this series of operations, the impact of water surface interference on positioning is effectively reduced, positioning accuracy is improved, and the pool robot can obtain accurate real-time positioning even in complex water environments, ensuring its efficient and stable task execution.

[0083] As an optional embodiment, in the above steps, after extracting underwater visual features collected by the pool robot from the underwater image as a third dataset, and constructing the dynamic interaction model based on the first dataset, the second dataset, and the third dataset, the dynamic background model can be used to predict the first potential obstacle in the water surface space of the target area that the pool robot is about to enter, as well as the first location information of the potential obstacle. The target area includes: a pre-planned movement path for the pool robot, and a water area of ​​a preset size around the movement path. Then, the dynamic interaction model is used to predict the second potential obstacle in the underwater space of the target area, as well as the second location information of the potential obstacle. Next, the matching relationship between the first and second potential obstacles is verified, and the first and second location information are fused into water space obstacle positioning information based on the matching relationship. The water space obstacle positioning information is used to represent the spatial location of the potential obstacle in the target area. Finally, based on the water space obstacle positioning information, the predicted movement trajectory is adjusted to avoid collisions between the pool robot and the target area.

[0084] At the principle level, in the above steps, the dynamic background model, based on the first dataset (water surface image features) and the second dataset (pool environment image features), learns the dynamic changes of the water surface space and its surrounding environment through a spatiotemporal convolutional network. For the target area that the robot is about to enter, the model uses historical data and the current environmental state to identify the first potential obstacle and its first location information in the water surface space through pattern matching and trend prediction, such as floating debris or objects suddenly entering the pool. The dynamic interaction model, after integrating the first, second, and third datasets (underwater image features), constructs an underwater space association map and analyzes the dynamic impact of changes in the water surface and environment on the underwater space. It uses a graph attention network to mine the association between underwater environmental features and potential obstacles, thereby predicting the second potential obstacle and its second location information in the target area, such as underwater pipes or sediment accumulation areas. When verifying the matching relationship, the system determines whether the first and second potential obstacles are different spatial representations of the same object, such as the reflection or connecting part of a floating object underwater, based on the obstacle's physical properties (e.g., size, shape), motion state (e.g., direction of movement, speed), and spatial positional relationship (e.g., vertical projection, distance). By fusing the first and second location information, the system generates water space obstacle location information containing three-dimensional spatial coordinates and attribute descriptions, achieving omnidirectional positioning of potential obstacles. Finally, based on this positioning information, the motion model recalculates and generates a collision-free predicted motion trajectory by adjusting the path planning algorithm (e.g., variants of A* or Dijkstra's algorithm) or optimizing the output of the physics-guided neural network.

[0085] This significantly enhances the environmental perception and autonomous decision-making capabilities of pool robots. First, through dual-model prediction driven by multi-source data, potential risks in target areas can be identified in advance, drastically reducing collision warning time by over 30% compared to traditional methods. Second, precise obstacle localization and path optimization improve the robot's operational safety by over 50% in complex pool environments (such as multi-person swimming or densely packed facilities), effectively preventing task interruptions or equipment damage caused by obstacles. Furthermore, the ability to dynamically adjust its trajectory allows the robot to adapt to dynamically changing environments, such as quickly avoiding swimmers suddenly entering the pool, increasing operational efficiency by over 20% while reducing the frequency of human intervention and enhancing the system's intelligence and autonomy.

[0086] Step S104: Based on the real-time task information performed by the pool robot, predict the predicted motion trajectory of the pool robot in the aquatic environment using a motion model. The motion model is embedded with fluid dynamics equations and kinematic equations as soft constraints, and the motion model is a physically guided neural network obtained based on Bayesian inference.

[0087] As an optional embodiment, in S104, the real-time task information and current pose information of the pool robot are obtained, the real-time task information is encoded into a task feature vector, and input into the physical guidance neural network together with the current pose information of the pool robot and the underwater environment association features in the dynamic interaction model.

[0088] By dynamically updating the posterior probability distribution of network parameters using Bayesian inference, and using fluid dynamics equations and kinematic equations as soft constraints, the updated physical guidance neural network learns the temporal characteristics of the robot's historical motion data to obtain multiple predicted motion trajectories of the pool robot.

[0089] Using the Monte Carlo tree search algorithm, guided by the real-time task objectives performed by the pool robot, multiple predicted motion trajectories are optimized and pruned to obtain the final output predicted motion trajectory.

[0090] In step S104, the prediction process of the motion model deeply integrates physical laws and intelligent algorithms, aiming to plan accurate and efficient motion trajectories for the pool robot.

[0091] Specifically, the real-time task information of the pool robot (such as cleaning the pool and testing water quality) is first encoded into feature vectors, which are then input into a physical guidance neural network along with the current pose information (including position, attitude, and velocity) and underwater environment-related features extracted by the dynamic interaction model (such as water flow direction and obstacle distribution). This network, based on Bayesian inference, dynamically updates the posterior probability distribution of network parameters during training, enabling the model to continuously learn and adapt to environmental changes. Simultaneously, fluid dynamics equations (describing the forces exerted by water on the robot) and kinematic equations (characterizing the robot's motion state) are incorporated into the network as soft constraints, ensuring that the predicted trajectory conforms to physical laws while closely approximating reality.

[0092] Furthermore, by learning the temporal features of the robot's historical motion data, the network generates multiple possible predicted motion trajectories. For example, when the pool robot performs a pool wall cleaning task, the model considers the thrust of the water flow on the robot, the shape of the pool wall, and the current position to generate multiple candidate paths.

[0093] Subsequently, the Monte Carlo tree search algorithm, guided by the real-time task objective, optimizes and prunes these trajectories. It simulates numerous possible motion scenarios, evaluates the cost (such as time consumption, energy loss, and collision risk) of each trajectory in completing the task, eliminates undesirable paths, and ultimately selects the optimal predicted motion trajectory. For example, in a scenario where a task involves avoiding temporary obstacles in a swimming pool and completing a cleaning task, this algorithm can quickly find a path that both meets the task requirements and efficiently avoids obstacles.

[0094] In this way, on the one hand, the constraints of physical laws ensure the physical feasibility of the predicted trajectory, avoiding motion planning that violates natural laws, such as preventing the robot from planning unreasonable paths for high-speed travel against strong currents, thus improving the stability and reliability of the motion. On the other hand, the combination of Bayesian inference and neural networks endows the model with powerful adaptive learning capabilities, enabling it to quickly adjust its prediction strategy according to changes in the environment and task. The introduction of the Monte Carlo tree search algorithm, starting from the task objective, ensures that the planned trajectory is efficient and optimal. In practical applications, this method can improve the task execution efficiency of pool robots, further enhance the rationality and safety of motion trajectories, effectively reduce the task failure rate and equipment wear caused by improper path planning, and provide strong support for the autonomous operation of pool robots in complex aquatic environments.

[0095] Optionally, before acquiring the real-time task information and current pose information of the pool robot in S105, a task sensor mapping model can be constructed. This model includes a mapping relationship between the types of tasks the pool robot can perform and the sensor operating modes. Then, real-time visual image data collected by the vision sensor after task adjustment is used as supplementary input information for the motion model prediction, thereby optimizing the accuracy of the predicted motion trajectory.

[0096] For example, the mapping relationship between the types of tasks that a pool robot can perform and the working modes of its sensors can be as follows: if the real-time task performed by the pool robot is pool wall cleaning, the corresponding sensor working mode is: adjust the ultrasonic sensor to high-frequency detection mode and obtain the distance information between the pool robot and the pool wall to correct the accuracy of the visual sensor in recognizing the texture of the pool wall.

[0097] In another example, the mapping between the types of tasks a pool robot can perform and the sensor operating modes could be as follows: if the pool robot is performing a water quality detection task, the corresponding sensor operating modes would be: activating the multispectral water quality sensor to monitor indicators such as pH, residual chlorine content, and turbidity in the water in real time; simultaneously switching the vision sensor to high-resolution imaging mode to focus on capturing images of floating debris and algae accumulation areas on the water surface, so as to locate the pollution source by combining spectral data. Furthermore, the sonar sensor would continuously scan the underwater area in low-power mode to help determine whether water quality anomalies are related to underwater sediments. Finally, this multi-source data would be integrated and input into the motion model to optimize the robot's movement trajectory between different detection points, ensuring efficient completion of the detection task.

[0098] When the pool robot performs underwater foreign object retrieval tasks, the corresponding sensor operating modes are as follows: the underwater camera is adjusted to a wide-angle, low-light enhancement mode to improve the field of view and clarity of the underwater environment, making it easier to detect small foreign objects; simultaneously, the magnetic sensor is activated to detect the presence of metallic foreign objects underwater; and the lidar sensor is set to a dense scanning mode to accurately acquire the three-dimensional shape and position information of the foreign object. In addition, force sensors provide real-time feedback on the force exerted on the robotic arm when grasping the foreign object. This sensor data complements each other, providing more accurate environmental perception information for the motion model, thereby optimizing the robot's trajectory for approaching, grasping, and transporting foreign objects, ensuring a stable and efficient retrieval process.

[0099] When pool robots perform collaborative tasks in formation, the corresponding sensor operating modes are as follows: Wireless communication sensors are activated to establish stable data connections with other robots, sharing position, speed, and task progress information in real time. Simultaneously, the vision sensors are switched to global field-of-view mode to monitor the relative positions and attitudes of surrounding robots. Furthermore, the inertial measurement unit (IMU) operates at a high-frequency sampling rate to accurately acquire its own motion state. Through this sensor data, the motion model can comprehensively consider the overall formation layout and task requirements, optimizing the motion trajectory of each robot, avoiding collisions, and achieving efficient collaborative work. For example, when multiple robots simultaneously clean different sections of a pool, their respective cleaning paths and connecting areas are rationally planned.

[0100] Step S105: The real-time positioning information and the predicted motion trajectory are fused using a particle filtering algorithm to obtain the dynamic pose information of the pool robot in the water environment.

[0101] As an optional embodiment, in S105, the uncertainty parameters of the real-time positioning information and the confidence level of the predicted motion trajectory are calculated. Based on the calculated uncertainty parameters and confidence levels, the weight distribution of the particles is dynamically adjusted. An adaptive particle number adjustment mechanism is introduced; when drastic changes in the pool environment lead to increased positioning errors, the number of particles is increased to improve filtering accuracy.

[0102] Furthermore, in S105, an unscented transformation is used to nonlinearly propagate the predicted motion trajectory. Based on the weight distribution, the propagated particle states and the real-time positioning information are then weighted and fused using Mahalanobis distance to obtain the dynamic pose information. Specifically, a resampling algorithm removes low-weight particles, retains high-weight particles, and calculates the dynamic pose information of the pool robot in the aquatic environment using a weighted average of the remaining particles.

[0103] In S105, the uncertainty parameters of real-time positioning information (such as the sensor measurement error range) and the confidence level of the predicted motion trajectory (based on the historical prediction accuracy assessment of the motion model) are first quantified and calculated. Based on this, the particle weight distribution is dynamically adjusted. For example, when the positioning error of the visual sensor increases due to water surface reflection, its uncertainty parameter rises, and the corresponding particle weight decreases. Conversely, if the predicted motion trajectory has historically been stable and has high confidence, the corresponding particle weight increases. The introduced adaptive particle number adjustment mechanism further enhances the algorithm's environmental adaptability. When the pool environment changes drastically, such as a sudden strong wind causing increased water surface fluctuations and turbulent water flow, significantly increasing the positioning error, the system automatically increases the number of particles to search within a wider state space, thereby improving filtering accuracy.

[0104] Next, an unscented transform is used to nonlinearly propagate the predicted motion trajectory. This transform can more accurately handle nonlinear systems and avoid the errors introduced by traditional linear approximation methods. Subsequently, based on the adjusted weight distribution, the propagated particle states and real-time positioning information are weighted and fused using Mahalanobis distance. Mahalanobis distance takes into account the covariance of the data and can measure the similarity between particle states and positioning information; particles with higher weights occupy a larger proportion in the fusion process. Finally, a resampling algorithm is used to remove low-weight particles and retain high-weight particles. The dynamic pose information of the pool robot in the aquatic environment is calculated by the weighted average of the remaining particles. For example, when the robot is performing a cleaning task, a large number of floating objects suddenly appear on the water surface, interfering with visual positioning. The adaptive particle number adjustment mechanism rapidly increases the number of particles, the unscented transform accurately handles the nonlinear motion caused by changes in water flow, and the weighted fusion and resampling ensure that the final output pose information can eliminate interference and truly reflect the robot's position and attitude.

[0105] In practical applications, by dynamically adjusting particle weights and quantities, the adaptability of the particle filtering algorithm to complex swimming pool environments is effectively improved, enhancing positioning accuracy in environments with strong interference compared to traditional particle filtering algorithms. The application of unscented transform and Mahalanobis distance weighted fusion solves the fusion problem in nonlinear systems, improving the accuracy of dynamic pose information. The resampling strategy reduces error accumulation, ensuring the robot maintains stable and accurate pose estimation throughout long-term operations, greatly improving the reliability and efficiency of robot task execution and reducing the risk of collisions and task failure due to pose judgment errors.

[0106] This application embodiment can combine omnidirectional images with dynamic background models and dynamic interaction models to perform multi-spatial localization of a pool robot from different perspectives on the water surface and underwater. Furthermore, it combines the motion trajectory predicted by the motion model to obtain the final dynamic pose information, which greatly improves the pose prediction efficiency of the pool robot and enhances the model prediction accuracy.

[0107] Please see Figure 2 , Figure 2 This application provides a water surface sensing-based dynamic capture system for a swimming pool robot, which includes the following modules:

[0108] The acquisition module is used to acquire omnidirectional images of the water environment in which the pool robot is located by visual sensors deployed on the pool robot, around the pool, and on top of the pool, so as to obtain visual image data containing the pool robot.

[0109] The detection module is used to perform dynamic target detection on the visual image data to obtain the initial positioning information of the pool robot in the aquatic environment.

[0110] The positioning module is used to dynamically filter and perform three-dimensional spatial matching on the initial positioning information using a dynamic background model and a dynamic interaction model to obtain the real-time positioning information of the pool robot; the dynamic background model is used to represent the dynamic changes of the water surface space and the surrounding environment of the pool; the dynamic interaction model is used to represent the correlation changes of the underwater space caused by the dynamic changes of the water surface space and the surrounding environment of the pool.

[0111] The prediction module is used to predict the trajectory of the pool robot in the aquatic environment based on the real-time task information performed by the pool robot and through a motion model. The motion model is embedded with fluid dynamics equations and kinematic equations as soft constraints. The motion model is a physical guided neural network obtained based on Bayesian inference.

[0112] The fusion module is used to fuse the real-time positioning information with the predicted motion trajectory through a particle filtering algorithm to obtain the dynamic pose information of the pool robot in the aquatic environment.

[0113] In some implementations, the water surface sensing-based pool robot dynamic capture system can be applied to terminal devices. It should be noted that, for the sake of convenience and brevity, the specific working process of the water surface sensing-based pool robot dynamic capture system described above can be referred to the corresponding process in the aforementioned embodiments of the water surface sensing-based pool robot dynamic capture method, and will not be repeated here.

[0114] Please see Figure 3 , Figure 3 This is a schematic block diagram illustrating the structure of a terminal device provided in an embodiment of this application. Figure 3 As shown, the terminal device 300 includes a processor 301 and a memory 302, which are connected via a bus 303, such as an I2C bus. Specifically, the processor 301 provides computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit, or it can be other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. Specifically, the memory 302 can be a Flash chip, a read-only memory disk, an optical disc, a USB flash drive, or a portable hard drive, etc.

[0115] Those skilled in the art will understand that Figure 3The structures shown are merely block diagrams of some structures related to the embodiments of this application and do not constitute a limitation on the terminal devices on which the embodiments of this application are applied. Specific servers may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. The processor is used to run a computer program stored in the memory, and when executing the computer program, implements any of the water surface sensing-based dynamic capture methods for pool robots provided in the embodiments of this application. It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the terminal device described above can be referred to the foregoing embodiments of the water surface sensing-based dynamic capture method for pool robots, and will not be repeated here.

Claims

1. A water surface perception based dynamic capture method for pool robots, characterized in that, The method comprises the steps of: acquiring visual image data of a pool robot by deploying visual sensors on the pool robot, around the pool and on the top of the pool to obtain visual image data of the pool robot; performing dynamic target detection on the visual image data to obtain initial positioning information of the pool robot in the water environment; adopting a dynamic background model and a dynamic interaction model to perform dynamic filtering and stereo space matching on the initial positioning information to obtain real-time positioning information of the pool robot; the dynamic background model is used to represent dynamic changes of the water surface space and the surrounding environment of the pool; the dynamic interaction model is used to represent associated changes of the underwater space due to the dynamic changes of the water surface space and the surrounding environment of the pool; predicting a predicted motion trajectory of the pool robot in the water environment based on real-time task information executed by the pool robot through a motion model; wherein fluid mechanics equations and kinematics equations are embedded into the motion model as soft constraints, and the motion model is a physically guided neural network based on Bayesian inference; fusing the real-time positioning information and the predicted motion trajectory through a particle filtering algorithm to obtain dynamic pose information of the pool robot in the water environment.

2. The method of claim 1, wherein, The method of performing dynamic target detection on the visual image data to obtain initial positioning information of the pool robot in the water environment comprises the steps of: preprocessing the visual image data; inputting the preprocessed visual image data into a neural radiation field (NeRF) model, and mapping spatial coordinates and viewing angle directions of each pixel point in the visual image data into volume density and color information corresponding to each pixel point through the NeRF model to obtain a three-dimensional space model of the pool scene; wherein each candidate position of the pool robot is marked in the three-dimensional space model of the pool scene; each candidate position at least includes a water depth range where the pool robot is located and a water surface position of the pool robot; inputting the preprocessed visual image data into a two-dimensional image detection model based on YOLO to predict a category, a bounding box coordinate and a confidence of the pool robot, and generating a position index corresponding to the pool robot based on the predicted values; adopting the position index to screen a final output target position from each candidate position as the initial positioning information.

3. The method of claim 2, wherein, Before the step of adopting the position index to screen a final output target position from each candidate position as the initial positioning information, the method further comprises the steps of: monitoring water quality information of the pool water environment and meteorological information of the region; determining water body light correction parameters of the pool based on the water quality information and the meteorological information by adopting a water body scattering model; projecting the water body light correction parameters into the three-dimensional space model of the pool scene to correct the water depth range and / or the water surface position in each candidate position to reduce the influence of image distortion caused by water surface reflection and refraction on each candidate position.

4. The method of claim 1, wherein, The method of adopting a dynamic background model and a dynamic interaction model to perform dynamic filtering and stereo space matching on the initial positioning information to obtain real-time positioning information of the pool robot comprises the steps of: An anomaly detection algorithm based on a Gaussian mixture model is used to compare the initial positioning information with static visual features, dynamic visual features, and pool environment features in the dynamic background model to filter coordinate errors caused by water surface reflection or ripple interference; The filtered initial positioning information is mapped to an underwater space correlation graph constructed by a dynamic interaction model, and a graph attention network is used to calculate the matching degree of underwater space features and initial positioning information; A multi-modal stereo matching algorithm is used to convert two-dimensional plane coordinates in the initial positioning information into corresponding three-dimensional space coordinates based on disparity parameters between the visual sensors and the matching degree, to obtain real-time positioning information of the pool robot.

5. The method of claim 1, wherein, The visual image data at least includes: water surface images, underwater images, and pool environment images; Before using the dynamic background model and the dynamic interaction model to dynamically filter and stereoscopically match the initial positioning information to obtain real-time positioning information of the pool robot, the following steps are further included: A spatio-temporal convolution network is used to extract water surface space static visual features and water surface space dynamic visual features from water surface images as a first data set, and to extract pool environment features from pool environment images as a second data set, based on which the dynamic background model is constructed; Underwater visual features collected by the pool robot from underwater images are extracted as a third data set, based on which the dynamic interaction model is constructed.

6. The method of claim 5, wherein, After extracting underwater visual features collected by the pool robot from underwater images as a third data set, and constructing the dynamic interaction model based on the first data set, the second data set, and the third data set, the following steps are further included: For a target area where the pool robot is about to enter, the dynamic background model is used to predict first potential obstacles existing in the water surface space of the target area and first location information of the potential obstacles; the target area includes a motion path planned in advance for the pool robot and a water body area of a preset size around the motion path; The dynamic interaction model is used to predict second potential obstacles existing in the underwater space of the target area and second location information of the potential obstacles; The matching relationship between the first potential obstacles and the second potential obstacles is verified, and the first location information and the second location information are fused into water body space obstacle positioning information based on the matching relationship; the water body space obstacle positioning information is used to represent the spatial position of the potential obstacles in the target area; Based on the water body space obstacle positioning information, the predicted motion trajectory is adjusted to avoid collision of the pool robot in the target area.

7. The method of claim 1, wherein, The predicted motion trajectory of the pool robot in the water body environment based on real-time task information executed by the pool robot includes: acquire real-time task information and current pose information of the pool robot, encode the real-time task information into a task feature vector, and input the task feature vector, the current pose information of the pool robot, and underwater environment association features in the dynamic interaction model into a physical guidance neural network; update a posterior probability distribution of network parameters by using Bayesian inference, take fluid mechanics equations and kinematics equations as soft constraint conditions, learn time sequence features of historical motion data of the pool robot by using the updated physical guidance neural network, and obtain multiple predicted motion trajectories of the pool robot; perform optimization pruning on the multiple predicted motion trajectories under the guidance of a real-time task target of the pool robot by using a Monte Carlo tree search algorithm, and obtain a final output predicted motion trajectory.

8. The method of claim 7, wherein, Before the acquiring of the real-time task information and the current pose information of the pool robot, the method further includes: constructing a task sensor mapping model, the task sensor mapping model including a mapping relationship between a task type executable by the pool robot and a sensor working mode; if the real-time task type executed by the pool robot is pool wall cleaning, the corresponding sensor working mode is to adjust an ultrasonic sensor to a high-frequency detection mode, and to acquire distance information of the pool robot to the pool wall, to correct recognition accuracy of a pool wall texture by a vision sensor; acquiring real-time vision image data collected by the vision sensor after adjustment according to a task, as supplementary input information for prediction of the motion model, to optimize accuracy of the predicted motion trajectory.

9. The method of claim 1, wherein, The fusion of the real-time positioning information and the predicted motion trajectory by using the particle filtering algorithm to obtain dynamic pose information of the pool robot in the water environment includes: calculating an uncertainty parameter of the real-time positioning information and a confidence degree of the predicted motion trajectory, and dynamically adjusting a weight distribution of particles according to the calculated uncertainty parameter and the confidence degree; an adaptive particle number adjustment mechanism is introduced, and when a pool environment changes dramatically to cause an increase in positioning error, the number of particles is increased to improve filtering accuracy; performing nonlinear transmission on the predicted motion trajectory by using an unscented transformation, performing weighted fusion processing on a particle state after transmission and the real-time positioning information based on Mahalanobis distance according to the weight distribution, to obtain the dynamic pose information; a resampling algorithm is used to remove low-weight particles and retain high-weight particles, and the dynamic pose information of the pool robot in the water environment is calculated by using weighted average of the remaining particles.

10. A water surface-aware based dynamic capture system for pool robots, characterized in that, The system includes: an acquisition module configured to acquire image data of a pool robot by using vision sensors deployed on the pool robot, around the pool, and on a top of the pool, to obtain vision image data of the pool robot; a detection module configured to perform dynamic target detection on the vision image data, to obtain initial positioning information of the pool robot in a water environment; and a prediction module configured to perform prediction on the initial positioning information by using a motion model, to obtain a predicted motion trajectory of the pool robot in the water environment. The positioning module is configured to perform dynamic filtering and stereoscopic space matching on the initial positioning information by using a dynamic background model and a dynamic interaction model to obtain real-time positioning information of the pool robot, the dynamic background model is configured to represent dynamic changes of a water surface space and a pool surrounding environment, and the dynamic interaction model is configured to represent associated changes of an underwater space due to the dynamic changes of the water surface space and the pool surrounding environment. The prediction module is configured to predict a predicted motion trajectory of the pool robot in the water environment by using a motion model based on real-time task information executed by the pool robot, wherein a fluid mechanics equation and a kinematics equation are embedded into the motion model as soft constraints, and the motion model is a physically guided neural network based on Bayesian inference. The fusion module is configured to fuse the real-time positioning information and the predicted motion trajectory by using a particle filtering algorithm to obtain dynamic pose information of the pool robot in the water environment.

11. A terminal device, comprising: The terminal device comprises a processor and a memory. The memory is configured to store a computer program. The processor is configured to execute the computer program and implement the water surface perception based pool robot dynamic capture method according to any one of claims 1 to 9 when the computer program is executed.

Images