Control method of automatic pool cleaning device and automatic pool cleaning device

Abstract

The invention provides a control method of an automatic pool cleaning device and the automatic pool cleaning device. The method comprises the following steps: respectively acquiring image information and depth information of an underwater object in a pool through an image sensor and a distance measuring sensor arranged on the automatic pool cleaning device; generating fusion data based on the image information and the depth information; based on the fusion data, identifying the position and the category of the underwater object; and controlling the automatic pool cleaning device to advance based on the position and the category of the underwater object.

Description

Technical Field

[0001] The present disclosure relates to the field of pool cleaning, and in particular to a control method for an automatic pool cleaning device and the automatic pool cleaning device. Background Art

[0002] Automatic pool cleaning devices are generally used to clean pools, for example, to collect and clean garbage / debris on the bottom, side walls and / or water surface of a pool such as a swimming pool, so as to filter and purify the water in the pool, and the filtered and purified water can be discharged into the pool. Summary of the Invention

[0003] According to one aspect of the present disclosure, a method for controlling an automatic pool cleaning device is proposed, comprising: obtaining image information and depth information of underwater objects in a pool using an image sensor and a ranging sensor equipped with the automatic pool cleaning device; generating fused data based on the image information and the depth information; identifying the position and category of the underwater objects based on the fused data; and controlling the movement of the automatic pool cleaning device based on the position and category of the underwater objects.

[0004] According to at least one embodiment of the present disclosure, in the above method, controlling the movement of the automatic pool cleaning device includes one of the following: controlling the automatic pool cleaning device to move in a direction to avoid the underwater object, controlling the automatic pool cleaning device to continue moving along the original path, or controlling the automatic pool cleaning device to move toward the underwater object.

[0005] According to at least one embodiment of the present disclosure, in the above method, controlling the automatic pool cleaning device to move toward the underwater object includes: replanning a path for the automatic pool cleaning device based on the position of the underwater object; and controlling the automatic pool cleaning device to move along the replanned path.

[0006] According to at least one embodiment of the present disclosure, in the above method, the type of the underwater object includes: garbage or obstacles to be cleaned; when the type of the underwater object is identified as garbage to be cleaned, the automatic pool cleaning device is controlled to move along the original path or towards the underwater object; when the type of the underwater object is identified as an obstacle, the automatic pool cleaning device is controlled to move in a direction to avoid the underwater object.

[0007] According to at least one embodiment of the present disclosure, in the above method, controlling the automatic pool cleaning device to move in a direction to avoid the underwater object includes: controlling the automatic pool cleaning device to perform steering or U-turn to change the direction of travel to avoid the underwater object.

[0008] According to at least one embodiment of the present disclosure, the method further includes: after avoiding the underwater object, continuing to move toward the original moving target.

[0009] According to at least one embodiment of the present disclosure, the method further includes: controlling a rotation direction of the automatic pool cleaning device according to a position of the underwater object to control the automatic pool cleaning device to move toward or avoid the underwater object.

[0010] According to at least one embodiment of the present disclosure, in the above method, the ranging sensor is a TOF sensor, a laser radar, or a 3D structured light sensor.

[0011] According to at least one embodiment of the present disclosure, in the above method, the image information acquired by the image sensor and the depth information acquired by the ranging sensor correspond to the same underwater object.

[0012] According to at least one embodiment of the present disclosure, in the above method, generating the fused data includes: preprocessing the acquired image information and depth information; and fusing the preprocessed image information and depth information to generate the fused data.

[0013] According to at least one embodiment of the present disclosure, in the above method, the original path includes multiple first sub-paths and multiple second sub-paths, and the length of the first sub-path is greater than the length of the second sub-path.

[0014] According to another aspect of the present disclosure, an automatic pool cleaning device is also proposed, which includes: an image sensor for acquiring image information of underwater objects in the pool; a ranging sensor for acquiring depth information of the underwater objects; and a processor configured to cause the automatic pool cleaning device to perform the above method when executing one or more instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 The figure schematically shows the appearance of an automatic pool cleaning device according to an embodiment of the present disclosure.

[0017] Figure 2 The working environment of the automatic pool cleaning device according to an embodiment of the present disclosure is schematically shown.

[0018] Figure 3The flowchart of a control method for an automatic pool cleaning device according to an embodiment of the present disclosure is schematically shown.

[0019] Figures 4A-4C The diagram schematically shows a scene in which an automatic pool cleaning device operates in a rectangular pool.

[0020] Figure 5 The schematic structure of the automatic pool cleaning device for identifying underwater objects according to an embodiment of the present disclosure is shown.

[0021] Figure 6 The structure block diagram of the automatic pool cleaning device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0022] The detailed description set forth below, in conjunction with the accompanying drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details intended to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details.

[0023] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", "one side", "the other side", "front end", "rear end" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present disclosure.

[0024] Furthermore, terms such as "first," "second," and "third" that relate to order are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features being referred to. Thus, features defined with terms such as "first," "second," and "third" that relate to order may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0025] Furthermore, in the drawings, for clarity of illustration, dimensions may be exaggerated and not drawn to scale. Throughout the drawings, like reference numerals generally refer to like elements.

[0026] Figure 1The figure schematically shows the appearance of an automatic pool cleaning device 100 according to an embodiment of the present disclosure. The automatic pool cleaning device 100 can clean the bottom, walls, water and surface of a pool (e.g., a swimming pool) as needed, for example, to remove garbage in the water, on the bottom and on the surface of the water, and to clean dirt on the bottom and walls of the pool. Figure 1 As shown, the automatic pool cleaning device 100 may include structures / components such as a shell 110, a travel mechanism 120, and a cleaning unit 130. As an example, a control compartment, a power compartment, and a filter compartment (not shown) may be provided in the shell 100, wherein a control circuit such as a microprocessor, a digital signal processor (DSP), or a microcontroller may be installed in the control compartment, a driving mechanism such as a water pump or a drive motor may be provided in the power compartment, and a filter unit such as a filter basket may be provided in the filter compartment to filter and purify the water entering the filter compartment through the water inlet, filter out impurities therein, and discharge the cleaned water from the automatic pool cleaning device through the water outlet. In addition, the automatic pool cleaning device may also be equipped with a rechargeable battery as a power source to power components such as the drive motor, the water pump, and the control circuit. As an example, Figure 1 The traveling mechanism 120 of the automatic pool cleaning device 100 shown is a crawler-type traveling mechanism. However, the automatic pool cleaning device may also adopt a wheel-type traveling mechanism, which is not limited here.

[0027] As an example, the automatic pool cleaning device may also be equipped with a water spray mechanism, such as a water pump and an impeller, so that the automatic pool cleaning device can use the water spray mechanism to spray water outward from the water spray port to assist the automatic pool cleaning device to move on the pool wall, in the water and / or on the water surface; for example, the water spray mechanism can be used to spray water from a water spray port (e.g., a nozzle with a direction opposite to the direction of travel) in the direction of the spray port. Figure 1 Alternatively, when the automatic pool cleaning device is climbing the pool wall, the water jetting mechanism can be used to eject water from the water jetting port (e.g., the water jetting port 140) on the top of the body of the automatic pool cleaning device. Figure 1 The water flow sprayed from the water nozzle 150 as shown generates pressure applied to the bottom surface of the automatic pool cleaning device to improve the adhesion between the travel mechanism of the automatic pool cleaning device and the pool wall, thereby maintaining the stability of its body in a vertical state.

[0028] It should be noted that the position, shape and / or number of the water spray ports 140-150 provided on the housing 110 of the automatic pool cleaning device 100 may be adjusted accordingly according to the actual operational requirements of the automatic pool cleaning device and are not limited here.

[0029] although Figure 1The overall appearance of an automatic pool cleaning device according to an embodiment of the present disclosure is schematically shown. It should be understood that this is merely schematic and does not constitute any limitation to the principles of the present disclosure.

[0030] According to embodiments of the present disclosure, the automatic pool cleaning device may also be equipped with various sensors to enable various operations such as detecting the underwater environment, determining a travel route, and / or performing cleaning operations. For example, the automatic pool cleaning device according to embodiments of the present disclosure may be equipped with an image sensor for acquiring image information of objects surrounding the automatic pool cleaning device; it may also be equipped with a distance sensor for acquiring distance information between the automatic pool cleaning device and the objects.

[0031] As an example, the image sensor may include an RGB image sensor.

[0032] As an example, the above-mentioned ranging sensor may include but is not limited to at least one of the following: a lidar sensor, a TOF sensor, and a 3D structured light sensor.

[0033] According to an embodiment of the present disclosure, the automatic pool cleaning device may also be equipped with an inertial measurement unit (IMU) for obtaining attitude data of the automatic pool cleaning device. For example, the automatic pool cleaning device 100 may utilize the equipped inertial measurement unit to collect the acceleration and angular velocity values of the automatic pool cleaning device about the X, Y, and Z axes in the three-dimensional space of the pool, thereby obtaining attitude data such as the pitch angle, yaw angle, and / or roll angle of the automatic pool cleaning device.

[0034] In order to improve the cleaning efficiency and cleaning quality of the automatic pool cleaning device, the automatic pool cleaning device can be controlled to move along a set route so that it can perform obstacle avoidance, cleaning and other operations during the movement to achieve all-round cleaning of the pool.

[0035] However, when the automatic pool cleaning device operates underwater, the performance of the sensors it is equipped with is greatly affected by the underwater environment. For example, due to the complexity of the underwater environment, and taking into account factors such as water quality and / or light changes, water surface reflections, and dynamic interference (such as water flow, bubbles, and the movement of floating objects in the water), the performance of the sensors decreases. For example, the detection accuracy of ultrasonic sensors decreases in turbid water environments or when there are a large number of bubbles in the water, and traditional image sensors cannot obtain effective image information in insufficient light, especially in low-light environments or when working at night, and perform poorly, resulting in a significant reduction in the accuracy and / or stability of obtaining environmental information.

[0036] In addition, each type of sensor has its own advantages and disadvantages: for example, lidar can be used for high-precision ranging and generate dense point cloud data, which is suitable for medium and long-range environmental perception, but its blind spot is large at close range; ultrasonic sensors can measure ranging at close range with high accuracy and low cost, and are suitable for detecting close-range obstacles, but their measurement range is limited and they are easily affected by environmental noise; image sensors can provide rich texture and color information, which is suitable for target recognition and scene understanding, but they are greatly affected by lighting conditions and lack depth information.

[0037] To this end, according to the embodiments of the present disclosure, it is proposed that the deficiencies of a single sensor can be compensated by fusing data from multiple sensors. On the one hand, the robustness of underwater environment perception can be enhanced, and the perception capability and anti-interference capability can be improved. For example, stable perception capability can be maintained under different environmental conditions (such as lighting changes and occlusions). On the other hand, a richer and more comprehensive description of the underwater environment can be provided.

[0038] As an example, when the automatic pool cleaning device is moving in the pool, it can obtain image information of underwater objects through the image sensor it is equipped with, and obtain distance information of the same underwater object through the ranging sensor it is equipped with. Based on the data fusion of image information and distance information, the position and type of underwater objects can be better identified, so as to control the operation of the automatic pool cleaning device accordingly, so as to improve the operation quality and efficiency of the automatic pool cleaning device.

[0039] Figure 2 The working environment of the automatic pool cleaning device according to the embodiment of the present disclosure is schematically shown, such as a pool 200 such as a swimming pool. As an example, Figure 2 As shown, the pool 200 is rectangular and includes facilities such as a platform 230 and steps 240. The automatic pool cleaning device 100 can travel on the pool bottom 210, water surface 220, platform 230, steps 240, pool wall 250 and / or in the water of the pool 200, and perform cleaning operations during the travel process, clearing garbage floating in the water and on the water surface, and removing dirt on the surface of the pool bottom 210, pool wall 250, platform 230 and / or steps 240, so as to provide users with a clean and hygienic environment. It should be noted that although Figure 2 In the example shown, the pool 200 as a swimming pool is rectangular in shape. However, the principles of the present disclosure may also be applied to pools of circular, elliptical, L-shaped, kidney-shaped or other shapes, without limitation.

[0040] like Figure 2As shown, the underwater environment of the pool 200 is complex. For example, there are facilities such as a drain 260 and a floor lamp 270 on the pool bottom 210, and there may be garbage 280 such as fallen leaves, mud, sand, and stones in the water and / or on the pool bottom. For example, when the automatic pool cleaning device 100 moves underwater to clean the pool bottom, it needs to avoid underwater facilities such as the drain 260, the floor lamp 270, and / or the edge of the platform 230 and the steps 240 that are part of the pool wall, and clean up the garbage 280 such as fallen leaves, mud, sand, and stones during the movement.

[0041] Figure 3 The following schematically illustrates a process of a control method for an automatic pool cleaning device according to an embodiment of the present disclosure. Figure 3 As shown, the method 300 may include: S310, respectively acquiring image information and depth information of underwater objects in the pool through an image sensor and a ranging sensor equipped with the automatic pool cleaning device; S320, generating fusion data based on the image information and the depth information; S330, identifying the position and category of the underwater object based on the fusion data; and, S340, controlling the movement of the automatic pool cleaning device based on the position and category of the underwater object.

[0042] The control method for the automatic pool cleaning device according to the embodiment of the present disclosure is described in detail below with reference to specific examples.

[0043] It should be noted that according to the principles of this disclosure, when performing data fusion based on image information and depth information, the types of data fusion employed include, but are not limited to, pixel-level fusion, feature-level fusion, and decision-level fusion. The following uses feature-level fusion as an example to explain the principles of this disclosure in detail.

[0044] Figure 4A The schematic diagram shows a scene in which an automatic pool cleaning device operates in a rectangular pool. As an example, Figure 4A As shown, the automatic pool cleaning device 400 can travel on the bottom of the pool 410 to perform a cleaning operation on the pool bottom. As an example, the automatic pool cleaning device 400 can move along a set bow-shaped path (such as Figure 4A) is used to clean the bottom of the pool. During the movement of the automatic pool cleaning device 400, the image sensor it is equipped with can obtain image information of surrounding objects. As an example, the image sensor may include an RGB image sensor, thereby obtaining color images of surrounding objects. In addition, during the movement of the automatic pool cleaning device 400, the ranging sensor it is equipped with, such as a TOF sensor, can obtain depth information of the same underwater object; based on the image information obtained by the image sensor and the depth information obtained by the ranging sensor, the control device of the automatic pool cleaning device, such as a control unit such as a central processing unit (CPU), a microprocessor (MPU) or a digital signal processor (DSP) equipped with the automatic pool cleaning device, can perform data fusion on the image information and the depth information to generate fused data, and based on the generated fused data, identify the position and type of the underwater object, and then control the automatic pool cleaning device to perform corresponding operations. For example, if the underwater object is identified as garbage to be cleaned, the automatic pool cleaning device can be controlled to change the original path and move toward the location of the underwater object based on the relative position relationship between the underwater object and the automatic pool cleaning device so as to perform the cleaning operation as soon as possible, or if the underwater object is located on the original path, the automatic pool cleaning device can be controlled to continue moving along the original path until it reaches the location of the underwater object and performs the cleaning operation; if the underwater object is identified as an obstacle, an obstacle avoidance operation can be performed, for example, when the automatic pool cleaning device moves to a certain safe distance from the obstacle, the automatic pool cleaning device is controlled to turn or make a U-turn to avoid the obstacle.

[0045] As an example, the ranging sensor includes, but is not limited to, a TOF sensor, a lidar, or a 3D structured light sensor.

[0046] Figure 5 The principle architecture of the automatic pool cleaning device for recognizing underwater objects according to an embodiment of the present disclosure is shown. As an example, Figure 5 As shown, the automatic pool cleaning device can be equipped with an image sensor 510, such as a camera, to capture images of the surrounding underwater environment, thereby obtaining image information about underwater objects. For example, an RGB camera can be used to obtain color image information of underwater objects, or a grayscale camera can be used to obtain grayscale image information of underwater objects. In addition, a ranging sensor 520, such as a TOF (Time of Flight) sensor, a 3D structured light sensor, or a lidar, equipped with the automatic pool cleaning device can be used to obtain depth information of the same underwater object, and a depth image of the underwater object can be obtained based on the obtained depth information of the underwater object.

[0047] The image information and depth information can then be preprocessed. For example, the color image of the underwater object can be preprocessed 530 to extract object features from the color image, as shown in 550. Furthermore, the depth information of the underwater object can be preprocessed 540 to extract object features from the depth image, as shown in 560. The extracted object features include, but are not limited to, features such as the object's outline, shape, edge, texture, and position.

[0048] Furthermore, if Figure 5 As shown in 570 , feature matching and feature fusion can be performed.

[0049] As an example, the above pre-processing may include denoising; for example, Gaussian filtering or median filtering may be used to perform denoising, thereby improving the signal-to-noise ratio of image information.

[0050] As an example, preprocessing may also include image enhancement. For example, image enhancement may include image enhancement in the spatial domain and / or frequency domain. Spatial domain image enhancement directly processes the pixels of an image. For example, histogram equalization, Laplace sharpening, and the like may be used for spatial domain image enhancement. In addition, Fourier transform may be used to enhance the image in the frequency domain. For example, high-pass filtering may be used to achieve frequency domain enhancement.

[0051] It should be noted that although Figure 5 As shown, the image information acquired by the image sensor and the depth information acquired by the TOF sensor are preprocessed separately. However, the principles of the present disclosure are not limited thereto, and the image information and the depth information may also be preprocessed synchronously.

[0052] According to an embodiment of the present disclosure, based on the preprocessed image information and depth information, object features can be extracted through a deep learning model such as a convolutional neural network (CNN). As an example, the preprocessed image and depth information can be input into a convolutional neural network (CNN) to extract object features corresponding to the image information and depth information. Furthermore, through multi-layer convolution and pooling operations, object features of different scales and levels corresponding to the image information / depth information can be extracted.

[0053] According to an embodiment of the present disclosure, feature matching can be performed based on the extracted object features of different scales and levels, and object features extracted based on image information and object features extracted based on depth information can be fused using data fusion.

[0054] As mentioned above, the performance of a single sensor type has its own advantages and disadvantages, and is significantly affected by the complex underwater environment. Therefore, by acquiring underwater object features from different sensor types and performing feature matching and fusion, the shortcomings of a single sensor type can be overcome, allowing the automatic pool cleaning device to more accurately detect the type and location of underwater objects.

[0055] As an example, when performing feature fusion, weighted fusion or attention mechanism can be used to enhance important features, such as enhancing the edge features of the object, so as to better identify the outline, shape, size and other features of the underwater object, so as to determine the type and / or location of the object.

[0056] like Figure 5 As further shown, data fusion is performed on the features of underwater objects extracted from image information and depth information respectively, and as shown in FIG. Figure 5 As shown in 580 , the type and location of underwater objects can be identified based on data fusion, and as shown in 590 , the travel path of the automatic pool cleaning device can be planned to better control the underwater operation of the automatic pool cleaning device and improve its operation efficiency and quality.

[0057] For example, the automatic pool cleaning device can determine whether the object is garbage to be cleaned or an obstacle to be avoided based on the recognition result of data fusion, and perform corresponding operations. Figure 4A Provide further explanation.

[0058] As an example, the automatic pool cleaning device 400 can travel along the bow-shaped planned path shown in dotted lines. Figure 4A As shown, the path includes a plurality of first sub-paths 4505 and a plurality of second sub-paths 4510 , wherein the length of the first sub-path 4505 is greater than the length of the second sub-path 4510 .

[0059] As another example, Figure 4B As shown, the automatic pool cleaning device 400 can travel along the zigzag planning path shown by the dotted line. Figure 4B As shown, the path includes a plurality of first sub-paths 4505 and a plurality of second sub-paths 4510 , wherein the length of the first sub-path 4505 is greater than the length of the second sub-path 4510 .

[0060] As another example, Figure 4C As shown, the automatic pool cleaning device 400 can travel along the U-shaped planning path shown by the dotted line. Figure 4C As shown, the path includes a plurality of first sub-paths 4505 and a plurality of second sub-paths 4510 , wherein the length of the first sub-path 4505 is greater than the length of the second sub-path 4510 .

[0061] It should be noted that the planned travel path of the automatic pool cleaning device described herein in conjunction with the accompanying drawings is only for the purpose of facilitating a better understanding of the principles of the present disclosure, and its specific type is not limited herein.

[0062] As an example, the automatic pool cleaning device can obtain image data about the surrounding environment through an equipped image sensor, such as an RGB camera or a grayscale camera, for example, it can obtain image information about underwater objects around it.

[0063] In addition, the automatic pool cleaning device can obtain depth information about surrounding underwater objects by being equipped with depth sensors, such as lidar, 3D structured light sensor, TOF sensor, such as DTOF (direct time of flight sensor) and / or ITOF (indirect time of flight sensor).

[0064] As an example, the underwater objects may include, but are not limited to, garbage such as fallen leaves, sand, paper, dirt, and stones, or obstacles such as pool walls, drains, platform edges, steps, ladders, wall lamps, and floor lamps.

[0065] As an example, a control device of an automatic pool cleaning device, such as a control unit such as a central processing unit (CPU), a microprocessor (MPU), a graphics processing unit (GPU) or a digital signal processor (DSP), performs preprocessing such as denoising and image enhancement on image information and depth information, and performs operations such as feature extraction, feature matching and feature fusion based on the preprocessed image information and depth information, thereby generating information including, for example, the outline, shape, texture, and position of an underwater object, and determining the type of the underwater object and its position relative to the automatic pool cleaning device, such as identifying whether the underwater object is garbage to be cleaned or an obstacle to be avoided, as well as the distance between the object and the automatic pool cleaning device and the relative position relationship between them.

[0066] According to an embodiment of the present disclosure, feature matching can be performed on object features extracted based on depth information obtained from a TOF sensor and object features extracted based on image information obtained from an image sensor; for example, geometric features of underwater objects, such as edge features of obstacles and / or garbage, can be extracted from the depth information obtained from the TOF sensor, and corresponding edge features can be extracted from the image information obtained from the image sensor, and the two can be matched, for example, the edge features of the object represented by the depth information are aligned with the edge features of the object represented in the image information and data fusion is performed to generate fused features of the object.

[0067] As an example, when performing data fusion, weighted fusion can be performed, where the weights can be set based on prior knowledge or adaptively adjusted. For example, based on indicators such as the accuracy and signal-to-noise ratio of information obtained by image sensors and ranging sensors, higher weights can be assigned to information obtained by sensors with higher accuracy and / or signal-to-noise ratio. For another example, the optimal weights can be determined by minimizing an error function (such as the mean square error) of the data fusion results, thereby adaptively adjusting the weights.

[0068] In addition, when performing data fusion, an attention mechanism can also be used for data fusion. For example, for important information about underwater objects (for example, the edge contours of underwater objects), the accuracy and robustness of the fusion results can be improved by dynamically allocating weights to focus on key areas or features such as edge contours.

[0069] Based on the results of object feature fusion, the type of underwater object can be determined, for example, whether the object is an obstacle or garbage to be cleaned. After the type of underwater object is determined based on the fusion results, the automatic pool cleaning device can be controlled to perform corresponding actions based on the type of object and its location. For example, if the object is determined to be an obstacle, the automatic pool cleaning device can be controlled to perform obstacle avoidance when it reaches a certain distance from the obstacle, such as 5-10 cm. As a result, the automatic pool cleaning device can be controlled to perform a turn or U-turn to change the direction of travel, thereby avoiding the obstacle.

[0070] For example, continue to refer to Figure 4A When the automatic pool cleaning device 400 identifies the front object 4401 as a pool bottom drain based on the feature fusion result of the image information and the depth information, it can turn to change the direction of travel when it reaches a certain threshold distance from the drain, for example, along Figure 4A Move in the direction indicated by arrow 1, avoid the drain outlet and continue to move forward to prevent the automatic pool cleaning device from being stuck due to the adsorption force generated by the drain outlet when it moves to the location of the drain outlet.

[0071] As another example, when the automatic pool cleaning device 400 identifies the front object 4402 as a floor lamp installed at the bottom of the pool based on the feature fusion result of the image information and the depth information, it can turn to change the direction of travel when it reaches a certain threshold distance from the floor lamp, for example, along Figure 4A Move in the direction indicated by arrow 2, avoid the ground lamp and continue to move forward to prevent the automatic pool cleaning device from being stuck at the ground lamp due to the bottom of its body contacting the ground lamp protruding from the bottom surface of the pool when it moves to the location of the ground lamp, causing it to be suspended and trapped.

[0072] As another example, when the automatic pool cleaning device 400 identifies the front object as the pool wall through the feature fusion result based on image information and depth information, for example, when it determines that the front object is the edge 4403 of the platform 420 that is part of the pool wall, it can make a U-turn or turn to change the direction of travel when it reaches a certain threshold distance from the pool wall, for example, along Figure 4A The vehicle moves in the direction of arrow 3 to continue cleaning the pool bottom along the originally set path.

[0073] As an example, after avoiding an obstacle, the automatic pool cleaning device may continue to move along the originally set path, for example, may continue to move toward the original travel target.

[0074] According to an embodiment of the present disclosure, when it is determined that the type of object is garbage to be cleaned based on the analysis of image information, the automatic pool cleaning device can be controlled to perform corresponding actions based on the relative position relationship between the object and the automatic pool cleaning device; for example, if the garbage is near the original travel path of the automatic pool cleaning device and the distance from the automatic pool cleaning device is less than a certain threshold distance, the travel path of the automatic pool cleaning device can be replanned so as to move towards the object to clean up the garbage.

[0075] As an example, Figure 4A As shown, when the automatic pool cleaning device recognizes that object 4404 belongs to garbage to be cleaned (such as fallen leaves, dirt, algae) based on the feature fusion result of image information and depth information, the control device of the automatic pool cleaning device, for example, a control unit such as a central processing unit (CPU), a microprocessor (MPU), a graphics processing unit (GPU) or a digital signal processor (DSP) equipped with the automatic pool cleaning device, can first determine whether object 4404 is on the travel path of the machine. If not, the travel path can be replanned, for example, along Figure 4A The re-planned travel path shown by the arrow 4 moves toward the object 4404 identified as garbage to complete the garbage cleaning operation, and after completing the garbage cleaning, continues to move along the originally planned path (such as the bow-shaped path) to traverse the entire bottom of the pool and clean the bottom of the pool in all directions.

[0076] As another example, when it is determined that the type of object is garbage to be cleaned, the automatic pool cleaning device can be controlled to perform a corresponding action based on the relative position relationship between the object and the automatic pool cleaning device; for example, if the distance between the garbage and the automatic pool cleaning device is not less than a certain threshold distance, the automatic pool cleaning device can be controlled to continue to move along the originally set path. Figure 4AAs shown, when the automatic pool cleaning device 400 travels to position P1, the object 4410 is identified as garbage to be cleaned based on the feature fusion result of the image information and the depth information, and it is determined that the distance between the object and the automatic pool cleaning device is greater than the threshold distance and is in the original travel path planned by the automatic pool cleaning device (i.e., Figure 4A When the automatic pool cleaning device is on the previously set path (for example, along the arched path shown by the dotted line), the automatic pool cleaning device can be controlled to continue along the originally set path (for example, along the arched path shown by the dotted line). Figure 4A ) until it moves along the originally set travel route to the vicinity of the object 4410, that is, the distance value to the object 4410 is not greater than the threshold distance, for example, Figure 4A At the position P2 shown, the object 4410 identified as garbage to be cleaned can be cleaned; that is, Figure 4A As shown, the object 4410 is on the travel path of the machine, and the machine can directly move along the original travel path to clean the object 4410.

[0077] According to an embodiment of the present disclosure, the rotation direction of the automatic pool cleaning device can be controlled based on the position of an underwater object, for example, based on the relative positional relationship between the underwater object and the automatic pool cleaning device. Furthermore, based on whether the object is garbage to be cleaned or an obstacle to be avoided, the automatic pool cleaning device can be controlled to move toward the object to be cleaned or avoid the obstacle. By way of example, controlling the rotation direction of the automatic pool cleaning device may include, but is not limited to, adjusting the wheel speed difference of the travel mechanism on both sides of the automatic pool cleaning device's body; and / or adjusting the thrust difference generated by the water spray mechanism equipped with the automatic pool cleaning device on both sides of the body.

[0078] Therefore, according to the embodiment of the present disclosure, the automatic pool cleaning device obtains image information of underwater objects through an image sensor, and obtains depth information of the object through a ranging sensor, determines the type and location of the underwater object based on the feature fusion of the image information and the depth information, and can perform corresponding operations based on the type and location of the underwater object. Therefore, the embodiment of the present disclosure, by combining visual information and depth information, makes up for the shortcomings of a single type of sensor, reduces the interference of complex underwater environments on sensor data, improves the real-time and reliability of object recognition, and makes the automatic pool cleaning device more accurate in identifying the type and location of underwater objects. In addition, ranging sensors, such as TOF sensors, do not rely on external light sources and can provide stable depth information in the dark or low-light environments, making up for the shortcomings of camera systems such as image sensors, so that the automatic pool cleaning device can quickly and accurately identify the type and location of obstacles in low-light environments, thereby being able to perform efficient cleaning work in different environments.

[0079] According to another aspect of the present disclosure, an automatic pool cleaning device is also provided. Figure 6 As shown, the automatic pool cleaning device 600 includes: an image sensor 610, which obtains image information of underwater objects in the pool; a ranging sensor 620, which obtains depth information of the underwater objects; and a processor 630, which is configured to enable the automatic pool cleaning device to perform the above method when executing one or more instructions.

[0080] As an example, the number, type, and installation location of image sensors, ranging sensors, and processors are not limited here.

[0081] For example, the image sensor may include at least one of the following: a monocular camera, a binocular camera, a multi-camera, or a panoramic camera. The image sensor may also include an RGB camera or a grayscale camera.

[0082] Through the equipped camera, the automatic pool cleaning device can obtain image information about surrounding objects, for example, it can obtain still images and / or video frames of surrounding objects, thereby obtaining information about the object such as edge, texture, size, color, etc.

[0083] As an example, the ranging sensor includes at least one of the following: a TOF sensor, a laser radar, or a 3D structured light sensor, to obtain depth information about surrounding objects.

[0084] For example, a distance measuring sensor and / or an image sensor may be installed on the front, rear, side, and / or top of the body of the automatic pool cleaning device to obtain depth information and image information of surrounding underwater objects.

[0085] As an example, the automatic pool cleaning device may further include an inertial measurement unit (IMU) to collect posture data of the automatic pool cleaning device, whereby the control device of the automatic pool cleaning device may control the moving posture of the automatic pool cleaning device based on the posture data.

[0086] According to the above-mentioned method and automatic pool cleaning device of the embodiments of the present disclosure, by obtaining image information and corresponding depth information of objects around the automatic pool cleaning device, the type and position of the object can be determined, and based on the type and position of the object, corresponding actions such as obstacle avoidance, cleaning operation and / or continuing to move are performed, thereby improving the flexibility of the automatic pool cleaning device in performing operations, and can dynamically adjust and optimize the operating process of the automatic pool cleaning device, thereby improving operating efficiency.

[0087] Thus, several aspects of the present disclosure are presented above with reference to various devices and methods. These devices and methods are illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or software depends on the specific application and the design constraints on the overall system.

[0088] Thus, in one or more example embodiments, the functions described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functions may be stored or encoded as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media. The storage medium may be any available medium that can be accessed by a computer. It should be understood that the specific order or hierarchy of blocks in the disclosed process / flowcharts is illustrative of example methods. Based on design preferences, it should be understood that the specific order or hierarchy of blocks in the process / flowcharts may be rearranged. In addition, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in a sample order and are not meant to be limited to the specific order or hierarchy presented.

[0089] The embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to perform equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of this application.

Claims

1. A method for controlling an automatic pool cleaning device, comprising: Acquire image information and depth information of underwater objects in the pool respectively through the image sensor and distance measuring sensor equipped with the automatic pool cleaning device; generating fused data based on the image information and the depth information; Based on the fused data, identifying the location and category of the underwater object; The automatic pool cleaning device is controlled to move based on the location and category of the underwater object.

2. The method according to claim 1, wherein Controlling the automatic pool cleaning device to move includes one of the following: controlling the automatic pool cleaning device to move in a direction to avoid the underwater object, controlling the automatic pool cleaning device to continue moving along the original path, or controlling the automatic pool cleaning device to move toward the underwater object.

3. The method according to claim 2, wherein: Controlling the automatic pool cleaning device to move toward the underwater object includes: replanning a path for the automated pool cleaning device based on the location of the underwater object; and The automatic pool cleaning device is controlled to move along the re-planned path.

4. The method according to claim 2, wherein: The types of underwater objects include: garbage or obstacles to be cleared; When the type of the underwater object is identified as garbage to be cleaned, controlling the automatic pool cleaning device to move along the original path or towards the underwater object; When the type of the underwater object is identified as an obstacle, the automatic pool cleaning device is controlled to move in a direction to avoid the underwater object.

5. The method according to claim 4, wherein The controlling the automatic pool cleaning device to move in a direction to avoid the underwater object comprises: The automatic pool cleaning device is controlled to perform steering or U-turn to change the direction of travel and avoid the underwater object.

6. The method according to claim 4, wherein: The method further comprises: After avoiding the underwater object, continue moving toward the original moving target.

7. The method according to claim 4, further comprising: The rotation direction of the automatic pool cleaning device is controlled according to the position of the underwater object, so as to control the automatic pool cleaning device to move toward the underwater object or avoid the underwater object.

8. The method according to any one of claims 1 to 7, wherein: The distance measuring sensor is a TOF sensor, a laser radar or a 3D structured light sensor.

9. The method according to claim 1, wherein The image information acquired by the image sensor and the depth information acquired by the ranging sensor correspond to the same underwater object.

10. The method according to claim 1, wherein Generating the fused data includes: Preprocessing the acquired image information and depth information; The pre-processed image information and depth information are fused to generate the fused data.

11. The method according to claim 1, wherein The original path includes a plurality of first sub-paths and a plurality of second sub-paths, and the length of the first sub-path is greater than the length of the second sub-path.

12. An automatic pool cleaning device comprising: An image sensor, which acquires image information of underwater objects in the pool; A ranging sensor for acquiring depth information of the underwater object; The processor is configured to cause the automatic pool cleaning device to perform the method according to any one of claims 1 to 11 when executing one or more instructions.

Images