Automatic station returning method of pool robot, pool robot and storage medium

Abstract

The invention provides an automatic station returning method of a pool robot, the pool robot and a computer readable storage medium, and relates to the field of robots. The automatic station returning method comprises the steps that an environment image of the environment where the pool robot is located is collected, wherein the environment image comprises a base station device; detecting a plurality of feature mark points of the base station device in the environment image; determining two-dimensional coordinate information of the plurality of feature mark points in the environment image; establishing a three-dimensional coordinate system according to the plurality of feature mark points, and determining three-dimensional coordinate information of the plurality of feature mark points in the three-dimensional coordinate system; determining pose information of the base station device relative to the pool robot based on the two-dimensional coordinate information and the three-dimensional coordinate information corresponding to the plurality of feature mark points and imaging parameters of the pool robot; and controlling the pool robot to move towards the base station device according to the pose information. According to the invention, the pool robot is accurately controlled to move towards the base station device for charging, and automatic cruise charging of the pool robot is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, in particular to an automatic back-to-base method of a pool robot, a pool robot and a computer readable storage medium. BACKGROUND

[0002] The pool robot can clean the pool bottom and pool wall in the underwater environment. In the related art, when the pool robot needs to be charged, the pool robot is generally controlled to move to a preset position and move to a base station device for charging according to a planned route, or the pool robot can collect pool image information and identify the position of the base station device according to the pool image information. However, the pool robot generally performs underwater operation, and due to the influence of water flow in the underwater environment, the features are not obvious enough, which leads to the inability to accurately identify the position of the base station device, that is, the pool robot in the related art cannot accurately identify and move to the position of the base station device for charging, and the intelligent degree is low. SUMMARY

[0003] The main purpose of the present application is to provide an automatic back-to-base method of a pool robot, a pool robot and a computer readable storage medium, which aims to accurately control the pool robot to move towards the base station device for charging, and realizes the automatic cruising charging of the pool robot.

[0004] In a first aspect, the present application provides an automatic back-to-base method of a pool robot, the automatic back-to-base method comprising: collecting an environment image of an environment in which the pool robot is located, the environment image containing a base station device; detecting a plurality of feature marker points of the base station device in the environment image; determining two-dimensional coordinate information of the plurality of feature marker points in the environment image; establishing a three-dimensional coordinate system according to the plurality of feature marker points, and determining three-dimensional coordinate information of the plurality of feature marker points in the three-dimensional coordinate system; determining pose information of the base station device relative to the pool robot based on two-dimensional coordinate information corresponding to the plurality of feature marker points, the three-dimensional coordinate information and imaging parameters of the pool robot; controlling the pool robot to move towards the base station device according to the pose information.

[0005] In a second aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the automatic back-to-base method of the pool robot as described above.

[0006] In a third aspect, the present application provides a pool robot, comprising: a robot body; a moving assembly, disposed on the robot body, for driving the robot body to move; a cleaning assembly, disposed on the robot body, for cleaning the pool bottom and / or pool wall; a vision assembly, disposed on the robot body, for collecting images; a controller for executing the automatic back-to-base method of the pool robot as described above.

[0007] The present application provides an automatic back-to-base method of a pool robot, a pool robot and a computer readable storage medium. An environment image of an environment where the pool robot is located is collected. The environment image contains a base station device. A plurality of feature marker points of the base station device in the environment image are detected. Two-dimensional coordinate information of the plurality of feature marker points in the environment image is determined. A three-dimensional coordinate system is established according to the plurality of feature marker points, and three-dimensional coordinate information of the plurality of feature marker points in the three-dimensional coordinate system is determined. Based on the two-dimensional coordinate information, the three-dimensional coordinate information corresponding to the plurality of feature marker points, and imaging parameters of the pool robot, pose information of the base station device relative to the pool robot is determined. According to the pose information, the pool robot is controlled to move towards the base station device. Thus, the plurality of feature marker points of the base station device can be detected to accurately determine the pose information of the base station device relative to the pool robot, so that the pool robot can be accurately controlled to move towards the base station device for charging. When the pool robot is low on power, the pool robot can be accurately positioned to the position of the base station device under the underwater environment and automatically cruise for charging without manually controlling the pool robot for charging. The intelligent degree of the pool robot is improved, and the cost and implementation difficulty are low. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0009] Figure 1 A step flowchart of an automatic back-to-base method of a pool robot provided by an embodiment of the present application; Figure 2 An environment image containing lamp beads provided by an embodiment of the present application; Figure 3 An environment image containing reflective markers provided by an embodiment of the present application; Figure 4 Another environment image containing reflective markers provided by an embodiment of the present application; Figure 5 This application provides a schematic diagram of the steps for determining whether a robot is in a water tank, as an embodiment of the present application. Figure 6 This application provides an embodiment of an environmental image schematic diagram for detecting multiple first feature markers of a base station device in an environmental image; Figure 7 This application provides an embodiment of an environmental image schematic diagram for detecting multiple second feature markers of a base station device in an environmental image; Figure 8 A flowchart illustrating the steps for determining whether a pool robot needs to re-collect environmental images of its environment, as provided in an embodiment of this application; Figure 9 A flowchart illustrating the steps of another automatic return-to-station method for a pool robot provided in an embodiment of this application; Figure 10 A schematic block diagram of a water tank cleaning system provided in one embodiment of this application; Figure 11 This is a schematic diagram of the structure of a pool robot provided in one embodiment of this application; Figure 12 This is a schematic diagram of the structure of a base station device provided in one embodiment of this application; Figure 13 This is a schematic diagram of another base station device provided in an embodiment of this application; Figure 14 This is a schematic block diagram of a pool robot provided in one embodiment of this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0012] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0013] Please refer toFigure 1 , Figure 1 This is a flowchart illustrating an automatic return-to-station method for a pool robot, provided as an embodiment of this application.

[0014] This automatic return-to-base method for the pool robot can be applied to pool robots, terminals, or servers. By detecting multiple feature markers on the base station device, the pose information of the base station device relative to the pool robot can be accurately determined. This allows for precise control of the pool robot to move toward the base station device for charging. When the pool robot's battery is low, it can accurately locate the base station device in the underwater environment and automatically cruise to charge, eliminating the need for manual control of the pool robot for charging. This improves the intelligence level of the pool robot and has low cost and low implementation difficulty.

[0015] Among them, the pool robot can be a robot capable of cleaning a pool; the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these; the server can be a standalone server, a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0016] For example, the pool robot may include a walking motor for driving the robot to move within the pool and climb walls. The base station device has the function of placing the pool robot, and in some base station devices, it also has the function of charging the pool robot.

[0017] like Figure 1 As shown, the automatic return method of the pool robot includes steps S101 to S106.

[0018] S101. Collect environmental images of the environment in which the water tank robot is located. The environmental images include base station devices.

[0019] Specifically, the pool robot can be equipped with a camera device to capture environmental images of the environment in which the pool robot operates. The camera device is typically positioned at the front of the robot's body, and the pool robot's working environment is generally underwater. In this embodiment, the camera device is generally capable of capturing environmental images that include a base station device.

[0020] For example, the shooting device may include a camera, video camera, mobile phone, or other device capable of capturing environmental images.

[0021] For example, during the operation or movement of the pool robot, an imaging device can continuously collect environmental images corresponding to the robot's surroundings.

[0022] For example, the pool robot can control the orientation of the shooting device so that the shooting device can collect corresponding environmental images in different orientations. The pool robot can filter the environmental images collected in different orientations, thereby retaining the environmental images containing the base station device for subsequent feature marker detection and extraction processing.

[0023] For example, the pool robot can control the camera to collect corresponding environmental images in the east, south, west and north directions. If the pool robot determines that the environmental image collected by the camera in the north direction contains a base station device, it will retain the environmental image collected in the north direction and filter out the other environmental images.

[0024] Specifically, the battery level of the water tank robot is obtained. If the battery level of the water tank robot is detected to be lower than a preset battery threshold, the automatic return method of the water tank robot provided in this application is executed.

[0025] The preset battery threshold can be any battery level, such as 10%, and can be set according to actual conditions. No specific restrictions are made here.

[0026] Since the pool robot typically only performs the automatic return-to-base operation when its power is low, the power level of the pool robot can be obtained and compared with a preset power threshold. When the power level of the pool robot is lower than the preset power threshold, the automatic return-to-base method for the pool robot provided in this application can be executed.

[0027] For example, if the battery level of the water tank robot is 5% and the preset battery threshold is 10%, it can be determined that the battery level of the water tank robot is lower than the preset battery threshold, and then the water tank robot is controlled to automatically return to the station.

[0028] Specifically, the working status of the pool robot is obtained. If the working status of the pool robot is detected to be idle, the automatic return method of the pool robot provided in this application is executed.

[0029] Since base station devices generally also have the function of placing water tank robots, the water tank robot can also execute the automatic return method of the water tank robot provided in this application when it is in an idle state, so that the water tank robot returns to the base station device.

[0030] S102. Detect multiple feature markers of the base station device in the environmental image.

[0031] The base station device includes multiple feature markers used for positioning. These feature markers may include a first feature marker and a second feature marker, which have different application scenarios. In this embodiment, the first feature marker is the feature point corresponding to a first marker, which may be an LED bead; the second feature marker is the feature point corresponding to a second marker, which may be a reflective marker or a QR code.

[0032] like Figure 2 As shown, since the base station device is generally located on the pool wall area, and the pool robot is located at the bottom of the pool, the distance between the pool robot and the pool wall is relatively far, i.e., the distance between the pool robot and the base station device is relatively far. Therefore, it is difficult to detect the reflective mark or QR code of the base station device from the environmental image. However, the LED beads will emit light, so it is relatively easy to detect the LED beads of the base station device from the environmental image. Therefore, when the pool robot is located at the bottom of the pool, the feature points corresponding to the LED beads can be detected from the environmental image.

[0033] At the same time, such as Figure 3 and Figure 4 As shown, the pool robot is located on the pool wall and is climbing the wall to get closer to the base station device. Because the distance between the pool robot and the base station device is relatively close, not all the LED beads may be visible in the environmental image. This will result in lower accuracy of the two-dimensional and three-dimensional coordinate information of the feature marker points. Therefore, when the pool robot is located on the pool wall, it can detect the feature points corresponding to reflective marks or QR codes that are distributed at a closer distance from the environmental image.

[0034] It should be noted that the base station device may be equipped with only the first marker or only the second marker. Preferably, the base station device may be equipped with both the first marker and the second marker, so that the location of the base station device can be accurately located in different scenarios.

[0035] In some embodiments, the position of the pool robot in the pool is determined; if the pool robot is determined to be at the bottom of the pool, multiple first feature markers of the base station device in the environmental image are detected; if the pool robot is determined to be on the pool wall, multiple second feature markers of the base station device in the environmental image are detected. Thus, based on the position of the pool robot in the pool, corresponding feature markers in the environmental image can be detected, thereby enabling more accurate identification of the two-dimensional and three-dimensional coordinate information of the feature markers.

[0036] The location of the pool robot in the pool can generally include the bottom and the walls of the pool, and the pool robot mainly cleans the bottom and the walls of the pool.

[0037] like Figure 5As shown, for example, step S102 may include steps S1021 to S1023.

[0038] S1021. Determine the position of the robot in the pool.

[0039] S1022. If it is determined that the robot in the pool is at the bottom of the pool, then detect multiple first feature markers of the base station device in the environmental image.

[0040] S1023. If it is determined that the robot in the pool is on the pool wall, then detect multiple second feature markers of the base station device in the environmental image.

[0041] For example, the position of the pool robot in the pool can be determined based on one or more of the posture information, pressure information, and task information. If it is determined that the pool robot is at the bottom of the pool, multiple first feature markers of the base station device in the environmental image are detected. If it is determined that the pool robot is on the pool wall, multiple second feature markers of the base station device in the environmental image are detected.

[0042] In this embodiment, the first feature marker is a feature point corresponding to a first marker, which is an LED bead; the second feature marker is a feature point corresponding to a second marker, which is a reflective mark or a QR code. Figure 2 As shown, since the base station device is generally located on the wall of the pool, and the pool robot is located at the bottom of the pool, the distance between the pool robot and the pool wall is relatively far, that is, the distance between the pool robot and the base station device is relatively far. Therefore, it is difficult to detect the reflective mark or QR code of the base station device from the environmental image. However, the LED beads will emit light, so it is relatively easy to detect the LED beads of the base station device from the environmental image. Therefore, when the pool robot is located at the bottom of the pool, the feature points corresponding to the LED beads can be detected from the environmental image.

[0043] At the same time, such as Figure 3 and Figure 4 As shown, the pool robot is located on the pool wall and is climbing the wall to get closer to the base station device. Because the distance between the pool robot and the base station device is relatively close, not all the LED beads may be visible in the environmental image. This will result in lower accuracy of the two-dimensional and three-dimensional coordinate information of the feature marker points. Therefore, when the pool robot is located on the pool wall, it can detect the feature points corresponding to reflective marks or QR codes that are distributed at a closer distance from the environmental image.

[0044] Specifically, after the robot in the pool collects environmental images, a keypoint detection model can be used to detect targets within the images, thereby identifying the locations of the base station device and markers (first or second markers) within the environmental image. The keypoint detection model outputs a bounding box (enclosing the base station device) and multiple feature markers. like Figure 6 As shown, taking the example of a robot in a pool located at the bottom of a pool, the robot will use a key point detection model to perform target detection on the environmental image, thereby detecting the area corresponding to the base station device. Figure 6 (the rectangle in the middle) and the first marker ( Figure 6 (The LED beads in the lamp).

[0045] like Figure 7 As shown, taking the example of a robot positioned on the wall of a pool, the robot will use a key point detection model to perform target detection on the environmental image, thereby detecting the area corresponding to the base station device. Figure 7 (the rectangle in the middle) and the second marker ( Figure 7 (The circular pieces in the middle).

[0046] For example, the task information of the pool robot can be obtained. If the task information of the pool robot is to clean the bottom of the pool, it can be determined that the pool robot is at the bottom of the pool, and multiple first feature markers of the base station device in the environmental image can be detected. If the task information of the pool robot is to clean the pool wall, it can be determined that the pool robot is at the pool wall, and multiple second feature markers of the base station device in the environmental image can be detected.

[0047] In some embodiments, the posture information of the pool robot in the pool is acquired; based on the posture information, the position of the pool robot in the pool is determined. Thus, the acquired posture information can be used to accurately determine the position of the pool robot in the pool.

[0048] Specifically, a pool robot can be equipped with an inertial measurement unit (IMU), which is a device that measures the three-axis attitude angles (or angular rates) and acceleration of an object. Therefore, the pool robot can obtain the attitude information of the pool robot in the pool through the inertial measurement unit.

[0049] Since the posture information of a robot in a pool is generally different at the bottom and the walls of the pool, the position of the robot in the pool can be accurately determined by the posture information collected by the inertial measurement unit.

[0050] For example, if the attitude information of the pool robot collected by the inertial measurement unit is a first IMU value, it can be determined that the pool robot is located at the bottom of the pool; if the attitude information of the pool robot collected by the inertial measurement unit is a second IMU value, it can be determined that the pool robot is located on the pool wall, wherein the first IMU value and the second IMU value are different.

[0051] In some embodiments, pressure information of the pool robot in the pool is obtained; based on the pressure information, the position of the pool robot in the pool is determined.

[0052] Specifically, the pool robot can be equipped with a pressure sensor, which is a device that can sense pressure signals and convert them into electrical signals. Therefore, the pool robot can obtain pressure information in the pool through the pressure sensor.

[0053] Because the forces acting on the robot body at the bottom and the walls of the pool are significantly different, the pressure information corresponding to the robot at the bottom and the walls of the pool is generally different. Therefore, the position of the robot in the pool can be accurately determined by the pressure information collected by the pressure sensor.

[0054] For example, if the pressure information of the pool robot collected by the pressure sensor is a first pressure value, it can be determined that the pool robot is located at the bottom of the pool; if the pressure information of the pool robot collected by the pressure sensor is a second pressure value, it can be determined that the pool robot is located on the pool wall, wherein the first pressure value is greater than the second pressure value.

[0055] In some embodiments, after detecting a plurality of second feature markers of the base station device in the environmental image, the method further includes: if no plurality of second feature markers are detected in the environmental image, or if the detected second feature markers do not meet preset conditions, then detecting a plurality of first feature markers in the environmental image that meet preset conditions.

[0056] Specifically, since it is determined that the robot is on the pool wall, it is necessary to detect multiple second feature markers of the base station device in the environmental image. If a preset number of second feature markers are not detected in the environmental image, then multiple first feature markers that meet the preset conditions in the environmental image need to be detected.

[0057] The preset quantity can be any number, such as 4, and can be set according to the actual situation. No specific limit is made here.

[0058] For example, taking a preset quantity of four as an example, if at least four second feature markers cannot be detected in the environmental image due to reasons such as the second marker (reflective marker or QR code) falling off or becoming invalid, it indicates that the number of detected second feature markers is insufficient, which may lead to lower accuracy in the subsequent identification of the two-dimensional and three-dimensional coordinate information of the feature markers. In order to improve the accuracy of the subsequent determination of the two-dimensional and three-dimensional coordinate information of the feature markers, it is necessary to re-detect multiple first feature markers in the environmental image that meet the preset conditions.

[0059] Specifically, since it is determined that the robot is on the pool wall, it is necessary to detect multiple second feature markers of the base station device in the environmental image. If the detected second feature markers do not meet the preset conditions, then multiple first feature markers that meet the preset conditions in the environmental image are detected.

[0060] The preset condition is that the number of non-collinear second feature markers among multiple second feature markers is greater than a preset number. The preset number can be any number, such as 4, and can be set according to the actual situation; no specific limit is made here.

[0061] For example, taking a preset quantity of four, if multiple second feature markers are collinear, it is impossible to construct a spatial pattern using these markers, thus failing to accurately establish a three-dimensional coordinate system and determine the three-dimensional coordinate information. Therefore, at least four non-collinear second feature markers must be detected to meet the condition of constructing a spatial pattern using multiple feature markers. To improve the accuracy of subsequently determining the two-dimensional and three-dimensional coordinate information of the feature markers, if the number of non-collinear second feature markers detected is not greater than the preset quantity, it is necessary to re-detect multiple first feature markers in the environmental image that meet the preset conditions.

[0062] It should be noted that multiple first feature markers meeting the preset conditions can be defined as follows: the number of non-collinear first feature markers among the multiple first feature markers is greater than a preset number. The preset number can be any number, such as four, and can be set according to actual circumstances; no specific limitation is made here.

[0063] In some embodiments, after detecting multiple feature markers of the base station device in the environmental image, the method further includes: detecting whether the area ratio of the base station device in the environmental image is greater than a preset ratio threshold, and detecting whether the number of non-collinear feature markers among the multiple feature markers is greater than a preset number; if the area ratio of the base station device in the environmental image is not greater than the preset ratio threshold, or if the number of non-collinear feature markers among the multiple feature markers is not greater than the preset number, then the environmental image of the environment where the pool robot is located is re-acquired.

[0064] The preset quantity can be any number, such as 4, and can be set according to actual needs; no specific limitation is made here. The preset ratio threshold can be any area percentage, such as 0.005, and can be set according to actual needs; no specific limitation is made here.

[0065] like Figure 8 As shown, for example, after step S102, steps S201 to S203 may also be included.

[0066] S201. Detect whether the area ratio of the base station device in the environmental image is greater than the preset ratio threshold.

[0067] S202. Detect whether the number of non-collinear feature markers among multiple feature markers is greater than the preset number.

[0068] S203. If the area ratio of the base station device in the environmental image is not greater than the preset ratio threshold, or if the number of non-collinear feature markers among multiple feature markers is not greater than the preset number, then the environmental image of the environment where the pool robot is located is re-acquired.

[0069] If the area of ​​the base station device in the environmental image is not greater than a preset threshold, it indicates that the area of ​​the base station device in the environmental image is small, making it generally difficult to accurately identify feature markers and determine their two-dimensional and three-dimensional coordinates. Therefore, it is necessary to re-acquire environmental images of the environment where the robot is located in the pool to obtain images where the area of ​​the base station device is greater than the preset threshold.

[0070] If multiple feature markers are collinear, it will be impossible to construct a spatial pattern using these markers, thus making it impossible to accurately establish a 3D coordinate system and determine 3D coordinate information. Therefore, if the number of non-collinear feature markers detected is not greater than a preset number, it is necessary to re-acquire environmental images of the pool robot's environment.

[0071] S103. Determine the two-dimensional coordinate information of multiple feature marker points in the environmental image.

[0072] Two-dimensional coordinate information is used to represent the positions of multiple feature marker points in the environmental image.

[0073] In some embodiments, a local image region corresponding to the feature marker point is extracted from the environmental image; pixels whose pixel values ​​meet preset conditions are selected within the local image region; the centroid position of the pixels that meet the preset conditions is calculated, and the centroid position is used as the two-dimensional coordinate information of the corresponding feature marker point. This ensures that the calculated two-dimensional coordinate information of the feature marker point in the environmental image is at the center of the marker.

[0074] For example, for any feature marker point, a local image region is extracted from the environmental image with the feature marker point as the center. The size of the local image region can be set according to the size of the detection frame corresponding to the base station device.

[0075] Specifically, the pixels that meet the preset conditions can be: pixels whose pixel value is greater than the preset pixel value threshold, or the top N pixels with the largest pixel value, where N is a positive integer greater than or equal to 1.

[0076] The preset pixel value threshold can be any pixel value and can be set according to the actual situation; no specific limitation is made here.

[0077] For example, for any feature marker, a local image region corresponding to the feature marker can be extracted from the environmental image. The local image region may include multiple pixels. It is determined whether the pixel values ​​of the multiple pixels in the local image region are greater than a preset pixel value threshold, and the pixels that are greater than the preset pixel value threshold are taken as pixels that meet the preset conditions.

[0078] For example, for any feature marker, a local image region corresponding to the feature marker can be extracted from the environmental image. The local image region may include multiple pixels. The pixel values ​​of the multiple pixels in the local image region are determined and sorted from largest to smallest. The top 10 pixels with the largest pixel values ​​are selected as pixels that meet the preset conditions, or the top 1% of pixels in the sorting are selected as pixels that meet the preset conditions.

[0079] Specifically, after obtaining pixels that meet the preset conditions, the centroid positions of these pixels can be calculated, and the calculated centroid positions are used as the two-dimensional coordinate information of the corresponding feature marker points. This ensures that the calculated two-dimensional coordinate information of the feature marker points in the environmental image is at the center of the marker.

[0080] S104. Establish a three-dimensional coordinate system based on multiple feature marker points, and determine the three-dimensional coordinate information of the multiple feature marker points in the three-dimensional coordinate system.

[0081] Establishing a three-dimensional coordinate system is used to determine the spatial relationship between multiple feature marker points. The three-dimensional coordinate information is used to represent the positions of multiple feature marker points in the three-dimensional coordinate system.

[0082] In some embodiments, the center position of the spatial pattern formed by multiple feature markers is determined and set as the origin of the three-dimensional coordinate system; the coordinate axis direction of the three-dimensional coordinate system is determined according to the spatial positional relationship of the multiple feature markers relative to the origin.

[0083] For example, since multiple feature markers are non-collinear, the multiple feature markers can be connected to form a spatial pattern, and the center position can be set as the origin of the three-dimensional coordinate system. Then, the coordinate axis directions of the three-dimensional coordinate system can be determined according to the spatial positional relationship of the multiple feature markers relative to the origin, thereby establishing the three-dimensional coordinate system.

[0084] Specifically, after establishing a three-dimensional coordinate system, the projection points of each feature marker point onto the coordinate axes of each three-dimensional coordinate system can be determined. Based on these projection points, the three-dimensional coordinate information of each feature marker point in the three-dimensional coordinate system can be determined. Therefore, a three-dimensional coordinate system can be established based on multiple feature marker points, and the three-dimensional coordinate information of multiple feature marker points in the three-dimensional coordinate system can be accurately calculated.

[0085] S105. Based on the two-dimensional coordinate information, three-dimensional coordinate information and imaging parameters of the pool robot corresponding to multiple feature marker points, determine the pose information of the base station device relative to the pool robot.

[0086] The imaging parameters of the pool robot can include the intrinsic parameters of the imaging device within the pool robot. The pose information of the base station device relative to the pool robot refers to the relative position and orientation relationship between the base station device and the imaging device within the pool robot.

[0087] Specifically, based on the PnP-RANSAC algorithm, the two-dimensional coordinate information, three-dimensional coordinate information, and intrinsic parameters of the imaging device in the pool robot corresponding to multiple feature marker points can be used as input information to accurately calculate the pose information of the base station device relative to the pool robot.

[0088] Taking multiple feature markers as the first feature marker as an example, since the first marker corresponding to the first feature marker is an LED bead, the PnP-RANSAC algorithm can be used to take the two-dimensional coordinate information, three-dimensional coordinate information, and intrinsic parameters of the camera device in the pool robot as input information, so as to accurately calculate the pose information of the base station device relative to the pool robot.

[0089] Taking multiple feature markers as the second feature markers as an example, since the second markers corresponding to the second feature markers are reflective markers or QR codes, the two-dimensional coordinate information, three-dimensional coordinate information, and intrinsic parameters of the shooting device in the pool robot can be used as input information based on the PnP-RANSAC algorithm to accurately calculate the pose information of the base station device relative to the pool robot.

[0090] S106. Based on the pose information, control the pool robot to move toward the base station device.

[0091] Specifically, by determining the pose information of the base station device relative to the pool robot, it is equivalent to determining the position information of the base station device relative to the pool robot. Therefore, the pool robot can be controlled to move towards the base station device so that the pool robot and the base station device can cooperate with each other, thereby enabling the base station device to transmit electrical energy to the pool robot to charge it.

[0092] likeFigure 9 As shown, the following is combined Figure 9 The overall process of the automatic return-to-station method for the pool robot provided in this application is introduced.

[0093] S301. Collect environmental images of the environment in which the robot is located in the pool.

[0094] S302. Determine the position of the robot in the pool; if it is at the bottom of the pool, proceed to S304; if it is on the pool wall, proceed to S303.

[0095] S303. Detect multiple second feature markers of the base station device in the environmental image and determine whether the second feature markers meet the preset conditions; if yes, proceed to S305; if no, proceed to S304. S304. Detect multiple first feature markers of the base station device in the environmental image and determine whether the first feature markers meet the preset conditions; if yes, proceed to S305; if no, proceed to S306. S305. Calculate the pose information of the base station device relative to the pool robot, so as to control the pool robot to move toward the base station device.

[0096] S306. Reacquire environmental images of the environment in which the robot is located in the pool.

[0097] This application embodiment can accurately determine the pose information of the base station device relative to the pool robot by detecting multiple feature marker points of the base station device. This enables accurate control of the pool robot to move toward the base station device for charging. When the pool robot's battery is low, it can accurately locate the position of the base station device in the underwater environment and automatically cruise to charge without the need for manual control of the pool robot for charging. This improves the intelligence level of the pool robot and has lower cost and implementation difficulty.

[0098] After introducing the automatic return method of the pool robot provided in this application, the pool cleaning system provided in this application will be introduced.

[0099] Please refer to Figure 10 , Figure 10 A schematic block diagram of a water tank cleaning system provided for an embodiment of this application.

[0100] like Figure 10 As shown, this application provides a pool cleaning system 1000, which includes a pool robot 100 and a base station device 200.

[0101] like Figure 11As shown in the figure, this application embodiment also provides a pool robot 100, which includes a robot body 10, a moving component 20, a cleaning component 30, a vision component 40, and a controller (not shown). The moving component 20 is used to drive the robot body 10 to move; the cleaning component 30 is disposed on the robot body 10 and is used to clean the bottom and / or walls of the pool; the controller is connected to the moving component 20 and the cleaning component 30.

[0102] For example, the pool robot 100 provided in this application embodiment can be used to perform cleaning operations on the bottom and / or walls of a pool during movement, but it is not limited to this. For example, the cleaning component 30 is disposed at the bottom of the robot body 10 for cleaning the bottom and / or walls of the pool. The moving component 20 is disposed on the robot body 10 for moving the robot body 10, so that the robot body 10 can drive the cleaning component 30 to clean the pool along a preset trajectory, thereby greatly reducing manual operation, saving time and effort, and truly freeing people from the labor of pool maintenance.

[0103] For example, the vision component 40 is disposed on the robot body 10. The vision component 40 can be a shooting device used to collect environmental images corresponding to the environment in which the robot is located in the pool.

[0104] In this embodiment, the base station device 200 is provided with multiple feature markers, which are used for positioning the base station device 200. The base station device 200 can be used to house the pool robot 100. For example, the base station device 200 can be installed on the bank, bottom wall or pool wall of the pool, and the base station device 200 can be set in the underwater area or the water surface area, without specific limitation.

[0105] Taking the base station device 200 installed on the wall of the pool as an example, the pool robot 100 can move to the charging area of ​​the base station device 200 so that the pool robot 100 and the base station device 200 can cooperate with each other, thereby enabling the base station device 200 to transmit electrical energy to the pool robot 100 to charge the pool robot 100.

[0106] The following describes the base station device provided in this application embodiment, taking the example of the base station device being installed on the bank of a pool. Since the base station device provided in this application embodiment is installed on the bank of a pool, it can also be called an off-water base station.

[0107] like Figure 12 and Figure 13As shown, the base station device 200 includes a base station body 210, a lifting component 220, a positioning component 230, and a connecting component 240. The base station body 210 is used to be set on the bank of the pool. The lifting component 220 is movably connected to the base station body 210. The connecting component 240 is set on the lifting component 220 and is used to connect with the pool robot 100. The positioning component 230 is set on the lifting component 220, and multiple markers are at least partially set on the positioning component 230.

[0108] For example, the base station body 210 is fixedly installed on the bank of the pool, and the lifting component 220 is movably connected to the base station body 210. The lifting component 220 is used to drag the pool robot 100 from the pool onto the bank, so that the pool robot 100 leaves the water surface; or, the lifting component 220 is also used to send the pool robot 100, which is in the bank state, into the pool so that the pool robot 100 can enter the water.

[0109] like Figure 12 As shown, when the lifting component 220 is in the first position, it is located above the base station body 210, meaning the lifting component 220 is in the landed state. If the pool robot 100 is located inside the receiving cavity of the lifting component 220, the pool robot is also in the landed state.

[0110] like Figure 13 As shown, when the lifting assembly 220 is in the second position, it is at least partially submerged below the water surface. For example, the lifting assembly 220 is attached to the pool wall to allow the pool robot 100 located within the receiving cavity to enter the water, or for the pool robot 100 within the pool to enter the lifting assembly 220. In other words, the second position is the position where the lifting assembly 220 is attached to the pool wall or other position where it is at least partially submerged, allowing the pool robot 100 to move from the pool wall to the lifting assembly 220, or to enter or exit the water from the lifting assembly 220.

[0111] Optionally, the lifting component 220 is an expandable and retractable structure. The lifting component 220 has an expanded position and a retracted position. When the lifting component 220 is in the retracted position, it is located on the base station body 210. When the lifting component 220 is in the expanded position, it extends at least partially below the water surface of the pool.

[0112] For example, the positioning component 230 is disposed on the lifting component 220, and multiple markers are at least partially disposed on the positioning component 230. When the positioning component 230 is underwater, the pool robot 100 can acquire environmental images containing the base station device 200 through a camera during operation or movement, so that the pool robot 100 can move to the position corresponding to the positioning component 230 through the above-mentioned automatic return method of the pool robot. After the pool robot 100 moves to the corresponding position, the pool robot 100 in the pool will dock with the lifting component 220. After docking, the lifting component 220 will drag the pool robot 100 from the pool to the shore, so that the pool robot 100 leaves the water surface and is placed on the base station body 210.

[0113] For example, the connecting component 240 is disposed on the lifting component 220 for connection with the pool robot 100. After the pool robot 100 moves to the position corresponding to the positioning component 230, the pool robot 100 will connect with the connecting component 240, thereby realizing the limiting cooperation between the pool robot 100 and the lifting component 220, so that the lifting component 220 can drag the pool robot 100 from the pool onto the shore to leave the water surface, thereby placing the pool robot 100 on the base station body 210.

[0114] For example, the base station device 200 may also include a charging component. When the lifting component 220 is in the first position and the pool robot 100 is disposed in the receiving cavity of the lifting component 220, the pool robot 100 can cooperate with the charging component, thereby enabling the charging component to transmit electrical energy to the pool robot 100 to charge the pool robot 100.

[0115] In some embodiments, the plurality of markers includes a plurality of first markers and a plurality of second markers, wherein the plurality of first markers are disposed on the positioning component 230 and the plurality of second markers are disposed on the connecting component 240.

[0116] The first and second markers have different application scenarios. In this embodiment, the feature point corresponding to the first marker is the first feature marker point in the above embodiment, and the first marker can be an LED bead. The feature point corresponding to the second marker is the second feature marker point in the above embodiment, and the second marker can be a reflective mark or a QR code.

[0117] For example, since several first markers are set on the positioning component 230, if it is determined that the pool robot is at the bottom of the pool, multiple first feature markers corresponding to several first markers in the environmental image containing the positioning component 230 can be detected so that the pool robot 100 can move to the position corresponding to the positioning component 230 through the above-mentioned automatic return method of the pool robot. After the pool robot 100 moves to the corresponding position, the pool robot 100 in the pool will dock with the lifting component 220. After docking, the lifting component 220 will drag the pool robot 100 from the pool to the shore, so that the pool robot 100 leaves the water surface and is placed on the base station body 210.

[0118] For example, since several second markers are set on the connection component 240, if it is determined that the pool robot is on the pool wall, multiple second feature markers corresponding to several second markers in the environmental image containing the connection component 240 can be detected so that the pool robot 100 can move to the position corresponding to the positioning component 230 through the above-mentioned automatic return method of the pool robot. After the pool robot 100 moves to the corresponding position, the pool robot 100 in the pool will dock with the lifting component 220. After docking, the lifting component 220 will drag the pool robot 100 from the pool to the shore, so that the pool robot 100 leaves the water surface and is placed on the base station body 210.

[0119] like Figure 13 As shown, in some embodiments, the positioning component 230 is tilted when the lifting component 220 moves it to the position in the pool where it docks with the pool robot.

[0120] Since several first markers are set on the positioning component 230, and these first markers are generally light-emitting markers such as LED beads, by setting the positioning component 230 at an angle, the field of view of the light emitted by the LED beads can be increased. This allows the pool robot 100 to determine and acquire an environmental image containing the base station device 200 from a greater distance using the light emitted by the LED beads. This enables the pool robot 100 to move to the position corresponding to the positioning component 230 from a greater distance using the automatic return-to-base method provided in this application. If the positioning component 230 is set perpendicular to the water surface, the field of view of the light emitted by the LED beads is smaller, and the pool robot 100 can only determine and acquire an environmental image containing the base station device 200 using the light emitted by the LED beads within its effective position range.

[0121] Please see Figure 14 , Figure 14 This is a schematic block diagram of the structure of a pool robot 100 provided in an embodiment of this application.Figure 14 In the pool robot 100, there are a processor 110 and a memory 120. The processor 110 and the memory 120 are connected by a bus, which can be any applicable bus such as I2C (Inter-integrated Circuit) bus.

[0122] The memory 120 may include a storage medium and internal memory. The storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause the processor to perform the automatic return-to-station method for the pool robot described in any embodiment.

[0123] The processor 110 provides computing and control capabilities to support the operation of the entire robot 100.

[0124] The processor 110 can be a Central Processing Unit (CPU), but it can also be a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or it can be any conventional processor.

[0125] The processor 110 is used to run a computer program stored in the memory 120, and performs the following steps when executing the computer program: An environmental image of the pool robot's environment, including a base station device, is acquired. Multiple feature markers of the base station device in the environmental image are detected. Two-dimensional coordinate information of the multiple feature markers in the environmental image is determined. A three-dimensional coordinate system is established based on the multiple feature markers, and the three-dimensional coordinate information of the multiple feature markers in the three-dimensional coordinate system is determined. Based on the two-dimensional coordinate information, the three-dimensional coordinate information, and the imaging parameters of the pool robot, the pose information of the base station device relative to the pool robot is determined. Based on the pose information, the pool robot is controlled to move towards the base station device.

[0126] In some embodiments, the processor 110, in implementing the detection of multiple feature markers of the base station device in the environmental image, is used to: The location of the pool robot in the pool is determined; if the pool robot is determined to be at the bottom of the pool, multiple first feature markers of the base station device in the environmental image are detected; if the pool robot is determined to be on the pool wall, multiple second feature markers of the base station device in the environmental image are detected.

[0127] In some embodiments, when determining the position of the pool robot in the pool, the processor 110 is configured to: Obtain the posture information of the pool robot in the pool; determine the position of the pool robot in the pool based on the posture information.

[0128] In some embodiments, when determining the position of the pool robot in the pool, the processor 110 is configured to: Obtain the pressure information of the pool robot in the pool; determine the position of the pool robot in the pool based on the pressure information.

[0129] In some embodiments, after detecting a plurality of second feature markers of a base station device in the environmental image, the processor 110 is further configured to: If the plurality of second feature markers are not detected in the environmental image, or if the detected second feature markers do not meet the preset conditions, then the plurality of first feature markers that meet the preset conditions in the environmental image are detected.

[0130] In some embodiments, after detecting multiple feature markers of the base station device in the environmental image, the processor 110 is further configured to: The system detects whether the area ratio of the base station device in the environmental image is greater than a preset ratio threshold, and whether the number of non-collinear feature markers among the multiple feature markers is greater than a preset number. If the area ratio of the base station device in the environmental image is not greater than the preset threshold, or if the number of non-collinear feature markers among the multiple feature markers is not greater than the preset number, then the environmental image of the environment where the pool robot is located is re-acquired.

[0131] In some embodiments, when determining the two-dimensional coordinate information of the plurality of feature marker points in the environmental image, the processor 110 is configured to: A local image region corresponding to the feature marker point is extracted from the environmental image; pixels whose pixel values ​​meet preset conditions are selected within the local image region; the centroid position of the pixels that meet the preset conditions is calculated, and the centroid position is used as the two-dimensional coordinate information of the corresponding feature marker point.

[0132] In some embodiments, when the processor 110 establishes a three-dimensional coordinate system based on the plurality of feature marker points, it is configured to: The center position of the spatial pattern formed by the plurality of feature markers is determined, and the center position is set as the origin of the three-dimensional coordinate system; the coordinate axis direction of the three-dimensional coordinate system is determined according to the spatial positional relationship of the plurality of feature markers relative to the origin.

[0133] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions. A processor executes these instructions to implement the automatic return-to-station method for the pool robot provided in any of the embodiments of this application. For example, when the computer program is loaded by a processor, it can perform the following steps: An environmental image of the pool robot's environment, including a base station device, is acquired. Multiple feature markers of the base station device in the environmental image are detected. Two-dimensional coordinate information of the multiple feature markers in the environmental image is determined. A three-dimensional coordinate system is established based on the multiple feature markers, and the three-dimensional coordinate information of the multiple feature markers in the three-dimensional coordinate system is determined. Based on the two-dimensional coordinate information, the three-dimensional coordinate information, and the imaging parameters of the pool robot, the pose information of the base station device relative to the pool robot is determined. Based on the pose information, the pool robot is controlled to move towards the base station device.

[0134] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0135] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0136] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0137] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An automatic return-to-station method for a pool robot, characterized in that, The automatic return method includes: Collect environmental images of the environment in which the water tank robot is located, and the environmental images include base station devices; Detect multiple feature markers of the base station device in the environmental image; Determine the two-dimensional coordinate information of the plurality of feature markers in the environmental image; A three-dimensional coordinate system is established based on the plurality of feature marker points, and the three-dimensional coordinate information of the plurality of feature marker points in the three-dimensional coordinate system is determined; Based on the two-dimensional coordinate information and the three-dimensional coordinate information corresponding to the multiple feature marker points, as well as the imaging parameters of the pool robot, the pose information of the base station device relative to the pool robot is determined. Based on the pose information, the water tank robot is controlled to move toward the base station device.

2. The automatic return-to-station method according to claim 1, characterized in that, The detection of multiple feature markers of the base station device in the environmental image includes: Determine the position of the robot in the pool; If it is determined that the pool robot is at the bottom of the pool, then multiple first feature markers of the base station device in the environmental image are detected; If it is determined that the pool robot is on the pool wall, then multiple second feature markers of the base station device in the environmental image are detected.

3. The automatic return-to-station method according to claim 2, characterized in that, Determining the position of the robot in the pool includes: Obtain the posture information of the pool robot in the pool; Based on the posture information, the position of the pool robot in the pool is determined.

4. The automatic return-to-station method according to claim 2, characterized in that, Determining the position of the robot in the pool includes: Obtain the pressure information of the water tank robot in the water tank; Based on the pressure information, the position of the pool robot in the pool is determined.

5. The automatic return-to-station method according to claim 2, characterized in that, After detecting multiple second feature markers of the base station device in the environmental image, the method further includes: If the plurality of second feature markers are not detected in the environmental image, or if the detected second feature markers do not meet the preset conditions, then the plurality of first feature markers that meet the preset conditions in the environmental image are detected.

6. The automatic return-to-station method according to claim 1, characterized in that, After detecting multiple feature markers of the base station device in the environmental image, the method further includes: The system detects whether the area ratio of the base station device in the environmental image is greater than a preset ratio threshold, and whether the number of non-collinear feature markers among the multiple feature markers is greater than a preset number. If the area ratio of the base station device in the environmental image is not greater than a preset threshold, or if the number of non-collinear feature markers among the multiple feature markers is not greater than a preset number, then the environmental image of the environment where the pool robot is located is re-acquired.

7. The automatic return-to-station method according to claim 1, characterized in that, Determining the two-dimensional coordinate information of the plurality of feature marker points in the environmental image includes: Extract a local image region corresponding to the feature marker points from the environmental image; Within the local image area, select pixels whose pixel values ​​meet preset conditions; The centroid position of the pixel that meets the preset conditions is calculated, and the centroid position is used as the two-dimensional coordinate information of the corresponding feature marker point.

8. The automatic return-to-station method according to claim 1, characterized in that, The step of establishing a three-dimensional coordinate system based on the plurality of feature marker points includes: Determine the center position of the spatial pattern formed by the plurality of feature marker points, and set the center position as the origin of the three-dimensional coordinate system; The coordinate axis directions of the three-dimensional coordinate system are determined based on the spatial positional relationship of the multiple feature marker points relative to the origin.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the automatic return-to-station method for the pool robot as described in any one of claims 1-8.

10. A pool robot, characterized in that, include: Robot body; A mobile component, disposed on the robot body, is used to drive the robot body to move; A cleaning component, mounted on the robot body, is used to clean the bottom and / or walls of the pool. A vision component, mounted on the robot body, is used to acquire images; A controller for performing the automatic return-to-station method for a pool robot as described in any one of claims 1-8.

11. A water tank cleaning system, characterized in that, It includes a base station device and a pool robot as described in claim 10, wherein the base station device is provided with a plurality of markers.

12. The water tank cleaning system according to claim 11, characterized in that, The base station device includes a base station body, a lifting component, a positioning component, and a connecting component. The base station body is used to be set on the bank of the pool. The lifting component is movably connected to the base station body. The connecting component is set on the lifting component and is used to connect with the pool robot. The positioning component is set on the lifting component. The plurality of markers are at least partially set on the positioning component.

13. The pool cleaning system according to claim 12, characterized in that, When the lifting component moves the positioning component to the position in the pool where it docks with the pool robot, the positioning component is tilted.

14. The water tank cleaning system according to claim 12, characterized in that, The plurality of feature markers include a plurality of first markers and a plurality of second markers, wherein the plurality of first feature markers are disposed on the positioning component and the plurality of second markers are disposed on the connecting component.

15. The pool cleaning system according to claim 14, characterized in that, The first marker is an LED bead, and the second marker is a QR code or a reflective disc.

Images