Recharging method for mobile robot, household appliance, and computer readable storage medium
By taking base station images and determining the motion path, the problem of low recharge efficiency caused by large base station depth is solved, and efficient recharge in the case of deep base station position is achieved.
Patent Information
- Application Number
- PCT/CN2024/081722
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-06
- Filing Date
- 2024-03-14
- Publication Date
- 2024-11-14
AI Technical Summary
In the prior art, the mobile robot recharge method based on infrared sensors and scanning code positioning is less efficient when the base station has a large depth. Due to the limitations of the signal range and viewing angle range, it is difficult to accurately return to the base station for charging.
By taking a base station image at a first reference position by the mobile robot, determining the base station profile position information using an edge feature extraction algorithm, calculating the first motion path, and controlling the robot to move along the path to the second reference position to perform the back-charge action to avoid signal range and limits on viewing angle range.
The recharge efficiency and accuracy of the mobile robot when the base station is deep is improved, ensuring that the robot can return to the base station quickly and accurately for charging.
Smart Images

Figure CN2024081722_14112024_PF_FP_ABST
Abstract
Description
Mobile robot recharging method, household appliance, and computer-readable storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This disclosure claims priority to Chinese patent application number 202310504549.3, filed on May 6, 2023, entitled “Recharging method for mobile robot, household appliance and computer-readable storage medium,” the entire contents of which are incorporated by reference into this disclosure. Technical Field
[0003] The present disclosure relates to the technical field of mobile robots, and in particular to a recharging method for a mobile robot, a household appliance, and a computer-readable storage medium. Background Art
[0004] With the development of science and technology, mobile robots have been widely used in social life, bringing convenience to people's lives. For example, the most common mobile robots are sweeping robots in homes, food delivery robots in hotels, and shopping guide robots in supermarkets.
[0005] For mobile robots, autonomously returning to the base station for recharging is a critical technology. Related technologies include methods such as infrared sensors and code scanning for positioning to achieve recharging of mobile robots. However, some mobile robots are large in size or, due to functional requirements, require base stations to be installed at a greater depth. In this case, the infrared emission angle of the recharging method based on infrared sensors will be very limited. Similarly, the greater depth of the base station will also affect the field of view angle of the code scanning positioning method. It can be seen that for base stations with greater depth, the recharging methods based on infrared sensors and code scanning for positioning in related technologies are less efficient.
[0006] Public content
[0007] The present disclosure aims to at least partially address one of the technical issues in the related art. To this end, one objective of the present disclosure is to provide a recharging method for a mobile robot that can determine a path back to the base station based on a base station image captured of the base station, thereby reaching a reference position for recharging the robot, without being restricted by signal range or viewing angle, thereby improving recharging efficiency.
[0008] The second objective of the present disclosure is to provide a charging device for a mobile robot.
[0009] The third objective of the present disclosure is to provide a household appliance.
[0010] A fourth object of the present disclosure is to provide a computer-readable storage medium.
[0011] To achieve the above objectives, the first embodiment of the present disclosure provides a recharging method for a mobile robot. The recharging method comprises: obtaining a base station image captured by a camera at a first reference position of the mobile robot; wherein the base station image includes a base station outline; determining a first motion path based on the position information of the base station outline in the base station image; and controlling the mobile robot to perform recharging along the first motion path.
[0012] In some embodiments of the present disclosure, controlling the mobile robot to move along the first motion path to perform recharging includes: controlling the mobile robot to move along the first motion path to a second reference position preset for the base station to perform recharging at the second reference position; wherein, the mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.
[0013] In some embodiments of the present disclosure, determining a first motion path based on the position information of the base station outline in the base station image includes: using the base station image to determine first depth information from the base station to the mobile robot; determining a first motion orientation of the first motion path according to the position information of the base station outline in the base station image; and determining the first motion path according to the first depth information and the first motion orientation.
[0014] In some embodiments of the present disclosure, before determining the first motion path based on the first depth information and the first motion orientation, the method also includes: determining the obstacle situation on the first motion path; determining the first motion path based on the first depth information and the first motion orientation, including: when there is no obstacle on the first motion path, determining the movement distance along the first motion orientation in a straight line based on the first depth information and the first motion orientation to obtain the first motion path; or when there is an obstacle on the first motion path, determining the first motion path based on the position information of the obstacle, the first depth information, and the first motion orientation.
[0015] In some embodiments of the present disclosure, the base station contour of the base station image is determined by: performing edge feature extraction on several first objects to be verified in the base station image to obtain contour features corresponding to the several first objects to be verified; among the contour features corresponding to the several first objects to be verified, determining a target contour feature that matches the base station contour feature preset for the base station; wherein the contour of the object to be verified corresponding to the target contour feature is the base station contour.
[0016] In some embodiments of the present disclosure, before obtaining the base station image captured by the mobile robot through the camera at the first reference position, the method further includes: controlling the mobile robot to move to the first reference position preset for the target object; wherein the position of the target object and the base station are relatively fixed.
[0017] In some embodiments of the present disclosure, controlling the mobile robot to move to a first reference position preset for a target object includes: if an object contour of the target object is extracted based on an image of the working environment of the mobile robot, determining a second motion path according to position information of the object contour in the working environment image; and controlling the mobile robot to move to the first reference position along the second motion path.
[0018] In some embodiments of the present disclosure, controlling the mobile robot to move along the second motion path to the first reference position includes: in the process of controlling the mobile robot to move along the second motion path, capturing an object environment image for the target object; wherein the object environment image includes a target object outline; adjusting the second motion path based on position information of the target object outline in the object environment image and second depth information from the target object to the mobile robot, and controlling the mobile robot to move along the adjusted second motion path to reach the first reference position.
[0019] In some embodiments of the present disclosure, before controlling the mobile robot to move to a first reference position preset for a target object, the method further includes: if an object contour of the target object is extracted based on an image of the working environment of the mobile robot, recording the coordinate position of the working environment image; controlling the mobile robot to move to the first reference position preset for the target object includes: controlling the mobile robot to move to the coordinate position; obtaining an object environment image obtained by photographing the target object at the coordinate position; wherein the object environment image includes a target object contour; determining a third motion path based on position information of the target object contour in the object environment image; and controlling the mobile robot to move to the first reference position along the third motion path.
[0020] In some embodiments of the present disclosure, extracting the object contour of a target object based on an image of a working environment of a mobile robot includes: performing edge feature extraction on several second objects to be verified in the working environment image to obtain contour features corresponding to the several second objects to be verified; and determining that the object contour of the target object is extracted based on the working environment image if, among the contour features corresponding to the several second objects to be verified, there is any one second object to be verified whose contour features match the object contour features preset for the target object.
[0021] In some embodiments of the present disclosure, the base station is set at the bottom of the target object or as a part of the target object, and the target object includes household appliances and furniture.
[0022] In some embodiments of the present disclosure, the household appliance includes a clothes treating appliance.
[0023] To achieve the above objectives, a second embodiment of the present disclosure provides a recharging device for a mobile robot. The recharging device comprises: an image acquisition module for acquiring a base station image captured by a camera of the mobile robot at a first reference position; wherein the base station image includes a base station outline; a path determination module for determining a first motion path based on the positional information of the base station outline in the base station image; and a recharging module for controlling the mobile robot to perform recharging along the first motion path.
[0024] To achieve the above objectives, a third aspect of the present disclosure provides a mobile robot. The mobile robot includes a memory, a processor, and a mobile robot recharge program stored in the memory and executable on the processor. When the processor executes the program, the mobile robot recharge method described in any one of the first aspects is implemented.
[0025] In some embodiments of the present disclosure, the base station corresponding to the mobile robot is set at the bottom of the target object or as a part of the target object, and the target object includes household appliances and furniture.
[0026] In some embodiments of the present disclosure, the household appliance includes a clothes treating appliance.
[0027] To achieve the above-mentioned purpose, a fourth embodiment of the present disclosure provides a computer-readable storage medium having a mobile robot recharging program stored thereon, which, when executed by a processor, implements the mobile robot recharging method as described in any one of the first aspects.
[0028] Through the above embodiment, the mobile robot is controlled to capture a base station image at a first reference position, and then a first motion path back to the base station is determined based on the position information of the base station outline in the base station image, and recharging is performed along the first motion path. The recharging method of the disclosed embodiment, which uses a camera to capture images, is not limited by signal range and viewing angle range, and improves the efficiency of the mobile robot's recharging when the base station is located deep inside.
[0029] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0031] FIG1 is a scene example diagram of the recharging method of a mobile robot according to an embodiment of the present disclosure.
[0032] FIG2 is a flow chart of a recharging method for a mobile robot according to an embodiment of the present disclosure.
[0033] FIG3 is a flow chart of a recharging method for a mobile robot according to an embodiment of the present disclosure.
[0034] FIG4 is a flowchart of a recharging method for a mobile robot according to another embodiment of the present disclosure.
[0035] FIG5 is a structural block diagram of a recharging device for a mobile robot according to an embodiment of the present disclosure.
[0036] FIG6 is a structural block diagram of a mobile robot according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0037] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.
[0038] Currently, mobile robots are widely used in people's social lives, bringing great convenience to people's lives. However, due to the large size or functional requirements of some mobile robots, the depth of their base stations is relatively large. For example, the sweeping robots in home life. In order to minimize manual cleaning work, some current sweeping robots have dual functions of sweeping and mopping. The base station of a sweeping robot that sweeps and mops all in one needs to provide not only charging functions, but also water supply and cleaning functions. Therefore, the base station of a sweeping robot that sweeps and mops all in one may be connected to the water pipes in the home and installed at the bottom of the home appliance or sink equipment. Its depth is large and can accommodate a large area or the entire body of the sweeping robot.
[0039] In the recharging method based on infrared sensors, it is necessary to arrange infrared sensors in the base station to emit infrared rays to the outside. However, when the depth of the base station is large, the emission angle of the infrared sensor will be very limited. Similarly, the large depth of the base station will also affect the field of view angle of the code scanning positioning method. It can be seen that for base stations with a large depth, the recharging method based on infrared sensors and code scanning positioning in the related art is less efficient. Therefore, in response to the above situation, the present disclosure proposes a recharging method for a mobile robot, a household appliance and a computer-readable storage medium, which can determine the path back to the base station based on the base station image taken of the base station, so as to reach the reference position of the return station to perform the recharging action, without being restricted by the signal range and the viewing angle, thereby improving the recharging efficiency.
[0040] The embodiment of the present disclosure provides a scenario example of a recharging method for a mobile robot, taking the example of a base station of a sweeping and mopping robot being set at the bottom of a washing machine. The recharging method of the mobile robot is applied to the mobile robot (sweeping robot) shown in Figure 1. Referring to Figure 1, the scenario example includes a mobile robot 110, a washing machine 120, and a base station 130 set at the bottom of the washing machine 120. Among them, the position of the base station 130 relative to the washing machine 120 is fixed. A monocular camera can be configured on the mobile robot 110 to shoot the surrounding environment. If the mobile robot 110 needs to return to the base station for charging or cleaning, it can first move to a first reference position preset for the washing machine 120, and then shoot the base station 130 at the first reference position to obtain a base station image, and determine the path back to the base station 130 by analyzing and processing the base station image, and then perform the recharging action.
[0041] For example, when the mobile robot triggers the recharging task conditions, the mobile robot 110 can be controlled to move to a first reference position preset for the washing machine 120. The first reference position can be a designated area in front of the washing machine 120. At any first reference position in the designated area, when the monocular camera of the mobile robot 110 shoots in an orientation facing the washing machine 120, a base station image with a complete base station outline can be captured. In the case where the mobile robot 110 has pre-stored an operating map of the operating environment and marked the first reference position, the mobile robot 110 can be directly controlled to move to the first reference position according to the marked position. In the case where the mobile robot 110 cannot know the specific location of the washing machine 120, it can also obtain an image of the operating environment by shooting the operating environment. When the outline of the washing machine 120 is extracted from the operating environment image, the position of the washing machine 120 is determined, and then the path to the first reference position is determined, and the first reference position is reached according to the determined path.
[0042] A base station image is obtained by capturing the base station 130 at a first reference position using a camera on the mobile robot 110. The base station image includes a base station outline. The camera can be any of a variety of camera types, such as a monocular camera, a binocular camera, or a multi-camera camera. Referring to Figure 1 , after the mobile robot 110 reaches the first reference position, in order to accurately reach a designated location in front of the base station and accurately perform a recharging operation, the mobile robot 110 can capture a base station image including the base station outline at the first reference position. The base station image is processed using an edge feature extraction algorithm to extract the base station outline from the base station image. The position of the base station 130 relative to the mobile robot 110 can then be determined based on the position information of the base station outline in the base station image. Subsequently, a first motion path is determined based on the position information of the base station outline in the base station image. The mobile robot 110 is controlled to move along the first motion path to a second reference position preset for the base station 130 to perform the recharging operation. It should be understood that the mobile robot 110 is typically positioned on the central axis of the base station 130 to perform the recharging operation. As can be seen, the second reference position is generally also located on the central axis of the base station 130. While controlling the mobile robot 110 to move along the first motion path to the second reference position, the base station image can also be captured in real time to update the first motion path based on the current position information of the base station outline in the base station image, so as to more accurately reach the second reference position preset for the base station 130.
[0043] Please refer to Figure 2, which shows a flow chart of a recharging method for a mobile robot according to an embodiment of the present disclosure. The recharging method may include the following steps:
[0044] S210: Acquire a base station image captured by a camera of the mobile robot at a first reference position.
[0045] The base station image includes the base station outline.
[0046] S220: Determine a first motion path based on position information of the base station outline in the base station image.
[0047] S230: Control the mobile robot to perform recharging along the first motion path.
[0048] In an embodiment of the present disclosure, a monocular camera can be configured on a mobile robot to identify base stations by capturing images of the operating environment. The first reference position can be any location within a designated area in front of the base station. At any first reference position within the designated area, the image captured by the mobile robot's monocular camera facing the base station can contain the complete outline of the base station.
[0049] The mobile robot is controlled to be located at the first reference position and photographs the base station in order to more accurately obtain the outline of the base station, thereby making the first motion path more accurate.
[0050] Specifically, after acquiring a base station image, an image recognition algorithm is used to obtain the base station outline within the image. Subsequently, a first motion path is determined based on the positional information of the base station outline within the base station image. The mobile robot can then follow the first motion path directly to the base station for recharging. Alternatively, a recharging backup point can be pre-set for the base station. The mobile robot can then be controlled along the first motion path to reach the recharging backup point, where it can then enter the base station for recharging.
[0051] Through the above embodiment, the mobile robot is controlled to capture a base station image at a first reference position, and then a first motion path back to the base station is determined based on the position information of the base station outline in the base station image, and recharging is performed along the first motion path. The recharging method of the disclosed embodiment, which uses a camera to capture images, is not limited by signal range and viewing angle range, and improves the efficiency of the mobile robot's recharging when the base station is located deep inside.
[0052] In some embodiments of the present disclosure, before obtaining an image of the base station captured by a camera of the mobile robot at a first reference position, the recharging method of the mobile robot further includes: controlling the mobile robot to move to a first reference position preset for a target object, wherein the target object is fixed relative to the base station.
[0053] In an embodiment of the present disclosure, a monocular camera may be configured on the mobile robot to identify the target object or base station by capturing images in the working environment. The target object is a device or object that is relatively fixed in position to the base station. The mobile robot may be set at the bottom of the target object or as a part of the target object. For example, for a mobile robot that can wash and mop at the same time, its base station may be set at the bottom of the washing machine to connect with the waterway, or at the bottom of the sink to connect with the waterway. It can be seen that for a mobile robot used for cleaning work at home, the target object may be a household appliance or furniture on which a base station can be fixedly set. Specifically, for example, a washing machine, a dishwasher, a refrigerator, an air conditioner or a dishwashing sink, an integrated stove, an air purifier, etc.
[0054] The first reference position may be any position in a designated area in front of the target object. At any first reference position in the designated area, the image captured by the monocular camera of the mobile robot facing the target object may include a complete base station outline.
[0055] When the mobile robot has pre-stored an operating map of the operating area and has marked the position of the target object, the mobile robot can be directly controlled to move to a first reference position preset for the target object based on the marked position. When the mobile robot does not know the position of the target object, a monocular camera can be used to capture the operating environment to obtain an image of the operating environment. When the outline of the target object is extracted from the operating environment image, the position of the target object is determined and the mobile robot is controlled to move to the first reference position based on the position of the outline of the target object in the operating environment image. The mobile robot can also capture the operating environment according to a preset shooting frequency while performing other tasks. When the target object appears in the image, the coordinate position of the image is recorded. Then, when a recharging task needs to be performed, the mobile robot is directly controlled to move to the recorded coordinate position, thereby guiding the mobile robot to move to the first reference position based on the captured image.
[0056] In some embodiments of the present disclosure, controlling a mobile robot to move along a first motion path to perform recharging includes controlling the mobile robot to move along the first motion path to a second reference position preset relative to a base station, so as to perform recharging at the second reference position. The mobile robot is capable of moving straight into the base station after rotating to a preset heading angle at the second reference position.
[0057] The mobile robot is controlled to reach the first reference position preset for the target object in order to be able to quickly reach the vicinity of the base station. However, since the base station of the embodiment of the present disclosure has a large depth, the mobile robot needs a more precise route when entering the base station to perform the recharging action to reduce the possibility of collisions. The second reference position can be understood as a position preset for the mobile robot to accurately perform the recharging action, which is generally preset to a position on the central axis close to the base station. The mobile robot is located at the second reference position and takes an image of the base station, and the base station is located in the center of the base station image. Therefore, after the mobile robot reaches the first reference position, the base station can be photographed using a monocular camera, and the mobile robot can be continuously adjusted using the base station image, so that the mobile robot can quickly and accurately reach the second reference position to smoothly perform the recharging action.
[0058] Specifically, when the mobile robot is at the first reference position, a base station image is obtained by photographing the base station using a monocular camera. The base station image must include the base station outline. An image recognition algorithm can be used for the base station image. For example, an edge feature extraction algorithm extracts the outline of the object or device included in the base station image to determine the base station outline. Based on the position information of the base station outline in the base station image, the time when the base station image was captured and the orientation of the base station relative to the mobile robot can be determined, thereby determining the first motion path based on the position information of the base station outline in the base station image. The mobile robot can reach a second reference position preset for the base station along the first motion path.
[0059] For example, when the mobile robot is at a first reference position and photographs the base station, a base station image is obtained, and features are extracted from the base station image to obtain a base station outline in the base station image. The base station outline is located to the left of the base station image. Based on this position information, it can be determined that the current base station is located in the left front position of the mobile robot. Therefore, the first motion path can be determined to be leftward and forward. The monocular camera can be set to a fixed viewing angle and resolution when capturing images. The distance between the mobile robot and the base station can be determined based on the size of the base station outline in the base station image, thereby more accurately determining the distance to the left or forward, so as to reach the second reference position more accurately and quickly.
[0060] Through the above embodiment, the mobile robot is first controlled to move to the first reference position preset for the target object, so as to achieve the purpose of quickly reaching the vicinity of the base station. Then, at the first reference position, the base station is photographed to obtain an image of the base station. According to the position information of the base station outline in the base station image, the first motion path to move to the second reference position preset for the base station can be determined. After controlling the robot to move to the second reference position along the first motion path, a recharging action can be performed at the second reference position to return to the base station. The recharging method of the embodiment of the present disclosure is not limited by the signal range and the viewing angle range, and improves the efficiency and accuracy of the mobile robot when recharging in the case of a deeper base station position.
[0061] In some embodiments of the present disclosure, determining a first motion path based on position information of a base station outline in a base station image includes: determining first depth information from the base station to a mobile robot using the base station image; determining a first motion orientation of the first motion path based on the position information of the base station outline in the base station image; and determining the first motion path based on the first depth information and the first motion orientation.
[0062] In an embodiment of the present disclosure, the purpose of the mobile robot moving along the first motion path is to reach the second reference position. If the moving direction is determined only based on the position information of the base station outline in the base station image, its movement efficiency is relatively slow. Therefore, the first depth information from the base station to the first reference position can also be determined based on the base station image taken by the monocular camera, so as to determine the current distance from the base station to the mobile robot. The first motion direction of the first motion path can be determined based on the position information, and the motion distance for the first motion direction can be determined in combination with the first depth information. For example, based on the position of the mobile robot, after determining that the first motion direction is first moving to the left and then moving forward based on the position information of the base station outline in the base station image. Based on the first depth information, it can be determined that the first motion path is moving 50 cm to the left and then moving 20 cm forward. In this way, after determining the motion distance for the first motion direction, the second reference position can be reached more quickly and accurately.
[0063] In some embodiments, after determining the first motion path, controlling the mobile robot to move along the first motion path to a second reference position preset for the base station may include: controlling the mobile robot to move along the first motion orientation by a motion distance corresponding to the motion orientation to reach the second reference position.
[0064] For example, based on the position information and the first depth information, it is determined that the first movement direction of the first movement path is leftward and then forward, and the movement distance for the first movement direction is 50 cm to the left and then 20 cm forward. The mobile robot can then be controlled to first move 50 cm to the left and then 20 cm forward to reach the second reference position.
[0065] It is understandable that while the mobile robot is moving along the first motion path, it can also photograph the base station in real time, and adjust the motion direction and distance of the first motion path according to the newly photographed base station image to reach the second reference position more accurately.
[0066] The mobile robot can move straight into the base station after rotating to a preset orientation angle at the second reference position.
[0067] After the mobile robot reaches the second reference position, it faces the base station. Since the charging device is typically located behind the mobile robot, it can rotate 180 degrees at the second reference position and then move straight back into the base station. Once the charging voltage is detected, the robot stops moving backward, completing the recharging process. In this scenario, the preset heading angle is the angle along the base station's central axis, with the robot facing away from the base station.
[0068] The depth information from the base station to the mobile robot can be obtained in any of the following ways: First, using a base station image captured by a monocular camera and radar data acquired by the mobile robot's 2D lidar to determine the depth information from the base station to the mobile robot; Second, configuring the mobile robot with a depth sensor, such as a depth lidar, depth camera, or binocular camera, to determine the depth information from the base station to the mobile robot. Since the embodiment of this disclosure uses a monocular camera to determine the position, the first method can be used to determine the depth information, which is more cost-effective and has sufficient accuracy.
[0069] Exemplarily, a monocular visual depth estimation algorithm can be used to calculate the depth information from the base station to the mobile robot. The algorithm can fuse the scale information of the 2D lidar and the texture structure information of the monocular image, so that the accuracy of the depth estimation is greatly improved. A trained monocular depth estimation network is configured in the mobile robot. When depth estimation is required, the input is an RGB image and a radar data mapped to a two-dimensional image. After processing by the monocular depth estimation network, a depth map (the depth value corresponding to each pixel position in the RGB image) can be obtained. In an embodiment of the present disclosure, the base station image captured by the monocular camera at the first reference position and the radar data obtained by the lidar at the first reference position are input into the monocular depth estimation network for processing, and the first depth information from the base station to the mobile robot, that is, the depth value from the base station to the mobile robot, can be obtained.
[0070] In some embodiments of the present disclosure, before determining the first motion path based on the first depth information and the first motion orientation, the recharging method of the mobile robot further includes: determining an obstacle situation on the first motion path.
[0071] Determining the first motion path based on the first depth information and the first motion orientation includes: determining a distance of movement along a straight line in the first motion orientation based on the first depth information and the first motion orientation to obtain the first motion path when there are no obstacles on the first motion path; or determining the first motion path based on position information of the obstacle, the first depth information, and the first motion orientation when there are obstacles on the first motion path.
[0072] In some cases, there may be obstacles in the area between the mobile robot's current location and the base station, or the second reference point. For example, the obstacle could be a water stain, a sock, or something similar. Therefore, before determining the first motion path, it is also necessary to first determine the presence of obstacles in the area. For example, this can be done by capturing images of the base station to identify the presence of obstacles, or by scanning the environment with a lidar radar.
[0073] When there are no obstacles along the first motion path, first depth information from the base station to a first reference position is determined based on the base station image captured by the monocular camera, thereby determining the current distance from the base station to the mobile robot. Based on this position information, a first motion orientation of the first motion path can be determined. Combined with the first depth information, the distance traveled along the first motion orientation can be determined, thereby obtaining the first motion path. The first motion orientation can include multiple orientations.
[0074] When an obstacle exists on the first motion path, a first motion path that can avoid the obstacle is determined by combining the distance determined according to the first depth information, the first motion direction, and the position information of the obstacle.
[0075] For example, if the second reference position is determined to be 50 cm to the left and front of the mobile robot based on the first depth information and the position of the base station outline in the base station image, and if there are no obstacles in the area between the mobile robot and the base station, the first motion path can be determined to be 50 cm straight ahead to the left to reach the second reference position. If there are obstacles in this area, the mobile robot can be controlled to move 30 cm to the left and then 40 cm forward to reach the second reference position.
[0076] In some embodiments of the present disclosure, a base station profile in a base station image is determined by performing edge feature extraction on a plurality of first objects to be verified in the base station image to obtain profile features corresponding to the plurality of first objects to be verified. A target profile feature is then determined from the profile features corresponding to the plurality of first objects to be verified that matches a preset base station profile feature for the base station. The profile of the object to be verified corresponding to the target profile feature is the base station profile.
[0077] Specifically, after acquiring the base station image, edge features of several first objects to be verified contained in the base station image are extracted based on an edge feature extraction algorithm to obtain an edge feature map corresponding to the base station image. The edge feature map includes contour features corresponding to several first objects to be verified. The contour features of the base station are pre-stored in the mobile robot. The contour features corresponding to the several first objects to be verified are respectively compared with the pre-stored base station contour features preset for the base station to determine the target contour features that match the base station contour features. The contour of the object to be verified corresponding to the target contour feature is determined to be the base station contour, that is, the object to be verified is determined to be a base station. Among them, edge feature extraction can also be performed based on a preset image recognition algorithm or image recognition model. For example, a deep learning algorithm, a neural network model, etc.
[0078] In some embodiments of the present disclosure, controlling a mobile robot to move to a first reference position preset for a target object includes: if an object outline of the target object is extracted based on an image of the mobile robot's working environment, determining a second motion path based on position information of the object outline in the image of the working environment, and controlling the mobile robot to move to the first reference position along the second motion path.
[0079] In some cases, the mobile robot is unaware of the location and room of the target object. When recharging is required, the mobile robot needs to use a monocular camera to photograph the working environment, determine the location of the target object based on the working environment image, and thus reach the first reference position.
[0080] Specifically, when a mobile robot triggers a recharge task, it collects real-time images of its work environment, starting from its current location. Different sampling points can be pre-set for different room types. Once the room type is identified based on the furniture in the image, the image can be captured based on the sampling points in that room. This reduces unnecessary image capture, improves image processing, and accelerates target object recognition.
[0081] When the target object's outline is extracted based on the work environment image, it can be determined that the location where the work environment image was captured is near the target object. Based on the positional information of the target object's outline in the work environment image, the relative orientation of the target object relative to the location of the mobile robot at the time the work environment image was captured can be determined. Similar to the method for determining the first motion path in the above-mentioned embodiment, based on the positional information of the object outline in the work environment image, a second motion path can be determined, i.e., the direction in which the mobile robot should move left, right, forward, and / or backward to reach the first reference position.
[0082] In some embodiments of the present disclosure, controlling a mobile robot to move along a second motion path to a first reference position includes: capturing an object environment image of a target object while controlling the mobile robot to move along the second motion path. The object environment image includes a target object outline. The second motion path is adjusted based on position information of the target object outline in the object environment image and second depth information from the target object to the mobile robot, and the mobile robot is controlled to move along the adjusted second motion path to reach the first reference position.
[0083] In some cases, the mobile robot is controlled to reach the first reference position first in order to be able to quickly reach the vicinity of the base station to speed up the recharging efficiency. In the embodiment of the present disclosure, the positions of the target object and the base station are relatively fixed. Therefore, the first reference position can be set to be located on the central axis of the target object, so that when the mobile robot is located at the first reference position, the working environment image obtained by shooting the target object is in the center of the working environment image. In this way, when shooting the base station based on the first reference position, the efficiency of the final moving second reference position will be improved.
[0084] Specifically, when the object outline of the target object is extracted based on the working environment image, the position where the mobile robot takes the working environment image must be located in a designated area where the target object can be photographed, but it is not necessarily located on the central axis of the target object. Therefore, in order to improve the recharging efficiency, when setting the first reference position to be located in the designated area and on the central axis of the target object, the initial second motion path can be determined based on the object outline in the working environment image. In the process of controlling the mobile robot to move along the second motion path, continue to take images of the target object to obtain an object environment image. At this time, the object environment image obtained must contain the outline of the target object. By continuously obtaining the object environment images during the movement, the orientation and distance of the target object relative to the mobile robot can be continuously updated. Therefore, the second motion path can be continuously adjusted based on the position information of the target object outline in the object environment image and the second depth information from the target object to the mobile robot, so that the process of the mobile robot reaching the first reference position along the adjusted second motion path is faster and more accurate.
[0085] The method for determining the second depth information from the target object to the mobile robot by using the object environment image may refer to the above-mentioned method for determining the first depth information, which will not be described in detail here.
[0086] In some embodiments of the present disclosure, before controlling the mobile robot to move to a first reference position preset for the target object, the recharging method further includes: if the object contour of the target object is extracted based on the working environment image of the mobile robot, recording the coordinate position of the captured working environment image.
[0087] Controlling the mobile robot to move to a first reference position preset for a target object includes: controlling the mobile robot to move to a coordinate position; acquiring an object environment image captured at the coordinate position and capturing the target object; wherein the object environment image includes a target object outline; determining a third motion path based on positional information of the target object outline in the object environment image; and controlling the mobile robot to move along the third motion path to the first reference position.
[0088] In some cases, to improve the recharging efficiency of the mobile robot, it can be set to take a picture of the working environment while the mobile robot is performing other tasks and determine the location of the target object based on the captured working environment image. In this case, to avoid the mobile robot from being stuck due to processing too much data due to performing multiple tasks at the same time, a slower shooting frequency can be set to take pictures or a working environment image can be taken and processed every time the rotation angle reaches a certain threshold. For example, when the sweeping robot is performing a cleaning task, it can be set to take a working environment image every 90 degrees of rotation in a room to determine whether the target object exists in the room.
[0089] Specifically, if the object outline of the target object is extracted based on the working environment image taken by the mobile robot during the performance of other tasks, it can be determined that the target object is located in the room, and the coordinate position of the working environment image can be recorded. When the mobile robot needs to perform a recharging task, the mobile robot is controlled to move to the coordinate position first. The target object is photographed from the coordinate position to obtain an object environment image. At this time, the object environment image must include the outline of the target object. Then, a third motion path to reach the first reference position is determined based on the object environment image. Among them, the method for determining the third motion path is the same as the principle for determining the second motion path. The third motion path can be determined based on the position information of the target object outline in the object environment image and the third depth information from the base station to the mobile robot. In the process of controlling the mobile robot to move along the third motion path, the object environment image can also be continuously photographed to adjust the third motion path, so that the mobile robot can move to the first reference position more quickly and accurately.
[0090] In some embodiments of the present disclosure, extracting an object contour of a target object based on an image of a mobile robot's working environment includes performing edge feature extraction on a plurality of second objects to be verified in the working environment image to obtain contour features corresponding to the plurality of second objects to be verified. If any of the contour features corresponding to the plurality of second objects to be verified matches a preset object contour feature for the target object, the object contour of the target object is determined to be extracted based on the working environment image.
[0091] In an embodiment of the present disclosure, the way in which the mobile robot extracts the object contour of the target object from the working environment image is the same as the way in which the base station contour is extracted from the base station image. The object contour of the target object is pre-stored in the mobile robot. After acquiring the working environment image, edge feature extraction is performed on the second object to be verified contained in the working environment image based on an edge feature extraction algorithm to obtain an edge feature map corresponding to the working environment image. The edge feature map includes contour features corresponding to several second objects to be verified. The contour features corresponding to the several second objects to be verified are respectively compared with the pre-stored contour features preset for the target object. If, among the contour features corresponding to the several second objects to be verified, there is one contour feature of the second object to be verified that matches the object contour feature preset for the target object, then the contour of the second object to be verified can be determined to be the object contour of the target object, that is, the second object to be verified is determined to be the target object.
[0092] In some embodiments of the present disclosure, the base station is disposed at the bottom of a target object or as a part of the target object, and the target object includes household appliances and furniture. The household appliances include laundry processing equipment.
[0093] The recharging method of the disclosed embodiments is applicable to situations where the base station is located at a greater depth. For example, in a household robot vacuum cleaner that washes and mops, its base station can be located at the bottom of a target object, which can be a household appliance used for laundry processing. For example, the appliance could be a washing machine. By placing the base station at the bottom of the washing machine, the washing machine's water supply can be connected, making the base station's cleaning work more convenient for the robot vacuum cleaner.
[0094] In some embodiments, the base station may also be a part of a household appliance; or when the base station of the mobile robot does not need to be connected to a waterway, the base station may also be set at the bottom of a piece of furniture or be a part of the furniture.
[0095] Home appliances may include laundry machines, such as washing machines, dryers, and washer-dryers. They may also include refrigerators, dishwashers, televisions, air purifiers, water purifiers, air conditioners, steamers, ovens, sinks, and integrated stoves. Furniture may include shoe cabinets, TV cabinets, bookcases, bathtubs, fish tanks, cabinets, toilets, fixed tables and chairs, bar counters, and sinks.
[0096] In some embodiments of the present disclosure, referring to FIG3 , a recharging method for a mobile robot may include the following steps:
[0097] S310 : If an object contour of the target object is extracted based on the working environment image of the mobile robot, a second motion path is determined according to position information of the object contour in the working environment image.
[0098] S320: Control the mobile robot to move along the second motion path to a first reference position preset for the target object.
[0099] The positions of the target object and the base station are relatively fixed.
[0100] S330: Acquire a base station image obtained by photographing the base station at a first reference position.
[0101] The base station image includes the base station outline.
[0102] S340: Determine first depth information from the base station to the mobile robot using the base station image.
[0103] S350: Determine a first motion direction of a first motion path according to position information of a base station outline in the base station image.
[0104] S360: Determine a movement distance for the first movement direction according to the first depth information.
[0105] S370, controlling the mobile robot to move along the first movement orientation by a movement distance corresponding to the movement orientation to reach a second reference position to perform a recharging action.
[0106] In some embodiments of the present disclosure, referring to FIG4 , the recharging method of the mobile robot may further include the following steps:
[0107] S402 : If the object contour of the target object is extracted based on the working environment image of the mobile robot, the coordinate position of the captured working environment image is recorded.
[0108] S404, controlling the mobile robot to move to the coordinate position.
[0109] S406: Acquire an object environment image obtained by photographing the target object at the coordinate position.
[0110] The object environment image includes the target object outline.
[0111] S408: Determine a third motion path based on the position information of the target object outline in the object environment image.
[0112] S410: Control the mobile robot to move along a third motion path to a first reference position preset for a target object.
[0113] S412: Acquire a base station image obtained by photographing the base station at a first reference position.
[0114] The base station image includes the base station outline.
[0115] S414: Determine first depth information from the base station to the mobile robot using the base station image.
[0116] S416: Determine a first motion direction of the first motion path according to position information of the base station outline in the base station image.
[0117] S418: Determine a movement distance for the first movement direction according to the first depth information.
[0118] S420: Control the mobile robot to move along the first movement orientation by a movement distance corresponding to the movement orientation to reach a second reference position to perform a recharging action.
[0119] Corresponding to the above embodiment, please refer to FIG5 , the embodiment of the present disclosure also proposes a recharging device for a mobile robot. The recharging device includes:
[0120] The image acquisition module 510 is used to acquire an image of the base station captured by a camera of the mobile robot at a first reference position.
[0121] The base station image includes the base station outline.
[0122] The path determination module 520 is configured to determine a first motion path based on position information of the base station outline in the base station image.
[0123] The recharging module 530 is configured to control the mobile robot to perform recharging along the first motion path.
[0124] According to the mobile robot recharging device of the disclosed embodiment, the mobile robot is controlled to capture a base station image at a first reference position, and then a first motion path back to the base station is determined based on the position information of the base station outline in the base station image. Recharging is then performed along the first motion path. The recharging method of the disclosed embodiment, which uses camera photography, is not limited by signal range and viewing angle, and improves the efficiency of the mobile robot's recharging when the base station is located deep inside.
[0125] The specific definition of the mobile robot recharging device can be found in the definition of the mobile robot recharging method above, and will not be repeated here. The various modules in the above-mentioned mobile robot recharging device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above-mentioned modules.
[0126] Figure 6 is a block diagram of a mobile robot according to one embodiment of the present disclosure. Referring to Figure 6 , mobile robot 600 includes a memory 604, a processor 602, and a mobile robot recharge program 606 stored in memory 604 and executable on processor 602. When processor 602 executes mobile robot recharge program 606, the mobile robot recharge method according to any of the aforementioned embodiments is implemented.
[0127] In some embodiments of the present disclosure, the base station corresponding to the mobile robot is set at the bottom of the target object, and the target object includes a household appliance.
[0128] In some embodiments of the present disclosure, the household appliance includes a clothes treating appliance.
[0129] According to the mobile robot of the embodiment of the present disclosure, when the processor 602 executes the recharging program 606 of the mobile robot, the mobile robot will not be limited by the signal range and the viewing angle range, and the efficiency of the mobile robot during recharging is improved in the case where the base station is located deeper.
[0130] Corresponding to the above embodiments, the present disclosure further provides a computer-readable storage medium on which a recharging program for a mobile robot is stored, and when the recharging program is executed by a processor, the recharging method for a mobile robot as described in any one of the above embodiments is implemented.
[0131] According to the computer-readable storage medium of the embodiment of the present disclosure, when the recharging program is executed by the processor, the mobile robot will not be limited by the signal range and the viewing angle range, and the efficiency of the mobile robot during recharging is improved in the case where the base station is located deeper.
[0132] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein. For example, it can be considered as a sequenced list of executable instructions for implementing a logical function, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or for use in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this disclosure, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.
[0133] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0134] In the description of this disclosure, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the disclosure. In this disclosure, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
[0135] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation to the present disclosure.
[0136] In addition, the terms "first" and "second" used in the embodiments of the present disclosure are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the embodiments. Therefore, the features defined by the terms "first" and "second" in the embodiments of the present disclosure may explicitly or implicitly indicate that the embodiment includes at least one such feature. In the description of the present disclosure, the word "multiple" means at least two or two or more. For example, two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.
[0137] In this disclosure, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected" and "fixed" appearing in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements, or the interaction relationship between two elements. For those skilled in the art, the specific meanings of the above terms in this disclosure can be understood based on the specific implementation.
[0138] In the present disclosure, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.
[0139] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are illustrative and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.
Claims
1. A recharging method for a mobile robot, the method comprising: Acquire a base station image captured by a camera of the mobile robot at a first reference position; wherein the base station image includes a base station outline; Determine a first motion path based on position information of the base station outline in the base station image; The mobile robot is controlled to perform recharging along the first motion path.
2. The method according to claim 1, wherein: The controlling the mobile robot to move along the first motion path to perform recharging includes: The mobile robot is controlled to move along the first motion path to a second reference position preset for the base station to perform recharging at the second reference position; wherein the mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.
3. The method according to claim 1, wherein: Determining a first motion path based on position information of the base station outline in the base station image includes: Determine first depth information from the base station to the mobile robot using the base station image; Determine a first motion orientation of the first motion path according to position information of the base station outline in the base station image; The first motion path is determined according to the first depth information and the first motion orientation.
4. The method according to claim 3, wherein: Before determining the first motion path according to the first depth information and the first motion orientation, the method further includes: Determining obstacles on the first motion path; Determining the first motion path according to the first depth information and the first motion orientation includes: In a case where there is no obstacle on the first motion path, determining a motion distance moving in a straight line along the first motion orientation according to the first depth information and the first motion orientation to obtain the first motion path; or In the case where there is an obstacle on the first motion path, the first motion path is determined according to the position information of the obstacle, the first depth information, and the first motion orientation.
5. The method according to claim 1, wherein: The base station contour of the base station image is determined in the following manner: Extracting edge features of a plurality of first objects to be verified in the base station image to obtain contour features corresponding to the plurality of first objects to be verified; Among the contour features corresponding to the plurality of first objects to be verified, a target contour feature matching a base station contour feature preset for the base station is determined; wherein the contour of the object to be verified corresponding to the target contour feature is the base station contour.
6. The method according to claim 1, wherein: Before acquiring the base station image captured by the mobile robot at the first reference position through the camera, the method further includes: The mobile robot is controlled to move to the first reference position preset for the target object; wherein the positions of the target object and the base station are relatively fixed.
7. The method according to claim 6, wherein: Controlling the mobile robot to move to the first reference position preset for the target object includes: If an object contour of the target object is extracted based on the working environment image of the mobile robot, a second motion path is determined according to position information of the object contour in the working environment image; The mobile robot is controlled to move along the second motion path to the first reference position.
8. The method according to claim 7, wherein: Controlling the mobile robot to move to the first reference position along the second motion path includes: In the process of controlling the mobile robot to move along the second motion path, capturing an object environment image for the target object; wherein the object environment image includes a contour of the target object; The second motion path is adjusted based on the position information of the target object outline in the object environment image and the second depth information from the target object to the mobile robot, and the mobile robot is controlled to move along the adjusted second motion path to reach the first reference position.
9. The method according to claim 6, wherein: Before controlling the mobile robot to move to a first reference position preset for the target object, the method further includes: If an object contour of the target object is extracted based on the working environment image of the mobile robot, a coordinate position of shooting the working environment image is recorded; The controlling the mobile robot to move to a first reference position preset for the target object comprises: Controlling the mobile robot to move to the coordinate position; Acquire an object environment image obtained by photographing the target object at the coordinate position; wherein the object environment image includes a contour of the target object; Determine a third motion path based on position information of the target object contour in the object environment image; The mobile robot is controlled to move along the third motion path to the first reference position.
10. The method according to claim 7 or 9, wherein: Extract the object contour of the target object based on the working environment image of the mobile robot, including: Extracting edge features of a plurality of second objects to be verified in the working environment image to obtain contour features corresponding to the plurality of second objects to be verified; In the case where any contour feature of the second objects to be verified matches the contour feature of the target object preset among the contour features corresponding to the plurality of second objects to be verified, determining the object to be verified based on the working environment map The image is extracted to the object contour of the target object.
11. The method according to any one of claims 6 to 9, wherein: The base station is arranged at the bottom of the target object or as a part of the target object, and the target object includes household electrical appliances and furniture.
12. The method according to claim 11, wherein: The household appliance includes a clothes processing device.
13. A mobile robot, comprising a memory, a processor, and a recharging program of the mobile robot stored in the memory and executable on the processor, wherein when the processor executes the program, the recharging method of the mobile robot as described in any one of claims 1 to 12 is implemented.
14. The mobile robot according to claim 13, wherein: The base station corresponding to the mobile robot is arranged at the bottom of the target object or as a part of the target object, and the target object includes household electrical appliances and furniture.
15. The mobile robot according to claim 14, wherein: The household appliance includes a clothes treating device.
16. A computer-readable storage medium having a mobile robot recharging program stored thereon, wherein when the program is executed by a processor, the mobile robot recharging method according to any one of claims 1 to 11 is implemented.
Citation Information
Patent Citations
Vision charging base and location method thereof, and re-charge method of vision charging base
CN108646727A
Robot automatic charging method, device and system, terminal equipment and storage medium
CN111061280A
Robot and automatic control method and device thereof
CN111546333A
Mobile robot autonomous charging method based on ArUco code guidance
CN112183133A
Robot recharging method and device, electronic equipment and readable storage medium
CN113467451A
