Mobile robot recharging method, household appliance and computer readable storage medium

By taking images at the base station location and using image recognition algorithms to determine the motion path, the problem of low recharge efficiency caused by large base station depth is solved, and efficient recharge in the case of deep base station location is achieved.

CN118915711APending Publication Date: 2024-11-08MIDEA ROBOZONE TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202310504549.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-06
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the prior art, the mobile robot recharge method based on infrared sensors and scanning code positioning is less efficient when the base station has a large depth. Due to the limitations of the signal range and viewing angle range, it is difficult to accurately return to the base station for charging.

Method used

By the mobile robot taking a base station image at the first reference position, the base station profile and depth information are determined using an image recognition algorithm, the first motion path is calculated, and the robot is moved along the path to a preset second reference position to perform the recharge action , avoid limitations in signal range and viewing angle range.

Benefits of technology

The recharge efficiency and accuracy of the mobile robot when the base station is deep is improved, ensuring that the robot can return to the base station quickly and accurately for charging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915711A_ABST
    Figure CN118915711A_ABST
Patent Text Reader

Abstract

The invention relates to a recharging method of a mobile robot, a household appliance and a computer readable storage medium. The recharging method comprises the following steps: acquiring a base station image shot by a mobile robot at a first reference position through a camera; wherein the base station image comprises a base station contour; determining a first motion path based on the position information of the base station contour in the base station image; and controlling the mobile robot to execute recharging along the first motion path. According to the embodiment of the invention, the path for returning to the base station can be determined based on the base station image shot for the base station, so that the recharging action is executed at the reference position for returning to the station without being limited by a signal range and a visual angle range, and the recharging efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present specification relates to the technical field of mobile robots, and in particular to a recharging method for a mobile robot, a household appliance, and a computer-readable storage medium. Background Art

[0002] With the development of science and technology, mobile robots have been widely used in social life, bringing convenience to people's lives. For example, the most common mobile robots are sweeping robots in home life, food delivery robots in hotels, and shopping guide robots in supermarkets.

[0003] For mobile robots, autonomously returning to the base station for energy replenishment is a critical technology. In the related technologies, there are methods based on infrared sensors, code scanning and positioning to realize the recharging of mobile robots. However, some mobile robots are large in size or have a large base station depth due to functional needs. In this case, the infrared emission angle of the recharging method based on infrared sensors will be very limited. Similarly, the large depth of the base station will also affect the field of view angle of the code scanning positioning method. It can be seen that for base stations with a large depth, the recharging methods based on infrared sensors and code scanning positioning in the related technology are less efficient. Summary of the invention

[0004] This specification aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of this specification is to propose a recharging method for a mobile robot, which can determine a path back to the base station based on a base station image taken of the base station, so as to reach a reference position of the back station to perform a recharging action, without being limited by the signal range and the viewing angle range, thereby improving the recharging efficiency.

[0005] The second purpose of this specification is to provide a charging device for a mobile robot.

[0006] The third objective of this specification is to provide a household appliance.

[0007] A fourth objective of this specification is to provide a computer-readable storage medium.

[0008] To achieve the above-mentioned purpose, the first embodiment of the present specification proposes a recharging method for a mobile robot. The recharging method comprises: obtaining a base station image taken by a camera of the mobile robot at a first reference position; wherein the base station image comprises a base station outline; determining a first motion path based on the position information of the base station outline in the base station image; and controlling the mobile robot to perform recharging along the first motion path.

[0009] In some embodiments of the present specification, controlling the mobile robot to move along the first motion path to perform recharging includes: controlling the mobile robot to move along the first motion path to a second reference position preset for the base station to perform recharging at the second reference position; wherein the mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.

[0010] In some embodiments of the present specification, determining a first motion path based on the position information of the base station outline in the base station image includes: using the base station image to determine first depth information from the base station to the mobile robot; determining a first motion orientation of the first motion path according to the position information of the base station outline in the base station image; determining the first motion path according to the first depth information and the first motion orientation.

[0011] In some embodiments of the present specification, before determining the first motion path based on the first depth information and the first motion direction, the method also includes: determining the obstacle situation on the first motion path; determining the first motion path based on the first depth information and the first motion direction, including: when there is no obstacle on the first motion path, determining the movement distance along the first motion direction in a straight line based on the first depth information and the first motion direction to obtain the first motion path; or when there is an obstacle on the first motion path, determining the first motion path based on the position information of the obstacle, the first depth information, and the first motion direction.

[0012] In some embodiments of the present specification, the base station contour of the base station image is determined in the following manner: edge feature extraction is performed on several first objects to be verified in the base station image to obtain contour features corresponding to the several first objects to be verified; among the contour features corresponding to the several first objects to be verified, a target contour feature that matches the base station contour feature preset for the base station is determined; wherein the contour of the object to be verified corresponding to the target contour feature is the base station contour.

[0013] In some embodiments of the present specification, before obtaining the base station image captured by the mobile robot through a camera at the first reference position, the method further includes: controlling the mobile robot to move to the first reference position preset for the target object; wherein the position of the target object and the base station are relatively fixed.

[0014] In some embodiments of the present specification, controlling the mobile robot to move to a first reference position preset for a target object includes: if an object contour of the target object is extracted based on an image of the working environment of the mobile robot, determining a second motion path according to position information of the object contour in the working environment image; and controlling the mobile robot to move to the first reference position along the second motion path.

[0015] In some embodiments of the present specification, controlling the mobile robot to move along the second motion path to the first reference position includes: in the process of controlling the mobile robot to move along the second motion path, capturing an object environment image for the target object; wherein the object environment image includes a target object contour; adjusting the second motion path based on position information of the target object contour in the object environment image and second depth information from the target object to the mobile robot, and controlling the mobile robot to move along the adjusted second motion path to reach the first reference position.

[0016] In some embodiments of the present specification, before controlling the mobile robot to move to a first reference position preset for the target object, the method further includes: if an object contour of the target object is extracted based on the working environment image of the mobile robot, recording the coordinate position of shooting the working environment image; controlling the mobile robot to move to the first reference position preset for the target object includes: controlling the mobile robot to move to the coordinate position; acquiring an object environment image obtained by shooting the target object at the coordinate position; wherein the object environment image includes a target object contour; determining a third motion path based on position information of the target object contour in the object environment image; and controlling the mobile robot to move to the first reference position along the third motion path.

[0017] In some embodiments of the present specification, extracting the object contour of the target object based on the working environment image of the mobile robot includes: performing edge feature extraction on several second objects to be verified in the working environment image to obtain contour features corresponding to the several second objects to be verified; and determining that the object contour of the target object is extracted based on the working environment image when, among the contour features corresponding to the several second objects to be verified, there is any one second object to be verified whose contour features match the object contour features preset for the target object.

[0018] In some embodiments of the present specification, the base station is disposed at the bottom of the target object or as a part of the target object, and the target object includes household appliances and furniture.

[0019] In some embodiments of the present specification, the household appliance includes a clothing processing device.

[0020] To achieve the above-mentioned purpose, the second embodiment of the specification proposes a recharging device for a mobile robot. The recharging device comprises: an image acquisition module, used to acquire a base station image taken by a camera of the mobile robot at a first reference position; wherein the base station image comprises a base station outline; a path determination module, used to determine a first motion path based on the position information of the base station outline in the base station image; and a recharging module, used to control the mobile robot to perform recharging along the first motion path.

[0021] To achieve the above-mentioned purpose, the third embodiment of the specification proposes a mobile robot. The mobile robot includes a memory, a processor, and a mobile robot recharging program stored in the memory and executable on the processor, and when the processor executes the program, the mobile robot recharging method as described in any one of the first aspects is implemented.

[0022] In some embodiments of the present specification, the base station corresponding to the mobile robot is arranged at the bottom of the target object or as a part of the target object, and the target object includes household appliances and furniture.

[0023] In some embodiments of the present specification, the household appliance includes a clothing processing device.

[0024] To achieve the above-mentioned purpose, the fourth aspect of the present specification proposes a computer-readable storage medium, on which a mobile robot recharging program is stored, and when the program is executed by a processor, the mobile robot recharging method as described in any one of the first aspects is implemented.

[0025] Through the above embodiment, the mobile robot is controlled to shoot the base station at the first reference position to obtain the base station image, and then the first motion path back to the base station is determined according to the position information of the base station outline in the base station image, and the recharging is performed along the first motion path. The recharging method of the embodiment of this specification, by taking pictures with a camera, is not limited by the signal range and the viewing angle range, and improves the efficiency of the mobile robot when recharging in the case of a deeper base station.

[0026] Additional aspects and advantages of the present specification will be given in part in the description below and in part will become apparent from the description below or will be learned through the practice of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a scene example diagram of the recharging method of the mobile robot in the embodiment of this specification.

[0028] Figure 2It is a flow chart of the recharging method of the mobile robot according to an embodiment of this specification.

[0029] Figure 3 It is a flow chart of a recharging method for a mobile robot according to an embodiment of the present specification.

[0030] Figure 4 It is a flow chart of a mobile robot recharging method according to another embodiment of the present specification.

[0031] Figure 5 It is a structural block diagram of the recharging device of the mobile robot according to the embodiment of this specification.

[0032] Figure 6 It is a structural block diagram of the mobile robot of the embodiment of this specification. DETAILED DESCRIPTION

[0033] The embodiments of the present specification are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present specification, and should not be construed as limiting the present specification.

[0034] At present, mobile robots are widely used in people's social life, bringing great convenience to people's lives. However, due to the large size or functional requirements of some mobile robots, the depth of their base stations is set relatively large. For example, the sweeping robot in home life. In order to reduce manual cleaning work to a greater extent, some current sweeping robots have dual functions of sweeping and mopping. The base station of the sweeping and mopping robot not only needs to provide charging function, but also needs to provide water supply and cleaning rag functions. Therefore, the base station of the sweeping and mopping robot may be connected to the waterway in the home and set at the bottom of the home appliance or sink equipment. It has a large depth and can accommodate a large range or all of the body of the sweeping robot.

[0035] In the recharging method based on infrared sensors, it is necessary to arrange infrared sensors in the base station to emit infrared rays to the outside. However, when the depth of the base station is large, the emission angle of the infrared sensor will be very limited. Similarly, the large depth of the base station will also affect the field of view angle of the code scanning positioning method. It can be seen that for base stations with a large depth, the recharging method based on infrared sensors and code scanning positioning in the related art is less efficient. Therefore, in response to the above situation, this specification proposes a recharging method for a mobile robot, a household appliance, and a computer-readable storage medium, which can determine the path back to the base station based on the base station image taken of the base station, so as to reach the reference position of the return station to perform the recharging action, which is not limited by the signal range and the viewing angle, thereby improving the recharging efficiency.

[0036] This specification provides a scenario example of a mobile robot recharging method by taking the example of a sweeping and mopping robot base station being arranged at the bottom of a washing machine. The mobile robot recharging method is applied to Figure 1 In the mobile robot (sweeping robot) shown in FIG. Figure 1 As shown, the scene example includes a mobile robot 110, a washing machine 120, and a base station 130 disposed at the bottom of the washing machine 120. Among them, the position of the base station 130 relative to the washing machine 120 is fixed. A monocular camera can be configured on the mobile robot 110 to shoot the surrounding environment. If the mobile robot 110 needs to return to the base station for charging or cleaning, it can first move to a first reference position preset for the washing machine 120, and then shoot the base station 130 at the first reference position to obtain a base station image, and determine the path back to the base station 130 by analyzing and processing the base station image, and then perform the recharging action.

[0037] Exemplarily, when the mobile robot triggers the recharging task conditions, the mobile robot 110 can be controlled to move to the first reference position preset for the washing machine 120. The first reference position can be a designated area in front of the washing machine 120. At any first reference position in the designated area, when the monocular camera of the mobile robot 110 shoots in an orientation facing the washing machine 120, a base station image with a complete base station outline can be captured. In the case where the mobile robot 110 has pre-stored an operating map of the operating environment and marked the first reference position, the mobile robot 110 can be directly controlled to move to the first reference position according to the marked position. In the case where the mobile robot 110 cannot know the specific position of the washing machine 120, the operating environment image can also be obtained by shooting the operating environment. When the operating environment image is extracted to include the outline of the washing machine 120, the position of the washing machine 120 is determined, and then the path to the first reference position is determined, and the first reference position is reached according to the determined path.

[0038] A base station image is obtained by capturing the base station 130 at a first reference position through a camera on the mobile robot 110. The base station image includes a base station outline. The camera may be, for example, a monocular camera, a binocular camera, a multi-camera camera, etc. Figure 1After the mobile robot 110 reaches the first reference position, in order to accurately reach the designated position in front of the base station so as to accurately perform the recharging action, the base station 130 can be photographed at the first reference position to obtain a base station image including the base station outline. The base station image is processed using an edge feature extraction algorithm to extract the base station outline in the base station image. Thus, the orientation of the base station 130 relative to the mobile robot 110 can be determined based on the position information of the base station outline in the base station image. Then, the first motion path is determined based on the position information of the base station outline in the base station image. The mobile robot 110 is controlled to move along the first motion path to the second reference position preset for the base station 130 to perform the recharging action. It can be understood that the mobile robot 110 is usually located on the central axis of the base station 130 to perform the recharging action. It can be seen that the second reference position is generally also located on the central axis of the base station 130. In the process of controlling the mobile robot 110 to move along the first motion path to the second reference position, the base station image can also be photographed in real time to update the first motion path according to the position information of the current base station outline in the base station image, so as to more accurately reach the second reference position preset for the base station 130.

[0039] like Figure 2 As shown, the present specification embodiment provides a flow chart of a recharging method for a mobile robot. Figure 2 , the recharging method includes:

[0040] S210, obtaining a base station image captured by a camera of the mobile robot at a first reference position.

[0041] The base station image includes the base station outline.

[0042] S220: Determine a first motion path based on position information of the base station outline in the base station image.

[0043] S230, controlling the mobile robot to perform recharging along the first motion path.

[0044] In the embodiments of the present specification, a monocular camera may be configured on the mobile robot to identify the base station by capturing images in the working environment. The first reference position may be any position in a designated area in front of the base station, and at any first reference position in the designated area, the image captured by the monocular camera of the mobile robot facing the base station may include a complete outline of the base station.

[0045] The mobile robot is controlled to be located at the first reference position and photographs the base station in order to more accurately obtain the outline of the base station, thereby making the first motion path more accurate.

[0046] Specifically, after the base station image is acquired, the base station outline in the base station image is acquired through an image recognition algorithm. Then, the first motion path is determined according to the position information of the base station outline in the base station image. The mobile robot can directly enter the base station along the first motion path for recharging. A back-to-station backup point can also be pre-set for the base station, and the mobile robot can be controlled to reach the back-to-station backup point along the first motion path, and then enter the base station through the back-to-station backup point to perform recharging.

[0047] Through the above embodiment, the mobile robot is controlled to shoot the base station at the first reference position to obtain the base station image, and then the first motion path back to the base station is determined according to the position information of the base station outline in the base station image, and the recharging is performed along the first motion path. The recharging method of the embodiment of this specification, by taking pictures with a camera, is not limited by the signal range and the viewing angle range, and improves the efficiency of the mobile robot when recharging in the case of a deeper base station.

[0048] In some embodiments of the present specification, before acquiring the base station image captured by the mobile robot through the camera at the first reference position, the recharging method of the mobile robot further includes: controlling the mobile robot to move to the first reference position preset for the target object, wherein the position of the target object and the base station is relatively fixed.

[0049] In an embodiment of the present specification, a monocular camera may be configured on the mobile robot to identify the target object or base station by capturing images in the working environment. The target object is a device or object that is relatively fixed to the base station. The mobile robot may be set at the bottom of the target object or as a part of the target object. Exemplarily, for a mobile robot that can wash and mop at the same time, its base station may be set at the bottom of the washing machine to connect with the waterway, or at the bottom of the sink to connect with the waterway. It can be seen that for a mobile robot used for cleaning work at home, the target object may be a household appliance or furniture on which a base station can be fixedly set, such as a washing machine, a dishwasher, a refrigerator, an air conditioner or a dishwashing sink, an integrated stove, an air purifier, etc.

[0050] The first reference position may be any position in a designated area in front of the target object. At any first reference position in the designated area, an image captured by the monocular camera of the mobile robot facing the target object may include a complete base station outline.

[0051] In the case where the mobile robot has pre-stored a work map of the work area and marked the position of the target object, the mobile robot can be directly controlled to move to the first reference position preset for the target object according to the marked position. In the case where the mobile robot is unknown to the position of the target object, a monocular camera can be used to photograph the work environment to obtain an image of the work environment. When the outline of the target object is extracted from the work environment image, the position of the target object is determined and the mobile robot is controlled to move to the first reference position according to the position of the outline of the target object in the work environment image. The mobile robot can also photograph the work environment according to a preset shooting frequency while performing other tasks. When the target object appears in the image, the coordinate position of the image is recorded. Then, when it is necessary to perform a recharging task, the mobile robot is directly controlled to move to the recorded coordinate position, thereby guiding the mobile robot to move to the first reference position according to the photographed image.

[0052] In some embodiments of the present specification, controlling the mobile robot to move along the first motion path to perform recharging includes: controlling the mobile robot to move along the first motion path to a second reference position preset for the base station to perform recharging at the second reference position. The mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.

[0053] The mobile robot is controlled to reach the first reference position preset for the target object in order to be able to quickly reach the vicinity of the base station. However, due to the large depth of the base station in the embodiment of the present specification, the mobile robot needs a more precise route when entering the base station to perform the recharging action to reduce the possibility of collisions. The second reference position can be understood as a preset position for the mobile robot to accurately perform the recharging action, which is generally preset to a position on the central axis close to the base station. The mobile robot is located at the second reference position to take an image of the base station, and the base station is located in the center of the base station image. Therefore, after the mobile robot reaches the first reference position, the base station can be photographed using a monocular camera, and the mobile robot can be continuously adjusted using the base station image, so that the mobile robot can quickly and accurately reach the second reference position to smoothly perform the recharging action.

[0054] Specifically, when the mobile robot is at the first reference position, a base station image is obtained by photographing the base station using a monocular camera. The base station image must include the base station outline. For the base station image, an image recognition algorithm, such as an edge feature extraction algorithm, can be used to extract the outline of the object or device included in the base station image to determine the base station outline. According to the position information of the base station outline in the base station image, the time when the base station image is photographed and the orientation of the base station relative to the mobile robot can be determined, so that the first motion path can be determined based on the position information of the base station outline in the base station image. The mobile robot can reach the second reference position preset for the base station along the first motion path.

[0055] Exemplarily, when the mobile robot is located at the first reference position, a base station image is obtained by photographing the base station, and features are extracted from the base station image to obtain the base station outline in the base station image. The base station outline is located at the left side of the base station image. According to this position information, it can be known that the current base station is located in the left front position of the mobile robot. Therefore, it can be determined that the first motion path is to the left and forward. The viewing angle and resolution of the monocular camera can be set to be fixed when shooting images, and the distance between the mobile robot and the base station can be determined based on the size of the base station outline in the base station image, so as to more accurately determine the distance to the left or forward, so as to reach the second reference position more accurately and quickly.

[0056] Through the above embodiments, the mobile robot is first controlled to move to a first reference position preset for the target object, so as to achieve the purpose of quickly reaching the vicinity of the base station. Then, at the first reference position, the base station is photographed to obtain an image of the base station. According to the position information of the base station outline in the base station image, the first motion path to move to the second reference position preset for the base station can be determined. After controlling the robot to move to the second reference position along the first motion path, a recharging action can be performed at the second reference position to return to the base station. The recharging method of the embodiment of the present specification will not be limited by the signal range and the viewing angle range. In the case of a deeper base station position, the efficiency and accuracy of the mobile robot during recharging are improved.

[0057] In some embodiments of the present specification, determining a first motion path based on position information of a base station outline in a base station image includes: using the base station image to determine first depth information from the base station to a mobile robot; determining a first motion orientation of the first motion path according to the position information of the base station outline in the base station image; and determining the first motion path according to the first depth information and the first motion orientation.

[0058] In an embodiment of the present specification, the purpose of the mobile robot moving along the first motion path is to reach the second reference position. If the moving direction is determined only according to the position information of the base station outline in the base station image, its moving efficiency is relatively slow. Therefore, the first depth information from the base station to the first reference position can also be determined based on the base station image taken by the monocular camera, so as to determine the current distance from the base station to the mobile robot. The first motion direction of the first motion path can be determined according to the position information, and the motion distance for the first motion direction can be determined in combination with the first depth information. Exemplarily, based on the position of the mobile robot, after determining that the first motion direction is first to move to the left and then to move forward according to the position information of the base station outline in the base station image. Based on the first depth information, it can be determined that the first motion path is to move 50cm to the left and then move 20cm forward. In this way, after determining the motion distance for the first motion direction, the second reference position can be reached more quickly and accurately.

[0059] In some embodiments, after determining the first motion path, controlling the mobile robot to move along the first motion path to a second reference position preset for the base station may include: controlling the mobile robot to move along the first motion orientation by a motion distance corresponding to the motion orientation to reach the second reference position.

[0060] For example, based on the position information and the first depth information, it is determined that the first movement direction of the first movement path is leftward and then forward, and the movement distance for the first movement direction is 50 cm to the left and then 20 cm forward. Then the mobile robot can be controlled to move 50 cm to the left and then 20 cm forward to reach the second reference position.

[0061] It is understandable that while the mobile robot is moving along the first motion path, the base station can also be photographed in real time, and the motion direction and distance of the first motion path can be adjusted according to the newly photographed base station image to reach the second reference position more accurately.

[0062] The mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.

[0063] After the mobile robot reaches the second reference position, it is currently facing the base station. In general, the charging device is configured behind the mobile robot. Therefore, it can rotate 180 degrees in situ at the second reference position, move backward and straight into the base station, and stop moving backward after detecting the charging voltage to complete the recharging action. In this scenario, the preset heading angle is the azimuth angle of the mobile robot facing away from the base station along the central axis of the base station.

[0064] The depth information from the base station to the mobile robot can be obtained in any of the following ways: first, using the base station image taken by a monocular camera and the radar data obtained by the 2D laser radar of the mobile robot to determine the depth information from the base station to the mobile robot; second, configuring a depth sensor on the mobile robot, such as a depth laser radar, a depth camera or a binocular camera, to determine the depth information from the base station to the mobile robot. Since the monocular camera is used to determine the position in the embodiment of this specification, the first method is preferably used to determine the depth information, which is lower in cost and has sufficient accuracy.

[0065] Exemplarily, a monocular visual depth estimation algorithm can be used to calculate the depth information from the base station to the mobile robot. The algorithm can fuse the scale information of the 2D laser radar and the texture structure information of the monocular image, so that the accuracy of the depth estimation is greatly improved. A trained monocular depth estimation network is configured in the mobile robot. When depth estimation is required, the input is an RGB image and a radar data mapped to a two-dimensional image. After processing by the monocular depth estimation network, a depth map (the depth value corresponding to each pixel position in the RGB image) can be obtained. In an embodiment of the present specification, the base station image taken by a monocular camera at a first reference position and the radar data obtained by a laser radar at the first reference position are input into the monocular depth estimation network for processing, and the first depth information from the base station to the mobile robot, that is, the depth value from the base station to the mobile robot, can be obtained.

[0066] In some embodiments of the present specification, before determining the first motion path according to the first depth information and the first motion orientation, the recharging method of the mobile robot further includes: determining an obstacle condition on the first motion path.

[0067] Determining the first motion path according to the first depth information and the first motion orientation includes: determining a motion distance along a straight line of the first motion orientation according to the first depth information and the first motion orientation to obtain the first motion path when there is no obstacle on the first motion path; or determining the first motion path according to position information of the obstacle, the first depth information, and the first motion orientation when there is an obstacle on the first motion path.

[0068] In some cases, there may be obstacles, such as water stains, socks, etc., in the area between the current position of the mobile robot and the base station, or to the second reference point. Therefore, before determining the first motion path, the situation of obstacles in the area may also be determined. For example, it is possible to identify whether there are obstacles by taking a base station image, or to determine whether there are obstacles by scanning the environment with a laser radar.

[0069] In the case where there are no obstacles on the first motion path, the first depth information from the base station to the first reference position is determined based on the base station image captured by the monocular camera, thereby determining the current distance from the base station to the mobile robot. The first motion orientation of the first motion path can be determined based on the position information, and the motion distance along the first motion orientation can be determined in combination with the first depth information, thereby obtaining the first motion path. The first motion orientation may include multiple orientations.

[0070] In the case where there is an obstacle on the first motion path, the first motion path that can avoid the obstacle is determined by combining the distance determined according to the first depth information, the first motion direction, and the position information of the obstacle.

[0071] For example, if the second reference position is determined by the first depth information and the position information of the base station outline in the base station image, it is located 50 cm in front of the left side of the mobile robot. If there are no obstacles in the area between the mobile robot and the base station, the first motion path can be determined to be 50 cm straight along the left front to reach the second reference position. If there are obstacles in the area, the mobile robot can be controlled to move 30 cm to the left first, and then move 40 cm forward to reach the second reference position.

[0072] In some embodiments of the present specification, the base station profile of the base station image is determined by: performing edge feature extraction on a plurality of first objects to be verified in the base station image to obtain profile features corresponding to the plurality of first objects to be verified. Among the profile features corresponding to the plurality of first objects to be verified, a target profile feature that matches the base station profile feature preset for the base station is determined. The profile of the object to be verified corresponding to the target profile feature is the base station profile.

[0073] Specifically, after acquiring the base station image, edge features of several first objects to be verified contained in the base station image are extracted based on an edge feature extraction algorithm to obtain an edge feature map corresponding to the base station image. The edge feature map includes contour features corresponding to several first objects to be verified. The contour features of the base station are pre-stored in the mobile robot. The contour features corresponding to several first objects to be verified are respectively compared with the pre-stored base station contour features preset for the base station to determine the target contour features that match the base station contour features. It is determined that the contour of the object to be verified corresponding to the target contour feature is the base station contour, that is, the object to be verified is determined to be a base station. Among them, edge feature extraction can also be performed based on a preset image recognition algorithm or image recognition model, such as a deep learning algorithm, a neural network model, etc.

[0074] In some embodiments of the present specification, controlling the mobile robot to move to a first reference position preset for a target object includes: if an object contour of the target object is extracted based on an image of the working environment of the mobile robot, determining a second motion path according to position information of the object contour in the image of the working environment, and controlling the mobile robot to move to the first reference position along the second motion path.

[0075] In some cases, the mobile robot is unaware of the location and room of the target object. When recharging is needed, the mobile robot needs to use a monocular camera to photograph the working environment, determine the location of the target object based on the working environment image, and thus reach the first reference position.

[0076] Specifically, when the mobile robot triggers the recharging task, it starts to collect the working environment image of its working environment in real time from the current position. Different shooting sampling points can be pre-set for different room types. When the room type is identified based on the furniture in the working environment image, the working environment image can be shot according to the shooting sampling points of the room, which can reduce a lot of invalid image shooting, improve image processing work, and speed up the recognition of target objects.

[0077] In the case where the object contour of the target object is extracted based on the working environment image, it can be determined that the position where the working environment image is taken is near the target object. Based on the position information of the object contour of the target object in the working environment image, the relative position of the target object relative to the position of the mobile robot when the working environment image is taken can be determined. In the same way as the determination of the first motion path in the above embodiment, the second motion path can be determined based on the position information of the object contour in the working environment image, that is, it is determined that the mobile robot should move left, right, forward and / or backward to reach the first reference position.

[0078] In some embodiments of the present specification, controlling the mobile robot to move along the second motion path to the first reference position includes: in the process of controlling the mobile robot to move along the second motion path, capturing an object environment image for the target object. The object environment image includes a target object outline. The second motion path is adjusted based on position information of the target object outline in the object environment image and second depth information from the target object to the mobile robot, and the mobile robot is controlled to move along the adjusted second motion path to reach the first reference position.

[0079] In some cases, the mobile robot is controlled to reach the first reference position first in order to be able to quickly reach the vicinity of the base station to speed up the recharging efficiency. In the embodiments of the present specification, the positions of the target object and the base station are relatively fixed. Therefore, the first reference position can be set to be located on the central axis of the target object, so that when the mobile robot is located at the first reference position, the target object is photographed to obtain an operating environment image, and the target object is in the center of the operating environment image. In this way, when shooting the base station based on the first reference position, the efficiency of the final moving second reference position will be improved.

[0080] Specifically, when the object contour of the target object is extracted based on the working environment image, the position where the mobile robot shoots the working environment image must be located in a designated area where the target object can be photographed, but it is not necessarily located on the central axis of the target object. Therefore, in order to improve the recharging efficiency, when the first reference position is set to be located in the designated area and on the central axis of the target object, the initial second motion path can be determined based on the object contour in the working environment image. In the process of controlling the mobile robot to move along the second motion path, continue to shoot images for the target object to obtain the object environment image. At this time, the captured object environment image must contain the contour of the target object. Through the object environment images continuously obtained during the movement, the orientation and distance of the target object relative to the mobile robot can be continuously updated. Therefore, the second motion path can be continuously adjusted based on the position information of the target object contour in the object environment image and the second depth information from the target object to the mobile robot, so that the process of the mobile robot reaching the first reference position along the adjusted second motion path is faster and more accurate.

[0081] The method for determining the second depth information from the target object to the mobile robot by using the object environment image may refer to the method for determining the first depth information described above, which will not be described in detail herein.

[0082] In some embodiments of the present specification, before controlling the mobile robot to move to a first reference position preset for the target object, the recharging method further includes: if the object contour of the target object is extracted based on the working environment image of the mobile robot, recording the coordinate position of the captured working environment image.

[0083] Controlling the mobile robot to move to a first reference position preset for a target object includes: controlling the mobile robot to move to a coordinate position. Acquiring an object environment image obtained by photographing the target object at the coordinate position. The object environment image includes a target object contour. Determining a third motion path based on position information of the target object contour in the object environment image. Controlling the mobile robot to move to the first reference position along the third motion path.

[0084] In some cases, in order to improve the recharging efficiency of the mobile robot, it can be set to take pictures of the working environment and determine the location of the target object based on the taken working environment image when the mobile robot performs other tasks. In this case, to avoid the mobile robot from being stuck due to processing too much data due to performing multiple tasks at the same time, a slower shooting frequency can be set to take pictures or set the rotation angle to take another working environment image for processing after reaching a certain threshold. For example, when the sweeping robot is performing a cleaning task, it can be set to take a working environment image every 90 degrees of rotation in a room to determine whether there is a target object in the room.

[0085] Specifically, if the object contour of the target object is extracted based on the working environment image taken by the mobile robot during the execution of other tasks, it can be determined that the target object is located in the room, and the coordinate position of the working environment image can be recorded. When the mobile robot needs to perform a recharging task, the mobile robot is controlled to move to the coordinate position first. The target object is photographed from the coordinate position to obtain an object environment image. At this time, the object environment image must include the contour of the target object. Then, a third motion path to reach the first reference position is determined based on the object environment image. Among them, the method for determining the third motion path is the same as the principle for determining the second motion path, and the third motion path can be determined based on the position information of the target object contour in the object environment image and the third depth information from the base station to the mobile robot. In the process of controlling the mobile robot to move along the third motion path, the object environment image can also be continuously photographed to adjust the third motion path, so that the mobile robot moves to the first reference position more quickly and accurately.

[0086] In some embodiments of the present specification, extracting the object contour of the target object based on the working environment image of the mobile robot includes: extracting edge features of a plurality of second objects to be verified in the working environment image to obtain contour features corresponding to the plurality of second objects to be verified. If any of the contour features corresponding to the plurality of second objects to be verified matches the object contour features preset for the target object, it is determined that the object contour of the target object is extracted based on the working environment image.

[0087] In an embodiment of the present specification, the way in which the mobile robot extracts the object contour of the target object through the working environment image is the same as the way in which the base station contour is extracted from the base station image. The object contour of the target object is pre-stored in the mobile robot. After acquiring the working environment image, the edge feature extraction algorithm is used to extract the edge feature of the second object to be verified contained in the working environment image to obtain an edge feature map corresponding to the working environment image. The edge feature map includes contour features corresponding to several second objects to be verified. The contour features corresponding to several second objects to be verified are respectively compared with the pre-stored contour features preset for the target object. If among the contour features corresponding to several second objects to be verified, there is one contour feature of the second object to be verified that matches the object contour feature preset for the target object, then the contour of the second object to be verified can be determined to be the object contour of the target object, that is, the second object to be verified is determined to be the target object.

[0088] In some embodiments of the present specification, the base station is disposed at the bottom of a target object or as a part of the target object, and the target object includes household appliances and furniture. The household appliances include clothing processing equipment.

[0089] The recharging method of the embodiment of this specification is applicable to the case where the base station has a large depth. For example, in a sweeping robot that can wash and mop at home, its base station can be set at the bottom of the target object, and the target object can be a household appliance for clothing processing, such as a washing machine. By setting the base station at the bottom of the washing machine, the water path of the washing machine can be connected, making the cleaning work of the sweeping robot more convenient.

[0090] In some embodiments, the base station may also be a part of a household appliance; or when the base station of the mobile robot does not need to be connected to a waterway, the base station may also be set at the bottom of a piece of furniture or be a part of the furniture.

[0091] Among them, the household appliances may include clothing processing equipment, which may be washing machines, dryers, and integrated washing and drying machines. The household appliances may also be any one of refrigerators, dishwashers, televisions, air purifiers, water purifiers, air conditioners, steaming and baking equipment, sinks, integrated stoves, etc. The furniture may be any one of shoe cabinets, TV cabinets, bookcases, bathtubs, fish tanks, cabinets, toilets, fixed tables and chairs, bars, washbasins, etc.

[0092] In some embodiments of this specification, please refer to Figure 3 , the recharging method of the mobile robot may include the following steps:

[0093] S310: If an object contour of the target object is extracted based on the working environment image of the mobile robot, a second motion path is determined according to position information of the object contour in the working environment image.

[0094] S320, controlling the mobile robot to move along the second motion path to a first reference position preset for the target object.

[0095] Among them, the positions of the target object and the base station are relatively fixed.

[0096] S330: Acquire a base station image obtained by photographing the base station at a first reference position.

[0097] The base station image includes the base station outline.

[0098] S340, using the base station image, determine first depth information from the base station to the mobile robot.

[0099] S350: Determine a first motion direction of a first motion path according to position information of a base station outline in the base station image.

[0100] S360: Determine a movement distance for the first movement direction according to the first depth information.

[0101] S370, controlling the mobile robot to move along the first movement orientation by a movement distance corresponding to the movement orientation to reach a second reference position to perform a recharging action.

[0102] In some embodiments of this specification, please refer to Figure 4 The recharging method of the mobile robot may further include the following steps:

[0103] S402: If the object contour of the target object is extracted based on the working environment image of the mobile robot, the coordinate position of the captured working environment image is recorded.

[0104] S404, controlling the mobile robot to move to the coordinate position.

[0105] S406, acquiring an object environment image obtained by photographing the target object at the coordinate position.

[0106] The object environment image includes the target object contour.

[0107] S408: Determine a third motion path based on the position information of the target object contour in the object environment image.

[0108] S410, controlling the mobile robot to move along a third motion path to a first reference position preset for a target object.

[0109] S412: Acquire a base station image obtained by photographing the base station at a first reference position.

[0110] The base station image includes the base station outline.

[0111] S414, using the base station image, determine first depth information from the base station to the mobile robot.

[0112] S416: Determine a first motion orientation of the first motion path according to position information of the base station outline in the base station image.

[0113] S418: Determine a movement distance for the first movement direction according to the first depth information.

[0114] S420, controlling the mobile robot to move along the first movement orientation by a movement distance corresponding to the movement orientation to reach a second reference position to perform a recharging action.

[0115] For the above embodiments, please refer to Figure 5 The embodiment of this specification also proposes a recharging device for a mobile robot. The recharging device includes:

[0116] The image acquisition module 510 is used to acquire the base station image taken by the mobile robot at the first reference position through the camera.

[0117] The base station image includes the base station outline.

[0118] The path determination module 520 is configured to determine a first motion path based on position information of the base station outline in the base station image.

[0119] The recharging module 530 is used to control the mobile robot to perform recharging along the first motion path.

[0120] According to the recharging device of the mobile robot of the embodiment of this specification, the mobile robot is controlled to shoot the base station at the first reference position to obtain the base station image, and then the first motion path of returning to the station is determined according to the position information of the base station outline in the base station image, and the recharging is performed along the first motion path. The recharging method of the embodiment of this specification, by taking pictures with a camera, is not limited by the signal range and the viewing angle range, and improves the efficiency of the mobile robot when recharging in the case of a base station located deep.

[0121] For the specific definition of the recharging device of the mobile robot, please refer to the definition of the recharging method of the mobile robot above, which will not be repeated here. Each module in the above-mentioned recharging device of the mobile robot can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0122] Figure 6 is a block diagram of a mobile robot according to an embodiment of the present specification. Figure 6As shown, the mobile robot 600 includes a memory 604, a processor 602, and a mobile robot recharging program 606 stored in the memory 604 and executable on the processor 602. When the processor 602 executes the mobile robot recharging program 606, the mobile robot recharging method of any one of the above-mentioned embodiments is implemented.

[0123] In some embodiments of the present specification, the base station corresponding to the mobile robot is arranged at the bottom of the target object, and the target object includes a household appliance.

[0124] In some embodiments of the present specification, the household appliance includes a clothing processing device.

[0125] According to the mobile robot of the embodiment of the present specification, when the processor 602 executes the recharging program 606 of the mobile robot, the mobile robot will not be limited by the signal range and the viewing angle range, and in the case where the base station is located deeper, the efficiency of the mobile robot during recharging is improved.

[0126] Corresponding to the above embodiments, the embodiments of this specification further propose a computer-readable storage medium, on which a recharging program of a mobile robot is stored, and when the recharging program is executed by a processor, the recharging method of the mobile robot as described in any one of the above is implemented.

[0127] According to the computer-readable storage medium of the embodiment of this specification, when the recharging program is executed by the processor, the mobile robot will not be limited by the signal range and the viewing angle range. In the case where the base station is located deeper, the efficiency of the mobile robot during recharging is improved.

[0128] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0129] It should be understood that various parts of this specification can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0131] In the description of this specification, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this specification and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on this specification.

[0132] In addition, the terms "first", "second", etc. used in the embodiments of this specification are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in this embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of this specification can explicitly or implicitly indicate that the embodiment includes at least one of the features. In the description of this specification, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0133] In this specification, unless otherwise clearly specified or limited in the embodiments, the terms "install", "connect", "connect" and "fix" etc. appearing in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can be understood that it can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements, or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this specification can be understood according to the specific implementation situation.

[0134] In this specification, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0135] Although the embodiments of the present specification have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limitations on the present specification. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present specification.

Claims

1. A recharging method for a mobile robot, characterized in that: The method comprises: Acquire a base station image captured by a camera of the mobile robot at a first reference position; wherein the base station image includes a base station outline; Determine a first motion path based on position information of the base station outline in the base station image; The mobile robot is controlled to perform recharging along the first motion path.

2. The method according to claim 1, characterized in that The controlling the mobile robot to move along the first motion path to perform recharging includes: The mobile robot is controlled to move along the first motion path to a second reference position preset for the base station to perform recharging at the second reference position; wherein the mobile robot can go straight into the base station after rotating to a preset orientation angle at the second reference position.

3. The method according to claim 1, characterized in that Determining a first motion path based on position information of the base station outline in the base station image includes: Determine first depth information from the base station to the mobile robot using the base station image; Determine a first motion orientation of the first motion path according to position information of the base station outline in the base station image; The first motion path is determined according to the first depth information and the first motion orientation.

4. The method according to claim 3, characterized in that Before determining the first motion path according to the first depth information and the first motion orientation, the method further includes: Determining obstacles on the first motion path; Determining the first motion path according to the first depth information and the first motion orientation includes: In a case where there is no obstacle on the first motion path, determining a motion distance moving in a straight line along the first motion orientation according to the first depth information and the first motion orientation to obtain the first motion path; or In the case where there is an obstacle on the first motion path, the first motion path is determined according to the position information of the obstacle, the first depth information, and the first motion orientation.

5. The method according to claim 1, characterized in that The base station contour of the base station image is determined in the following manner: Extracting edge features of a plurality of first objects to be verified in the base station image to obtain contour features corresponding to the plurality of first objects to be verified; Among the contour features corresponding to the plurality of first objects to be verified, a target contour feature matching a base station contour feature preset for the base station is determined; wherein the contour of the object to be verified corresponding to the target contour feature is the base station contour.

6. The method according to claim 1, characterized in that Before acquiring the base station image captured by the mobile robot at the first reference position through the camera, the method further includes: The mobile robot is controlled to move to the first reference position preset for the target object; wherein the positions of the target object and the base station are relatively fixed.

7. The method according to claim 6, characterized in that Controlling the mobile robot to move to the first reference position preset for the target object includes: If an object contour of the target object is extracted based on the working environment image of the mobile robot, a second motion path is determined according to position information of the object contour in the working environment image; The mobile robot is controlled to move along the second motion path to the first reference position.

8. The method according to claim 7, characterized in that Controlling the mobile robot to move to the first reference position along the second motion path includes: In the process of controlling the mobile robot to move along the second motion path, capturing an object environment image for the target object; wherein the object environment image includes a contour of the target object; The second motion path is adjusted based on the position information of the target object outline in the object environment image and the second depth information from the target object to the mobile robot, and the mobile robot is controlled to move along the adjusted second motion path to reach the first reference position.

9. The method according to claim 6, characterized in that Before controlling the mobile robot to move to a first reference position preset for the target object, the method further includes: If an object contour of the target object is extracted based on the working environment image of the mobile robot, a coordinate position of shooting the working environment image is recorded; The controlling the mobile robot to move to a first reference position preset for the target object comprises: Controlling the mobile robot to move to the coordinate position; Acquire an object environment image obtained by photographing the target object at the coordinate position; wherein the object environment image includes a contour of the target object; Determine a third motion path based on position information of the target object contour in the object environment image; The mobile robot is controlled to move along the third motion path to the first reference position.

10. The method according to claim 7 or 9, characterized in that: Extract the object contour of the target object based on the working environment image of the mobile robot, including: Extracting edge features of a plurality of second objects to be verified in the working environment image to obtain contour features corresponding to the plurality of second objects to be verified; When any contour feature of the second objects to be verified among the contour features corresponding to the plurality of second objects to be verified matches an object contour feature preset for the target object, it is determined that the object contour of the target object is extracted based on the working environment image.

11. The method according to any one of claims 6 to 9, characterized in that: The base station is arranged at the bottom of the target object or as a part of the target object, and the target object includes household electrical appliances and furniture.

12. The method according to claim 11, characterized in that The household appliance includes a clothes processing device.

13. A mobile robot, characterized in that: It includes a memory, a processor, and a recharging program for a mobile robot stored in the memory and executable on the processor. When the processor executes the program, it implements the recharging method for the mobile robot as described in any one of claims 1 to 12.

14. The mobile robot according to claim 13, characterized in that: The base station corresponding to the mobile robot is arranged at the bottom of the target object or as a part of the target object, and the target object includes household electrical appliances and furniture.

15. The mobile robot according to claim 14, characterized in that: The household appliance includes a clothes processing device.

16. A computer-readable storage medium, characterized in that: A recharging program of the mobile robot is stored thereon, and when the program is executed by the processor, the recharging method of the mobile robot as described in any one of claims 1-11 is implemented.

Citation Information

Patent Citations

  • Vision charging base and location method thereof, and re-charge method of vision charging base

    CN108646727A

  • Return-charging control method, device and system for indoor mobile robot

    CN108733048A

  • Robot automatic charging method, device and system, terminal equipment and storage medium

    CN111061280A

  • Mobile robot autonomous charging method based on ArUco code guidance

    CN112183133A

  • Robot recharging method and device, electronic equipment and readable storage medium

    CN113467451A