Cleaning method and device, cleaning equipment, computer program product and storage medium

Abstract

The embodiment of the invention is suitable for the technical field of smart home, and provides a cleaning method and device, cleaning equipment, a computer program product and a storage medium. When objects such as obstacles are encountered for cleaning the ground, whether the mechanical arm of the cleaning robot can carry the objects or not can be judged according to the object information, and the objects are carried to the storage area through the mechanical arm of the cleaning robot under the condition that the judgment result is yes, and then cleaning is continued; therefore, the purpose of automatically storing objects such as obstacles on the ground is achieved, the time cost of manual storage by a user is saved, the product function of the cleaning robot is expanded, and the use experience of the user using the cleaning robot is improved. And on the other hand, after the articles are placed in the storage area, the bottoms of the articles such as obstacles and the surrounding ground can be further cleaned, so that the ground cleaning coverage rate is increased, and the ground cleaning effect and the user experience are improved.

Description

Technical Field

[0001] The embodiments of the present application belong to the technical field of smart home, and particularly relate to a cleaning method, device, cleaning equipment, computer program product and storage medium. Background Art

[0002] Cleaning robots (such as floor scrubbers, sweepers, etc.) are special robots engaged in cleaning, washing and other work, mainly serving fields such as home, commerce, medical care and industry. In recent years, with the deepening of the concept of smart home and the continuous progress of technology, cleaning robots have shown a trend of rapid growth.

[0003] However, existing cleaning robots can only complete the function of floor cleaning. When encountering obstacles during floor cleaning, the cleaning robot will actively avoid the obstacles. On the one hand, there is a need to store the obstacles, but existing cleaning robots cannot complete the storage of the obstacles. On the other hand, there is a problem of missed sweeping at the bottom and around the obstacles of the ground, resulting in a low cleaning coverage rate of the cleaning robot in a complex environment with multiple obstacles, and it is difficult to obtain satisfactory cleaning results for users. Summary of the Invention

[0004] The embodiments of the present application provide a cleaning method, device, cleaning equipment, computer program product and storage medium, which can solve the problems of low cleaning coverage rate and inability to handle obstacles when the cleaning robot cleans the ground in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a cleaning method, which is applied to a cleaning robot, and the cleaning robot includes a robotic arm. The cleaning method includes:

[0006] Obtain the item information of an item;

[0007] Judge whether the robotic arm of the cleaning robot can carry the item according to the item information, and obtain a judgment result;

[0008] When the judgment result is yes, control the robotic arm of the cleaning robot to carry the item to a storage area.

[0009] In a possible implementation manner of the first aspect, the storage area includes at least one of the following:

[0010] Add the storage area to the map of the cleaning area;

[0011] Determine the position of the storage container as the storage area;

[0012] The corner of the cleaning area.

[0013] In a possible implementation manner of the first aspect, the cleaning method further includes:

[0014] Detect whether there is a storage container in the cleaning area or the storage area;

[0015] If so, control the robotic arm of the cleaning robot to place the item into the storage container;

[0016] If not, control the robotic arm of the cleaning robot to place the item in the storage area.

[0017] In a possible implementation of the first aspect, the detecting whether there is a storage container in the cleaning area or the storage area includes:

[0018] Scan the space where the cleaning area or the storage area is located, and obtain the point cloud whose point cloud height exceeds the first height threshold in the space;

[0019] Fit the point cloud into one or more line segments, and detect whether the one or more line segments are the edges of the storage container;

[0020] If so, determine that there is a storage container in the storage area.

[0021] In a possible implementation of the first aspect, the detecting whether the one or more line segments are the edges of the storage container includes:

[0022] According to the lengths of the one or more line segments, determine the filtered line segments whose lengths exceed the first length threshold;

[0023] Determine whether the filtered line segments are the edges of the storage container by means of image recognition or preset two-dimensional code recognition.

[0024] In a possible implementation of the first aspect, the detecting whether there is a storage container in the cleaning area or the storage area includes:

[0025] Detect whether a preset two-dimensional code is pasted on the object in the cleaning area or the storage area;

[0026] If so, determine the object with the preset two-dimensional code pasted on it as the storage container in the storage area.

[0027] In a possible implementation of the first aspect, the controlling the robotic arm of the cleaning robot to carry the item into the storage container in the storage area includes:

[0028] Detect whether the height of the storage opening of the storage container is within the operating height range of the robotic arm;

[0029] If so, the robotic arm places the held item into the storage container through the storage opening of the storage container.

[0030] In a possible implementation of the first aspect, controlling the robotic arm of the cleaning robot to carry the item to the storage area includes:

[0031] According to the item type of the item, controlling the robotic arm to carry the item to the storage area corresponding to the item type.

[0032] In a second aspect, an embodiment of the present application provides a cleaning device. The cleaning device is applied to a cleaning robot, and the cleaning robot includes a robotic arm. The cleaning device includes:

[0033] A first acquisition module, configured to acquire item information of an item;

[0034] A first judgment module, configured to judge whether the robotic arm of the cleaning robot can carry the item according to the item information, and obtain a judgment result;

[0035] A first control module, configured to control the robotic arm of the cleaning robot to carry the item to the storage area when the judgment result is yes.

[0036] In a possible implementation of the second aspect, the cleaning device further includes:

[0037] A first addition module, configured to add the storage area to the map of the cleaning area; and / or

[0038] A first recording module, configured to determine the position of the storage container or a corner of the cleaning area as the storage area.

[0039] In a possible implementation of the second aspect, the cleaning device further includes:

[0040] A first recognition module, configured to detect whether there is a storage container in the cleaning area or the storage area;

[0041] The first control module is configured to:

[0042] Control the robotic arm of the cleaning robot to place the item into the storage container or place the item in the storage area.

[0043] In a possible implementation of the second aspect, the first recognition module is configured to:

[0044] Scan the space where the cleaning area or the storage area is located, and acquire the point cloud whose point cloud height exceeds the first height threshold in the space;

[0045] Fit the point cloud into one or more line segments, and detect whether the one or more line segments are the edges of the storage container;

[0046] If so, determine that the storage container exists in the storage area.

[0047] In a possible implementation manner of the second aspect, the first recognition module is further configured to:

[0048] Determine the filtered line segments with lengths exceeding a first length threshold according to the lengths of the one or more line segments;

[0049] Determine whether the filtered line segments are the edges of the storage container by means of image recognition or preset two-dimensional code recognition.

[0050] In a possible implementation manner of the second aspect, the first recognition module is configured to:

[0051] Detect whether a preset two-dimensional code is pasted on an object in the cleaning area or the storage area;

[0052] If so, determine the object pasted with the preset two-dimensional code as the storage container in the storage area.

[0053] In a possible implementation manner of the second aspect, the first control module is configured to:

[0054] Detect whether the height of the storage opening of the storage container is within the operating height range of the robotic arm;

[0055] If so, the robotic arm places the item held by it into the storage container through the storage opening of the storage container.

[0056] In a possible implementation manner of the second aspect, the first control module is configured to:

[0057] Control the robotic arm to carry the item to the storage area corresponding to the item type according to the item type of the item.

[0058] In a third aspect, an embodiment of the present application provides a cleaning device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the cleaning device implements the cleaning method according to any one of the above first aspects.

[0059] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is run, the cleaning method according to any one of the above first aspects is executed.

[0060] Fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the cleaning method as described in any one of the above first aspects.

[0061] The beneficial effects of the first aspect of the embodiments of the present application compared with the prior art are as follows:

[0062] The embodiments of the present application obtain item information; determine whether the robotic arm of the cleaning robot can carry the item according to the item information to obtain a determination result; in the case where the determination result is yes, carry the item to the storage area by the robotic arm of the cleaning robot; and continue cleaning. Existing cleaning robots can only perform the function of floor cleaning. The cleaning method provided by the embodiments of the present application is implemented by a cleaning robot, and the cleaning robot includes a robotic arm. When encountering an obstacle or other item during floor cleaning, the cleaning method provided by the embodiments of the present application can determine whether the robotic arm of the cleaning robot can carry the item according to the item information, and in the case where the determination result is yes, carry the item to the storage area by the robotic arm of the cleaning robot, and then continue cleaning. On the one hand, the cleaning method provided by the embodiments of the present application can realize the automatic storage of obstacles and other items on the floor, saving the time cost of manual storage by the user, expanding the product function of the cleaning robot, and improving the user experience of using the cleaning robot. On the other hand, after the item is placed in the storage area, the bottom and surrounding floor of the obstacle and other items can be further cleaned, thereby improving the cleaning coverage rate of the floor and enhancing the floor cleaning effect and user experience.

[0063] It can be understood that the beneficial effects of the above second to fifth aspects can refer to the relevant descriptions in the above first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0065] Figure 1 is a schematic flowchart of the cleaning method provided by an embodiment of the first embodiment of the present application;

[0066] Figure 2 is another schematic flowchart of the cleaning method provided by an embodiment of the first embodiment of the present application;

[0067] Figure 3It is another schematic flowchart of the cleaning method provided in the first embodiment of this application;

[0068] Figure 4 It is a schematic diagram of the effect of fitting point clouds into line segments provided in the first embodiment of this application;

[0069] Figure 5 It is a schematic diagram of the cleaning robot scanning and dotting provided in the first embodiment of this application;

[0070] Figure 6 It is a schematic diagram of a graphical user interface displaying a room map provided in the first embodiment of this application;

[0071] Figure 7 It is a schematic structural diagram of the cleaning device provided in the first embodiment of this application;

[0072] Figure 8 It is a schematic structural diagram of the cleaning equipment provided in the embodiment of this application. Detailed implementation manners

[0073] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that the embodiments of this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the embodiments of this application.

[0074] It should be understood that when used in the specification of the embodiments of this application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0075] It should also be understood that the term "and / or" as used in the specification of the embodiments of this application and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0076] As used in the specification of the embodiments of this application and the appended claims, the term "if" can be interpreted as "when...", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" can be interpreted as meaning "once determined", "in response to determining", "once detecting [the described condition or event]", or "in response to detecting [the described condition or event]" according to the context.

[0077] In addition, in the description of the specification of the embodiments of the present application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0078] Referring to "one embodiment" or "some embodiments" described in the specification of the embodiments of the present application means that in one or more embodiments of the embodiments of the present application, specific features, structures or characteristics described in connection with that embodiment are included. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0079] Figure 1 It is a schematic flow chart of the cleaning method provided by an embodiment of the embodiments of the present application.

[0080] The cleaning method provided by the embodiments of the present application is applied to a cleaning robot, and the cleaning robot includes a robotic arm.

[0081] See Figure 1 As shown, the cleaning method in the embodiments of the present application includes step S100 to step S300.

[0082] In step S100, item information of an item is obtained.

[0083] The item information of the item includes but is not limited to information such as the item name, item type, spatial information where the item is located, item volume, etc. The item name includes but is not limited to shoes, clothes, paper, pens, etc. The item type includes but is not limited to agglomerates, garbage, etc. The spatial information where the item is located includes but is not limited to the two-dimensional position point where the item is located, the three-dimensional space area where the item is located, etc.

[0084] In step S200, it is determined whether the robotic arm of the cleaning robot can carry the item according to the item information, and a determination result is obtained.

[0085] The conditions under which the robotic arm of the cleaning robot can carry an item can be preset, or the conditions under which the robotic arm of the cleaning robot cannot carry an item can be preset, and it is determined whether the above conditions are met according to the item information, and a determination result of whether the robotic arm of the cleaning robot can carry the item is obtained.

[0086] The conditions under which the robotic arm of the cleaning robot can carry an item include but are not limited to at least any one of the following conditions 11 and 12.

[0087] Condition 11: The volume of the object is smaller than the first volume threshold. If the volume of the object is smaller than the first volume threshold, a judgment result is obtained that the mechanical arm of the cleaning robot can carry the object.

[0088] Condition 12: The object type belongs to the object type that the robot arm supports grasping. For example, if the object type belongs to garbage, a judgment result is obtained that the robot arm of the cleaning robot can carry the object.

[0089] The conditions under which the robotic arm of the cleaning robot cannot carry objects include but are not limited to at least one of the following conditions 21 and 2.

[0090] Condition 21: The spatial information of the object belongs to an ungraspable area. For example, the two-dimensional location of the object belongs to the non-graspable area marked by the user, or the three-dimensional space where the object is located includes a grasping obstacle, and the cleaning robot's mechanical arm cannot carry the object.

[0091] Condition 22: The item is an item that the user is using. For example, if the item is a pair of shoes that the user is wearing, the result of the judgment that the robot arm of the cleaning robot cannot carry the item is obtained.

[0092] In step S300, when the judgment result is yes, the robot arm of the cleaning robot is controlled to move the items to the storage area.

[0093] If it is determined based on the item information that the cleaning robot's mechanical arm can carry the item, the cleaning robot can control the mechanical arm to grab the item. After grabbing the item, the cleaning robot moves the item to the storage area. After moving to the storage area, the cleaning robot controls the mechanical arm to release the item and place the item in the storage area.

[0094] After step S300 , the cleaning robot continues cleaning.

[0095] After moving the items to the storage area, the cleaning robot can clean according to the previously predetermined cleaning order, or it can re-determine the area to be cleaned and clean it, or it can perform additional cleaning on the location of the items or the area blocked by the items to improve the cleaning coverage.

[0096] The embodiments of the present application obtain item information; determine whether the robotic arm of the cleaning robot can carry the item according to the item information, and obtain a judgment result; in the case where the judgment result is yes, carry the item to the storage area through the robotic arm of the cleaning robot; and continue cleaning. Existing cleaning robots can only perform the function of floor cleaning. The cleaning method provided by the embodiments of the present application is implemented by a cleaning robot, and the cleaning robot includes a robotic arm. When encountering an obstacle or other item during floor cleaning, the cleaning method provided by the embodiments of the present application can determine whether the robotic arm of the cleaning robot can carry the item according to the item information, and in the case where the judgment result is yes, carry the item to the storage area through the robotic arm of the cleaning robot, and then continue cleaning. On the one hand, the cleaning method provided by the embodiments of the present application can achieve automatic storage of obstacles and other items on the floor, saving the time cost of manual storage by the user, expanding the product function of the cleaning robot, and enhancing the user experience of using the cleaning robot. On the other hand, after the item is placed in the storage area, the bottom and surrounding floor of the obstacle and other items can be further cleaned, thereby improving the cleaning coverage rate of the floor and enhancing the floor cleaning effect and the user experience.

[0097] In one embodiment, the storage area includes at least one of the following.

[0098] Storage area 1: Add the storage area to the map of the cleaning area.

[0099] The terminal device can display a graphical user interface, and the map of the cleaning area is displayed in the graphical user interface. The user can add a storage area on the graphical user interface displaying the map of the cleaning area by means of touch. For example, the user can input the shoe storage area and its location on the graphical user interface displaying the map of the cleaning area by means of touch. Another example is that the user can input the garbage storage area and its location on the graphical user interface displaying the map. Among them, the terminal device includes, but is not limited to, terminal devices such as mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality / virtual reality devices, laptop computers, ultra-mobile personal computers, netbooks, and personal digital assistants.

[0100] Storage area 2: Determine the position of the storage container as the storage area.

[0101] During the movement of the cleaning robot, the storage container can be identified through the sensors of the cleaning robot, and the storage container in the image can be identified by means of image recognition or by using an object recognition model. For example, the cleaning robot can identify storage containers such as trash cans, shoe racks, and laundry baskets through the sensors of the cleaning robot. And when the storage container is identified, the cleaning robot can record the position of the storage container and determine the position of the storage container as the storage area. Among them, the sensors include but are not limited to laser distance sensors (Laser Distance Sensor, abbreviated as LDS), time-of-flight sensors (Time of Flight, abbreviated as ToF), image acquisition and recognition sensors, etc.

[0102] Storage area 3: The corner of the cleaning area.

[0103] During the movement of the cleaning robot, the cleaning area can be scanned and recorded through the sensors of the cleaning robot. According to the spatial information of the cleaning area, the corner of the cleaning area can be determined as the storage area.

[0104] In the embodiments of the present application, the storage area includes at least one of the following: adding a storage area to the map of the cleaning area, determining the position of the storage container as the storage area, and the corner of the cleaning area. The storage area is determined in at least one of the above ways. The embodiments of the present application provide an implementation method for determining the storage area and its position, which can realize determining the storage area and its position based on the user's settings, and can also obtain the determination of the storage area and its position by identifying the storage container through the sensors of the cleaning robot. The corner of the cleaning area can also be determined as the storage area, so as to better meet the actual needs of users and improve the user experience.

[0105] In one embodiment, see Figure 2 As described, the cleaning method in this embodiment further includes step S400, step S500, and step S501.

[0106] In step S400, it is detected whether there is a storage container in the cleaning area or the storage area.

[0107] In step S500, if so, the robot arm of the cleaning robot is controlled to place the item in the storage container.

[0108] In step S501, if not, the robot arm of the cleaning robot is controlled to place the item in the storage area.

[0109] The cleaning robot can identify the storage container in the cleaning area or the storage area through a laser range finder sensor or a time-of-flight sensor. If a storage container is detected, the manipulator of the cleaning robot transports the item to a position near the storage container in the storage area and places the item into the storage container. If no storage container is detected, the manipulator of the cleaning robot can be controlled to place the item in the storage area. By placing the item in the storage container or the storage area, better item storage can be achieved.

[0110] In one embodiment, refer to Figure 3 As shown, in step S400, detecting whether there is a storage container in the cleaning area or the storage area includes: step S410, step S420, and step S430.

[0111] In step S410, scan the space where the cleaning area or the storage area is located to obtain the point cloud whose point cloud height exceeds the first height threshold in the space.

[0112] In step S420, fit the point cloud into one or more line segments and detect whether the one or more line segments are the edges of the storage container.

[0113] In step S430, if so, determine that there is a storage container in the storage area.

[0114] The cleaning robot can scan the space where the cleaning area or the storage area is located by using a laser distance sensor (abbreviated as LDS) or a time-of-flight sensor (abbreviated as ToF) and obtain the point cloud whose point cloud height exceeds the first height threshold in the space. The first height threshold can be, for example, 5 cm to 20 cm. Preferably, the first height threshold is 7 cm. Subsequently, the cleaning robot can calculate the point cloud of the optical sensor hitting the object to obtain the point cloud coordinates. Then, the cleaning robot uses the least squares method or other algorithms to fit the point cloud into one or more line segments, such as straight line segments or curve segments. As Figure 4 shown, Figure 4 The multiple dots in are the point cloud. The cleaning robot can fit the point cloud into multiple line segments, as shown by 41, 42, and 43 in Figure 4 Subsequently, the cleaning robot moves to a position near each line segment and determines whether the line segment is the edge of the storage container by means of image recognition or the like. If so, the storage container is recognized.

[0115] In an embodiment of the present application, by scanning the space where the cleaning area or the storage area is located, point clouds with a point cloud height exceeding a first height threshold in the space are obtained; the point clouds are fitted into one or more line segments, and it is detected whether the one or more line segments are the edges of the storage container; if so, it is determined that the storage container exists in the storage area. The embodiments of the present application realize detecting whether there is a storage container in the cleaning area or the storage area, provide an implementation manner for identifying the storage container, and improve the accuracy of cleaning and item storage.

[0116] In one embodiment, in step S420, detecting whether the one or more line segments are the edges of the storage container includes: step S421 and step S422.

[0117] In step S421, according to the lengths of the one or more line segments, filtered line segments with lengths exceeding a first length threshold are determined;

[0118] In step S422, by means of image recognition or preset QR code recognition, it is determined whether the filtered line segments are the edges of the storage container.

[0119] The first length threshold is, for example, 5 cm to 25 cm. Preferably, the first length threshold is 10 cm. After determining the filtered line segments with lengths exceeding the first length threshold, the cleaning robot can move to a position near the filtered line segments with lengths exceeding the first length threshold, for example, a position 20 cm to 30 cm near this part of the line segments, and use image recognition technology to perform object recognition on the images captured by the RGB camera to confirm whether the objects near each line segment are storage containers. For example, the cleaning robot can identify the line segments fitted by the point clouds as the edges of the trash can through image recognition technology. In addition, the cleaning robot can also perform object recognition by scanning a preset QR code. For example, the user can paste a predetermined preset QR code sticker on the trash can. When the cleaning robot moves to a position near the filtered line segments, it further scans the preset QR code through the RGB camera. When the predetermined preset QR code is scanned, the cleaning robot can confirm that the object near the line segment is a storage container.

[0120] In one embodiment, refer to Figure 3 As shown, in step S400, detecting whether there is a storage container in the cleaning area or the storage area includes: step S440 and step S450.

[0121] In step S440, it is detected whether a preset QR code is pasted on an object in the cleaning area or the storage area;

[0122] In step S450, if so, the object pasted with the preset QR code is determined to be the storage container in the storage area.

[0123] The user can paste a pre - set two - dimensional code sticker onto a storage container such as a trash can. After the cleaning robot moves to the storage area, it can use an RGB camera to scan whether a pre - set two - dimensional code is pasted on an object in the cleaning area or the storage area. When the pre - set two - dimensional code is scanned, the cleaning robot can determine the object with the pre - set two - dimensional code pasted on it as the storage container (such as a trash can) in the storage area, and the cleaning robot can move to a position near the storage container.

[0124] The embodiment of the present application scans whether a pre - set two - dimensional code is pasted on an object in the cleaning area or the storage area; if the pre - set two - dimensional code is scanned, the object with the pre - set two - dimensional code pasted on it is determined as the storage container in the storage area. The embodiment of the present application realizes detecting whether there is a storage container in the cleaning area or the storage area, provides an implementation method for identifying the storage container, and improves the accuracy of cleaning and item storage.

[0125] In one embodiment, as shown in Figure 3 In step S500, controlling the robotic arm of the cleaning robot to place the item into the storage container includes: step S510 and step S520.

[0126] In step S510, it is detected whether the height of the storage opening of the storage container is within the operating height range of the robotic arm.

[0127] In step S520, if so, the robotic arm places the item held by it into the storage container through the storage opening of the storage container.

[0128] After identifying the storage container in the storage area and moving to a position near the storage container, the cleaning robot can use a time - of - flight sensor to confirm the height of the storage opening of the storage container. If the height of the storage opening of the storage container is within the height range for dropping items corresponding to the robotic arm, the cleaning robot can raise the robotic arm to perform a forward dropping operation on the item held by it so that the item is placed into the storage container in the storage area. If the height of the storage opening of the storage container exceeds the height range for dropping items corresponding to the robotic arm, or if the cleaning robot fails to effectively identify the storage container, the cleaning robot can also place the item held by the robotic arm on the open ground near the storage area, or place the item held by the robotic arm in the corner closest to the storage area.

[0129] More specifically, the cleaning robot can scan the space near the storage container, obtain the point cloud again, and fit the point cloud into a line segment. The cleaning robot can select the line segment closest to the storage container, calculate the throwing distance, and then forward - drop the item held by the robotic arm. As shown in Figure 5As shown, the point 53 on the trash can 52 hit by the time-of-flight sensor of the cleaning robot 51 is as shown in the figure.

[0130] In the embodiment of the present application, when the height of the storage opening of the storage container is within the operating height range of the robotic arm, the cleaning robot places the held item into the storage container in the storage area by the robotic arm. The embodiment of the present application realizes controlling the robotic arm to place the item into the storage container in the storage area, expands the implementation method of placing the item into the storage container, and improves the accuracy of item storage.

[0131] In one embodiment, in step S300, controlling the robotic arm of the cleaning robot to carry the item to the storage area includes: controlling the robotic arm to carry the item to the storage area corresponding to the item type according to the item type of the item.

[0132] Among them, the item types include but are not limited to: shoes, fabrics, lumps, garbage, etc. Among them, the lumps include but are not limited to paper lumps, etc.

[0133] Each item type can respectively correspond to a specific storage area. For example, shoes correspond to the shoe storage area. Another example is that fabrics correspond to the fabric storage area. Another example is that lumps correspond to the lump storage area. Still another example is that garbage corresponds to the garbage storage area.

[0134] Alternatively, multiple item types can also correspond to a specific storage area. For example, lumps and garbage both correspond to the lump storage area.

[0135] Alternatively, one item type can also correspond to multiple storage areas. For example, garbage can correspond to two garbage storage areas. Correspondingly, the garbage can be placed in any one of the two storage areas. Or, the way of garbage classification can be realized according to the user's setting.

[0136] In the above illustrative content of the correspondence between item types and storage areas, the positions of each storage area can all be different. Or, the positions of some or all of the storage areas can also be the same.

[0137] Those skilled in the art should understand that the correspondence between the above-listed and other unlisted item types and storage areas all fall within the protection scope of the embodiments of the present application.

[0138] When the cleaning robot is cleaning in the area of the room map, it can scan obstacles and other items on the ground through a camera. Moreover, the cleaning robot can control the robotic arm to grab the item in order to obtain the item and achieve the storage of the item. Further, the cleaning robot can also obtain an image of the item through the camera and identify the item type of the item image by means of image recognition. After identifying the item type of the item held by the robotic arm, the cleaning robot can move to the position of the storage area corresponding to the item type according to the item type of the item held by the robotic arm and the corresponding relationship between the item type and the storage area, so as to move the item to the storage area corresponding to the item type.

[0139] In one embodiment, the position of the storage area corresponding to the item type is determined according to the position of the storage area set by the user on the map.

[0140] The terminal device can display a graphical user interface, and the map is displayed in the graphical user interface. The user can input the position of the storage area corresponding to the item type on the graphical user interface displaying the map by means of touch. For example, the user can input the position of the shoe storage area corresponding to shoes on the graphical user interface displaying the map by means of touch. Also, for example, the user can input the position of the garbage storage area corresponding to garbage on the graphical user interface displaying the map by means of touch. Among them, the terminal device includes but is not limited to terminal devices such as mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality / virtual reality devices, laptop computers, ultra-mobile personal computers, netbooks, and personal digital assistants.

[0141] Figure 6 It is a schematic diagram of a graphical user interface showing a room map provided by an embodiment of Embodiment 1 of this application.

[0142] As Figure 6As shown in the figure, the terminal device provides an interactive function of a graphical user interface, and the graphical user interface includes a first prompt card 21, a room map card 22, a cleaning robot icon 23, a first area control 241, a second area control 242, a third area control 243, and a storage area position icon 25. The text on the first prompt card 21 can be displayed as, for example, "Please add the position of the storage area" or "Please set the position of the storage area in an open area". The room map card 22 is used to display the room map established by the cleaning robot through scanning. The cleaning robot icon 23 is used to display the current position of the cleaning robot in the room map. The first area control 241, the second area control 242, and the third area control 243 are used to distinguish different types of storage areas. The text on the first area control 241 can be displayed as, for example, "Shoe storage area". When the first area control 241 is triggered, the position of the storage area input by the user on the graphical user interface displayed on the terminal device is determined as the position of the "Shoe storage area". The text on the second area control 242 can be displayed as, for example, "Fabric storage area". When the second area control 242 is triggered, the position of the storage area input by the user on the graphical user interface displayed on the terminal device is determined as the position of the "Fabric storage area". The text on the third area control 243 can be displayed as, for example, "Ball-shaped object storage area". When the third area control 243 is triggered, the position of the storage area input by the user on the graphical user interface displayed on the terminal device is determined as the position of the "Ball-shaped object storage area". The storage area position icon 25 is used to display the position and size of the current storage area. The user can move the storage area position icon 25 through a moving operation to adjust the position of the current storage area. The user can also change the size of the storage area position icon 25 through a stretching operation to adjust the size of the current storage area.

[0143] In the embodiment of the present application, by determining the position of the storage area corresponding to the item type according to the position of the storage area set by the user on the map, the position of the storage area corresponding to one or more item types in the map is obtained, which expands the implementation method of obtaining the position of the storage area corresponding to the item type, can realize obtaining the position of the storage area corresponding to the item type based on the user's setting method, can better meet the actual needs of the user, and improves the user experience.

[0144] In one embodiment, the cleaning method in the embodiment of the present application further includes step S600 and step S700.

[0145] In step S600, when the cleaning robot cleans in the area where the room map is located, one or more storage containers corresponding to one or more item types are identified;

[0146] In step S700, when at least one storage container corresponding to an item type is recognized, the position of the storage container is determined as the position of the storage area corresponding to the item type in the room map.

[0147] Here, different item types can correspond to different storage containers. For example, shoes correspond to shoe boxes, shoe racks, etc. Fabrics correspond to laundry baskets, laundry boxes, etc. Garbage corresponds to trash cans, dustbins, etc.

[0148] When the cleaning robot cleans the area where the room map is located, it can collect images of objects in the room through a camera, and use an object recognition model to determine which storage container the object in the image belongs to through image recognition. For example, the cleaning robot can recognize storage containers such as trash cans, shoe racks, and laundry baskets according to the object recognition model. Further, for example, when a trash can (the storage container corresponding to garbage) is recognized, the cleaning robot can determine the position coordinates of the trash can as the position of the garbage storage area corresponding to garbage in the room map.

[0149] In the embodiment of the present application, by recognizing the storage container corresponding to the item type during the cleaning by the cleaning robot and determining the position of the storage container as the position of the storage area corresponding to the item type, the position of the storage area corresponding to one or more item types in the room map is obtained, which expands the implementation method of obtaining the position of the storage area corresponding to the item type, can automatically recognize the position of the storage area corresponding to the item type in the room map, can more quickly meet the user's usage needs, saves the user's operation cost, and improves the user's usage experience.

[0150] Figure 7 It is a schematic structural diagram of a cleaning device provided by an embodiment of the present application.

[0151] As Figure 7 shown, the cleaning device 5 is applied to a cleaning robot. The cleaning robot includes a robotic arm. The cleaning device 5 includes:

[0152] A first acquisition module 51 for acquiring item information of an item;

[0153] A first judgment module 52 for judging whether the robotic arm of the cleaning robot can carry the item according to the item information, and obtaining a judgment result;

[0154] A first control module 53 for controlling the robotic arm of the cleaning robot to carry the item to the storage area when the judgment result is yes.

[0155] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the cleaning device further includes:

[0156] A first adding module, configured to add the storage area to a map of a cleaning area; and / or

[0157] A first recording module, configured to determine a position of a storage container or a corner of a cleaning area as the storage area.

[0158] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the cleaning device 5 further includes:

[0159] A first identifying module, configured to detect whether there is a storage container in the cleaning area or the storage area;

[0160] The first control module 53 is further configured to:

[0161] Control a robotic arm of the cleaning robot to place the item into the storage container or place the item in the storage area.

[0162] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the first identifying module is configured to:

[0163] Scan a space where the cleaning area or the storage area is located, and obtain point clouds in the space whose point cloud height exceeds a first height threshold;

[0164] Fit the point clouds into one or more line segments, and detect whether the one or more line segments are edges of the storage container;

[0165] If so, determine that there is a storage container in the storage area.

[0166] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the first identifying module is further configured to:

[0167] Determine filtered line segments whose length exceeds a first length threshold according to the length of the one or more line segments;

[0168] Determine whether the filtered line segments are edges of the storage container by means of image recognition or preset two-dimensional code recognition.

[0169] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the first identifying module is specifically configured to:

[0170] Detect whether a preset two-dimensional code is pasted on an object in the cleaning area or the storage area;

[0171] If so, determine the object with the preset two-dimensional code pasted thereon as the storage container in the storage area.

[0172] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the first control module 53 is further configured to:

[0173] Detect whether the height of the storage opening of the storage container is within the operating height range of the robotic arm;

[0174] If so, the robotic arm places the held item into the storage container through the storage opening of the storage container.

[0175] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the first control module 53 is configured to:

[0176] According to the item type of the item, control the robotic arm to carry the item to the storage area corresponding to the item type.

[0177] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the position of the storage area corresponding to the item type is determined according to the position of the storage area set by the user on the map.

[0178] Another embodiment of the present invention discloses a cleaning device 5. On the basis of the above Figure 5 corresponding embodiment, the cleaning device 5 further includes:

[0179] A second recognition module, configured to recognize one or more storage containers respectively corresponding to one or more item types when the cleaning robot cleans in the area where the room map is located;

[0180] A first determination module, configured to determine the position of the storage container as the position of the storage area corresponding to the item type in the room map when at least one storage container corresponding to the item type is recognized.

[0181] It should be noted that for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part specifically, and details will not be elaborated here.

[0182] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments and will not be elaborated herein.

[0183] The embodiment of the present application also provides a cleaning device, such as Figure 8 shown. The cleaning device 7 includes: at least one processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70. When the processor 70 executes the computer program 72, the steps in any of the foregoing method embodiments are implemented.

[0184] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

[0185] The embodiment of the present application provides a computer program product, including a computer program, and when the computer program is run, the steps in the foregoing method embodiments are executed.

[0186] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0187] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0188] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of this application.

[0189] In the embodiments provided by the embodiments of this application, it should be understood that the disclosed device / electronic device and method can be implemented in other ways. For example, the device / electronic device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0190] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0191] The above-described embodiments are only used to illustrate the technical solutions of the embodiments of the present application, and are not intended to limit the same; although the embodiments of the present application have been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the embodiments of the present application.

Claims

1. A cleaning method, characterized in that: The cleaning method is applied to a cleaning robot, the cleaning robot includes a mechanical arm, and the cleaning method includes: Get item information of an item; Determine whether the mechanical arm of the cleaning robot can carry the object according to the object information, and obtain a determination result; When the judgment result is yes, the robot arm of the cleaning robot is controlled to move the article to the storage area.

2. The cleaning method according to claim 1, characterized in that The storage area includes at least one of the following: Adding the storage area to a map of the cleaning area; Determine the position of the storage container as the storage area; Clean the corners of the area.

3. The cleaning method according to claim 2, characterized in that: The cleaning method further comprises: Detecting whether there is a storage container in the cleaning area or the storage area; If yes, controlling the mechanical arm of the cleaning robot to place the object in the storage container; If not, the mechanical arm of the cleaning robot is controlled to place the article in the storage area.

4. The cleaning method according to claim 3, characterized in that: The detecting whether there is a storage container in the cleaning area or the storage area includes: Scanning the space where the cleaning area or the storage area is located, and acquiring a point cloud in the space whose point cloud height exceeds a first height threshold; Fitting the point cloud into one or more line segments, and detecting whether the one or more line segments are edges of the storage container; If so, it is determined that the storage container exists in the storage area.

5. The cleaning method according to claim 4, characterized in that: The detecting whether the one or more line segments are edges of the storage container includes: Determine, according to the length of the one or more line segments, the filtered line segments whose length exceeds a first length threshold; By means of image recognition or preset QR code recognition, it is determined whether the filtered line segment is the edge of the storage container.

6. The cleaning method according to claim 3, characterized in that: The detecting whether there is a storage container in the cleaning area or the storage area includes: Detecting whether a preset QR code is attached to an object in the cleaning area or the storage area; If so, the object with the preset two-dimensional code affixed thereto is determined as the storage container in the storage area.

7. The cleaning method according to claim 3, characterized in that: The step of controlling the mechanical arm of the cleaning robot to place the article in the storage container comprises: Detecting whether the height of the storage opening of the storage container is within the operating height range of the robotic arm; If so, the robotic arm places the held article into the storage container through the storage opening of the storage container.

8. The cleaning method according to any one of claims 1 to 7, characterized in that: The controlling the mechanical arm of the cleaning robot to move the article to the storage area includes: According to the article type of the article, the robot arm is controlled to move the article to a storage area corresponding to the article type.

9. A cleaning device, characterized in that: The cleaning device is applied to a cleaning robot, the cleaning robot comprises a mechanical arm, and the cleaning device comprises: A first acquisition module, used to acquire item information of an item; A first judgment module is used to judge whether the mechanical arm of the cleaning robot can carry the object according to the object information, and obtain a judgment result; The first control module is used to control the mechanical arm of the cleaning robot to move the article to the storage area when the judgment result is yes.

10. The cleaning device according to claim 9, characterized in that The cleaning device also includes: A first adding module is used to add the storage area to the map of the cleaning area; and / or The first recording module is used to determine the position of the storage container or the corner of the cleaning area as the storage area.

11. The cleaning device according to claim 10, characterized in that The cleaning device also includes: A first recognition module is used to detect whether there is a storage container in the cleaning area or the storage area; The first control module is also used for: The mechanical arm of the cleaning robot is controlled to place the article in the storage container or to place the article in the storage area.

12. The cleaning device according to claim 11, characterized in that The first identification module is specifically used for: Scanning the space where the cleaning area or the storage area is located, and acquiring a point cloud in the space whose point cloud height exceeds a first height threshold; Fitting the point cloud into one or more line segments, and detecting whether the one or more line segments are edges of the storage container; If so, it is determined that the storage container exists in the storage area.

13. The cleaning device according to claim 12, characterized in that The first identification module is also used for: Determine, according to the length of the one or more line segments, the filtered line segments whose length exceeds a first length threshold; By means of image recognition or preset QR code recognition, it is determined whether the filtered line segment is the edge of the storage container.

14. The cleaning device according to claim 11, characterized in that The first identification module is specifically used for: Detecting whether a preset QR code is attached to an object in the cleaning area or the storage area; If so, the object with the preset two-dimensional code affixed thereto is determined as the storage container in the storage area.

15. The cleaning device according to claim 11, characterized in that The first control module is also used for: Detecting whether the height of the storage opening of the storage container is within the operating height range of the robotic arm; If so, the robotic arm places the held article into the storage container through the storage opening of the storage container.

16. The cleaning device according to any one of claims 9 to 15, characterized in that The first control module is also used for: According to the article type of the article, the robot arm is controlled to move the article to a storage area corresponding to the article type.

17. A cleaning device, characterized in that: The cleaning device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the cleaning device implements the cleaning method according to any one of claims 1 to 8.

18. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables the cleaning method according to any one of claims 1 to 8 to be performed.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cleaning method according to any one of claims 1 to 8 is implemented.

Images