Article grabbing method and device and self-moving cleaning equipment

Abstract

The invention provides an article grabbing method and device and self-moving cleaning equipment, and relates to the technical field of cleaning equipment. For each article to be grabbed, according to the category of the article to be grabbed, the grabbing direction of the self-moving cleaning equipment relative to the article to be grabbed is determined when the self-moving cleaning equipment grabs the article to be grabbed; and according to the position coordinates of all the to-be-grabbed objects in the target area, whether corresponding obstacle objects exist within the preset distance of each to-be-grabbed object facing the corresponding grabbing direction or not is determined. And all the to-be-grabbed objects in the to-be-grabbed object set are sequenced, so that the object grabbing sequence is obtained, and the sequence of the to-be-grabbed objects with the obstacle objects is located after the sequence of the corresponding obstacle objects. According to the object grabbing sequence, all the to-be-grabbed objects in the to-be-grabbed object set are grabbed in sequence, and efficient and reliable grabbing of all the to-be-grabbed objects in the to-be-grabbed object set is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of cleaning equipment, and in particular to an object grabbing method, device and self-moving cleaning equipment. Background Art

[0002] With the improvement of living standards and the development of science and technology, consumers' demand for self-mobile cleaning equipment continues to increase; especially in the field of smart homes, self-mobile cleaning equipment has gradually become the mainstream choice for household cleaning.

[0003] At present, the self-mobile cleaning device includes a body for cleaning and a mechanical arm with adjustable posture for grasping. The self-mobile cleaning device can grasp multiple objects on the ground through the mechanical arm to complete functions such as object storage or object removal. For any object to be grasped, the self-mobile cleaning device needs to move to the grasping position associated with the object to be grasped so that the mechanical arm can grasp the object to be grasped more reliably (for example, when the object to be grasped is a shoe, it needs to move to the side of the heel of the shoe; when the object to be grasped is a stick, it needs to move to the side of the geometric center of the stick).

[0004] However, if there are obstacles in the path of the self-mobile cleaning device to the grabbing position, the self-mobile cleaning device needs to control the mechanical arm to move the obstacles away before grabbing the objects to be grabbed. When the number of objects to be grabbed is large enough, the grabbing efficiency of the self-mobile cleaning device is very low. Summary of the invention

[0005] The present application provides an object grabbing method, device and self-moving cleaning equipment, which are used to solve the problem of very low efficiency of grabbing objects by self-moving cleaning equipment in the prior art.

[0006] In a first aspect, the present application provides an object grabbing method, which is applied to a self-moving cleaning device. The method provided by the present application includes:

[0007] Obtaining a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area;

[0008] For each object to be grasped, according to the category of the object to be grasped, determining the grasping orientation of the self-moving cleaning device relative to the object to be grasped when grasping the object to be grasped;

[0009] According to the position coordinates of each object to be grasped in the target area, determine whether there is a corresponding obstacle object within a preset distance of each object to be grasped toward the corresponding grasping direction;

[0010] Sorting the items to be grasped in the set of items to be grasped to obtain an order of grasping the items, wherein the order of the items to be grasped that have obstacle objects is placed after the order of the corresponding obstacle objects;

[0011] According to the order of grabbing items, each item to be grabbed in the set of items to be grabbed is grabbed in turn.

[0012] In some implementations, sorting the items to be grasped in the set of items to be grasped to obtain an item grasping order includes:

[0013] If there is a corresponding obstacle object within a preset distance of any object to be grasped toward the corresponding grasping direction, the object to be grasped that depends on the obstacle object is recorded, and the objects to be grasped that depend on the obstacle object are summarized to obtain a dependency relationship data set;

[0014] According to the dependency data set, the items to be grasped in the set of items to be grasped are sorted to obtain an item grasping order, wherein the order of any dependent obstacle item is before the order of the items to be grasped of the dependent obstacle item.

[0015] In some implementations, sorting the items to be grabbed in the set of items to be grabbed according to the dependency data set to obtain the order of grabbing the items includes:

[0016] From the dependency data set, extract obstacle objects that are not dependent on other objects to be grasped, and obtain a set of obstacle objects that can be grasped directly;

[0017] From the set of obstacles that can be directly grasped, one obstacle that can be directly grasped is extracted and included in the preset sequence table;

[0018] According to the dependency data set, find the object to be grasped that depends on an extracted obstacle object that can be grasped directly;

[0019] If the found object to be grasped also depends on other obstacle objects that can be grasped directly, then the other obstacle objects that can be grasped directly are sequentially added to the preset sequence table;

[0020] Eliminate directly graspable obstacle objects that have been included in a preset sequence table from the set of directly graspable obstacle objects;

[0021] If there are still directly graspable obstacle objects remaining in the directly graspable obstacle object set, return to the step of extracting one directly graspable obstacle object from the directly graspable obstacle object set and adding it to the preset sequence table until all directly graspable obstacle objects in the directly graspable obstacle object set are extracted;

[0022] Remove the obstacle items that have been extracted and can be directly grasped from the dependency data set to update the dependency data set, and return to execute the operation of extracting obstacle items that do not depend on other items to be grasped from the dependency data set to obtain a set of obstacle items that can be directly grasped, until all obstacle items that do not depend on other items to be grasped in the dependency data set are extracted;

[0023] The items to be grasped that have not been extracted from the dependency data set are added to the end of the preset sequence table to obtain the order in which the items are grasped.

[0024] In some implementations, according to the dependency data set, an object to be grasped that depends on an extracted obstacle object that can be grasped directly is found; if the object to be grasped that is found also depends on other obstacle objects that can be grasped directly, then the other obstacle objects that can be grasped directly are sequentially added to a preset sequence table, including:

[0025] According to the dependency data set, find the object to be grasped that depends on an extracted obstacle object that can be grasped directly;

[0026] Extract an object to be grasped that is an obstacle object that can be grasped directly, and put it into a preset sequence table;

[0027] For the items to be grasped that depend on the extracted directly graspable obstacle item, the number of remaining directly graspable obstacle items that depend on it is reduced by 1;

[0028] When it is determined that the number of remaining dependent obstacle objects that can be directly grasped is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is included in the preset sequence table until the number of remaining dependent obstacle objects that can be directly grasped is zero.

[0029] In some implementations, sorting the items to be grasped in the set of items to be grasped to obtain an item grasping order includes:

[0030] Extract the i-th item to be grasped from the set of items to be grasped;

[0031] According to the position coordinates of each object to be grasped in the target area, determine whether there is an obstacle within a preset distance from the i-th object to be grasped to the corresponding grasping direction, wherein the obstacle belongs to the set of objects to be grasped;

[0032] If there are obstacles, the priority of grabbing the obstacles is higher than the priority of the i-th item to be grabbed;

[0033] Add 1 to i, and return to the step of extracting the i-th item to be grabbed from the set of items to be grabbed, until the priorities of all items to be grabbed in the set of items to be grabbed are configured;

[0034] The items to be grasped in the set of items to be grasped are sorted to obtain the order of grasping the items, including: sorting the items to be grasped in the set of items to be grasped in order from high to low priority to obtain the order of grasping the items.

[0035] In some embodiments, the grasping orientation includes a grasping direction and a grasping distance relative to the object to be grasped, and according to the position coordinates of each object to be grasped in the target area, determining whether there is a corresponding obstacle object within a preset distance from each object to be grasped to the corresponding grasping orientation includes:

[0036] Corresponding to each object to be grasped, when the self-moving cleaning device is outside the preset distance of the object to be grasped toward the corresponding grasping direction, determine whether there is an object whose distance from the edge of the self-moving cleaning device perpendicular to the grasping direction is less than a preset threshold based on the position coordinates of each object to be grasped in the target area; if so, it is determined that there is a corresponding obstacle object.

[0037] In some embodiments, the self-moving cleaning device further includes a body for cleaning and embedded with a main controller, a mechanical arm capable of adjusting posture, and a clamping claw, wherein the starting end of the mechanical arm is connected to the body, and the end of the mechanical arm is provided with a clamping claw, and each of the items to be grasped in the set of items to be grasped is grasped in sequence according to the order of grasping the items, including:

[0038] Plan the object grabbing path according to the current position information of the self-mobile cleaning device, the position coordinates of each object to be grabbed in the target area and the order of grabbing the objects;

[0039] According to the object grabbing path, the fuselage is controlled to move toward the grabbing position of each object to be grabbed in sequence;

[0040] For each object to be grasped, if the movement is successful, the robot arm is controlled to drive the gripper to move to one side of the object to be grasped at the grasping position corresponding to the object to be grasped, and the gripper is controlled to grasp the object to be grasped.

[0041] In some embodiments, the self-moving cleaning device is further provided with a camera, which sequentially controls the body to move toward the grabbing position of each object to be grabbed according to the object grabbing path, including:

[0042] For each object to be grasped, if the movement fails, a first environment image in front captured by the camera is received;

[0043] If it is identified that there is a blocking object in the first environment image and the blocking object belongs to the set of objects to be grasped, then in the object grasping order of the ungrasped objects to be grasped, the order of the blocking object and the current object to be grasped is swapped.

[0044] In some embodiments, for each object currently to be grasped, if the movement fails, after receiving the first environment image in front captured by the camera, the method provided by the present application further includes:

[0045] If it is identified that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grasped, move to the grasping position of the blocking object;

[0046] The mechanical arm is controlled to drive the clamping claw to move to one side of the blocking object, and the clamping claw is controlled to grab the blocking object and place the blocking object at a position away from the grabbing position.

[0047] In some embodiments, the self-moving cleaning device further includes a camera and a body for cleaning and embedded with a main controller, and obtains a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area, including:

[0048] The machine body is controlled to move according to the preset cleaning path, and the cleaning components of the machine body are controlled to perform cleaning operations during the movement.

[0049] During the movement, receiving a second environment image collected by the camera;

[0050] Whenever an object belonging to a preset target object set is identified in the second environment image, the identified object is included in the set of objects to be grasped, and the position coordinates of the identified object in the target area are recorded.

[0051] In a second aspect, the present application further provides an article grabbing device, which is configured on a self-moving cleaning device. The device provided by the present application includes:

[0052] A data acquisition unit, used to acquire a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area;

[0053] A grasping orientation determination unit, for determining, for each object to be grasped, a grasping orientation of the self-moving cleaning device relative to the object to be grasped when grasping the object to be grasped according to the category of the object to be grasped;

[0054] An obstacle object determination unit is used to determine whether there is a corresponding obstacle object within a preset distance of each object to be grasped toward a corresponding grasping direction according to the position coordinates of each object to be grasped in the target area;

[0055] A grasping order determination unit, used for sorting the items to be grasped in the set of items to be grasped to obtain an order of grasping the items, wherein the order of the items to be grasped with obstacle objects is placed after the order of the corresponding obstacle objects;

[0056] The item grabbing unit is used to grab each item to be grabbed in the item set to be grabbed in sequence according to the item grabbing order.

[0057] In some implementations, the crawling order determination unit includes:

[0058] A dependency relationship acquisition subunit is used to record the dependency of the object to be grasped on the obstacle object if there is a corresponding obstacle object within a preset distance from any object to be grasped to the corresponding grasping direction, and to summarize the objects to be grasped that depend on the obstacle object to obtain a dependency relationship data set;

[0059] The sorting subunit is used to sort the items to be grasped in the set of items to be grasped according to the dependency data set to obtain the order of grasping the items, wherein the order of any dependent obstacle item is placed before the order of the items to be grasped of the dependent obstacle item.

[0060] In some embodiments, the sorting subunit comprises:

[0061] The object extraction submodule is used to extract obstacle objects that are not dependent on other objects to be grasped from the dependency relationship data set, and obtain a set of obstacle objects that can be grasped directly;

[0062] The data inclusion submodule is used to extract one obstacle object that can be directly grasped from the set of obstacle objects that can be directly grasped, and include it in a preset sequence table; wherein 1≤m≤n, n is the total number of obstacle objects that can be directly grasped;

[0063] The object search submodule is used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set;

[0064] The data inclusion submodule is used to include other directly graspable obstacle objects in a preset sequence table in turn if the found object to be grasped also depends on other directly graspable obstacle objects;

[0065] The object removal submodule is used to remove the directly graspable obstacle objects that have been included in the preset sequence table from the set of directly graspable obstacle objects;

[0066] The first step is a return submodule, which is used to return to the step of extracting one directly graspable obstacle object from the set of directly graspable obstacle objects and adding it to a preset sequence table if there are still directly graspable obstacle objects remaining in the set of directly graspable obstacle objects, until all directly graspable obstacle objects in the set of directly graspable obstacle objects are extracted;

[0067] The item removal submodule is used to remove directly graspable obstacle items that have been extracted from the dependency data set to update the dependency data set;

[0068] The second step is a return submodule, which is used to return to the execution of the operation of extracting obstacle objects that are not dependent on other objects to be grasped from the dependency data set to obtain a set of obstacle objects that can be directly grasped, until all obstacle objects in the dependency data set that are not dependent on other objects to be grasped are extracted;

[0069] The items to be grasped that have not been extracted from the dependency data set are added to the end of the preset sequence table to obtain the order in which the items are grasped.

[0070] In some embodiments, the object search submodule is specifically used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set;

[0071] The data inclusion submodule is specifically used to release the dependency relationship between the found object to be grasped and the extracted 1 directly graspable obstacle object, and incorporate the extracted 1 directly graspable obstacle object with the dependency released into a preset sequence table; for the object to be grasped that depends on the extracted 1 directly graspable obstacle object, the number of remaining dependent directly graspable obstacle objects is reduced by 1; when it is determined that the number of remaining dependent directly graspable obstacle objects is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is included in the preset sequence table until the number of remaining dependent directly graspable obstacle objects is zero.

[0072] In some embodiments, the obstacle object determination unit is specifically used to correspond to each object to be grasped. When the self-moving cleaning device is outside the preset distance of the object to be grasped toward the corresponding grasping direction, it is determined according to the position coordinates of each object to be grasped in the target area whether there is an object whose distance from the edge of the self-moving cleaning device is less than a preset threshold in the perpendicular to the grasping direction; if so, it is determined that there is a corresponding obstacle object.

[0073] In some embodiments, the self-moving cleaning device further includes a body for cleaning and embedded with a main controller, a mechanical arm capable of adjusting posture, and a clamping claw, the starting end of the mechanical arm is connected to the body, and the end of the mechanical arm is provided with a clamping claw, and the object grabbing unit specifically includes:

[0074] A path planning subunit, used to plan the object grabbing path according to the current position information of the self-moving cleaning device, the position coordinates of each object to be grabbed in the target area and the order of grabbing the objects;

[0075] The mobile control subunit is used to control the movement of the fuselage toward the grasping position of each object to be grasped in sequence according to the object grasping path;

[0076] The grasping control subunit is used to control the mechanical arm to drive the gripper to move to one side of the object to be grasped at the grasping position corresponding to the object to be grasped, and control the gripper to grasp the object to be grasped if the movement is successful.

[0077] In some embodiments, the device provided by the present application further includes:

[0078] A data receiving unit, for each object to be grasped currently, if the movement fails, receives a first environment image in front captured by the camera;

[0079] The sequence exchange unit is used to exchange the sequence of the blocking object and the current object to be grasped in the object grasping sequence of the ungrasped objects to be grasped if it is recognized that there is a blocking object in the first environment image and the blocking object belongs to the set of objects to be grasped.

[0080] In some embodiments, the movement control subunit is further configured to move to a grabbing position of the blocking object if it is identified that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grabbed;

[0081] The grabbing control subunit is also used to control the robot arm to drive the clamping claw to move to one side of the blocked object, and control the clamping claw to grab the blocked object and place the blocked object at a position away from the grabbing position.

[0082] In some embodiments, the self-moving cleaning device further includes a camera and a body for cleaning and embedded with a main controller, and a data acquisition unit, including:

[0083] The cleaning subunit is used to control the movement of the fuselage according to a preset cleaning path, and control the cleaning components of the fuselage to perform cleaning operations during the movement.

[0084] A data receiving subunit, used for receiving a second environment image collected by the camera during the movement;

[0085] The data recording subunit is used to include the identified object into the set of objects to be grasped and record the position coordinates of the identified object in the target area whenever an object belonging to the preset target object set is identified in the second environment image.

[0086] In a second aspect, the present application also provides a self-moving cleaning device, comprising a memory, a main controller, and a computer program stored in the memory and executable on the main controller. When the main controller executes the computer program, the main controller executes the method provided in the first aspect of the present application.

[0087] In a third aspect, the present application further provides a computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the self-moving cleaning device, the self-moving cleaning device can execute an object grabbing method as provided in the first aspect.

[0088] In a fourth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, is used to implement an object grabbing method as provided in the first aspect.

[0089] The present application provides an object grabbing method, device and self-moving cleaning equipment. For each object to be grabbed, according to the category of the object to be grabbed, the grabbing direction of the self-moving cleaning equipment relative to the object to be grabbed when grabbing the object to be grabbed is determined; according to the position coordinates of each object to be grabbed in the target area, it is determined whether there is a corresponding obstacle object within a preset distance from each object to be grabbed to the corresponding grabbing direction. The objects to be grabbed in the set of objects to be grabbed are sorted to obtain the order of grabbing objects, wherein the order of the objects to be grabbed with obstacle objects is located after the order of the corresponding obstacle objects. According to the order of grabbing objects, the objects to be grabbed in the set of objects to be grabbed are grabbed in sequence. It can be understood that since the order of the objects to be grabbed with obstacle objects is located after the order of the corresponding obstacle objects, when grabbing any object to be grabbed, there is no obstacle object blocking the moving path of the self-moving cleaning equipment, and there is no need for the self-moving cleaning equipment to control the mechanical arm to move the obstacle object away before grabbing the object to be grabbed. In this way, each object to be grasped in the set of objects to be grasped can be grasped efficiently and reliably. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0091] Figure 1 A schematic diagram of the structure of a self-moving cleaning device provided in an embodiment of the present application;

[0092] Figure 2 A schematic diagram of the structure of the body of the self-moving cleaning device provided in an embodiment of the present application;

[0093] Figure 3 A flowchart of an object grabbing method provided in an embodiment of the present application;

[0094] Figure 4 A schematic diagram of a scenario for determining whether there is a corresponding obstacle object provided in an embodiment of the present application;

[0095] Figure 5 for Figure 3 The specific flow chart of S305 in FIG.

[0096] Figure 6 for Figure 5 The specific flow chart of S503 in FIG.

[0097] Figure 7 A block diagram of the functional modules of the object grabbing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0099] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0100] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element or an intervening layer / element may exist therebetween. In addition, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed.

[0101] The following is a detailed description of the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0102] Explanation of terms involved in this application:

[0103] Self-moving cleaning equipment: an electronic device that automatically cleans, which can be a sweeping robot, a mopping robot, a sweeping and mopping robot, etc., without limitation. Figure 1 As shown, the self-moving cleaning device 101 provided in the embodiment of the present application mainly includes two parts, specifically including a body 102 and a mechanical arm 103 arranged on the body 102 .

[0104] The robot arm 103 is arranged on the fuselage 102. The robot arm 103 is a mechanical axis module that can realize multi-degree-of-freedom posture adjustment. The end of the robot arm 103 is set at the clamp 104, and the clamp 104 can grab or push some small objects. For example, during the process of cleaning or mopping the floor by the self-mobile cleaning device 101, socks or small toys are detected on the ground. The robot arm 103 can grab the socks or small toys and move them to the corresponding storage area; or move them to another location so that the self-mobile cleaning device 101 can clean the area covered by the socks or small toys. The embodiment of the present application does not limit the specific structure of the robot arm 103.

[0105] The self-moving cleaning device 101 also includes a housing chamber 106 for accommodating the robot arm 103. When the robot arm 103 is not in use, the robot arm 103 can be stored in the housing chamber 106 to minimize the space occupied by the self-moving cleaning device 101 and the robot arm 103. At this time, the robot arm 103 is in a warehoused state. Figure 2 As shown, the shaded area on the surface of the self-moving cleaning device 101 is the area corresponding to the accommodating cavity 106, and the robot arm 103 is accommodated in the accommodating cavity 106. Figure 1 and Figure 2In addition to arranging the robotic arm 103 on the top of the self-moving cleaning device 101, the robotic arm 103 can also be arranged on the side of the self-moving cleaning device 101. The specific arrangement position can be selected according to actual conditions. The embodiment of the present application only takes the arrangement of the robotic arm 103 on the top of the self-moving cleaning device 101 as an example for subsequent explanation.

[0106] Of course, the self-mobile cleaning device 101 may not be provided with the accommodating cavity 106, and the mechanical arm 103 may be arranged on the outer surface of the self-mobile cleaning device 101. When the mechanical arm 103 is not in use, the mechanical arm 103 may be stored on the surface of the self-mobile cleaning device 101. The storage method of the mechanical arm 103 on the body 102 may be adjusted according to actual conditions. The embodiment of the present application only takes the method of providing the accommodating cavity 106 in the self-mobile cleaning device 101 to accommodate the mechanical arm 103 as an example for subsequent description.

[0107] See also Figure 3 The embodiment of the present application also provides an object grabbing method, which is applied to the self-moving cleaning device 101. The method provided by the embodiment of the present application includes:

[0108] S301: Acquire a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area.

[0109] The target area may be a living room, bedroom or kitchen in a house, and the user may set it according to actual needs. The set of objects to be captured may be, but not limited to, shoes, socks, cat toys, paper balls and the like, which are not limited here.

[0110] Further, as described above, the self-moving cleaning device 101 may include a camera 104 and a body 102 for cleaning and embedded with a main controller. The camera 104 may be disposed at the front or top of the body 102, etc., which is not limited here. S301 may be specifically implemented as follows:

[0111] Step 1: Control the body 102 to move according to a preset cleaning path, and control the cleaning components of the body 102 to perform a cleaning operation during the movement.

[0112] For example, the body 102 may be provided with running wheels and a cleaning assembly for driving the self-mobile cleaning device 101. The cleaning assembly may include, but is not limited to, a dust suction port and a rotatable cleaning mop brush. The dust suction port is used to suck in dust on the ground and crush garbage on the ground, and the rotatable cleaning mop brush is used to clean up garbage.

[0113] Step 2: During the movement, a second environment image captured by the camera 104 is received.

[0114] It can be understood that the self-moving cleaning device 101 can collect the second environment image while performing the cleaning operation.

[0115] Step 3: Whenever an object belonging to the preset target object set is identified in the second environment image, the identified object is included in the set of objects to be grasped, and the position coordinates of the identified object in the target area are recorded.

[0116] For example, when the second environment image collected by the mobile cleaning device 101 includes “shoes” and “shoes” belong to the items in the preset target item set, the position coordinates of “shoes” in the target area are recorded.

[0117] The second environment image can be input into a pre-trained image recognition model to identify the type of items contained in the second environment image. The image recognition model is trained by inputting multiple training samples into a neural network, each training sample including an item image and a corresponding item type.

[0118] Based on the above steps 1 to 3, a set of objects to be grasped can be obtained when the self-mobile cleaning device 101 performs a cleaning operation, which saves time cost and is highly efficient.

[0119] S302: For each object to be grasped, according to the category of the object to be grasped, determining the grasping orientation of the self-moving cleaning device 101 relative to the object to be grasped when grasping the object to be grasped.

[0120] It should be noted that in order to reliably and stably grasp the items to be grasped, the grasping orientation of the items to be grasped corresponding to different types of items is different. For example, when the items to be grasped are shoes, the self-mobile cleaning device 101 needs to move to the side of the heel of the shoe, so that it is convenient for the mechanical arm 103 of the self-mobile cleaning device 101 to extend into the shoe opening of the shoe to grasp the instep of the shoe, so as to reliably and stably grasp the shoe; when the item to be grasped is a stick, it needs to be moved to one side of the geometric center of the stick, so that it is convenient for the mechanical arm 103 of the self-mobile cleaning device 101 to perform a grasping operation along the circumference corresponding to the geometric center of the stick, so as to reliably and stably grasp the stick. It should be noted that the grasping orientation of the self-mobile cleaning device 101 relative to the item to be grasped can be understood as the direction and distance of the self-mobile cleaning device 101 relative to the item to be grasped.

[0121] S303: Determine, based on the position coordinates of each object to be grasped in the target area, whether there is a corresponding obstacle object within a preset distance from each object to be grasped to the corresponding grasping direction.

[0122] Specifically, corresponding to each object to be grasped, when the self-mobile cleaning device 101 is outside the preset distance of the object to be grasped toward the corresponding grasping direction, it is determined based on the position coordinates of each object to be grasped in the target area whether there is an object whose distance from the edge of the self-mobile cleaning device 101 is less than a preset threshold in the perpendicular direction to the grasping direction; if so, it is determined that there is a corresponding obstacle object.

[0123] like Figure 4 As shown, assuming that the grasping direction corresponding to the object O1 to be grasped is Q1, the self-mobile cleaning device 101 is currently at position P1, and position P1 is outside the grasping distance D1 under the grasping direction of the grasping direction. The direction pointed by the line segment S1 between position P1 and the object O1 to be grasped is the grasping direction. Figure 4 There is an object O2 in the direction perpendicular to the grasping direction, and it is determined whether the distance d1 between the edge of the object O2 and the edge of the self-moving cleaning device 101 is less than a preset threshold (such as 1 cm, 2 cm, etc.). If the distance d1 between the edge of the object O2 and the edge of the self-moving cleaning device 101 is less than the preset threshold, it means that the object O2 will hinder the self-moving cleaning device 101 from moving to the position to be grasped Q1, and therefore it can be determined that there is a corresponding obstacle object (i.e., object O2).

[0124] S304: sorting the items to be grasped in the set of items to be grasped to obtain an order of grasping the items, wherein the order of the items to be grasped that have obstacle objects is placed after the order of the corresponding obstacle objects.

[0125] Specifically, the items to be grasped in the set of items to be grasped may be sorted according to the judgment result in S303. Figure 4 As shown, when the set of objects to be grasped includes the object O1 to be grasped and the object O2 to be grasped, the order of the object O2 to be grasped is set before the object O1 to be grasped.

[0126] S305: Grab each of the items to be grabbed in the set of items to be grabbed in sequence according to the order of grabbing the items.

[0127] Specifically, it can be seen from the above that the self-moving cleaning device 101 includes a body 102 for cleaning and embedded with a main controller, a mechanical arm 103 capable of adjusting posture, and a clamp 105. The starting end of the mechanical arm 103 is connected to the body 102, and the end of the mechanical arm 103 is provided with a clamp 105. S305 can be specifically implemented as follows:

[0128] Step A: planning the object grabbing path according to the current position information of the self-moving cleaning device 101, the position coordinates of each object to be grabbed in the target area and the order of grabbing the objects.

[0129] For example, if the order of grabbing items is item A, item B, and item C, then the planned item grabbing path is from the current position information to the grabbing position corresponding to item A, the grabbing position corresponding to item A to the grabbing position corresponding to item B, the grabbing position corresponding to item B, and then to the grabbing position corresponding to item C.

[0130] Step B: According to the object grabbing path, the body 102 is controlled to move toward the grabbing position of each object to be grabbed.

[0131] Step C1: For each object to be grasped, if the movement is successful, the robot arm 103 is controlled to drive the gripper 105 to move to one side of the object to be grasped at the grasping position corresponding to the object to be grasped, and the gripper 105 is controlled to grasp the object to be grasped.

[0132] For example, when the current item to be grasped is item A, if the current position information is successfully moved to the grasping position corresponding to item A, at the grasping position corresponding to item A, the robot arm 103 is controlled to drive the gripper 105 to move to one side of item A, and the gripper 105 is controlled to grasp item A.

[0133] It can be understood that, based on the above-mentioned steps A to C1, each object to be grasped can be grasped accurately and efficiently.

[0134] It should be noted that the set of items to be grabbed can be items that need to be stored. If the self-mobile cleaning device 101 carries a storage area (such as a storage basket), each time an item is grabbed, the robot arm 103 is controlled to put the grabbed item into the storage area; if the self-mobile cleaning device 101 does not carry a storage area, each time an item is grabbed, the robot arm 103 is controlled to put the grabbed item into the external storage area; for example, when the item is a shoe, the external storage area is the shoe placement area; when the item is a paper ball, the external storage area is the area where the trash can is located. In addition, the set of items to be grabbed can also be items in a position that needs to be moved, so that the self-mobile cleaning device 101 can clean the area covered by the items to be grabbed, and each time an item is grabbed, the grabbed item is moved to another position.

[0135] In addition, after the above step B, the method provided in the embodiment of the present application may further include:

[0136] Step C21: For each object to be grasped, if the movement fails, a first environment image in front captured by the camera 104 is received.

[0137] Step C22: If it is identified that there is a blocking object in the first environment image and the blocking object belongs to the set of objects to be grasped, then in the grasping order of the ungrasped objects to be grasped, the order of the blocking object and the current object to be grasped is swapped.

[0138] Understandably, when it is recognized that there are blocking objects in the first environment image, it means that the above-mentioned movement failure is caused by the blocking object, rather than other reasons such as the failure of the self-mobile cleaning device 101 itself; if it is recognized that the blocking object belongs to the set of objects to be grasped, it means that there is an error in the previously set object grasping order, so in the object grasping order of the ungrabbed objects to be grasped, the order of the blocking object and the current object to be grasped is swapped. In this way, the object grasping order can be adjusted in real time to further improve the grasping efficiency.

[0139] It should be noted that the principle of identifying the blocking object is similar to the principle of identifying the obstacle object in the above S303, and will not be described in detail here.

[0140] Step C23: if it is identified that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grasped, move to the grasping position of the blocking object.

[0141] Step C24: Control the robot arm 103 to drive the gripper 105 to move to one side of the blocking object, and control the gripper 105 to grab the blocking object and place the blocking object at a position away from the grabbing position.

[0142] It can be understood that when the blocking object does not belong to the set of objects to be grasped, it means that the blocking object does not need to be stored or the area covered by the blocking object does not need to be cleaned. Therefore, the clamp 105 can be controlled to grasp the blocking object and place the blocking object at a position away from the grasping position to facilitate the movement of the mobile cleaning device 101 to one side of the grasping position of the current object to be grasped.

[0143] In summary, the embodiment of the present application provides an object grabbing method. For each object to be grabbed, according to the category of the object to be grabbed, the grabbing orientation of the self-mobile cleaning device 101 relative to the object to be grabbed when grabbing the object to be grabbed is determined; according to the position coordinates of each object to be grabbed in the target area, it is determined whether there is a corresponding obstacle object within a preset distance of each object to be grabbed toward the corresponding grabbing orientation. The objects to be grabbed in the set of objects to be grabbed are sorted to obtain an order of grabbing objects, wherein the order of the objects to be grabbed with obstacle objects is located after the order of the corresponding obstacle objects. According to the order of grabbing objects, the objects to be grabbed in the set of objects to be grabbed are grabbed in turn. It can be understood that since the order of the objects to be grasped that have obstacles is located after the order of the corresponding obstacles, when grasping any object to be grasped, there is no obstacle blocking the moving path of the self-moving cleaning device 101, and the self-moving cleaning device 101 does not need to control the mechanical arm 103 to move the obstacle away before grasping the object to be grasped. In this way, each object to be grasped in the set of objects to be grasped is grasped efficiently and reliably.

[0144] In addition, in the above Figure 3 On the basis of Figure 5 As shown, S305 can be specifically implemented as follows:

[0145] S501: Determine whether there is a corresponding obstacle object within a preset distance of any object to be grasped toward the corresponding grasping direction. If yes, execute S502.

[0146] The preset distance may be in the range of, but not limited to, 0.5m-2m.

[0147] S502: Record the objects to be grasped that depend on the obstacle objects, and summarize the objects to be grasped that depend on the obstacle objects to obtain a dependency relationship data set.

[0148] For example, if there is a corresponding item B to be grasped (i.e., an obstacle item) within the preset distance of the grasping direction corresponding to the direction of the item A to be grasped, then it is recorded that the item A to be grasped depends on the item B to be grasped; if there is a corresponding item C to be grasped (i.e., an obstacle item) within the preset distance of the grasping direction corresponding to the direction of the item B to be grasped, then it is recorded that the item B to be grasped depends on the item C to be grasped; if there is a corresponding item E to be grasped (i.e., an obstacle item) within the preset distance of the grasping direction corresponding to the direction of the item D to be grasped, then it is recorded that the item D to be grasped depends on the item E to be grasped, and so on.

[0149] S503: Sort the items to be grasped in the set of items to be grasped according to the dependency relationship data set to obtain an item grasping order, wherein the order of any dependent obstacle item is before the order of the items to be grasped of the dependent obstacle item.

[0150] For example, if the item A to be grasped depends on the item B to be grasped, and the item B to be grasped depends on the item C to be grasped, then the order of grasping the items is the item A to be grasped, the item B to be grasped, and the item C to be grasped.

[0151] In some embodiments, Figure 5 On the corresponding basis, such as Figure 6 As shown, Figure 5 S503 in the above method can be specifically implemented as follows:

[0152] S601: Extracting obstacle objects that are not dependent on other objects to be grasped from the dependency relationship data set, and obtaining a set of obstacle objects that can be grasped directly.

[0153] S602: Extracting one obstacle object that can be directly grasped from the set of obstacle objects that can be directly grasped, and adding it to a preset sequence table.

[0154] S603: According to the dependency relationship data set, find an object to be grasped that depends on an extracted obstacle object that can be grasped directly.

[0155] S604: If the found object to be grasped is still dependent on other obstacle objects that can be grasped directly, then the other obstacle objects that can be grasped directly are sequentially added into a preset sequence table.

[0156] S605: Eliminate directly graspable obstacle objects that have been included in a preset sequence table from the set of directly graspable obstacle objects.

[0157] S606: Determine whether there are any obstacle objects that can be directly grasped in the set of obstacle objects that can be directly grasped. If yes, return to execute S602; if no, execute S607.

[0158] S607: Remove the directly graspable obstacle objects that have been extracted from the dependency data set to update the dependency data set.

[0159] S608: Determine whether the obstacle items in the dependency data set that are not dependent on other items to be captured have been extracted. If not, return to execute S601; if yes, execute S609.

[0160] S609: Add the to-be-grabbed items that have not been extracted from the dependency data set to the end of the preset sequence table to obtain the item grabbing sequence.

[0161] It can be understood that based on the above S601-S609, the order of grabbing items can be accurately obtained.

[0162] Furthermore, the above S603-S604 may be specifically implemented as follows:

[0163] Step M1: According to the dependency relationship data set, find the object to be grasped that depends on an extracted obstacle object that can be grasped directly.

[0164] Step M2: removing the dependency between the found object to be grasped and the extracted one directly graspable obstacle object, and adding the extracted one directly graspable obstacle object with the dependency removed into a preset sequence table.

[0165] Step M3: For the items to be grasped that depend on the extracted directly graspable obstacle object, the number of remaining directly graspable obstacle objects that depend on it is reduced by 1.

[0166] Step M4: When it is determined that the number of remaining dependent obstacle objects that can be directly grasped is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is added to the preset sequence table until the number of remaining dependent obstacle objects that can be directly grasped is zero.

[0167] It can be understood that, based on the above steps M1 to M4, each obstacle object can be accurately included in the preset sequence table.

[0168] Additionally, in some other embodiments, Figure 4 S403 in the above method can also be specifically implemented as follows:

[0169] Step N1: extract the i-th item to be grasped from the set of items to be grasped.

[0170] Step N2: According to the position coordinates of each object to be grasped in the target area, determine whether there is an obstacle within a preset distance from the i-th object to be grasped to the corresponding grasping direction, wherein the obstacle belongs to the set of objects to be grasped; if yes, execute step N3.

[0171] Step N3: configure the grabbing priority of the obstacle object to be higher than the priority of the i-th object to be grabbed;

[0172] Step N4: add 1 to i, and repeat steps N1-N3 until the priorities of all the items to be grasped in the set of items to be grasped are configured.

[0173] Step N5: sorting the items in the set of items to be grasped to obtain an item grasping order, including: sorting the items in the set of items to be grasped in order from high to low priority to obtain an item grasping order.

[0174] It can be understood that the order of grabbing items can be accurately obtained based on the above steps N1 to N5.

[0175] In addition, the embodiment of the present application also provides an article grabbing device, which is configured on the self-moving cleaning device 101. It should be noted that the basic principle and technical effects of the article grabbing device provided in the embodiment of the present application are the same as those in the above embodiment. For the sake of brief description, for the parts not mentioned in the embodiment of the present application, please refer to the corresponding contents in the above embodiment. Figure 7 As shown, the device provided in the embodiment of the present application includes a data acquisition unit, a grasping orientation determination unit, an obstacle object determination unit, a grasping sequence determination unit, and an object grasping unit, wherein:

[0176] A data acquisition unit, used to acquire a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area;

[0177] A grasping orientation determination unit, for each object to be grasped, according to the category of the object to be grasped, to determine the grasping orientation of the self-moving cleaning device 101 relative to the object to be grasped when grasping the object to be grasped;

[0178] An obstacle object determination unit is used to determine whether there is a corresponding obstacle object within a preset distance of each object to be grasped toward a corresponding grasping direction according to the position coordinates of each object to be grasped in the target area;

[0179] A grasping order determination unit, used for sorting the items to be grasped in the set of items to be grasped to obtain an order of grasping the items, wherein the order of the items to be grasped with obstacle objects is placed after the order of the corresponding obstacle objects;

[0180] The item grabbing unit is used to grab each item to be grabbed in the item set to be grabbed in sequence according to the item grabbing order.

[0181] In some implementations, the crawling order determination unit includes:

[0182] A dependency relationship acquisition subunit is used to record the dependency of the object to be grasped on the obstacle object if there is a corresponding obstacle object within a preset distance from any object to be grasped to the corresponding grasping direction, and to summarize the objects to be grasped that depend on the obstacle object to obtain a dependency relationship data set;

[0183] The sorting subunit is used to sort the items to be grasped in the set of items to be grasped according to the dependency data set to obtain the order of grasping the items, wherein the order of any dependent obstacle item is placed before the order of the items to be grasped of the dependent obstacle item.

[0184] In some embodiments, the sorting subunit comprises:

[0185] The object extraction submodule is used to extract obstacle objects that are not dependent on other objects to be grasped from the dependency relationship data set, and obtain a set of obstacle objects that can be grasped directly;

[0186] The data inclusion submodule is used to extract one obstacle object that can be directly grasped from the set of obstacle objects that can be directly grasped, and include it in a preset sequence table; wherein 1≤m≤n, n is the total number of obstacle objects that can be directly grasped;

[0187] The object search submodule is used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set;

[0188] The data inclusion submodule is used to include other directly graspable obstacle objects in a preset sequence table in turn if the found object to be grasped also depends on other directly graspable obstacle objects;

[0189] The object removal submodule is used to remove the directly graspable obstacle objects that have been included in the preset sequence table from the set of directly graspable obstacle objects;

[0190] The first step is a return submodule, which is used to return to the step of extracting one directly graspable obstacle object from the set of directly graspable obstacle objects and adding it to a preset sequence table if there are still directly graspable obstacle objects remaining in the set of directly graspable obstacle objects, until all directly graspable obstacle objects in the set of directly graspable obstacle objects are extracted;

[0191] The item removal submodule is used to remove directly graspable obstacle items that have been extracted from the dependency data set to update the dependency data set;

[0192] The second step is a return submodule, which is used to return to the execution of the operation of extracting obstacle objects that are not dependent on other objects to be grasped from the dependency data set to obtain a set of obstacle objects that can be directly grasped, until all obstacle objects in the dependency data set that are not dependent on other objects to be grasped are extracted;

[0193] The items to be grasped that have not been extracted from the dependency data set are added to the end of the preset sequence table to obtain the order in which the items are grasped.

[0194] In some embodiments, the object search submodule is specifically used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set;

[0195] The data inclusion submodule is specifically used to release the dependency relationship between the found object to be grasped and the extracted 1 directly graspable obstacle object, and incorporate the extracted 1 directly graspable obstacle object with the dependency released into a preset sequence table; for the object to be grasped that depends on the extracted 1 directly graspable obstacle object, the number of remaining dependent directly graspable obstacle objects is reduced by 1; when it is determined that the number of remaining dependent directly graspable obstacle objects is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is included in the preset sequence table until the number of remaining dependent directly graspable obstacle objects is zero.

[0196] In some embodiments, the obstacle object determination unit is specifically used to correspond to each object to be grasped. When the self-moving cleaning device is outside the preset distance of the object to be grasped toward the corresponding grasping direction, it is determined according to the position coordinates of each object to be grasped in the target area whether there is an object whose distance from the edge of the self-moving cleaning device is less than a preset threshold in the perpendicular to the grasping direction; if so, it is determined that there is a corresponding obstacle object.

[0197] In some embodiments, the obstacle object determination unit is specifically used to correspond to each object to be grasped. When the self-moving cleaning device 101 is outside the preset distance of the object to be grasped toward the corresponding grasping direction, it is determined according to the position coordinates of each object to be grasped in the target area whether there is an object whose distance from the edge of the self-moving cleaning device 101 is less than a preset threshold in the perpendicular to the grasping direction; if so, it is determined that there is a corresponding obstacle object.

[0198] In some embodiments, the self-moving cleaning device 101 further includes a body 102 for cleaning and embedded with a main controller, a mechanical arm 103 capable of adjusting posture, and a clamp 105, the starting end of the mechanical arm 103 is connected to the body 102, and the end of the mechanical arm 103 is provided with a clamp 105, and the object grabbing unit specifically includes:

[0199] A path planning subunit, used to plan an object grabbing path according to the current position information of the self-moving cleaning device 101, the position coordinates of each object to be grabbed in the target area and the order of grabbing the objects;

[0200] A movement control subunit, used to sequentially control the body 102 to move toward the grasping position of each object to be grasped according to the object grasping path;

[0201] The grasping control subunit is used to control the robot arm 103 to drive the gripper 105 to move to one side of the object to be grasped at the grasping position corresponding to the object to be grasped, and control the gripper 105 to grasp the object to be grasped if the movement is successful.

[0202] In some embodiments, the device provided by the present application further includes:

[0203] A data receiving unit, for each object to be grasped currently, if the movement fails, receives a first environment image in front captured by the camera 104;

[0204] The sequence exchange unit is used to exchange the sequence of the blocking object and the current object to be grasped in the object grasping sequence of the ungrasped objects to be grasped if it is recognized that there is a blocking object in the first environment image and the blocking object belongs to the set of objects to be grasped.

[0205] In some embodiments, the movement control subunit is further configured to move to a grabbing position of the blocking object if it is identified that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grabbed;

[0206] The grabbing control subunit is also used to control the robot arm 103 to drive the gripper 105 to move to one side of the blocked object, and control the gripper 105 to grab the blocked object and place the blocked object at a position away from the grabbing position.

[0207] In some embodiments, the self-moving cleaning device 101 further includes a camera 104 and a body 102 for cleaning and embedded with a main controller, and a data acquisition unit, including:

[0208] The cleaning subunit is used to control the movement of the body 102 according to a preset cleaning path, and control the cleaning components of the body 102 to perform cleaning operations during the movement.

[0209] A data receiving subunit, used for receiving a second environment image acquired by the camera 104 during the movement;

[0210] The data recording subunit is used to include the identified object into the set of objects to be grasped and record the position coordinates of the identified object in the target area whenever an object belonging to the preset target object set is identified in the second environment image.

[0211] In addition, the embodiment of the present application also provides a self-moving cleaning device, which can be a sweeping robot, a mopping robot, a sweeping and mopping robot, etc., which is not limited here. The self-moving cleaning device includes a main controller, a fuselage for cleaning and embedded with the main controller, a mechanical arm capable of adjusting the posture, and a clamp. The starting end of the mechanical arm is connected to the fuselage, and the end of the mechanical arm is provided with a clamp and a camera. Optionally, the self-moving cleaning device also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc.

[0212] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0213] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0214] The main controller reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming an object grabbing device at the logical level. The main controller executes the program stored in the memory and is specifically used to execute the object grabbing method provided in the above embodiment.

[0215] The above application Figure 3The method performed by the article grabbing device disclosed in the illustrated embodiment can be applied to the main controller or implemented by the main controller. The main controller may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the main controller or the instructions in the form of software. The above main controller can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in this application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in this application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0216] Of course, in addition to software implementation methods, the self-moving cleaning device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0217] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a self-moving cleaning device including a plurality of application programs, enable the self-moving cleaning device to execute Figure 3 The item grabbing method shown.

[0218] In addition, computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM) or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0219] In addition, the embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the present application. Figure 3 The item grabbing method shown.

[0220] In the above description, the technical details such as the patterning of each layer are not described in detail. However, those skilled in the art should understand that various technical means can be used to form layers, regions, etc. of desired shapes. In addition, in order to form the same structure, those skilled in the art can also design methods that are not completely the same as the methods described above. In addition, although the various embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage.

[0221] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for grabbing an object, characterized in that: Applied to a self-propelled cleaning device, the method comprises: Acquire a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area; For each of the objects to be grasped, determining, according to the category of the objects to be grasped, a grasping orientation of the self-moving cleaning device relative to the objects to be grasped when grasping the objects to be grasped; According to the position coordinates of each object to be grasped in the target area, determining whether there is a corresponding obstacle object within a preset distance of each object to be grasped toward a corresponding grasping orientation; Sorting each of the to-be-grabbed objects in the to-be-grabbed object set to obtain an object grabbing order, wherein the order of the to-be-grabbed objects with the obstacle objects is located after the order of the corresponding obstacle objects; According to the item grabbing sequence, each item to be grabbed in the set of items to be grabbed is grabbed in sequence.

2. The method according to claim 1, characterized in that The step of sorting the items to be grasped in the set of items to be grasped to obtain an item grasping order includes: If there is a corresponding obstacle object within a preset distance of any object to be grasped toward the corresponding grasping direction, then the dependency of the object to be grasped on the obstacle object is recorded, and the objects to be grasped that depend on the obstacle object are summarized to obtain a dependency relationship data set; According to the dependency data set, the items to be grasped in the set of items to be grasped are sorted to obtain an item grasping order, wherein the order of any dependent obstacle item is before the order of the items to be grasped that depend on the obstacle item.

3. The method according to claim 2, characterized in that The step of sorting the items to be grasped in the set of items to be grasped according to the dependency relationship data set to obtain an item grasping order includes: Extracting obstacle objects that are not dependent on other objects to be grasped from the dependency relationship data set to obtain a set of obstacle objects that can be grasped directly; Extracting one obstacle object that can be directly grasped from the set of obstacle objects that can be directly grasped, and adding it to a preset sequence table; According to the dependency relationship data set, searching for an object to be grasped that is dependent on an extracted obstacle object that can be grasped directly; If the found object to be grasped is still dependent on other obstacle objects that can be grasped directly, then the other obstacle objects that can be grasped directly are sequentially added to the preset sequence table; Eliminating the directly graspable obstacle objects that have been included in the preset sequence table from the set of directly graspable obstacle objects; If there are still directly graspable obstacle objects remaining in the set of directly graspable obstacle objects, return to the step of extracting one directly graspable obstacle object from the set of directly graspable obstacle objects and adding it to the preset sequence table, until all the directly graspable obstacle objects in the set of directly graspable obstacle objects are extracted; Eliminate the directly graspable obstacle objects that have been extracted from the dependency data set to update the dependency data set, and return to perform the operation of extracting obstacle objects that do not depend on other objects to be grasped from the dependency data set to obtain a directly graspable obstacle object set, until all obstacle objects that do not depend on other objects to be grasped in the dependency data set are extracted; The items to be grabbed that have not been extracted from the dependency data set are added to the end of the preset sequence table to obtain the order in which the items are grabbed.

4. The method according to claim 3, characterized in that The step of searching, according to the dependency relationship data set, an object to be grasped that depends on the extracted obstacle object that can be grasped directly; if the object to be grasped that is found also depends on other obstacle objects that can be grasped directly, sequentially adding the other obstacle objects that can be grasped directly into the preset sequence table includes: According to the dependency relationship data set, searching for an object to be grasped that is dependent on an extracted obstacle object that can be grasped directly; Extract an object to be grasped that is an obstacle object that can be grasped directly, and put it into a preset sequence table; For the items to be grasped that depend on the extracted one directly graspable obstacle object, the number of remaining dependent directly graspable obstacle objects is reduced by 1; When it is determined that the number of remaining dependent obstacle objects that can be directly grasped is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is included in the preset sequence table until the number of remaining dependent obstacle objects that can be directly grasped is zero.

5. The method according to claim 1, characterized in that: The step of sorting the items to be grasped in the set of items to be grasped to obtain an item grasping order includes: Extracting the i-th item to be grasped from the set of items to be grasped; Determine, according to the position coordinates of each object to be grasped in the target area, whether there is an obstacle object within a preset distance from the i-th object to be grasped to the corresponding grasping direction, wherein the obstacle object belongs to the set of objects to be grasped; If the obstacle object exists, the grabbing priority of the obstacle object is configured to be higher than the priority of the i-th object to be grabbed; Add 1 to i, and return to the step of extracting the i-th item to be grasped from the set of items to be grasped, until the priorities of all items to be grasped in the set of items to be grasped are configured; The step of sorting the items to be grasped in the set of items to be grasped to obtain the order of grasping the items includes: sorting the items to be grasped in the set of items to be grasped in order from high to low priority to obtain the order of grasping the items.

6. The method according to claim 1, characterized in that The grasping orientation includes a grasping direction and a grasping distance relative to the object to be grasped, and determining whether there is a corresponding obstacle object within a preset distance from each object to be grasped to the corresponding grasping orientation according to the position coordinates of each object to be grasped in the target area, includes: Corresponding to each object to be grasped, when the self-moving cleaning device is outside the preset distance of the object to be grasped toward the corresponding grasping direction, determine whether there is an object in the target area whose distance from the edge of the self-moving cleaning device is less than a preset threshold in the direction perpendicular to the grasping direction based on the position coordinates of each object to be grasped. If so, it is determined that there is a corresponding obstacle object.

7. The method according to claim 1, characterized in that The self-moving cleaning device also includes a body for cleaning and embedded with a main controller, a mechanical arm capable of adjusting posture, and a clamping claw, wherein the starting end of the mechanical arm is connected to the body, and the end of the mechanical arm is provided with a clamping claw, and the items to be grasped in the set of items to be grasped are grasped in sequence according to the order of grasping the items, including: Planning an object grabbing path according to the current position information of the self-moving cleaning device, the position coordinates of each object to be grabbed in the target area, and the order in which the objects are grabbed; According to the object grabbing path, sequentially controlling the body to move toward the grabbing position of each object to be grabbed; For each item to be grasped currently, if the movement is successful, the robot arm is controlled to drive the gripper to move to one side of the item to be grasped currently at the grasping position corresponding to the item to be grasped currently, and the gripper is controlled to grasp the item to be grasped currently.

8. The method according to claim 7, characterized in that The self-moving cleaning device is also provided with a camera, and sequentially controls the body to move toward the grabbing position of each of the objects to be grabbed according to the object grabbing path, including: For each object currently to be grasped, if the movement fails, receiving a first environment image in front captured by the camera; If it is identified that there is an obstructing object in the first environment image and the obstructing object belongs to the set of objects to be grasped, then in the object grasping order of the ungrasped objects to be grasped, the order of the obstructing object and the current object to be grasped is swapped.

9. The method according to claim 8, characterized in that After receiving the first environment image in front captured by the camera for each object to be grasped currently, if the movement fails, the method further includes: If it is identified that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grasped, move to the grasping position of the blocking object; The mechanical arm is controlled to drive the clamp to move to one side of the blocking object, and the clamp is controlled to grab the blocking object and place the blocking object at a position away from the grabbing position.

10. The method according to any one of claims 1 to 9, characterized in that: The self-moving cleaning device further includes a camera and a body for cleaning and embedded with a main controller, and the acquisition of a set of objects to be grasped in a target area and the position coordinates of each object to be grasped in the set of objects to be grasped in the target area includes: The body is controlled to move according to a preset cleaning path, and the cleaning components of the body are controlled to perform cleaning operations during the movement. During the movement, receiving a second environment image captured by the camera; Whenever an object belonging to a preset target object set is identified in the second environment image, the identified object is included in the set of objects to be grasped, and the position coordinates of the identified object in the target area are recorded.

11. An object grabbing device, characterized in that: Configured in a self-moving cleaning device, the device comprises: A data acquisition unit, used to acquire a set of objects to be grasped in a target area and position coordinates of each object to be grasped in the set of objects to be grasped in the target area; A grasping orientation determining unit, for determining, for each of the objects to be grasped, a grasping orientation of the self-moving cleaning device relative to the object to be grasped when grasping the object to be grasped according to the category of the object to be grasped; an obstacle object determination unit, configured to determine whether there is a corresponding obstacle object within a preset distance from each of the objects to be grasped to a corresponding grasping orientation according to the position coordinates of each of the objects to be grasped in the target area; a grasping order determining unit, configured to sort the objects to be grasped in the set of objects to be grasped to obtain an object grasping order, wherein the order of the objects to be grasped that have the obstacle objects is located after the order of the corresponding obstacle objects; The item grabbing unit is used to grab each item to be grabbed in the set of items to be grabbed in sequence according to the item grabbing sequence.

12. The device according to claim 11, characterized in that The grabbing order determination unit comprises: A dependency relationship acquisition subunit is used to record the dependency of any object to be grasped on the obstacle object if there is a corresponding obstacle object within a preset distance from any object to be grasped to the corresponding grasping direction, and to summarize the objects to be grasped that depend on the obstacle object to obtain a dependency relationship data set; A sorting subunit is used to sort the items to be grasped in the set of items to be grasped according to the dependency data set to obtain an order of grasping the items, wherein the order of any dependent obstacle item is placed before the order of the items to be grasped that depend on the obstacle item.

13. The device according to claim 12, characterized in that The sorting subunit comprises: An object extraction submodule is used to extract obstacle objects that are not dependent on other objects to be grasped from the dependency relationship data set, and obtain a set of obstacle objects that can be grasped directly; The data inclusion submodule is used to extract one directly graspable obstacle object from the set of directly graspable obstacle objects and include it in a preset sequence table; wherein 1≤m≤n, n is the total number of the directly graspable obstacle objects; An object search submodule is used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set; The data inclusion submodule is used to include other directly graspable obstacle objects in the preset sequence table in sequence if the found object to be grasped also depends on other directly graspable obstacle objects; An object removal submodule, used to remove the directly graspable obstacle objects that have been included in the preset sequence table from the set of directly graspable obstacle objects; The first step is a return submodule, which is used to return to the step of extracting one directly graspable obstacle object from the set of directly graspable obstacle objects and adding it to a preset sequence table if there are still directly graspable obstacle objects remaining in the set of directly graspable obstacle objects, until all directly graspable obstacle objects in the set of directly graspable obstacle objects are extracted; An item removal submodule, used to remove directly graspable obstacle items that have been extracted from the dependency data set, so as to update the dependency data set; The second step is a return submodule, which is used to return to the operation of extracting obstacle objects that are not dependent on other objects to be grasped from the dependency data set to obtain a set of obstacle objects that can be directly grasped, until all obstacle objects that are not dependent on other objects to be grasped in the dependency data set are extracted; The items to be grabbed that have not been extracted from the dependency data set are added to the end of the preset sequence table to obtain the order in which the items are grabbed.

14. The device according to claim 13, characterized in that The object search submodule is specifically used to search for an object to be grasped that depends on an extracted obstacle object that can be grasped directly according to the dependency relationship data set; The data inclusion submodule is specifically used to release the dependency between the found object to be grasped and the extracted one directly graspable obstacle object, and incorporate the extracted one directly graspable obstacle object with the dependency released into a preset sequence table; for the object to be grasped that depends on the extracted one directly graspable obstacle object, the number of remaining dependent directly graspable obstacle objects is reduced by 1; when it is determined that the number of remaining dependent directly graspable obstacle objects is not zero, one of the remaining dependent obstacle objects of the found object to be grasped is included in the preset sequence table until the number of remaining dependent directly graspable obstacle objects is zero.

15. The device according to claim 11, characterized in that The grasping orientation includes a grasping direction and a grasping distance relative to the object to be grasped. The obstacle object determination unit is specifically used to correspond to each object to be grasped. When the self-moving cleaning device is outside the preset distance of the object to be grasped toward the corresponding grasping orientation, determine whether there is an object whose distance from the edge of the self-moving cleaning device is less than a preset threshold in the direction perpendicular to the grasping direction according to the position coordinates of each object to be grasped in the target area; if so, determine that there is a corresponding obstacle object.

16. The device according to claim 11, characterized in that The self-moving cleaning device also includes a body for cleaning and embedded with a main controller, a mechanical arm capable of adjusting posture, and a clamping claw, wherein the starting end of the mechanical arm is connected to the body, and the end of the mechanical arm is provided with a clamping claw, and the object grabbing unit specifically includes: A path planning subunit, used for planning an object grabbing path according to the current position information of the self-moving cleaning device, the position coordinates of each object to be grabbed in the target area and the object grabbing order; A movement control subunit, used for sequentially controlling the body to move toward the grasping position of each of the objects to be grasped according to the object grasping path; The grasping control subunit is used for controlling the robot arm to drive the gripper to move to one side of the object to be grasped at the grasping position corresponding to the object to be grasped, and controlling the gripper to grasp the object to be grasped if the movement is successful.

17. The device according to claim 16, characterized in that The self-moving cleaning device is also provided with a camera, and the device further comprises: A data receiving unit, for each object to be grasped currently, if the movement fails, receives a first environment image in front captured by the camera; A sequence exchange unit is used to exchange the order of the blocking object and the current object to be grasped in the object grasping order of the uncaught objects to be grasped if it is recognized that there is a blocking object in the first environment image and the blocking object belongs to the set of objects to be grasped.

18. The device according to claim 17, characterized in that The movement control subunit is further configured to move to a grasping position of the blocking object if it is recognized that there is a blocking object in the first environment image and the blocking object does not belong to the set of objects to be grasped; The grabbing control subunit is also used to control the robot arm to drive the clamp to move to one side of the blocking object, and control the clamp to grab the blocking object and place the blocking object at a position away from the grabbing position.

19. The device according to any one of claims 11 to 18, characterized in that: The self-moving cleaning device further includes a camera and a body for cleaning and embedded with a main controller, and the data acquisition unit includes: a cleaning subunit, used to control the movement of the body according to a preset cleaning path, and to control the cleaning components of the body to perform cleaning operations during the movement, A data receiving subunit, used for receiving a second environment image collected by the camera during the movement; The data recording subunit is used to include the identified object into the set of objects to be grasped and record the position coordinates of the identified object in the target area whenever an object belonging to the preset target object set is identified in the second environment image.

20. A self-propelled cleaning device, characterized in that: The method comprises a memory, a main controller and a computer program stored in the memory and executable on the main controller, wherein when the main controller executes the computer program, the main controller executes the method according to any one of claims 1 to 10.

21. A computer-readable storage medium, when instructions in the storage medium are executed by a processor of a self-moving cleaning device, the self-moving cleaning device is enabled to implement an object grabbing method according to any one of claims 1 to 10.

22. A computer program product, characterized in that The method comprises a computer program, which is used to implement the object grabbing method according to any one of claims 1 to 10 when the computer program is executed by a processor.

Images