Information processing method and apparatus for robot, robot, program product, and medium

Abstract

The present disclosure provides an information processing method and apparatus for a robot, a robot, a program product, and a medium. The method comprises: controlling a first mechanism of a robot to process a first object, wherein the first mechanism processing the first object comprises the first mechanism obtaining the first object from a first position, or the first mechanism placing the first object at a target placement point; controlling an environmental perception sensor of the robot to face the first position or the target placement point and acquire environmental information; and, on the basis of the acquired environmental information, updating at least one of obstacle avoidance information at a corresponding position and related display information at the corresponding position, wherein the related display information at the corresponding position can be displayed on an environmental map.

Description

Information processing methods and devices for robots, robots, program products, media Cross-references to related applications

[0001] This disclosure is based on and claims priority to Chinese Patent Application No. 202510308574.3, filed on March 14, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to robotics technology, including but not limited to information processing methods and devices for robots, robots, program products, and media. Background Technology

[0003] Some robots (such as cleaning robots with robotic arms) have the ability to move obstacles. The robot can identify obstacle types using cameras and detect the environment in front of it using line lasers / area arrays to determine the type and size of obstacles. After identifying an obstacle, it generates obstacle avoidance information and performs obstacle avoidance and path planning accordingly. Simultaneously, the robot's screen displays an icon of the identified object at the corresponding location on a map; users can click on the icon to view a real-world photo of the object at that location. Summary of the Invention

[0004] In a first aspect, embodiments of this disclosure provide an information processing method for a robot, the method comprising: controlling a first mechanism of the robot to process a first object; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from a first position, or the first mechanism placing the first object at a target placement point; controlling the robot's environmental perception sensors to collect environmental information toward the first position or the target placement point; updating at least one of obstacle avoidance information at a corresponding position and relevant display information at a corresponding position based on the collected environmental information; wherein, the relevant display information at the corresponding position can be displayed on an environmental map.

[0005] In some embodiments, updating at least one of obstacle avoidance information and related display information of the first location based on the first environmental information collected at the first location includes: clearing the related display information of the first location when it is determined based on the first environmental information that there is no object at the first location; wherein the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

[0006] For example, in some embodiments, the relevant display information of the first location includes a first icon displayed at the first location on the environment map, the first icon being used to identify the presence of an object at the first location.

[0007] In other embodiments, updating at least one of the obstacle avoidance information and the related display information of the first location based on the collected first environmental information at the first location includes: retaining the obstacle avoidance information and the related display information of the first location when it is determined from the first environmental information that a second object exists at the first location and the type of the second object is the same as the type of the first object; wherein the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

[0008] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location includes: determining first obstacle avoidance information based on the first environmental information when it is determined that a third object exists at the first location and the type of the third object is different from the type of the first object, and updating the obstacle avoidance information of the first location using the first obstacle avoidance information; and / or determining first relevant display information based on the first environmental information and updating the relevant display information of the first location using the first relevant display information; wherein the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

[0009] For example, in some embodiments, the first related display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map; wherein the second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location.

[0010] In some embodiments, controlling the first mechanism to place the first object at a target placement point includes: after controlling the environmental perception sensor to collect environmental information toward the first position, controlling the second mechanism of the robot to drive the robot to move to a second position; after the robot moves to the second position, controlling the first mechanism to perform a placement action to place the first object at the target placement point.

[0011] In some embodiments, the method further includes: backing up the obstacle avoidance information and related display information of the first location before updating the obstacle avoidance information and related display information of the first location.

[0012] Furthermore, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: if it is determined based on the second environmental information that there is no object at the target placement point, determining second obstacle avoidance information based on the backed-up obstacle avoidance information of the first location and the target placement point, and setting the second obstacle avoidance information at the target placement point; and / or setting at least one of the backed-up relevant display information of the first location at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0013] In some embodiments, the method further includes: after the robot moves to the second position and before the first mechanism performs the placement action, controlling the environmental perception sensor to collect third environmental information toward the target placement point; updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point, including: if it is determined that an object exists at the target placement point based on the second environmental information and that no object exists at the target placement point based on the third environmental information, determining third obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first position, and setting the third obstacle avoidance information at the target placement point; and / or, determining second relevant display information based on the second environmental information and setting the second relevant display information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0014] In other embodiments, the method further includes: after the robot moves to the second position and before the first mechanism performs the placement action, controlling the environmental perception sensor to collect third environmental information toward the target placement point; updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point, including: if it is determined that an object exists at the target placement point based on the second environmental information, and a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, determining fourth obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first position, and adding the fourth obstacle avoidance information to the target placement point; and / or, determining second relevant display information based on the second environmental information, and adding the second relevant display information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0015] For example, in some embodiments, the second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; wherein the third icon is used to identify a first object where the target placement point exists, and the second image is an environmental image of the first object at the target placement point.

[0016] In some other embodiments, the method further includes: after the robot moves to the second position and before the first mechanism performs the placement action, controlling the environmental perception sensor to collect third environmental information toward the target placement point; updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point, including: if it is determined that an object exists at the target placement point based on the second environmental information, and a fifth object exists at the target placement point based on the third environmental information, and the fifth object is of the same type as the first object, keeping the obstacle avoidance information and the relevant display information of the target placement point unchanged; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0017] Secondly, embodiments of this disclosure provide an information processing device for a robot, the device comprising: a first control unit configured to control a first mechanism of the robot to process a first object; and to control the robot's environmental perception sensors to collect environmental information toward the first position or the target placement point; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from the first position, or the first mechanism placing the first object at the target placement point; and an information processing unit configured to update at least one of obstacle avoidance information at a corresponding position and relevant display information at a corresponding position based on the collected environmental information; wherein, the relevant display information at a corresponding position can be displayed on an environmental map.

[0018] In some embodiments, updating at least one of obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: clearing the related display information of the first location when it is determined from the first environmental information that there is no object at the first location.

[0019] For example, in some embodiments, the relevant display information of the first location includes a first icon displayed at the first location on the environment map, the first icon being used to identify the presence of an object at the first location.

[0020] In other embodiments, updating at least one of the obstacle avoidance information and the related display information of the first location based on the collected first environmental information at the first location includes: if it is determined from the first environmental information that a second object exists at the first location and the type of the second object is the same as the type of the first object, retaining the obstacle avoidance information and the related display information of the first location.

[0021] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location includes: determining first obstacle avoidance information based on the first environmental information when it is determined that a third object exists at the first location and the type of the third object is different from the type of the first object, and updating the obstacle avoidance information of the first location using the first obstacle avoidance information; and / or determining first relevant display information based on the first environmental information, and updating the relevant display information of the first location using the first relevant display information.

[0022] For example, in some embodiments, the first related display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map; wherein the second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location.

[0023] In some embodiments, controlling the first mechanism to place the first object at a target placement point includes: after controlling the environmental perception sensor to collect environmental information toward the first position, controlling the second mechanism of the robot to drive the robot to move to a second position; after the robot moves to the second position, controlling the first mechanism to perform a placement action to place the first object at the target placement point.

[0024] In some embodiments, the device further includes a backup unit; the backup unit is configured to back up the obstacle avoidance information and the related display information of the first location before updating the obstacle avoidance information and the related display information of the first location.

[0025] Furthermore, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: if it is determined based on the second environmental information that there is no object at the target placement point, determining second obstacle avoidance information based on the backed-up obstacle avoidance information of the first location and the target placement point, and setting the second obstacle avoidance information at the target placement point; and / or setting at least one of the backed-up relevant display information of the first location at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0026] In some embodiments, the first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; update at least one of the obstacle avoidance information and the relevant display information of the target placement point according to the collected second environmental information of the target placement point, including: if it is determined that there is an object at the target placement point according to the second environmental information and that there is no object at the target placement point according to the third environmental information, determine third obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first position, and set the third obstacle avoidance information at the target placement point; and / or, determine second relevant display information based on the second environmental information and set the second relevant display information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0027] In other embodiments, the first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; update at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point, including: if it is determined that an object exists at the target placement point based on the second environmental information, and a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, determine fourth obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first position, and add the fourth obstacle avoidance information to the target placement point; and / or determine second relevant display information based on the second environmental information, and add the second relevant display information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0028] For example, in some embodiments, the second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; wherein the third icon is used to identify a first object where the target placement point exists, and the second image is an environmental image of the first object at the target placement point.

[0029] In some other embodiments, the first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; update at least one of the obstacle avoidance information and the relevant display information of the target placement point according to the collected second environmental information of the target placement point, including: if it is determined according to the second environmental information that an object exists at the target placement point, and according to the third environmental information that a fifth object exists at the target placement point, and the fifth object is of the same type as the first object, keep the obstacle avoidance information and the relevant display information of the target placement point unchanged; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0030] Thirdly, embodiments of this disclosure provide a robot including a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the method as described in the first aspect of this disclosure.

[0031] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor or a robot, implements the method described in the first aspect of this disclosure.

[0032] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program or instructions, which, when executed by a processor or robot, implement the method described in the first aspect of this disclosure.

[0033] In a sixth aspect, embodiments of this disclosure provide a computer program that causes a processor or robot to perform the method described in the first aspect.

[0034] It is understood that in the robot provided in this embodiment, after the first mechanism controlling the robot acquires the first object at the first location, the environment at the first location may change accordingly; after the first mechanism controlling the robot places the acquired first object at the target placement point, the environment at the target placement point may also change accordingly. Therefore, after the first mechanism controlling the robot acquires the first object at the first location, or after the first mechanism controlling the robot places the acquired first object at the target placement point, the environment at the first location or the target placement point is collected by an environmental perception sensor, and the obstacle avoidance information and / or the relevant display information at the corresponding location are updated based on the collected environmental information; thus, the obstacle avoidance information and / or the relevant display information at the corresponding location change with the environment at that location; wherein, the obstacle avoidance information at the corresponding location changes with the environment at that location, which is beneficial to improving the real-time performance of the robot's obstacle avoidance behavior, thereby improving the robot's task execution efficiency; and the relevant display information at the corresponding location changes with the environment at that location, so that the relevant display information displayed on the environmental map at that location is up-to-date and consistent with the real-time environment at that location, thereby improving the user experience.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to illustrate the technical solutions of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0037] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0038] Figure 1 is a schematic diagram of the structure of the robot provided in an embodiment of this disclosure;

[0039] Figure 2 is a schematic diagram of the implementation flow of the robot information processing method provided in the embodiments of this disclosure;

[0040] Figure 3 is a schematic diagram of the structure of the robot provided in an embodiment of this disclosure;

[0041] Figure 4 is a schematic diagram of the structure of the robot provided in the embodiment of this disclosure;

[0042] Figure 5 is a schematic diagram of the structure of the robot control device provided in an embodiment of this disclosure;

[0043] Figure 6 is a schematic diagram of the structure of the robot provided in the embodiment of this disclosure. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the specific technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings of the embodiments of this disclosure. The following embodiments are used to illustrate this disclosure, but are not intended to limit the scope of this disclosure.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0046] In the following description, references to "some embodiments," "this embodiment," "this disclosure embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0047] The descriptions such as "first," "second," and "third" appearing in the embodiments of this disclosure do not have a specific meaning (such as no order distinction, nor do they indicate a special limitation on the number of devices in the embodiments of this disclosure), but are merely for the purpose of clearly describing the embodiments of this disclosure and do not constitute any limitation on the embodiments of this disclosure.

[0048] Figure 1 is a schematic diagram of the structure of a robot provided in an embodiment of this disclosure. As shown in Figure 1, the robot 100 includes a main body 101, a first mechanism 102, a screen 103, an environmental perception sensor 104, and a processor 105; wherein,

[0049] The first mechanism 102 is connected to the main body 101 and is used at least for acquiring and placing objects;

[0050] Screen 103 is located on the main body 101 of the device and is used to display an environment map;

[0051] An environmental sensing sensor 104 is installed on the main body 101 of the device to collect environmental information.

[0052] Figure 2 is a schematic flowchart illustrating the implementation of the robot information processing method provided in this embodiment of the present disclosure. This control method can be applied to the processor 105. As shown in Figure 2, the method includes the following steps 201 to 203:

[0053] Step 201: Control the first mechanism of the robot to process the first object; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from a first position, or the first mechanism placing the first object at a target placement point;

[0054] Step 202: Control the robot's environmental perception sensors to move towards the first position or the target placement point to collect environmental information.

[0055] It should be noted that, in step 202, controlling the robot's environmental perception sensor to move toward the first position to collect environmental information includes: after controlling the first mechanism to acquire the first object at the first position, controlling the robot's environmental perception sensor to move toward the first position to collect environmental information.

[0056] It should also be noted that, in step 202, controlling the robot's environmental perception sensor to collect environmental information toward the target placement point includes: after controlling the first mechanism of the robot to place the acquired first object at the target placement point, controlling the robot's environmental perception sensor to collect environmental information toward the target placement point.

[0057] Step 203: Update at least one of the obstacle avoidance information and the relevant display information at the corresponding location based on the collected environmental information; wherein the relevant display information at the corresponding location can be displayed on the environmental map on the screen.

[0058] It should be noted that in step 203, at least one of the obstacle avoidance information at the corresponding location and the relevant display information at the corresponding location is updated based on the collected environmental information. That is, at least one of the obstacle avoidance information at the first location and the relevant display information at the first location is updated based on the collected environmental information at the first location (hereinafter referred to as the first environmental information); at least one of the obstacle avoidance information at the target placement point and the relevant display information at the target placement point is updated based on the collected environmental information at the target placement point (hereinafter referred to as the second environmental information).

[0059] It is understood that in the robot provided in this embodiment, after the first mechanism controlling the robot acquires the first object at the first location, the environment at the first location may change accordingly; after the first mechanism controlling the robot places the acquired first object at the target placement point, the environment at the target placement point may also change accordingly. Therefore, after the first mechanism controlling the robot acquires the first object at the first location, or after the first mechanism controlling the robot places the acquired first object at the target placement point, the environment at the first location or the target placement point is collected by an environmental perception sensor, and the obstacle avoidance information and / or the relevant display information at the corresponding location are updated based on the collected environmental information; thus, the obstacle avoidance information and / or the relevant display information at the corresponding location change with the environment at that location; wherein, the obstacle avoidance information at the corresponding location changes with the environment at that location, which is beneficial to improving the real-time performance of the robot's obstacle avoidance behavior, thereby improving the robot's task execution efficiency; and the relevant display information at the corresponding location changes with the environment at that location, so that the relevant display information displayed on the environmental map at that location is up-to-date and consistent with the real-time environment at that location, thereby improving the user experience.

[0060] Furthermore, in some embodiments, as shown in FIG3, the robot 100 further includes a second mechanism 301; the second mechanism 301 is connected to the main body 101 and is used to move the robot 100 when driven.

[0061] It is understood that the second mechanism 301 is a component used to control the movement of the robot 100. In this embodiment of the disclosure, the second mechanism 301 is not limited, as long as the mechanism can enable the robot 100 to move when driven. Exemplarily, the second mechanism 301 includes, but is not limited to: drive wheel, driven wheel, omnidirectional wheel, spherical wheel, track, track wheel, mechanical leg, etc.

[0062] Regarding step 202 in the above embodiment, after controlling the first mechanism to acquire the first object at the first position, the robot's environmental perception sensor is controlled to move toward the first position to collect first environmental information.

[0063] In some embodiments, step 202 may further include: after controlling the first mechanism to acquire the first object at the first position, controlling the second mechanism to drive the robot to move away from the first position (e.g., move a first distance); after controlling the robot to move away from the first position, controlling the environmental perception sensor to collect first environmental information toward the first position.

[0064] It is understandable that before the environmental perception sensor collects the first environmental information at the first position, the robot is controlled to move away from the first position. The purpose is to enable the environmental perception sensor to obtain a larger field of view (i.e., observation angle), thereby obtaining first environmental information over a larger range, which in turn helps to improve the accuracy of the obstacle avoidance information and related display information update results at the first position.

[0065] However, considering that there may not be enough space for the robot to move in the direction away from the first position in a real environment, in some embodiments, controlling the second mechanism to drive the robot to move in the direction away from the first position includes: if it is determined that there is movable space in the direction away from the first position, controlling the second mechanism to drive the robot to move in the direction away from the first position. In some embodiments, the method further includes: if it is determined that there is no movable space in the direction away from the first position, controlling the environmental perception sensor to move towards the first position to collect first environmental information. That is, if it is determined that there is no movable space in the direction away from the first position, the robot no longer moves in the direction away from the first position, but directly collects the first environmental information of the first position.

[0066] The above embodiments describe how, after acquiring a first object at a first location, the robot needs to update the obstacle avoidance information and / or related display information at that first location. Specifically, after controlling the first mechanism to acquire the first object at the first location, the robot controls the environmental perception sensor to move towards the first location to collect first environmental information; and updates at least one of the obstacle avoidance information and related display information at the first location based on the collected first environmental information; wherein the related display information at the first location can be displayed on the environmental map.

[0067] It should be understood that after the first mechanism acquires the first object at the first location, there may be no object at the first location, or there may be other objects, which may be of the same or different type as the first object. The following will describe the update strategies for obstacle avoidance information and related display information at the first location for these different situations.

[0068] For step 203, update at least one of the obstacle avoidance information at the corresponding location and the relevant display information at the corresponding location based on the collected environmental information; wherein, the relevant display information at the corresponding location can be displayed on the environmental map on the screen.

[0069] It is understood that Scheme 1 in step 203 involves updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location. Further, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information includes: clearing the relevant display information of the first location if it is determined based on the first environmental information that no object exists at the first location; wherein the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

[0070] For example, if it is determined based on the first environmental information that no object exists at the first location, the first icon displayed at the first location on the environmental map is cleared. This first icon is used to identify the presence of an object at the first location. Alternatively, if it is determined based on the first environmental information that no object exists at the first location, the first icon and a third image at the first location are cleared. This third image is an environmental image of the first object.

[0071] In some embodiments, the first icon is used to identify an object at a first location and the type of the object.

[0072] In some embodiments, a user can display a third image at a first location on an environmental map by clicking a first icon at that location. That is, the method further includes displaying the third image on the screen in response to a touch operation received from the first icon on the robot's screen.

[0073] For step 203, update at least one of the obstacle avoidance information at the corresponding location and the relevant display information at the corresponding location based on the collected environmental information; wherein, the relevant display information at the corresponding location can be displayed on the environmental map on the screen.

[0074] It is understood that Scheme 1 in step 203 involves updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location. Further, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information includes: retaining the obstacle avoidance information and the relevant display information of the first location if it is determined from the first environmental information that a second object exists at the first location and the type of the second object is the same as the type of the first object; wherein the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

[0075] It should be noted that the first object and the second object do not refer to specific objects. The "first" in "first object" and the "second" in "second object" are merely for the purpose of clearly describing this technical solution. Therefore, the determination that a second object exists at the first location can also be understood as the existence of other objects at the first location.

[0076] It should be understood that controlling the first mechanism to acquire the first object at the first position may result in either the first mechanism successfully acquiring the first object or it may fail to acquire it. If the first mechanism successfully acquires the first object, the first object and the second object are two different objects. If the first mechanism fails to acquire the first object, the second object is the first object.

[0077] For step 203 of scheme 1, further, in some other embodiments, updating at least one of the obstacle avoidance information of the first location and the related display information of the first location based on the collected first environmental information may include: if it is determined based on the first environmental information that there is a third object at the first location and the type of the third object is different from the type of the first object, determining the first obstacle avoidance information based on the first environmental information and updating the obstacle avoidance information of the first location using the first obstacle avoidance information.

[0078] In some embodiments, obstacle avoidance information is used to identify obstacles, plan paths, and optimize cleaning routes. Exemplarily, the first obstacle avoidance information includes at least one of the following: object type, object location, and obstacle avoidance map. It is understood that if the type of the third object is different from the type of the first object, it indicates that the environment at the first location has changed as the first object is removed by the robot. In this case, new obstacle avoidance information (i.e., the first obstacle avoidance information) is determined based on the new environmental information (i.e., the first environmental information). For example, the first obstacle avoidance information is determined based on the first location, the type of the third object, and the obstacle avoidance information at the first location (i.e., the obstacle avoidance information before the update).

[0079] For step 203 of scheme 1, further, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location includes: if it is determined based on the first environmental information that a third object exists at the first location and the type of the third object is different from the type of the first object, determining first relevant display information based on the first environmental information and updating the relevant display information of the first location using the first relevant display information; wherein, the first environmental information is collected after controlling the first mechanism to acquire the first object at the first location.

[0080] For example, in some embodiments, the first relevant display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map;

[0081] Wherein, the second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location. Further, in some embodiments, the second icon is used to identify the presence of an object at the first location and the type of object present.

[0082] It should be understood that the reason why the first relevant display information is said to be the information to be displayed is because it will only be displayed after the relevant display information at the first position is updated using the first relevant display information.

[0083] For step 203 of scheme 1, further, in some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location includes: determining first obstacle avoidance information based on the first environmental information when it is determined that a third object exists at the first location and the type of the third object is different from the type of the first object, and updating the obstacle avoidance information of the first location using the first obstacle avoidance information; and determining first relevant display information based on the first environmental information and updating the relevant display information of the first location using the first relevant display information; wherein the first environmental information is collected after controlling the first mechanism to acquire the first object at the first location.

[0084] It is understood that after the first mechanism acquires the first object at the first location, it needs to update the obstacle avoidance information and / or related display information at that first location. The above embodiments describe different update strategies for the obstacle avoidance information and related display information at the first location after the first mechanism acquires the first object at the first location, taking into account different situations that may occur at the first location.

[0085] In step 201, a first mechanism controlling the robot processes the first object; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from a first position, or the first mechanism placing the first object at a target placement point.

[0086] Based on one or more of the above embodiments, further, in some embodiments, controlling the first mechanism to place the first object at the target placement point includes: after controlling the robot's environmental perception sensor to collect environmental information toward the first position, controlling the second mechanism to drive the robot to move to the second position; after the robot moves to the second position, controlling the first mechanism to perform a placement action to place the first object at the target placement point.

[0087] Furthermore, in some embodiments, the method further includes: after controlling the first mechanism to perform the placement action, controlling the environmental perception sensor to collect second environmental information toward the target placement point; and updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point according to the collected second environmental information; wherein the relevant display information of the target placement point can be displayed on the environmental map.

[0088] In some embodiments of the above embodiments, the step of controlling the environmental perception sensor to collect second environmental information toward the target placement point after controlling the first mechanism to perform the placement action may further include: after controlling the first mechanism 102 to perform the placement action, controlling the second mechanism to drive the robot to move in the opposite direction to the target placement point (e.g., move a second distance); and after controlling the robot to move in the opposite direction to the target placement point, controlling the environmental perception sensor 104 to collect second environmental information toward the target placement point.

[0089] It is understandable that controlling the robot to move in the opposite direction of the target placement point before the environmental perception sensor collects the second environmental information of the target placement point is intended to enable the environmental perception sensor to obtain a larger field of view (i.e., observation angle), thereby obtaining second environmental information over a wider range, which in turn helps to improve the accuracy of the obstacle avoidance information and related display information update results of the target placement point.

[0090] However, considering that there may not be enough space for the robot to move in the opposite direction of the target placement point in a real environment, in some embodiments, controlling the second mechanism to drive the robot to move in the opposite direction of the target placement point includes: if it is determined that there is movable space in the opposite direction of the target placement point, controlling the second mechanism to drive the robot to move in the opposite direction of the target placement point. In some embodiments, the method further includes: if it is determined that there is no movable space in the opposite direction of the target placement point, controlling the environmental perception sensor to collect second environmental information toward the target placement point. That is, if it is determined that there is no movable space in the opposite direction of the target placement point, the robot no longer moves in the opposite direction of the target placement point, but directly collects the second environmental information of the target placement point.

[0091] In some embodiments, the method further includes: backing up the obstacle avoidance information and related display information of the first location before updating the obstacle avoidance information and related display information of the first location. Exemplarily, in some embodiments, the backed-up obstacle avoidance information of the first location includes at least one of an obstacle avoidance map, the type of a first object, and the location of the first object (i.e., the first location). Exemplarily, in some embodiments, the backed-up related display information of the first location includes a third image, which is an environmental image of the first object at the first location.

[0092] The above embodiments describe the need for the robot to update the obstacle avoidance information and / or related display information of the target placement point after placing the first object there. Specifically, after controlling the first mechanism to perform the placement action, the robot controls the environmental perception sensor to move towards the target placement point to collect second environmental information; and updates at least one of the obstacle avoidance information and related display information of the target placement point based on the collected second environmental information; wherein the related display information of the target placement point can be displayed on the environmental map.

[0093] It should be understood that after the first mechanism places the first object at the target placement point, the target placement point may not contain any objects, or it may contain objects but not exist before the first object was placed, or it may contain objects and also contain objects before the first object was placed, and so on. The following sections will describe the update strategies for obstacle avoidance information and related display information at the target placement point for these different situations.

[0094] In some embodiments, updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: when it is determined based on the second environmental information that there is no object at the target placement point, determining second obstacle avoidance information based on the backed-up obstacle avoidance information of the first location and the target placement point, and setting the second obstacle avoidance information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0095] Further, in some embodiments, the second obstacle avoidance information includes at least one of the following: object type, object location, and obstacle avoidance map; wherein, the object type in the second obstacle avoidance information is the object type in the backup obstacle avoidance information of the first location; the object location in the second obstacle avoidance information is the target placement point; the obstacle avoidance map in the second obstacle avoidance information is a map obtained by updating the obstacle avoidance map in the backup obstacle avoidance information of the first location based on the existence of obstacles at the target placement point, or the obstacle avoidance map in the second obstacle avoidance information is the obstacle avoidance map in the backup obstacle avoidance information of the first location.

[0096] It should be understood that determining that no object exists at the target placement point based on the second environmental information may be because the robot failed to acquire the first object at the first location, or it may be because although the robot successfully acquired the first object at the first location and successfully placed it at the target placement point, the second environmental information fails to identify the object. Therefore, in the above embodiment, when it is determined that no object exists at the target placement point based on the second environmental information, second obstacle avoidance information is determined based on the backed-up obstacle avoidance information of the first location and the target placement point, and the second obstacle avoidance information is set at the target placement point. In this way, even if the robot successfully acquires the first object at the first location and successfully places it at the target placement point, but the second environmental information fails to identify the object, it is beneficial for the robot to better plan obstacle avoidance paths, thereby improving the robot's task execution efficiency.

[0097] In other embodiments, updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: setting the backed-up relevant display information of the first position at the target placement point when it is determined from the second environmental information that there is no object at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0098] In some other embodiments, updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: when it is determined based on the second environmental information that there is no object at the target placement point, determining second obstacle avoidance information based on the backed-up obstacle avoidance information of the first location and the target placement point, and setting the second obstacle avoidance information at the target placement point; and setting the backed-up relevant display information of the first location at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0099] In some embodiments, the method further includes: after the robot moves to the second position and before the first mechanism performs the placement action, controlling an environmental perception sensor to collect third environmental information toward the target placement point.

[0100] In some embodiments, updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: when it is determined that an object exists at the target placement point based on the second environmental information and that no object exists at the target placement point based on the third environmental information, determining third obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first location, and setting the third obstacle avoidance information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0101] It is understood that if the second environmental information determines that an object exists at the target placement point, and the third environmental information determines that no object exists at the target placement point, it means that the robot has successfully placed the first object at the target placement point. Therefore, the environment at the target placement point has changed compared to before the first object was placed. At this time, setting the third obstacle avoidance information at the target placement point is beneficial for the robot to avoid obstacles in a timely manner, thereby improving the robot's task execution efficiency.

[0102] Further, in some embodiments, the third obstacle avoidance information includes at least one of the following: object type, object location, and obstacle avoidance map; wherein, the object type in the third obstacle avoidance information is the object type in the backup obstacle avoidance information of the first location; the object location in the third obstacle avoidance information is the target placement point; the obstacle avoidance map in the third obstacle avoidance information is a map obtained by updating the obstacle avoidance map in the backup obstacle avoidance information of the first location based on the existence of obstacles at the target placement point, or, the obstacle avoidance map in the third obstacle avoidance information is the obstacle avoidance map in the backup obstacle avoidance information of the first location.

[0103] In other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information includes: determining second relevant display information based on the second environmental information and setting the second relevant display information at the target placement point when it is determined that an object exists at the target placement point based on the second environmental information and that no object exists at the target placement point based on the third environmental information; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0104] For example, in some embodiments, the second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; wherein the third icon is used to identify a first object present at the target placement point, and the second image is an environmental image of the first object at the target placement point. Further, in some embodiments, the third icon is used to identify the presence of a first object at the target placement point and the type of object present.

[0105] It is understandable that setting the second relevant display information after the target placement point allows the second relevant display information at that target placement point to be displayed on the environment map.

[0106] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information includes: when it is determined that an object exists at the target placement point based on the second environmental information and that no object exists at the target placement point based on the third environmental information, determining third obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first location, and setting the third obstacle avoidance information at the target placement point; and determining second relevant display information based on the second environmental information, and setting the second relevant display information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0107] In some other embodiments, updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: if it is determined that an object exists at the target placement point based on the second environmental information, and a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, determining fourth obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backed-up first location, and adding the fourth obstacle avoidance information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0108] Further, in some embodiments, the fourth obstacle avoidance information includes at least one of the following: object type, object location, and obstacle avoidance map; wherein, the object type in the fourth obstacle avoidance information is the object type in the obstacle avoidance information of the backed-up first location; the object location in the fourth obstacle avoidance information is the target placement point; the obstacle avoidance map in the fourth obstacle avoidance information is a map obtained by updating the obstacle avoidance map in the obstacle avoidance information of the backed-up first location based on the existence of obstacles at the target placement point, or, the obstacle avoidance map in the fourth obstacle avoidance information is the obstacle avoidance map in the obstacle avoidance information of the backed-up first location.

[0109] In other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information includes: determining second relevant display information based on the second environmental information when it is determined that an object exists at the target placement point, and determining that a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, and adding the second relevant display information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0110] For example, in some embodiments, the second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; wherein the third icon is used to identify a first object and the type of object present at the target placement point, and the second image is an environmental image of the first object at the target placement point.

[0111] It is understandable that adding the second relevant display information after the target placement point supports displaying the second relevant display information at that target placement point on the environment map.

[0112] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information includes: if it is determined that an object exists at the target placement point based on the second environmental information, and a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, determining fourth obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backed-up first location, and adding the fourth obstacle avoidance information to the target placement point; and determining second relevant display information based on the second environmental information, and adding the second relevant display information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0113] In some further embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information includes: if it is determined that an object exists at the target placement point based on the second environmental information, and a fifth object exists at the target placement point based on the third environmental information, and the fifth object is of the same type as the first object, then keeping the obstacle avoidance information and the relevant display information of the target placement point unchanged; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action. For example, both the fifth object and the first object are crumpled paper.

[0114] As mentioned earlier, the first mechanism 102, connected to the main body 101, is at least used for acquiring and placing objects. In this embodiment, the specific structure of the first mechanism 102 is not limited; in short, the mechanism must have the ability to acquire and place objects. For example, the first mechanism 102 may include a robotic arm, a robotic hand, a robotic gripper, etc. The specific structure of the first mechanism 102 is not limited; in short, the mechanism must have the ability to acquire and place objects. For example, referring to the robot 100 shown in Figure 4, the first mechanism 102 may be a robotic arm, which can be stored in a robotic arm compartment when not in use.

[0115] It should also be noted that the number of first mechanisms 102 can be one or more. For example, referring to the robot 100 shown in FIG4, one or more first mechanisms 102 can be provided on its main body 101. This disclosure embodiment does not limit the number of first mechanisms 102.

[0116] As mentioned earlier, the environmental perception sensor 104 is disposed on the main body 101 of the device and is used to collect environmental information. In this embodiment, the environmental perception sensor 104 is not limited. Exemplarily, in some embodiments, the environmental perception sensor 104 includes a camera; in other embodiments, it includes a camera and a laser distance sensor (LDS). In still other embodiments, it includes a camera, a laser distance sensor (LDS), and a structured light sensor. In yet another embodiment, it includes a camera and a structured light sensor.

[0117] It should be noted that the number of environmental perception sensors 104 can be one or more. When there are multiple environmental perception sensors 104, they can be set at different positions on the main body 101 of the robot to sense environmental information in different directions. For example, referring to Figure 4, two environmental perception sensors 104A and 104B are provided on the main body 101 of the robot 100. Environmental perception sensor 104A is a forward sensing sensor used to sense forward environmental information in the direction of travel of the robot 100. Environmental perception sensor 104B is a top sensing sensor used to sense environmental information on the top of the robot 100.

[0118] In this embodiment, the type of robot 100 is not limited; it can be a cleaning robot, a handling robot, or other types of robots. For a cleaning robot, the main body of robot 100 is further provided with a third mechanism for cleaning surfaces it contacts. For example, the third mechanism includes a side brush and a side mop.

[0119] The following examples illustrate possible implementation schemes for one or more of the above embodiments.

[0120] Robots equipped with robotic arms possess the ability to move obstacles. As an obstacle is moved to a new location, the environment's accessibility for the robot changes accordingly. Robotic vacuum cleaners can identify obstacle types using cameras and detect the environment in front of the machine using line lasers and / or area arrays to determine the type and size of obstacles. Upon obstacle detection, obstacle avoidance information is generated (such as object / obstacle type, object / obstacle location, and an obstacle avoidance map). When the machine approaches an obstacle, it triggers a collision signal, allowing it to avoid the detected object. Simultaneously, the robotic vacuum cleaner's app displays an icon of the detected object at the corresponding location on the map. If the user has enabled the real-view photo function, clicking the icon will display an image of the object's environment—a photo of the object at that location.

[0121] The following solution aims to provide a strategy for processing obstacle avoidance information when a robot with a robotic arm is moving obstacles, as well as a method for selecting images of objects to display on an app.

[0122] Step 1. After the robot successfully picks up the obstacle, it first backs up the type, location, obstacle avoidance map, and object-based icon display image of the currently picked-up object, and then enters the recognition and confirmation logic.

[0123] Step 2. The robot will usually back up first to gain a wider field of view; if there is no space to back up, it may not need to. Then, it will use the camera to identify the area in front of it.

[0124] If no object is recognized (i.e. no object is detected), the icon of the corresponding object displayed on the APP will be cleared.

[0125] If a new object is identified (i.e., an object is detected), and the type of the new object is the same as the original one, the obstacle avoidance information of the original object (including object type, object location, and obstacle avoidance map) will be retained, as will the icon displayed at the corresponding location of the original object on the app.

[0126] If a new object is identified, but the new object is of a different type than the original object, the obstacle avoidance information of the original object is updated with the obstacle avoidance information of the new object, the object icon is replaced on the APP, and the first frame image of the newly identified object is used as the image for displaying the object icon.

[0127] 3. Once the robot has grasped the object and moved it to the vicinity of the placement location (i.e., the target placement point), the robotic arm performs the placement action. After successfully placing the object, the robot will usually move backward first to obtain a wider viewing angle. If there is no space to move backward, it may not move backward. Then, it will use the camera to identify the area in front of it.

[0128] If no object is identified (i.e., no object is detected), the obstacle avoidance information of the backed-up object's original position and the image used to display the object icon are set at the placement location. Here, when setting the obstacle avoidance information of the backed-up object's original position at the placement location, the object's original position in the obstacle avoidance information needs to be updated to the placement location. Optionally, the obstacle avoidance map in the obstacle avoidance information of the backed-up object's original position can also be updated according to the placement location, and the updated obstacle avoidance map is set at the placement location.

[0129] If an object is identified (i.e., an object is detected), and no object was detected at that location before placement, then the obstacle avoidance information of the original location of the backed-up object is set at the placement location, and the first frame image of the newly detected object is used as the image for displaying the object icon. Here, when setting the obstacle avoidance information of the original location of the backed-up object at the placement location, the original location of the object in the obstacle avoidance information needs to be updated to the placement location. Optionally, the obstacle avoidance map in the obstacle avoidance information of the original location of the backed-up object can also be updated according to the placement location, and the updated obstacle avoidance map is set at the placement location.

[0130] If an object exists, and other types of objects already exist at that location before placement, the obstacle avoidance information of the backed-up object's original location is added to the placement location, a new object icon is added to the placement location, and the first frame image of the newly identified object is used as the display image for the object icon. Here, when adding the obstacle avoidance information of the backed-up object's original location to the placement location, the object's original location in the obstacle avoidance information needs to be updated to the placement location. Optionally, the obstacle avoidance map in the obstacle avoidance information of the backed-up object's original location can also be updated according to the placement location, and the updated obstacle avoidance map is added to the placement location.

[0131] If a recognizable object exists, and there is already a recognizable object of the same type at that location before placement, then the obstacle avoidance information and object icon at the placement location will not be processed.

[0132] It is understandable that in the above solution, when the robot with the robotic arm is transporting an object that needs to avoid obstacles, the obstacle avoidance information and the object icon and display image shown in the APP can change accordingly, so as to improve the real-time performance of the robot's obstacle avoidance behavior and allow users to intuitively feel the result of the object being transported.

[0133] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0134] Based on the foregoing embodiments, this disclosure provides a robot control device, which can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be an AI acceleration engine (such as an NPU), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0135] Figure 5 is a schematic diagram of the structure of the robot control device provided in an embodiment of this disclosure. As shown in Figure 5, the robot's information processing device 500 includes a first control unit 501 and an information processing unit 502, wherein:

[0136] The first control unit 501 is configured to control a first mechanism of the robot to process a first object; and to control the robot's environmental perception sensors to collect environmental information toward the first position or the target placement point; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from the first position, or the first mechanism placing the first object at the target placement point.

[0137] The information processing unit 502 is configured to update at least one of the obstacle avoidance information of the first location or the target placement point and the relevant display information of the first location or the target placement point based on the collected environmental information; wherein the relevant display information of the first location or the target placement point can be displayed on an environmental map.

[0138] In some embodiments, updating at least one of obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: clearing the related display information of the first location when it is determined from the first environmental information that there is no object at the first location.

[0139] For example, in some embodiments, the relevant display information of the first location includes a first icon displayed at the first location on the environment map, the first icon being used to identify the presence of an object at the first location.

[0140] In other embodiments, updating at least one of the obstacle avoidance information and the related display information of the first location based on the collected first environmental information at the first location includes: if it is determined from the first environmental information that a second object exists at the first location and the type of the second object is the same as the type of the first object, retaining the obstacle avoidance information and the related display information of the first location.

[0141] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the first location based on the collected first environmental information at the first location includes: determining first obstacle avoidance information based on the first environmental information when it is determined that a third object exists at the first location and the type of the third object is different from the type of the first object, and updating the obstacle avoidance information of the first location using the first obstacle avoidance information; and / or determining first relevant display information based on the first environmental information, and updating the relevant display information of the first location using the first relevant display information.

[0142] For example, in some embodiments, the first related display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map; wherein the second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location.

[0143] In some embodiments, controlling the first mechanism to place the first object at a target placement point includes: after controlling the environmental perception sensor to collect environmental information toward the first position, controlling the second mechanism of the robot to drive the robot to move to a second position; after the robot moves to the second position, controlling the first mechanism to perform a placement action to place the first object at the target placement point.

[0144] In some embodiments, the robot's information processing device 500 further includes a backup unit; the backup unit is configured to back up the obstacle avoidance information and the related display information of the first position before updating the obstacle avoidance information and the related display information of the first position.

[0145] Furthermore, in some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: if it is determined based on the second environmental information that there is no object at the target placement point, determining second obstacle avoidance information based on the backed-up obstacle avoidance information of the first location and the target placement point, and setting the second obstacle avoidance information at the target placement point; and / or setting at least one of the backed-up relevant display information of the first location at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0146] In some embodiments, the first control unit 501 is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point.

[0147] In some embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point includes: if it is determined that an object exists at the target placement point based on the second environmental information, and it is determined that no object exists at the target placement point based on the third environmental information, determining third obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backup first location, and setting the third obstacle avoidance information at the target placement point; and / or, determining second relevant display information based on the second environmental information, and setting the second relevant display information at the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0148] In other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the collected second environmental information of the target placement point includes: if it is determined that an object exists at the target placement point based on the second environmental information, and a fourth object exists at the target placement point based on the third environmental information, and the fourth object is of a different type from the first object, determining fourth obstacle avoidance information based on the obstacle avoidance information of the target placement point and the backed-up first location, and adding the fourth obstacle avoidance information to the target placement point; and / or, determining second relevant display information based on the second environmental information, and adding the second relevant display information to the target placement point; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0149] For example, in some embodiments, the second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; wherein the third icon is used to identify a first object where the target placement point exists, and the second image is an environmental image of the first object at the target placement point.

[0150] In some other embodiments, updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the target placement point includes: if it is determined that an object exists at the target placement point based on the second environmental information, and a fifth object exists at the target placement point based on the third environmental information, and the fifth object is of the same type as the first object, then keeping the obstacle avoidance information and the relevant display information of the target placement point unchanged; wherein the second environmental information is collected after controlling the first mechanism to perform the placement action.

[0151] The description of the above apparatus embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the description of the method embodiments of this disclosure for understanding.

[0152] It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit by two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0153] It should be noted that, in the embodiments of this disclosure, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware and software combination.

[0154] This disclosure provides a robot. Figure 6 is a schematic diagram of the structure of the robot provided in this disclosure. As shown in Figure 6, the robot 600 includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602. When the processor 602 executes the program, it implements the steps in the method provided in the above embodiments.

[0155] It should be noted that the memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or already processed in the processor 602 and the various modules in the robot 600 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0156] This disclosure also provides a computer-readable storage medium for storing computer programs.

[0157] Optionally, the computer-readable storage medium may be applied to a processor or robot in the embodiments of this disclosure, and the computer program causes the processor or robot to perform various methods of the embodiments of this disclosure, which will not be described in detail here for the sake of brevity.

[0158] This disclosure also provides a computer program product, including computer program instructions.

[0159] Optionally, the computer program product can be applied to the processor or robot in the embodiments of this disclosure, and the computer program instructions cause the processor or robot to perform the various methods of the embodiments of this disclosure, which will not be described in detail here for the sake of brevity.

[0160] This disclosure also provides a computer program.

[0161] Optionally, the computer program can be applied to the processor or robot in the embodiments of this disclosure. When the computer program runs on the processor or robot, it causes the processor or robot to perform the various methods of the embodiments of this disclosure. For the sake of brevity, these will not be described in detail here.

[0162] It should be noted that the descriptions of the storage medium, computer program product, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, computer program product, and computer program embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0163] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0164] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0165] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0167] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, each functional module in the various embodiments of this disclosure can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0169] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0170] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0171] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0172] The features disclosed in the several product embodiments provided in this disclosure can be combined arbitrarily without conflict to obtain new product embodiments.

[0173] The features disclosed in the several method or device embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0174] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An information processing method for a robot, comprising: A first mechanism controlling the robot processes a first object; wherein, the first mechanism processing the first object includes the first mechanism acquiring the first object from a first position, or the first mechanism placing the first object at a target placement point; The robot's environmental perception sensors are directed toward the first position or the target placement point to collect environmental information. The obstacle avoidance information and the relevant display information at the corresponding location are updated based on the collected environmental information; wherein the relevant display information at the corresponding location can be displayed on the environmental map.

2. The method according to claim 1, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If it is determined that there is no object at the first location based on the first environmental information, the relevant display information for the first location is cleared; wherein, the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

3. The method according to claim 2, wherein, The relevant display information for the first location includes a first icon displayed at the first location on the environmental map, which is used to identify the presence of an object at the first location.

4. The method according to claim 1, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If, based on the first environmental information, it is determined that a second object exists at the first location, and the type of the second object is the same as the type of the first object, the obstacle avoidance information and related display information of the first location are retained; wherein, the first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

5. The method according to claim 1, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If, based on the first environmental information, it is determined that a third object exists at the first location, and the type of the third object is different from the type of the first object, then first obstacle avoidance information is determined based on the first environmental information, and the obstacle avoidance information at the first location is updated using the first obstacle avoidance information. And / or, determine first relevant display information based on the first environmental information, and update the relevant display information of the first location using the first relevant display information; The first environmental information is collected after the first mechanism is controlled to acquire the first object at the first location.

6. The method according to claim 5, wherein, The first relevant display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map; The second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location.

7. The method according to any one of claims 1-6, wherein, Controlling the first mechanism to place the first object at the target placement point includes: After controlling the environmental perception sensor to collect environmental information at the first position, the second mechanism of the robot is controlled to drive the robot to move to the second position; After the robot moves to the second position, the first mechanism is controlled to perform a placement action to place the first object at the target placement point.

8. The method according to claim 7, wherein, The method further includes: Before updating the obstacle avoidance information and related display information of the first location, back up the obstacle avoidance information and related display information of the first location.

9. The method according to claim 8, wherein, Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that there is no object at the target placement point based on the second environmental information, second obstacle avoidance information is determined based on the obstacle avoidance information of the backed-up first location and the target placement point, and the second obstacle avoidance information is set at the target placement point; and / or, at least one of the relevant display information of the backed-up first location is set at the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

10. The method according to claim 8, wherein, The method further includes: After the robot moves to the second position and before the first mechanism performs the placement action, the environmental perception sensor is controlled to move toward the target placement point to collect third environmental information; Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that there is an object at the target placement point based on the second environmental information, and it is determined that there is no object at the target placement point based on the third environmental information, then the third obstacle avoidance information is determined based on the obstacle avoidance information of the target placement point and the backup first position, and the third obstacle avoidance information is set at the target placement point. And / or, determine second relevant display information based on the second environmental information, and set the second relevant display information at the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

11. The method according to claim 8, wherein, The method further includes: After the robot moves to the second position and before the first mechanism performs the placement action, the environmental perception sensor is controlled to move toward the target placement point to collect third environmental information; Updating at least one of the obstacle avoidance information of the target placement point and the relevant display information of the target placement point based on the collected second environmental information includes: If, based on the second environmental information, it is determined that an object exists at the target placement point, and based on the third environmental information, it is determined that a fourth object exists at the target placement point, and the fourth object is of a different type from the first object, then, based on the obstacle avoidance information of the target placement point and the backed-up obstacle avoidance information of the first location, the fourth obstacle avoidance information is determined and added to the target placement point. And / or, determine second relevant display information based on the second environmental information, and add the second relevant display information to the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

12. The method according to claim 10 or 11, wherein, The second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; The third icon is used to identify the first object present at the target placement point, and the second image is an environmental image of the first object at the target placement point.

13. The method according to claim 8, wherein, The method further includes: After the robot moves to the second position and before the first mechanism performs the placement action, the environmental perception sensor is controlled to move toward the target placement point to collect third environmental information; Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that an object exists at the target placement point based on the second environmental information, and a fifth object exists at the target placement point based on the third environmental information, and the fifth object is of the same type as the first object, the obstacle avoidance information and related display information of the target placement point shall remain unchanged. The second environmental information is collected after the first mechanism is controlled to perform the placement action.

14. An information processing device for a robot, comprising: The first control unit is configured to control the first mechanism of the robot to process the first object; The first mechanism controls the robot's environmental perception sensors to collect environmental information by moving them toward a first position or a target placement point; wherein, the first mechanism processes the first object by either acquiring the first object from the first position or placing the first object at the target placement point. The information processing unit is configured to update at least one of obstacle avoidance information and relevant display information at the corresponding location based on the collected environmental information; wherein the relevant display information at the corresponding location can be displayed on an environmental map.

15. The apparatus according to claim 14, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If it is determined from the first environmental information that there is no object at the first location, the relevant display information at the first location is cleared.

16. The apparatus according to claim 14, wherein, The relevant display information for the first location includes a first icon displayed at the first location on the environmental map, which is used to identify the presence of an object at the first location.

17. The apparatus according to claim 14, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If, based on the first environmental information, it is determined that a second object exists at the first location, and the type of the second object is the same as the type of the first object, the obstacle avoidance information and related display information of the first location are retained.

18. The apparatus according to claim 14, wherein, Updating at least one of the obstacle avoidance information and related display information of the first location based on the collected first environmental information at the first location includes: If, based on the first environmental information, it is determined that a third object exists at the first location, and the type of the third object is different from the type of the first object, then first obstacle avoidance information is determined based on the first environmental information, and the obstacle avoidance information at the first location is updated using the first obstacle avoidance information. And / or, determine first relevant display information based on the first environmental information, and update the relevant display information of the first location using the first relevant display information.

19. The apparatus according to claim 18, wherein, The first relevant display information includes at least one of a second icon to be displayed at a first location on the environment map and a first image to be displayed at the first location on the environment map; The second icon is used to identify the presence of an object at the first location, and the first image is an environmental image of the third object at the first location.

20. The apparatus according to any one of claims 14-19, wherein, Controlling the first mechanism to place the first object at the target placement point includes: After controlling the environmental perception sensor to collect environmental information at the first position, the second mechanism of the robot is controlled to drive the robot to move to the second position; After the robot moves to the second position, the first mechanism is controlled to perform a placement action to place the first object at the target placement point.

21. The apparatus according to claim 20, wherein, The device further includes a backup unit; the backup unit is configured to: Before updating the obstacle avoidance information and related display information of the first location, back up the obstacle avoidance information and related display information of the first location.

22. The apparatus according to claim 21, wherein, Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that there is no object at the target placement point based on the second environmental information, second obstacle avoidance information is determined based on the obstacle avoidance information of the backed-up first location and the target placement point, and the second obstacle avoidance information is set at the target placement point; and / or, at least one of the relevant display information of the backed-up first location is set at the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

23. The apparatus according to claim 21, wherein, The first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that there is an object at the target placement point based on the second environmental information, and it is determined that there is no object at the target placement point based on the third environmental information, then the third obstacle avoidance information is determined based on the obstacle avoidance information of the target placement point and the backup first position, and the third obstacle avoidance information is set at the target placement point. And / or, determine second relevant display information based on the second environmental information, and set the second relevant display information at the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

24. The apparatus according to claim 21, wherein, The first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If, based on the second environmental information, it is determined that an object exists at the target placement point, and based on the third environmental information, it is determined that a fourth object exists at the target placement point, and the fourth object is of a different type from the first object, then, based on the obstacle avoidance information of the target placement point and the backed-up obstacle avoidance information of the first location, the fourth obstacle avoidance information is determined and added to the target placement point. And / or, determine second relevant display information based on the second environmental information, and add the second relevant display information to the target placement point; The second environmental information is collected after the first mechanism is controlled to perform the placement action.

25. The apparatus according to claim 23 or 24, wherein, The second related display information includes at least one of a third icon to be displayed at the target placement point on the environment map and a second image to be displayed at the target placement point on the environment map; The third icon is used to identify the first object present at the target placement point, and the second image is an environmental image of the first object at the target placement point.

26. The apparatus according to claim 21, wherein, The first control unit is further configured to: after the robot moves to the second position and before the first mechanism performs the placement action, control the environmental perception sensor to collect third environmental information toward the target placement point; Updating at least one of the obstacle avoidance information and the relevant display information of the target placement point based on the second environmental information of the collected target placement point includes: If it is determined that an object exists at the target placement point based on the second environmental information, and a fifth object exists at the target placement point based on the third environmental information, and the fifth object is of the same type as the first object, the obstacle avoidance information and related display information of the target placement point shall remain unchanged. The second environmental information is collected after the first mechanism is controlled to perform the placement action.

27. A robot comprising a memory and a processor, wherein, The memory stores a computer program that can run on a processor, which, when executing the program, implements the steps of the method as described in any one of claims 1 to 13.

28. A computer program product comprising a computer program or instructions, wherein, When the computer program or instructions are executed by a processor or robot, they implement the steps of the method as described in any one of claims 1 to 13.

29. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor or robot, it implements the steps of the method as described in any one of claims 1 to 13.

Images